Native UI testing / automation with node.js

nut.js is a cross-platform native UI testing tool.

It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.

The following snippet shows a valid NUT example (on macOS)

"use strict";

const { keyboard, Key, mouse, left, right, up, down } = require("@nut-tree/nut-js");

const square = async () => {
  await mouse.move(right(500));
  await mouse.move(down(500));
  await mouse.move(left(500));
  await mouse.move(up(500));
};

const openSpotlight = async () => {
  await keyboard.pressKey(Key.LeftSuper);
  await keyboard.pressKey(Key.Space);
  await keyboard.releaseKey(Key.Space);
  await keyboard.releaseKey(Key.LeftSuper);
};

describe("Basic test", () => {
  it("Should run a simple test", async () => {
    await square();
    await openSpotlight();
    await keyboard.type("calculator");
    await keyboard.type(Key.Return);
  });
});

While nut.js comes with a pre-built version of OpenCV for your respective target platform, it currently still requires some 3rd-party dependencies to generate node bindings. This setup will be enhanced in the future so no additional packages have to be installed on a target system.

Until then, the following tools / libraries need to be installed in order to use nut.js:

The installation process comes with a pre-built version of OpenCV for the respective target platform. It's not required to manually build or install OpenCV, but the installation process has some 3rd-party dependencies.

In order to install nut.js on Windows, Windows Build Tools and Python 2 are required. You can either set them up manually, or install them via npm:

npm install --global windows-build-tools

or

yarn global add windows-build-tools

On macOS, Xcode command line tools are required. You can install them by running

xcode-select --install

Depending on your distribution, Linux setups may differ.

In general, nut.js requires

• Python 2
• g++
• make
• libXtst
• libPng

Installation on *buntu distributions:

sudo apt-get install build-essential python libxtst-dev libpng++-dev

Setups on other distributions might differ.

For reference, please see:

• opencv4nodejs
• robotjs

Running

npm i @nut-tree/nut-js

or

yarn add @nut-tree/nut-js

will install nut.js and its required dependencies.

nut-tree/trailmix contains a set of ready to use examples which demo the usage ot nut.js.

This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.

• Copy text to clipboard
• Paste text from clipboard

• Support for standard US keyboard layout
• Support for German special characters

• Support for mouse movement
• Support for mouse scroll
• Configurable movement speed
• Mouse drag

• Retrieve the region of a process window

• findOnScreen
• waitFor
• Hooks to trigger actions based on images

• Jest