Native UI testing / automation with node.js

nut.js is a cross-platform native UI testing tool.

It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.

The following snippet shows a valid NUT example (on macOS)

"use strict";

const { keyboard, Key, mouse, left, right, up, down } = require("@nut-tree/nut-js");

const square = async () => {
  await mouse.move(right(500));
  await mouse.move(down(500));
  await mouse.move(left(500));
  await mouse.move(up(500));
};

const openSpotlight = async () => {
  await keyboard.pressKey(Key.LeftSuper);
  await keyboard.pressKey(Key.Space);
  await keyboard.releaseKey(Key.Space);
  await keyboard.releaseKey(Key.LeftSuper);
};

describe("Basic test", () => {
  it("Should run a simple test", async () => {
    await square();
    await openSpotlight();
    await keyboard.type("calculator");
    await keyboard.type(Key.Return);
  });
});

Running

npm i @nut-tree/nut-js

or

yarn add @nut-tree/nut-js

will install nut.js and its required dependencies.

The installation process assumes you do not have an existing OpenCV installation and will try to build it from source via opencv4nodejs.

Building OpenCV from scratch requires a cmake installation.

In case you already have an OpenCV installation (version 3.x.x required, e.g. via brew install opencv@3 or else), you can disable the build process via environment variable:

export OPENCV4NODEJS_DISABLE_AUTOBUILD=1

or

set OPENCV4NODEJS_DISABLE_AUTOBUILD=1

Please make sure to also install all required peer dependencies:

• opencv4nodejs
• robotjs

nut-tree/trailmix contains a set of ready to use examples which demo the usage ot nut.js.

This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.

• Copy text to clipboard
• Paste text from clipboard

• Support for standard US keyboard layout
• Support for German special characters

• Support for mouse movement
• Support for mouse scroll
• Configurable movement speed
• Mouse drag

• Retrieve the region of a process window

• findOnScreen
• waitFor
• Hooks to trigger actions based on images

• Jest