Skip to content

screen.find neglects offsets when providing a search region #160

Description

@s1hofmann

Version
1.4.3-next.20200813090951

Short overview
When providing a search region to screen.find the resulting Region is off.

Issue occurs on

  • Virtual machine
  • Docker container
  • Dev/Host system

Detailed error description
When providing a search region to screen.find, the returned Region is relative to the provided search region.
It does not incorporate offsets provided by the search region, so the result does not properly translate onto desktop space.

Steps to reproduce error

  • Define a new Region containing your target image, e.g. const searchRegion = new Region(300, 200, 500, 500);
  • Search for the target image, providing a search region: const detectedRegion = await screen.find("target_image.png", {searchRegion})
  • Highlight the detected region: highlight(detectedRegion)
  • -> Highlighted region is off by x and y coordinates of our searchRegion
  • Add the offset to the result region:
const correctedRegion = new Region(searchRegion.left + detectedRegion.left, searchRegion.top + detectedRegion.top, detectedRegion.width, detectedRegion.height);
highlight(correctedRegion);
  • -> Correct region is highlighted
    Additional content

Please provide any (mandatory) additional data to reproduce the error (Dockerfiles etc.)

const { screen, Region } = require("@nut-tree/nut-js");

(async () => {
    const searchRegion = new Region(300, 400, 500, 500);
    const detectedRegion = await screen.find("image.png", {searchRegion})
    await screen.highlight(detectedRegion);
    const correctedRegion = new Region(
        searchRegion.left + detectedRegion.left,
        searchRegion.top + detectedRegion.top,
        detectedRegion.width,
        detectedRegion.height
    );
    await screen.highlight(correctedRegion);
})();

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions