FindText - Capture screen image into text and then find it

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: FindText - Capture screen image into text and then find it

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 23 Sep 2018, 17:48

@FastLearner, Don't copy the form of automatically generated code. :headwall:
I believe you can learn to set search scope if you read the introduction. :beer:

;————————————————————

returnArray := FindText(
X --> upper left corner X coordinates
, Y --> upper left corner Y coordinates
, W --> the search scope's Width
, H --> the search scope's Height
, Character "0" fault-tolerant in percentage --> 0.1=10%
, Character "_" fault-tolerant in percentage --> 0.1=10%
, text --> The Base64 encoding string for the text to find
)

The range used by AHK is determined by the upper left
corner and the lower right corner: (x1, y1, x2, y2),
it can be converted to: (x1, y1, x2-x1+1, y2-y1+1).

;————————————————————

If you want to search the full screen, you can do this:
FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0,0,Text)

If you want to search window scope, you can do this:
WinGetPos, x, y, w, h, Window Title
FindText(x, y, w, h, 0,0,Text)

Re: FindText - Capture screen image into text and then find it

Post by FastLearner » 22 Sep 2018, 21:51

Pardon me, I just figured I did something dumb :oops: :crazy:

This is the Gui coordinates (triangle for the part of the screen I'm using but the image is somewhere in between and it's in different coordinates almost each time its found(but ofc it's not found by the tool because coordinates changes all the time)

Code: [Select all]GeSHi © Codebox Plus

;screen part I'm using
Gui, Show, x450 y34 w430 h670, TexT


if I put it like this, the first two coordinates (450+430//2,34+670//2) means the image I'm looking for is located there, right?

Code: [Select all]GeSHi © Codebox Plus

FindText(450+430//2,34+670//2,430//2,670//2,0,0,Text)


so in the script it will be like this

Code: [Select all] [Expand]GeSHi © Codebox Plus



but I guess it shouldn't be that way...how to make a rectangle instead of the whole screen and make it search in between? :cry:

if the gui placing is different then please guide me on this :roll:

Re: FindText - Capture screen image into text and then find it

Post by FastLearner » 22 Sep 2018, 21:27

After reading a bit in the comments I found the way and tested with gui to see if the rectangle is placed right or not and I got this
FindText(450+430//2,34+670//2,430//2,670//2,0,0,Text)

but my main issue still stands, when the image coordinates is changed what can we do? :headwall:

Re: FindText - Capture screen image into text and then find it

Post by FastLearner » 22 Sep 2018, 20:56

Hello Feiyue,

I have been using your awesome tool for months now maybe a year :D and it's pretty much AWESOME! :clap:

I just encountered a problem few days ago, it happens when the image moves from its place..so for example this code here

Code: [Select all] [Expand]GeSHi © Codebox Plus



It happens with anything of this kind...if the image is moved (searching tiles in a game so the coordinates will be different almost each time the image is found) I get failed Msgbox.

how can I make it search whole screen or for speed (I guess) how can I make it search a rectangle for example...I have no idea what coordinates to put where on script to make a rectangle for example (the game screen is a small rectangle in the middle of the screen)

I have a couple of coordinate finder including your tool when pressing capture..but how to specify a specific part of the screen (if it makes any difference in speed) and how to make the first two coordinates (662-150000//2, 409-150000//2) not fixed so it can get the image even if not found in these two specific coordinates (I guess)

I tried making them 0 or 150000, but no luck...once I move a bit in any direction so the image's coordinates changes...I get a fail

Any help would be appreciated

Thank you for your awesome work :salute:

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 21 Sep 2018, 23:41

nwr425 wrote:---------images 1 to 8 all identical-----------

---1st action/click---
[1]234
5678

--2nd action/click--
x123
[4]567

--3rd action/click---
x[1]23
x456

--4th action/click--
xx12
x[3]45

--5th action/click---
xx[1]2
xx34

--6th action/click--
xxx1
xx[2]3

--7th action/click--
xxx[1]
xxx2

--8th action/click--
xxxx
xxx[1]


Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 21 Sep 2018, 22:15

Thank you Feiyue,

Also what part of the code would I edit to click in a random location within the image's dimensions?
For example, If I did not want to click in the same location on the found image each time.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 21 Sep 2018, 04:43

nwr425 wrote:Was hoping this would register multiple identical images and possibly number them, if numbered, I'm sure what I displayed would work fine


I wrote a simple function SortOK() to sort the array by coordinates. See the example: :dance:

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 21 Sep 2018, 01:53

feiyue wrote:@nwr425,
Although this function FindText() returns all the locations found,
because the lookup is from top to bottom,
sometimes the middle image is higher than the first image,
so it returns first, so the position returned may be disordered and difficult to sort..

I find that the location of these pictures looks unchanged,
so get the coordinates of the first image as the origin,
you could use relative coordinates to click on other images. Like this:

Code: [Select all] [Expand]GeSHi © Codebox Plus





Ye I'm running something similar atm, thank you!
Was hoping this would register multiple identical images and possibly number them, if numbered, I'm sure what I displayed would work fine

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 20 Sep 2018, 14:27

Updated to the v6.0 version. :beer:
1. Slightly modified the machine code.
The image that has been found will be cleaned up in the same color.
Avoid matching the next row (column) when using tolerance lookup.

2. After upgrading to v6.0, the search area uses WinAPI's
upper left corner X, Y coordinates, and width, height.
This will be better understood and used.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 20 Sep 2018, 13:55

@nwr425,
Although this function FindText() returns all the locations found,
because the lookup is from top to bottom,
sometimes the middle image is higher than the first image,
so it returns first, so the position returned may be disordered and difficult to sort..

I find that the location of these pictures looks unchanged,
so get the coordinates of the first image as the origin,
you could use relative coordinates to click on other images. Like this:

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 20 Sep 2018, 02:37

@feiyue

I want to put this to code using findtext. I want to select a different image other than the one in the top left slot.
Does this program order identical images in a way to do that? i.e x.1 x.2 x.3 x.4 etc etc

Hope this helps clarify! Thanks for help!
https://imgur.com/a/oXmewZy

Default:
Image

Step 1:
Image

Step 2:
Image

Step 3:
Image

Step 4:
Image

Step 5:
Image

Step 6:
Image

Step 7:
Image

Step 8:
Image

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 20 Sep 2018, 02:19

The numbers represent identical images that are being focused by FindText
My end goal is to:

-Click the image1 then immediately after that happens, the image1 that has been clicked will disappear.

-Then I would like to click the image that was directly below the image that previous disappeared.

-Below is an example of image [1] being clicked, and then the next Image to be clicked will be demonstrated using [brackets]

-All images that have disappeared previously will be shown using 'x'

-When 1-8 identical images are present, I would like to click the images in the order shown using brackets.

---DEFAULT BEFORE ANY CLICKS ARE SENT---
1234
5678
---------images 1 to 8 all identical-----------

---1st action/click---
[1]234
5678

--2nd action/click--
x123
[4]567

--3rd action/click---
x[1]23
x456

--4th action/click--
xx12
x[3]45

--5th action/click---
xx[1]2
xx34

--6th action/click--
xxx1
xx[2]3

--7th action/click--
xxx[1]
xxx2

--8th action/click--
xxxx
xxx[1]

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 19 Sep 2018, 20:21

@nwr425, Your description is too vague, I still don't understand what you mean.
You can upload the work picture screenshots, and then explain how to do manual work step by step.

If the position of the image remains unchanged after clicking,
you can search for the unique image next to it as the origin,
and then use the offset coordinates relative to the origin to perform the click.

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 19 Sep 2018, 20:05

leosouza85 wrote:
nwr425 wrote:
feiyue wrote:Hi, @nwr425, Although this function FindText() returns all the locations found,
it is recommended to look for images that appear only once in the full screen.

1. When you write web page automation scripts,
you should try to avoid looking for multiple identical images,
try to use a unique image, to avoid unforeseen errors.
Finding another unique image and then using offset coordinates
can easily replace the coordinates of the original image.

2. If you have to use an image that appears in more than one place,
you can narrow the search area to ensure that no two original images
appear in each search area. Don't use full screen lookup.


Any way to select which order/skip the image order?
I have 20 small identical images within close proximity to one another. Hoping to select/click images in the following order:
1>5
1>4
1>3
1>2
1>5
1>4
1>3
repeating/etc...


Those images are in order vertically or horizontally? or in random places?


Visual Example:::
[1][2][3][4]
[5]repeating

Theory of working script[focused images]:::
[1]234
[5]678
--------
x[1]23
x[4]56
--------
xx[1]2
xx[3]4
-------
xxx[1]
xxx[2]
--------
[1]234
[5]678
-------
repeating

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 19 Sep 2018, 15:25

nwr425 wrote:
feiyue wrote:Hi, @nwr425, Although this function FindText() returns all the locations found,
it is recommended to look for images that appear only once in the full screen.

1. When you write web page automation scripts,
you should try to avoid looking for multiple identical images,
try to use a unique image, to avoid unforeseen errors.
Finding another unique image and then using offset coordinates
can easily replace the coordinates of the original image.

2. If you have to use an image that appears in more than one place,
you can narrow the search area to ensure that no two original images
appear in each search area. Don't use full screen lookup.


Any way to select which order/skip the image order?
I have 20 small identical images within close proximity to one another. Hoping to select/click images in the following order:
1>5
1>4
1>3
1>2
1>5
1>4
1>3
repeating/etc...


Those images are in order vertically or horizontally? or in random places?

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 19 Sep 2018, 05:37

feiyue wrote:Hi, @nwr425, Although this function FindText() returns all the locations found,
it is recommended to look for images that appear only once in the full screen.

1. When you write web page automation scripts,
you should try to avoid looking for multiple identical images,
try to use a unique image, to avoid unforeseen errors.
Finding another unique image and then using offset coordinates
can easily replace the coordinates of the original image.

2. If you have to use an image that appears in more than one place,
you can narrow the search area to ensure that no two original images
appear in each search area. Don't use full screen lookup.


Any way to select which order/skip the image order?
I have 20 small identical images within close proximity to one another. Hoping to select/click images in the following order:
1>5
1>4
1>3
1>2
1>5
1>4
1>3
repeating/etc...

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 18 Sep 2018, 20:41

Hi, @nwr425, Although this function FindText() returns all the locations found,
it is recommended to look for images that appear only once in the full screen.

1. When you write web page automation scripts,
you should try to avoid looking for multiple identical images,
try to use a unique image, to avoid unforeseen errors.
Finding another unique image and then using offset coordinates
can easily replace the coordinates of the original image.

2. If you have to use an image that appears in more than one place,
you can narrow the search area to ensure that no two original images
appear in each search area. Don't use full screen lookup.

Re: FindText - Capture screen image into text and then find it

Post by nwr425 » 18 Sep 2018, 18:17

feiyue wrote:Hi

@feiyue or Anyone else.

I looked through this thread but did not find simple terms of how to complete the following:

When finding multiple identical images on a page, my end goal is to click images in the following order:
1>5
1>4
1>3
1>2
1>5
1>4
1>3
repeating/etc...

I know the normal order is top and left image first. Looking for a work around to skip images 2-4 on the first action, then 2-3, then 2, then repeat.
I can be reached on this thread of the offical AutoHotkey Discord under the name "o0PkVids"

Thanks ahead of time!

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 10 Sep 2018, 21:39

Hi, @feiyue, firstly thank you so much for your work, can I ask you something vey very simple? Could you make the script generate the code already inside a verification loop? (loop search this graphics until it finds it). I think it will help us in most use cases! Thank you!

Re: FindText - Capture screen image into text and then find it

Post by leosouza85 » 09 Sep 2018, 23:31

Keep up the good work! It is awesome!

Top