FindText - Capture screen image into text and then find it

Post a reply


In an effort to prevent automatic submissions, we require that you type the text displayed into the field underneath.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: FindText - Capture screen image into text and then find it

Re: FindText - Capture screen image into text and then find it

Post by songdg » 08 Oct 2017, 02:41

Thanks for sharing,this is an awesome tool!I wonder how to turn off the always on top feature,it cause a little bit inconvenient,thanks again.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 03 Oct 2017, 04:47

Frosty Fridge wrote:I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Updated to the v5.4 version, Realized your idea. :superhappy:
However, even if the returned object contains multiple locations,
You also need loop processing to find the top or leftmost one.

Re: FindText - Capture screen image into text and then find it

Post by Frosty Fridge » 02 Oct 2017, 20:06

Hello feiyue,

First off I'd like to say that this script is amazing! It's both powerful and easy to use.

I noticed that you added support for returning info in an array for multiple different images

feiyue wrote:Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5


I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Thanks for making this script!

Re: FindText - Capture screen image into text and then find it

Post by ozzii » 02 Oct 2017, 04:15

@Joe Glines
Do you need to update your custom scripts for the latest update ???

Re: FindText - Capture screen image into text and then find it

Post by ali80 » 01 Oct 2017, 17:57

Hi feiyue, thanks for the great work
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 29 Sep 2017, 11:41

Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, Comment:=ok[i].5

Re: FindText - Capture screen image into text and then find it

Post by Helgef » 29 Sep 2017, 01:50

Joe Glines wrote:Great job feiyue!
Indeed it seems very nice.
Joe Glines wrote:I created a tutorial showing how to use the basics of the script.
Hello Joe Glines :wave:
I watched your video when you posted it, but forgot to comment. It is very nice and helpful. I hope feiyue links to it in the first post so others can find.

Cheers.

Re: FindText - Capture screen image into text and then find it

Post by vava » 28 Sep 2017, 17:19

hello All...
I like this program. but have no luck to test all the functions.
each time click on the "test" function get an error "Can't run AHK script!" .
Running windows 7 64bit with the latest Autohotkey. hope some can help.

Re: FindText - Capture screen image into text and then find it

Post by Reloaded » 28 Aug 2017, 15:17

Wow best Script i ever see :D

Re: FindText - Capture screen image into text and then find it

Post by Joe Glines » 28 Aug 2017, 12:40

Great job feiyue! This script rocks!
I created a tutorial showing how to use the basics of the script.


I also took a stab at writing some functions to click/send text, etc. to the image once it was found.

Code: [Select all] [Expand]GeSHi © Codebox Plus



You can check out a video I did using the custom scripts and more on my page here

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 26 Aug 2017, 02:10

The following code is for your reference.

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by ozyank » 25 Aug 2017, 16:08

First, thank you feiyue for the really nice script. I'm very new to scripting and AHK, but I've been able to use your example below and it works great. Much faster than trying to do the same thing in imagesearch.

As I said, I'm new to all of this so looking for advice on how to modify your script to conduct a nested search within the initial loop. The flow would look something like this:

1. Search for images using your original multi-image search loop.
2. If Image 1 found, click ;; I have this working
3. If Image 2 found, click ;; I have this working
4. If Image 3 found, click And
4a. While Image 3a, 3b and 3c are on the screen, click each until all have been clicked. ;; When Image 3 is clicked, it brings up another screen with 3 images on it. When an image is clicked, it is removed from the screen. I want this nested loop to click each of these until none are remaining. There may be 1, 2 or 3 images present so it needs to click until all of them are gone.

Images in initial loop:
Image 1
Image 2
Image 3

Images in nested loop initiated when Image 3 is clicked:
Image 3a
Image 3b
Image 3c

Pseudocode:
If Image 1 is found, Click
If Image 2 is found, Click
If Image 3 is found, Click
While (Image 3a and/or 3b and/or 3c) are on screen, Click each until none remain

I'm not sure I've explained this very well so please let me know if I can clarify. Thanks for any assistance!


feiyue wrote:
rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?

This problem is easy to solve, you can do it like this:

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by guest3456 » 13 Jun 2017, 10:04

stealzy wrote:In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.


+1

Re: FindText - Capture screen image into text and then find it

Post by 20170201225639 » 13 Jun 2017, 09:57

aack! my bad for not reading the discussions closely. thanks stealzy for pointing that out.
i realized another poster also warned against this misunderstanding earlier (https://autohotkey.com/boards/viewtopic ... 319#p94319)

Re: FindText - Capture screen image into text and then find it

Post by stealzy » 13 Jun 2017, 02:58

20170201225639, I use VimFx firefox addon and understand your idea, but it won't work because different windows have different fonts/sizes. Even in one window may be dozens combinations.
In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.

Re: FindText - Capture screen image into text and then find it

Post by 20170201225639 » 11 Jun 2017, 13:04

edit: please ignore —— i misunderstood what the script does

it just occurred to me there's an excellent use case for this script.

i've been working outside in the park the last couple of days (thanks to a really capacious laptop power bank i just got recently) and noticed that the fresh air and sun made a huge difference to productivity. i doubt i'll ever (voluntarily) go back to my office again!

anyway, since using mouse on a park bench or grass is inconvenient (and i don't like touchpad either), i've been trying to use autohotkey to make everything i needed to do on my computer, doable via keyboard

i find that although defined hotkeys are an important part of the workflow, it's also really important to have a convenient way to move mouse pointer with your keyboard, because a lot of applications these days are not designed to be in principle operable with keyboard only.

i had some rudimentary scripts to do this, and also tried a couple of scripts i found on the board. but none of them can really move mouse pointer as fast as with an actual mouse.

then it occurred to me .. you know how many vim users become accustomed to move caret not by hjkl, but by search (typing 'f' then type some relevant char)? and i realized that with FindText it may be possible to do that on a system-wide level! so that, say, i might be able to move mouse pointer to the OK button, just by typing 'f' followed by OK, then the pointer will jump straight to that button. This will save you the trouble of having to literally simulated any kind of mouse 'movement' with your keyboard.

this is just an idea of course (just popped into my head half an hour ago actually)... just throwing it out there in case anyone else is interested in using the script for this kind of purpose.

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 10 Jun 2017, 22:57

rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?

This problem is easy to solve, you can do it like this:

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: FindText - Capture screen image into text and then find it

Post by rupeshrulzz » 10 Jun 2017, 09:34

I am writing a script to automate some tasks in an online game I play. I am using this image search tool to make my script recognize images in my game and click at right places.
There are some images which are searched at multiple places in my script. I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
For example I want to search for image x, image y and image z at multiple points in my script. So I'll write the details for those 3 images which I got from find text tool at the beginning of the script. So now whenever I want to search for any image say image x I'll write a function to make the script search for image x instead of writing the detail for image x again. I don't know how to make a function to search for find text image.

Re: FindText - Capture screen image into text and then find it

Post by stealzy » 09 Jun 2017, 00:34

hal123, it was possible in version 4.7. Example.
But I think this is not best idea, you can use Tesseract OCR for such goals.

Top