Updated https://autohotkey.com/boards/search.ph ... 7&sr=posts to correctly handle controlclicks in internet explorer.Luffy wrote: It doesn't Click , only selects the browser. Sometimes it brings the desired window "This is a new title" window to the top , but doesn't do anything else.
FindText - Capture screen image into text and then find it
Re: FindText - Capture screen image into text and then find it
Re: FindText - Capture screen image into text and then find it
Hi,
This is a really cool script
Would it somehow be possible to make it work on dynamic text (Taking the text to find from a file)
My idea is somehow to generate the image to find. First I capture all the letters in the alphabet, based on the text and captured letters I then generate the image.
How can I merge the letters into an "image word" that I can search for?
Thanks a lot.
This is a really cool script
Would it somehow be possible to make it work on dynamic text (Taking the text to find from a file)
My idea is somehow to generate the image to find. First I capture all the letters in the alphabet, based on the text and captured letters I then generate the image.
Code: Select all
Pseudo code:
LetterA.="|<A>*144$10.60Q1EBUW28Tl1A6kO"
LetterB.="|<B>*144$8.z8O2Vjm6Uc+6z8"
LetterC.="|<C>*144$10.7UX44E1040E112A7W"
TextToFindDynamic = "ABC"
dynImage = generateImage(TextToFindDynamic)
ok:=FindText(0,0,150000,150000,0,0,dynImage)
generateImage(txt)
{
return = LetterA + LetterB + LetterC
}
Thanks a lot.
Re: FindText - Capture screen image into text and then find it
hal123, it was possible in version 4.7. Example.
But I think this is not best idea, you can use Tesseract OCR for such goals.
But I think this is not best idea, you can use Tesseract OCR for such goals.
-
- Posts: 3
- Joined: 07 Jun 2017, 02:23
- Contact:
Re: FindText - Capture screen image into text and then find it
I am writing a script to automate some tasks in an online game I play. I am using this image search tool to make my script recognize images in my game and click at right places.
There are some images which are searched at multiple places in my script. I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
For example I want to search for image x, image y and image z at multiple points in my script. So I'll write the details for those 3 images which I got from find text tool at the beginning of the script. So now whenever I want to search for any image say image x I'll write a function to make the script search for image x instead of writing the detail for image x again. I don't know how to make a function to search for find text image.
There are some images which are searched at multiple places in my script. I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
For example I want to search for image x, image y and image z at multiple points in my script. So I'll write the details for those 3 images which I got from find text tool at the beginning of the script. So now whenever I want to search for any image say image x I'll write a function to make the script search for image x instead of writing the detail for image x again. I don't know how to make a function to search for find text image.
Re: FindText - Capture screen image into text and then find it
This problem is easy to solve, you can do it like this:rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
Code: Select all
; This function import text images through the comments in < >
Pic(comments, add_new=0) {
static Lib:=[]
if (add_new)
{
re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
Loop, Parse, comments, |
if RegExMatch(A_LoopField,re,r)
Lib[Trim(r1)]:=r
}
else
{
text:=""
Loop, Parse, comments, |
text.="|" . Lib[Trim(A_LoopField)]
return, text
}
}
; Usage For example:
; You can put the text library at the beginning of the script
CoordMode, Mouse, Screen
Text:="|<comment1>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment2>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment3>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(Text,1)
;.....................
;.....................
;.....................
;-- Find one picture at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1"))
{
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2
;------------------------------
Click, X, Y
}
;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1|comment2|comment3"))
{
Loop, % ok.MaxIndex()//5
{
i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
;------------------------------
if (Comment="comment1")
{
Click, X, Y
}
else if (Comment="comment2")
{
Click, X, Y
}
else if (Comment="comment3")
{
Click, X, Y
}
}
}
-
- Posts: 144
- Joined: 01 Feb 2017, 22:57
Re: FindText - Capture screen image into text and then find it
edit: please ignore —— i misunderstood what the script does
it just occurred to me there's an excellent use case for this script.
i've been working outside in the park the last couple of days (thanks to a really capacious laptop power bank i just got recently) and noticed that the fresh air and sun made a huge difference to productivity. i doubt i'll ever (voluntarily) go back to my office again!
anyway, since using mouse on a park bench or grass is inconvenient (and i don't like touchpad either), i've been trying to use autohotkey to make everything i needed to do on my computer, doable via keyboard
i find that although defined hotkeys are an important part of the workflow, it's also really important to have a convenient way to move mouse pointer with your keyboard, because a lot of applications these days are not designed to be in principle operable with keyboard only.
i had some rudimentary scripts to do this, and also tried a couple of scripts i found on the board. but none of them can really move mouse pointer as fast as with an actual mouse.
then it occurred to me .. you know how many vim users become accustomed to move caret not by hjkl, but by search (typing 'f' then type some relevant char)? and i realized that with FindText it may be possible to do that on a system-wide level! so that, say, i might be able to move mouse pointer to the OK button, just by typing 'f' followed by OK, then the pointer will jump straight to that button. This will save you the trouble of having to literally simulated any kind of mouse 'movement' with your keyboard.
this is just an idea of course (just popped into my head half an hour ago actually)... just throwing it out there in case anyone else is interested in using the script for this kind of purpose.
Last edited by 20170201225639 on 13 Jun 2017, 10:00, edited 1 time in total.
Re: FindText - Capture screen image into text and then find it
20170201225639, I use VimFx firefox addon and understand your idea, but it won't work because different windows have different fonts/sizes. Even in one window may be dozens combinations.
In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.
In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.
Last edited by stealzy on 13 Jun 2017, 15:17, edited 1 time in total.
-
- Posts: 144
- Joined: 01 Feb 2017, 22:57
Re: FindText - Capture screen image into text and then find it
aack! my bad for not reading the discussions closely. thanks stealzy for pointing that out.
i realized another poster also warned against this misunderstanding earlier (https://autohotkey.com/boards/viewtopic ... 319#p94319)
i realized another poster also warned against this misunderstanding earlier (https://autohotkey.com/boards/viewtopic ... 319#p94319)
Re: FindText - Capture screen image into text and then find it
+1stealzy wrote: In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.
Re: FindText - Capture screen image into text and then find it
First, thank you feiyue for the really nice script. I'm very new to scripting and AHK, but I've been able to use your example below and it works great. Much faster than trying to do the same thing in imagesearch.
As I said, I'm new to all of this so looking for advice on how to modify your script to conduct a nested search within the initial loop. The flow would look something like this:
1. Search for images using your original multi-image search loop.
2. If Image 1 found, click ;; I have this working
3. If Image 2 found, click ;; I have this working
4. If Image 3 found, click And
4a. While Image 3a, 3b and 3c are on the screen, click each until all have been clicked. ;; When Image 3 is clicked, it brings up another screen with 3 images on it. When an image is clicked, it is removed from the screen. I want this nested loop to click each of these until none are remaining. There may be 1, 2 or 3 images present so it needs to click until all of them are gone.
Images in initial loop:
Image 1
Image 2
Image 3
Images in nested loop initiated when Image 3 is clicked:
Image 3a
Image 3b
Image 3c
Pseudocode:
If Image 1 is found, Click
If Image 2 is found, Click
If Image 3 is found, Click
While (Image 3a and/or 3b and/or 3c) are on screen, Click each until none remain
I'm not sure I've explained this very well so please let me know if I can clarify. Thanks for any assistance!
As I said, I'm new to all of this so looking for advice on how to modify your script to conduct a nested search within the initial loop. The flow would look something like this:
1. Search for images using your original multi-image search loop.
2. If Image 1 found, click ;; I have this working
3. If Image 2 found, click ;; I have this working
4. If Image 3 found, click And
4a. While Image 3a, 3b and 3c are on the screen, click each until all have been clicked. ;; When Image 3 is clicked, it brings up another screen with 3 images on it. When an image is clicked, it is removed from the screen. I want this nested loop to click each of these until none are remaining. There may be 1, 2 or 3 images present so it needs to click until all of them are gone.
Images in initial loop:
Image 1
Image 2
Image 3
Images in nested loop initiated when Image 3 is clicked:
Image 3a
Image 3b
Image 3c
Pseudocode:
If Image 1 is found, Click
If Image 2 is found, Click
If Image 3 is found, Click
While (Image 3a and/or 3b and/or 3c) are on screen, Click each until none remain
I'm not sure I've explained this very well so please let me know if I can clarify. Thanks for any assistance!
feiyue wrote:This problem is easy to solve, you can do it like this:rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
Code: Select all
; This function import text images through the comments in < > Pic(comments, add_new=0) { static Lib:=[] if (add_new) { re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}" Loop, Parse, comments, | if RegExMatch(A_LoopField,re,r) Lib[Trim(r1)]:=r } else { text:="" Loop, Parse, comments, | text.="|" . Lib[Trim(A_LoopField)] return, text } } ; Usage For example: ; You can put the text library at the beginning of the script CoordMode, Mouse, Screen Text:="|<comment1>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U" Text.="|<comment2>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U" Text.="|<comment3>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U" ;-- Use Pic(Text,1) to add the text library to Pic()'s Lib Pic(Text,1) ;..................... ;..................... ;..................... ;-- Find one picture at a time if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1")) { X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2 ;------------------------------ Click, X, Y } ;-- Find more pictures at a time if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1|comment2|comment3")) { Loop, % ok.MaxIndex()//5 { i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2] W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2 ;------------------------------ if (Comment="comment1") { Click, X, Y } else if (Comment="comment2") { Click, X, Y } else if (Comment="comment3") { Click, X, Y } } }
Re: FindText - Capture screen image into text and then find it
The following code is for your reference.
Code: Select all
CoordMode, Mouse
Loop {
;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("pic3a|pic3b|pic3c"))
{
Loop, % ok.MaxIndex()//5
{
i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
;------------------------------
Click, % X, % Y
Sleep, 500
}
Continue
}
;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0
,Pic("pic1|pic2|pic3"))
{
Loop, % ok.MaxIndex()//5
{
i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
;------------------------------
Click, % X, % Y
Sleep, 500
}
}
}
- Joe Glines
- Posts: 770
- Joined: 30 Sep 2013, 20:49
- Location: Dallas
- Contact:
Re: FindText - Capture screen image into text and then find it
Great job feiyue! This script rocks!
I created a tutorial showing how to use the basics of the script.
I also took a stab at writing some functions to click/send text, etc. to the image once it was found.
You can check out a video I did using the custom scripts and more on my page here
I created a tutorial showing how to use the basics of the script.
I also took a stab at writing some functions to click/send text, etc. to the image once it was found.
Code: Select all
FindText_Left_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
Click, %X%,%Y%
if (Move_Back){
Sleep, 30
MouseMove, %Orig_X%,%Orig_Y%
}
Return
}
;***********Right click*******************
FindText_Right_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
Click Right, %X%,%Y%
if (Move_Back){
Sleep, 30
MouseMove, %Orig_X%,%Orig_Y%
}
Return
}
;***********Double click*******************
FindText_Double_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
Click 2
if (Move_Back){
Sleep, 30
MouseMove, %Orig_X%,%Orig_Y%
}
}
;***********Move mouse to location*******************
FindText_Move_Mouse(Obj,X_Adj=0,Y_Adj=0){
CoordMode, Mouse
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
}
;***********Send text to location*******************
FindText_Send_Text(Obj,Text,Enter=0,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
Sleep 30
Click, %X%,%Y%
Sleep 30
SendInput , %Text%
if (Enter=1)
SendInput , {Enter}
if (Move_Back){
Sleep, 30
MouseMove, %Orig_X%,%Orig_Y%
}
}
Sign-up for the HK Newsletter
AHK Tutorials:Web Scraping | | Webservice APIs | AHK and Excel | Chrome | RegEx | Functions
Training: AHK Webinars Courses on AutoHotkey
YouTube
Quick Access Popup, the powerful Windows folders, apps and documents launcher!
AHK Tutorials:Web Scraping | | Webservice APIs | AHK and Excel | Chrome | RegEx | Functions
Training: AHK Webinars Courses on AutoHotkey
YouTube
Quick Access Popup, the powerful Windows folders, apps and documents launcher!
Re: FindText - Capture screen image into text and then find it
Wow best Script i ever see
Re: FindText - Capture screen image into text and then find it
hello All...
I like this program. but have no luck to test all the functions.
each time click on the "test" function get an error "Can't run AHK script!" .
Running windows 7 64bit with the latest Autohotkey. hope some can help.
I like this program. but have no luck to test all the functions.
each time click on the "test" function get an error "Can't run AHK script!" .
Running windows 7 64bit with the latest Autohotkey. hope some can help.
Re: FindText - Capture screen image into text and then find it
Indeed it seems very nice.Joe Glines wrote:Great job feiyue!
Hello Joe GlinesJoe Glines wrote:I created a tutorial showing how to use the basics of the script.
I watched your video when you posted it, but forgot to comment. It is very nice and helpful. I hope feiyue links to it in the first post so others can find.
Cheers.
Re: FindText - Capture screen image into text and then find it
Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation.
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation.
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
Last edited by feiyue on 03 Oct 2017, 05:17, edited 2 times in total.
Re: FindText - Capture screen image into text and then find it
Hi feiyue, thanks for the great work
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc
Re: FindText - Capture screen image into text and then find it
@Joe Glines
Do you need to update your custom scripts for the latest update ???
Do you need to update your custom scripts for the latest update ???
Re: FindText - Capture screen image into text and then find it
Hello feiyue,
First off I'd like to say that this script is amazing! It's both powerful and easy to use.
I noticed that you added support for returning info in an array for multiple different images
I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?
Thanks for making this script!
First off I'd like to say that this script is amazing! It's both powerful and easy to use.
I noticed that you added support for returning info in an array for multiple different images
feiyue wrote:Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation.
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?
Thanks for making this script!
Re: FindText - Capture screen image into text and then find it
Updated to the v5.4 version, Realized your idea.Frosty Fridge wrote:I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?
However, even if the returned object contains multiple locations,
You also need loop processing to find the top or leftmost one.
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: No registered users and 109 guests