Page 4 of 55

Re: FindText - Capture screen image into text and then find it

Posted: 23 May 2017, 09:46
by c4p
Luffy wrote: It doesn't Click , only selects the browser. Sometimes it brings the desired window "This is a new title" window to the top , but doesn't do anything else.
Updated https://autohotkey.com/boards/search.ph ... 7&sr=posts to correctly handle controlclicks in internet explorer.

Re: FindText - Capture screen image into text and then find it

Posted: 08 Jun 2017, 09:15
by hal123
Hi,

This is a really cool script :-)

Would it somehow be possible to make it work on dynamic text (Taking the text to find from a file)

My idea is somehow to generate the image to find. First I capture all the letters in the alphabet, based on the text and captured letters I then generate the image.

Code: Select all

Pseudo code:
LetterA.="|<A>*144$10.60Q1EBUW28Tl1A6kO"
LetterB.="|<B>*144$8.z8O2Vjm6Uc+6z8"
LetterC.="|<C>*144$10.7UX44E1040E112A7W"

TextToFindDynamic = "ABC"

dynImage = generateImage(TextToFindDynamic)

ok:=FindText(0,0,150000,150000,0,0,dynImage)

generateImage(txt)
{
	return = LetterA + LetterB + LetterC
}
How can I merge the letters into an "image word" that I can search for?

Thanks a lot.

Re: FindText - Capture screen image into text and then find it

Posted: 09 Jun 2017, 00:34
by stealzy
hal123, it was possible in version 4.7. Example.
But I think this is not best idea, you can use Tesseract OCR for such goals.

Re: FindText - Capture screen image into text and then find it

Posted: 10 Jun 2017, 09:34
by rupeshrulzz
I am writing a script to automate some tasks in an online game I play. I am using this image search tool to make my script recognize images in my game and click at right places.
There are some images which are searched at multiple places in my script. I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
For example I want to search for image x, image y and image z at multiple points in my script. So I'll write the details for those 3 images which I got from find text tool at the beginning of the script. So now whenever I want to search for any image say image x I'll write a function to make the script search for image x instead of writing the detail for image x again. I don't know how to make a function to search for find text image.

Re: FindText - Capture screen image into text and then find it

Posted: 10 Jun 2017, 22:57
by feiyue
rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
This problem is easy to solve, you can do it like this:

Code: Select all


; This function import text images through the comments in < >
Pic(comments, add_new=0) {
  static Lib:=[]
  if (add_new)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}

; Usage For example:

; You can put the text library at the beginning of the script
CoordMode, Mouse, Screen
Text:="|<comment1>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment2>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment3>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"

;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(Text,1)

;.....................

;.....................

;.....................

;-- Find one picture at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1"))
{
  X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2
  ;------------------------------
  Click, X, Y
}

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1|comment2|comment3"))
{
  Loop, % ok.MaxIndex()//5
  {
    i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
    W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
    if (Comment="comment1")
    {
      Click, X, Y
    }
    else if (Comment="comment2")
    {
      Click, X, Y
    }
    else if (Comment="comment3")
    {
      Click, X, Y
    }
  }
}


Re: FindText - Capture screen image into text and then find it

Posted: 11 Jun 2017, 13:04
by 20170201225639
edit: please ignore —— i misunderstood what the script does
it just occurred to me there's an excellent use case for this script.

i've been working outside in the park the last couple of days (thanks to a really capacious laptop power bank i just got recently) and noticed that the fresh air and sun made a huge difference to productivity. i doubt i'll ever (voluntarily) go back to my office again!

anyway, since using mouse on a park bench or grass is inconvenient (and i don't like touchpad either), i've been trying to use autohotkey to make everything i needed to do on my computer, doable via keyboard

i find that although defined hotkeys are an important part of the workflow, it's also really important to have a convenient way to move mouse pointer with your keyboard, because a lot of applications these days are not designed to be in principle operable with keyboard only.

i had some rudimentary scripts to do this, and also tried a couple of scripts i found on the board. but none of them can really move mouse pointer as fast as with an actual mouse.

then it occurred to me .. you know how many vim users become accustomed to move caret not by hjkl, but by search (typing 'f' then type some relevant char)? and i realized that with FindText it may be possible to do that on a system-wide level! so that, say, i might be able to move mouse pointer to the OK button, just by typing 'f' followed by OK, then the pointer will jump straight to that button. This will save you the trouble of having to literally simulated any kind of mouse 'movement' with your keyboard.

this is just an idea of course (just popped into my head half an hour ago actually)... just throwing it out there in case anyone else is interested in using the script for this kind of purpose.

Re: FindText - Capture screen image into text and then find it

Posted: 13 Jun 2017, 02:58
by stealzy
20170201225639, I use VimFx firefox addon and understand your idea, but it won't work because different windows have different fonts/sizes. Even in one window may be dozens combinations.
In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.

Re: FindText - Capture screen image into text and then find it

Posted: 13 Jun 2017, 09:57
by 20170201225639
aack! my bad for not reading the discussions closely. thanks stealzy for pointing that out.
i realized another poster also warned against this misunderstanding earlier (https://autohotkey.com/boards/viewtopic ... 319#p94319)

Re: FindText - Capture screen image into text and then find it

Posted: 13 Jun 2017, 10:04
by guest3456
stealzy wrote: In fact, the function should be called FindMonochromePattern, because this absolutely not about OCR.
+1

Re: FindText - Capture screen image into text and then find it

Posted: 25 Aug 2017, 16:08
by ozyank
First, thank you feiyue for the really nice script. I'm very new to scripting and AHK, but I've been able to use your example below and it works great. Much faster than trying to do the same thing in imagesearch.

As I said, I'm new to all of this so looking for advice on how to modify your script to conduct a nested search within the initial loop. The flow would look something like this:

1. Search for images using your original multi-image search loop.
2. If Image 1 found, click ;; I have this working
3. If Image 2 found, click ;; I have this working
4. If Image 3 found, click And
4a. While Image 3a, 3b and 3c are on the screen, click each until all have been clicked. ;; When Image 3 is clicked, it brings up another screen with 3 images on it. When an image is clicked, it is removed from the screen. I want this nested loop to click each of these until none are remaining. There may be 1, 2 or 3 images present so it needs to click until all of them are gone.

Images in initial loop:
Image 1
Image 2
Image 3

Images in nested loop initiated when Image 3 is clicked:
Image 3a
Image 3b
Image 3c

Pseudocode:
If Image 1 is found, Click
If Image 2 is found, Click
If Image 3 is found, Click
While (Image 3a and/or 3b and/or 3c) are on screen, Click each until none remain

I'm not sure I've explained this very well so please let me know if I can clarify. Thanks for any assistance!

feiyue wrote:
rupeshrulzz wrote: I want to put the details for all the required images in the beginning of the script and then call the desired image search wherever I need it in the script. How could I do it?
This problem is easy to solve, you can do it like this:

Code: Select all


; This function import text images through the comments in < >
Pic(comments, add_new=0) {
  static Lib:=[]
  if (add_new)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}

; Usage For example:

; You can put the text library at the beginning of the script
CoordMode, Mouse, Screen
Text:="|<comment1>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment2>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"
Text.="|<comment3>*138$24.10480U98zz9884Nz48O848u82Ec81U/z1U88CQ88k788U"

;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(Text,1)

;.....................

;.....................

;.....................

;-- Find one picture at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1"))
{
  X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2
  ;------------------------------
  Click, X, Y
}

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1|comment2|comment3"))
{
  Loop, % ok.MaxIndex()//5
  {
    i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
    W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
    if (Comment="comment1")
    {
      Click, X, Y
    }
    else if (Comment="comment2")
    {
      Click, X, Y
    }
    else if (Comment="comment3")
    {
      Click, X, Y
    }
  }
}


Re: FindText - Capture screen image into text and then find it

Posted: 26 Aug 2017, 02:10
by feiyue
The following code is for your reference.

Code: Select all

CoordMode, Mouse

Loop {

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("pic3a|pic3b|pic3c"))
{
  Loop, % ok.MaxIndex()//5
  {
    i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
    W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
    Click, % X, % Y
    Sleep, 500
  }
  Continue
}

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0
,Pic("pic1|pic2|pic3"))
{
  Loop, % ok.MaxIndex()//5
  {
    i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
    W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
    Click, % X, % Y
    Sleep, 500
  }
}

}

Re: FindText - Capture screen image into text and then find it

Posted: 28 Aug 2017, 12:40
by Joe Glines
Great job feiyue! This script rocks!
I created a tutorial showing how to use the basics of the script.


I also took a stab at writing some functions to click/send text, etc. to the image once it was found.

Code: Select all

FindText_Left_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
Click, %X%,%Y%
if (Move_Back){
  Sleep, 30
  MouseMove, %Orig_X%,%Orig_Y%
}
Return
}

;***********Right click*******************
FindText_Right_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
Click Right, %X%,%Y%
if (Move_Back){
  Sleep, 30
  MouseMove, %Orig_X%,%Orig_Y%
}
Return
}

;***********Double click*******************
FindText_Double_Click(Obj,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
Click 2
if (Move_Back){
  Sleep, 30
  MouseMove, %Orig_X%,%Orig_Y%
}
}

;***********Move mouse to location*******************
FindText_Move_Mouse(Obj,X_Adj=0,Y_Adj=0){
  CoordMode, Mouse
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
}

;***********Send text to location*******************
FindText_Send_Text(Obj,Text,Enter=0,X_Adj=0,Y_Adj=0,Move_Back=0){
CoordMode, Mouse
MouseGetPos,Orig_X,Orig_Y
x:=(obj.1+obj.3//2)+X_Adj
y:=(obj.2+obj.4//2)+Y_Adj
MouseMove, %X%,%Y%
Sleep 30
Click, %X%,%Y%
Sleep 30
SendInput , %Text%
if (Enter=1)
  SendInput , {Enter}
if (Move_Back){
  Sleep, 30
  MouseMove, %Orig_X%,%Orig_Y%
}
}
You can check out a video I did using the custom scripts and more on my page here

Re: FindText - Capture screen image into text and then find it

Posted: 28 Aug 2017, 15:17
by Reloaded
Wow best Script i ever see :D

Re: FindText - Capture screen image into text and then find it

Posted: 28 Sep 2017, 17:19
by vava
hello All...
I like this program. but have no luck to test all the functions.
each time click on the "test" function get an error "Can't run AHK script!" .
Running windows 7 64bit with the latest Autohotkey. hope some can help.

Re: FindText - Capture screen image into text and then find it

Posted: 29 Sep 2017, 01:50
by Helgef
Joe Glines wrote:Great job feiyue!
Indeed it seems very nice.
Joe Glines wrote:I created a tutorial showing how to use the basics of the script.
Hello Joe Glines :wave:
I watched your video when you posted it, but forgot to comment. It is very nice and helpful. I hope feiyue links to it in the first post so others can find.

Cheers.

Re: FindText - Capture screen image into text and then find it

Posted: 29 Sep 2017, 11:41
by feiyue
Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5

Re: FindText - Capture screen image into text and then find it

Posted: 01 Oct 2017, 17:57
by ali80
Hi feiyue, thanks for the great work
I have a minor problem, I include nearly all my scripts in one main script, and in that script I have #Warn flag on which gives useful warnings, when I include your code I get three warnings from it, all of them uninitialized variable usage warning, I tries fixing your code by initializing the variables first but it stopped working,
the three warnings are associated with following variables:
ppvBits
myFunc

Re: FindText - Capture screen image into text and then find it

Posted: 02 Oct 2017, 04:15
by ozzii
@Joe Glines
Do you need to update your custom scripts for the latest update ???

Re: FindText - Capture screen image into text and then find it

Posted: 02 Oct 2017, 20:06
by Frosty Fridge
Hello feiyue,

First off I'd like to say that this script is amazing! It's both powerful and easy to use.

I noticed that you added support for returning info in an array for multiple different images
feiyue wrote:Upgrade to version 5.3, changing the returned value of the object.
Previously, when looking for multiple images,
it would be difficult to extract each matching information if multiple images were found.
Now, a two level array is used to improve the situation. :beer:
But the original:
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
It has now been changed to:
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5
Or (variable): i:=A_Index
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5


I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?

Thanks for making this script!

Re: FindText - Capture screen image into text and then find it

Posted: 03 Oct 2017, 04:47
by feiyue
Frosty Fridge wrote:I was wondering if it was possible to have the function find and return data for multiple instances of the same image. For example, if the user's screen contains two identical pictures that are being searched for, could the function return info on both of them?
Updated to the v5.4 version, Realized your idea. :superhappy:
However, even if the returned object contains multiple locations,
You also need loop processing to find the top or leftmost one.