FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

20 Sep 2018, 14:27

Updated to the v6.0 version. :beer:
1. Slightly modified the machine code.
The image that has been found will be cleaned up in the same color.
Avoid matching the next row (column) when using tolerance lookup.

2. After upgrading to v6.0, the search area uses WinAPI's
upper left corner X, Y coordinates, and width, height.
This will be better understood and used.
Last edited by feiyue on 22 Sep 2018, 13:08, edited 1 time in total.
nwr425
Posts: 11
Joined: 08 Aug 2018, 15:41

Re: FindText - Capture screen image into text and then find it

21 Sep 2018, 01:53

feiyue wrote:@nwr425,
Although this function FindText() returns all the locations found,
because the lookup is from top to bottom,
sometimes the middle image is higher than the first image,
so it returns first, so the position returned may be disordered and difficult to sort..

I find that the location of these pictures looks unchanged,
so get the coordinates of the first image as the origin,
you could use relative coordinates to click on other images. Like this:

Code: Select all

F1::

CoordMode, Mouse

Text:="|<>*171$71.zz0000000Tzzzz0000000zzzzz0000003zzzzz000000Dzzzzz000000Tzzzzy000000zzzzzy000003zzzzzw000007zzzzzs00000Tzzzzzk00001zzzzzzk00003zzzzzzk00007zzzzzzU0000Dzzzzzz00000Tzzzzzy00000zzzzzzw1zzs3zzzzzzs7zzsDzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"
ok:=FindText(0, 0, 150000, 150000, 0.1, 0.1, Text)

; Get the coordinates of the first image as the origin.
For k,v in ok
{
  x:=v.1, y:=v.2, w:=v.3, h:=v.4, x+=w//2, y+=h//2
  len:=Sqrt(x**2+y**2)
  if (A_Index=1) or (len<min)
    min:=len, rx:=x, ry:=y
}

; Use relative coordinates to click
x:=rx, y:=ry+40, dx:=190, dy:=140
While not GetKeyState("Ctrl","P")
{
  ; 1>5
  MouseClick,, x, y
  Sleep, 1000
  MouseClick,, x, y+dy
  Sleep, 1000

  ; 2>6
  MouseClick,, x+dx, y
  Sleep, 1000
  MouseClick,, x+dx, y+dy
  Sleep, 1000

  ; 3>7
  MouseClick,, x+dx*2, y
  Sleep, 1000
  MouseClick,, x+dx*2, y+dy
  Sleep, 1000

  ; 4>8
  MouseClick,, x+dx*3, y
  Sleep, 1000
  MouseClick,, x+dx*3, y+dy
  Sleep, 1000
}
return



Ye I'm running something similar atm, thank you!
Was hoping this would register multiple identical images and possibly number them, if numbered, I'm sure what I displayed would work fine
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

21 Sep 2018, 04:43

nwr425 wrote: Was hoping this would register multiple identical images and possibly number them, if numbered, I'm sure what I displayed would work fine
I wrote a simple function SortOK() to sort the array by coordinates. See the example: :dance:

Code: Select all


SortOK(ok, dy=10) {
  if !IsObject(ok)
    return, ok
  ok2:=[]
  For k,v in ok
  {
    x:=v.1+v.3//2, y:=v.2+v.4//2
    y:=A_Index>1 and Abs(y-lasty)<dy ? lasty : y, lasty:=y
    n:=y*150000+x, s:=A_Index=1 ? n : s "-" n, ok2[n]:=v
  }
  ok:=[]
  Sort, s, N D-
  For k,n in StrSplit(s,"-")
    ok.Push( ok2[n] )
  return, ok
}

F1::

CoordMode, Mouse

Text:="|<>*173$71.zzk00000Dzzzzzk00000zzzzzzk00003zzzzzzk0000Dzzzzzzk0000zzzzzzzk0001zzzzzzzk0007zzzzzzzU000DzzzzzzzU000zzzzzzzz0001zzzzzzzy0007zzzzzzzy000Dzzzzzzzw000Tzzzzzzzs000zzzzzzzzkTz3zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"

if !(ok:=FindText(0, 0, 150000, 150000, 0.1, 0.1, Text))
{
  MsgBox, 4096, Err, Can't Find Picture !
  return
}
ok:=SortOK(ok)

While not GetKeyState("Ctrl","P")
{
  ; 1>5
  Click(1), Click(5)

  ; 2>6
  Click(2), Click(6)

  ; 3>7
  Click(3), Click(7)

  ; 4>8
  Click(4), Click(8)
}

MsgBox, 4096, Tip, Work Done !
return

Click(i) {
  global ok
  if not GetKeyState("Ctrl","P")
  {
    MouseClick,, ok[i].1+ok[i].3//2, ok[i].2+ok[i].4//2+20
    Sleep, 1000
  }
}

nwr425
Posts: 11
Joined: 08 Aug 2018, 15:41

Re: FindText - Capture screen image into text and then find it

21 Sep 2018, 22:15

Thank you Feiyue,

Also what part of the code would I edit to click in a random location within the image's dimensions?
For example, If I did not want to click in the same location on the found image each time.
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

21 Sep 2018, 23:41

nwr425 wrote: ---------images 1 to 8 all identical-----------

---1st action/click---
[1]234
5678

--2nd action/click--
x123
[4]567

--3rd action/click---
x[1]23
x456

--4th action/click--
xx12
x[3]45

--5th action/click---
xx[1]2
xx34

--6th action/click--
xxx1
xx[2]3

--7th action/click--
xxx[1]
xxx2

--8th action/click--
xxxx
xxx[1]

Code: Select all


SortOK(ok, dy=10) {
  if !IsObject(ok)
    return, ok
  ok2:=[]
  For k,v in ok
  {
    x:=v.1+v.3//2, y:=v.2+v.4//2
    y:=A_Index>1 and Abs(y-lasty)<dy ? lasty : y, lasty:=y
    n:=y*150000+x, s:=A_Index=1 ? n : s "-" n, ok2[n]:=v
  }
  ok:=[]
  Sort, s, N D-
  For k,n in StrSplit(s,"-")
    ok.Push( ok2[n] )
  return, ok
}

F1::

CoordMode, Mouse

Text:="|<>*173$71.zzk00000Dzzzzzk00000zzzzzzk00003zzzzzzk0000Dzzzzzzk0000zzzzzzzk0001zzzzzzzk0007zzzzzzzU000DzzzzzzzU000zzzzzzzz0001zzzzzzzy0007zzzzzzzy000Dzzzzzzzw000Tzzzzzzzs000zzzzzzzzkTz3zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"

While not GetKeyState("Ctrl","P")
{
  Sleep, 100
  ok:=SortOK(FindText(0, 0, 150000, 150000, 0.1, 0.1, Text))
  ; 1>5
  if ok.MaxIndex()=8
    Click(1)
  if ok.MaxIndex()=7
    Click(4)

  ; 2>6
  if ok.MaxIndex()=6
    Click(1)
  if ok.MaxIndex()=5
    Click(3)

  ; 3>7
  if ok.MaxIndex()=4
    Click(1)
  if ok.MaxIndex()=3
    Click(2)

  ; 4>8
  if ok.MaxIndex()=2
    Click(1)
  if ok.MaxIndex()=1
    Click(1)
}

MsgBox, 4096, Tip, Work Done !
return

Click(i) {
  global ok
  if not GetKeyState("Ctrl","P")
  {
    MouseClick,, ok[i].1+ok[i].3//2, ok[i].2+ok[i].4//2+20
    Sleep, 1000
  }
}

FastLearner

Re: FindText - Capture screen image into text and then find it

22 Sep 2018, 20:56

Hello Feiyue,

I have been using your awesome tool for months now maybe a year :D and it's pretty much AWESOME! :clap:

I just encountered a problem few days ago, it happens when the image moves from its place..so for example this code here

Code: Select all

t1:=A_TickCount

Text:="|<>231@0.70$59.M00001Xy7yA00003aQTw0000076sTs00000SNkRk00000kvUvE"

if (ok:=FindText(662-150000//2, 409-150000//2, 150000, 150000, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}
It happens with anything of this kind...if the image is moved (searching tiles in a game so the coordinates will be different almost each time the image is found) I get failed Msgbox.

how can I make it search whole screen or for speed (I guess) how can I make it search a rectangle for example...I have no idea what coordinates to put where on script to make a rectangle for example (the game screen is a small rectangle in the middle of the screen)

I have a couple of coordinate finder including your tool when pressing capture..but how to specify a specific part of the screen (if it makes any difference in speed) and how to make the first two coordinates (662-150000//2, 409-150000//2) not fixed so it can get the image even if not found in these two specific coordinates (I guess)

I tried making them 0 or 150000, but no luck...once I move a bit in any direction so the image's coordinates changes...I get a fail

Any help would be appreciated

Thank you for your awesome work :salute:
FastLearner

Re: FindText - Capture screen image into text and then find it

22 Sep 2018, 21:27

After reading a bit in the comments I found the way and tested with gui to see if the rectangle is placed right or not and I got this
FindText(450+430//2,34+670//2,430//2,670//2,0,0,Text)

but my main issue still stands, when the image coordinates is changed what can we do? :headwall:
FastLearner

Re: FindText - Capture screen image into text and then find it

22 Sep 2018, 21:51

Pardon me, I just figured I did something dumb :oops: :crazy:

This is the Gui coordinates (triangle for the part of the screen I'm using but the image is somewhere in between and it's in different coordinates almost each time its found(but ofc it's not found by the tool because coordinates changes all the time)

Code: Select all

;screen part I'm using
Gui, Show, x450 y34 w430 h670, TexT
if I put it like this, the first two coordinates (450+430//2,34+670//2) means the image I'm looking for is located there, right?

Code: Select all

FindText(450+430//2,34+670//2,430//2,670//2,0,0,Text)
so in the script it will be like this

Code: Select all

Text:="|<>167@0.64$42.000DbU700037wD0003aQT0003bQD000LaQ700067Q700037E700032A6U"


if (ok:=FindText(450+430//2,34+670//2,430//2,670//2,0,0,Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}

; or in my version of the tool it will be like this

Text:="|<>167@0.64$42.000DbU700037wD0003aQT0003bQD000LaQ700067Q700037E700032A6U"

if (ok:=FindText(450,34,430,670,0,0,Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}

but I guess it shouldn't be that way...how to make a rectangle instead of the whole screen and make it search in between? :cry:

if the gui placing is different then please guide me on this :roll:
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

23 Sep 2018, 17:48

@FastLearner, Don't copy the form of automatically generated code. :headwall:
I believe you can learn to set search scope if you read the introduction. :beer:

;————————————————————

returnArray := FindText(
X --> upper left corner X coordinates
, Y --> upper left corner Y coordinates
, W --> the search scope's Width
, H --> the search scope's Height
, Character "0" fault-tolerant in percentage --> 0.1=10%
, Character "_" fault-tolerant in percentage --> 0.1=10%
, text --> The Base64 encoding string for the text to find
)

The range used by AHK is determined by the upper left
corner and the lower right corner: (x1, y1, x2, y2),
it can be converted to: (x1, y1, x2-x1+1, y2-y1+1).

;————————————————————

If you want to search the full screen, you can do this:
FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0,0,Text)

If you want to search window scope, you can do this:
WinGetPos, x, y, w, h, Window Title
FindText(x, y, w, h, 0,0,Text)
leosouza85
Posts: 10
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

01 Oct 2018, 18:57

feiyue wrote:@FastLearner, Don't copy the form of automatically generated code. :headwall:
I believe you can learn to set search scope if you read the introduction. :beer:

;————————————————————

returnArray := FindText(
X --> upper left corner X coordinates
, Y --> upper left corner Y coordinates
, W --> the search scope's Width
, H --> the search scope's Height
, Character "0" fault-tolerant in percentage --> 0.1=10%
, Character "_" fault-tolerant in percentage --> 0.1=10%
, text --> The Base64 encoding string for the text to find
)

The range used by AHK is determined by the upper left
corner and the lower right corner: (x1, y1, x2, y2),
it can be converted to: (x1, y1, x2-x1+1, y2-y1+1).

;————————————————————

If you want to search the full screen, you can do this:
FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0,0,Text)

If you want to search window scope, you can do this:
WinGetPos, x, y, w, h, Window Title
FindText(x, y, w, h, 0,0,Text)
Hi, it is possible to configure the color correspondent to the "_" character to not be white? to be like red?
In one very specific use case it would be the solution of my problem...

Thank you so much!
paulpma
Posts: 12
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

05 Oct 2018, 12:21

Hello, feiyue

Very good script you have. A+. :bravo:

I have read through most of the 9 pages of text on this forum to get information about it. I was wondering why do you have x32 and x64 encoding in the script? Is this for security of your script or translation from C code that is posted on the bottom of script, or something else? Good job. Thank you.
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

06 Oct 2018, 19:24

Updated to the v6.1 version. :beer:
The specific color mode has been restored, so there are three modes.
feiyue wrote:Updated to the v5.5 version, in order to identify a variety of color verification code, I improved the color model, now it can adapt to various colors. :D
When upgrading to the v5.5 version, the specific color mode is modified to color location mode.

@leosouza85, I upgraded the function, which can solve your problem.
@paulpma, x32 and x64 are machine codes for C language code.
leosouza85
Posts: 10
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

06 Oct 2018, 21:05

feiyue wrote:Updated to the v6.1 version. :beer:
The specific color mode has been restored, so there are three modes.
feiyue wrote:Updated to the v5.5 version, in order to identify a variety of color verification code, I improved the color model, now it can adapt to various colors. :D
When upgrading to the v5.5 version, the specific color mode is modified to color location mode.

@leosouza85, I upgraded the function, which can solve your problem.
@paulpma, x32 and x64 are machine codes for C language code.
thank you so much!!!
leosouza85
Posts: 10
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

07 Oct 2018, 10:38

feiyue wrote:Updated to the v6.1 version. :beer:
The specific color mode has been restored, so there are three modes.
feiyue wrote:Updated to the v5.5 version, in order to identify a variety of color verification code, I improved the color model, now it can adapt to various colors. :D
When upgrading to the v5.5 version, the specific color mode is modified to color location mode.

@leosouza85, I upgraded the function, which can solve your problem.
@paulpma, x32 and x64 are machine codes for C language code.
Hi, when we separate text via the | character, the script searches for this AND that text, or searches for this OR that text? If it is a AND, it is possible to in one "if" expression to search 1 OR other text?!

Thanks!!!
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

07 Oct 2018, 22:08

This function finds and returns every image in every location of the search area (It's AND's relationship).

For example:

Code: Select all


Text := "|<pic1>231@0.70$59.M00001Xy7yA00003aQTw0000076sTs00000SNkRk00000kvUvE"
Text .= "|<pic2>231@0.70$59.M00001Xy7yA00003aQTw0000076sTs00000SNkRk00000kvUvE"
Text .= "|<pic3>231@0.70$59.M00001Xy7yA00003aQTw0000076sTs00000SNkRk00000kvUvE"

if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text))
{
  CoordMode, Mouse
  For i,v in ok
  if (v.5="pic1")  ; in one "if" expression to search 1 OR other text
  {
    X:=v.1, Y:=v.2, W:=v.3, H:=v.4, Comment:=v.5, X+=W//2, Y+=H//2
    MsgBox, Find %Comment% in Pos: %X%`, %Y%
    MouseMove, X, Y
  }
}
paulpma
Posts: 12
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

09 Oct 2018, 10:29

Hello, feiyue

Once again thank you for such amazing script. I am trying to implement this script into my process, and it seems that I have ran into a conflict. I am drying to draw boxes around searched area using GDIp_All. It seem there some kind of conflict, when FindTextOCR() and GDip all is used. Drawing is excecuted but not FindTextOCR(). Any ideas? Thank you.


Sorry for simplicity if code, but this is my current level of coding at AHK, but I am still learning. I am sure your code would much more compact...
Here is the code for reference:

Code: Select all


#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
;#Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
#SingleInstance, Force
#Include <GDIp_All>

; Start gdi+
If !pToken := Gdip_Startup()
{
	MsgBox, 48, gdiplus error!, Gdiplus failed to start. Please ensure you have gdiplus on your system
	ExitApp
}
OnExit, Exit

Width := 1360, Height := 768 ; Set the width and height we want as our drawing area, to draw everything in. This will be the dimensions of our bitmap

DrawBoxes:


Gui, Boxes: -Caption +E0x80000 +LastFound +AlwaysOnTop +ToolWindow +OwnDialogs ; Create a layered window (+E0x80000 : must be used for UpdateLayeredWindow to work!) that is always on top (+AlwaysOnTop), has no taskbar entry or caption
Gui, Boxes: Show, NA ; Show the window
hwnd1 := WinExist() ; Get a handle to this window we have created in order to update it later
hbm := CreateDIBSection(Width, Height) ; Create a gdi bitmap with width and height of what we are going to draw into it. This is the entire drawing area for everything
hdc := CreateCompatibleDC() ; Get a device context compatible with the screen
obm := SelectObject(hdc, hbm) ; Select the bitmap into the device context
G := Gdip_GraphicsFromHDC(hdc) ; Get a pointer to the graphics of the bitmap, for use with drawing functions
Gdip_SetSmoothingMode(G, 4) ; Set the smoothing mode to antialias = 4 to make shapes appear smother (only used for vector drawing and filling)
;

NGDOBC := [181, 180, 57, 14] ;NG DOB Cords xywh


;MsgBox % NGNC[1]
return


Text:=""
Text.="|<1>*155$4.009e8W802"
Text.="|<2>*156$4.00yF5gw02"
Text.="|<3>*156$6.000Sm2C3nS000U"
Text.="|<4>*127$6.26+GGWS27U"
Text.="|<5>*140$4.00yDY9w02"
Text.="|<6>*140$6.000SHynlHS000U"
Text.="|<7>*127$5.z8EW48EY"
Text.="|<8>*140$4.00uNONw02"
Text.="|<9>*156$6.000SnnT3nS000U"
Text.="|<0>*157$7.0001sgmNAaFk000E"



CoordMode, Mouse
MouseGetPos, x, y
t1:=A_TickCount
;------------------------------
OCR:=FindTextOCR(170, 170, 150, 150, 0, 0, Text)
;------------------------------
t1:=A_TickCount-t1
MsgBox, 4096, OCR, OCR Result: [%OCR%] in %t1% ms.
Return


<#d:: ;win+d

Gosub, DrawBoxes
;
  

	dobPen := Gdip_CreatePen(0x66161616, 1) 
	Gdip_DrawRectangle(G, dobPen, NGDOBC[1], NGDOBC[2], NGDOBC[3], NGDOBC[4]) 
	Gdip_DeletePen(dobPen)  
 
 


UpdateLayeredWindow(hwnd1, hdc, 0, 0, Width, Height) ; Update the specified window we have created (hwnd1) with a handle to our bitmap (hdc), specifying the x,y,w,h we want it positioned on our screen ; So this will position our gui at (0,0) with the Width and Height specified earlier
;
SelectObject(hdc, obm) ; Select the object back into the hdc
DeleteObject(hbm) ; Now the bitmap may be deleted
DeleteDC(hdc) ; Also the device context related to the bitmap may be deleted
Gdip_DeleteGraphics(G) ; The graphics may now be deleted

return




;===== Copy The Following Functions To Your Own Code Just once =====


; FindText() used to find images restored by Base64 text on screen.
; X is upper left corner X coordinates
; Y is upper left corner Y coordinates
; W is the search scope's Width
; H is the search scope's Height.
; err1 is the character "0" fault-tolerant in percentage.
; err0 is the character "_" fault-tolerant in percentage.
; Text can be a lot of text to find, separated by "|".
; ruturn is a array, contains the [X,Y,W,H,Comment] results of Each Find.

FindText(x,y,w,h,err1,err0,text)
{
  xywh2xywh(x,y,w,h,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  sx:=0, sy:=0, sw:=w, sh:=h, arr:=[]
  Loop, Parse, text, |
  {
    v:=A_LoopField
    IfNotInString, v, $, Continue
    comment:="", e1:=err1, e0:=err0
    ; You Can Add Comment Text within The <>
    if RegExMatch(v,"<([^>]*)>",r)
      v:=StrReplace(v,r), comment:=Trim(r1)
    ; You can Add two fault-tolerant in the [], separated by commas
    if RegExMatch(v,"\[([^\]]*)]",r)
    {
      v:=StrReplace(v,r), r1.=","
      StringSplit, r, r1, `,
      e1:=r1, e0:=r2
    }
    StringSplit, r, v, $
    color:=r1, v:=r2
    StringSplit, r, v, .
    w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
    if (r0<2 or h1<1 or w1>sw or h1>sh or StrLen(v)!=w1*h1)
      Continue
    ;--------------------------------------------
    mode:=InStr(color,"*") ? 1:0
    color:=StrReplace(color,"*") . "@"
    StringSplit, r, color, @
    color:=mode=1 ? r1 : ((r1-1)//w1)*Stride+Mod(r1-1,w1)*4
    n:=Round(r2,2)+(!r2), n:=Floor(255*3*(1-n))
    StrReplace(v,"1","",len1), len0:=StrLen(v)-len1
    VarSetCapacity(allpos, 1024*4, 0), k:=StrLen(v)*4
    VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
    ;--------------------------------------------
    if (ok:=PicFind(mode,color,n,Scan0,Stride,sx,sy,sw,sh
      ,v,s1,s0,Round(len1*e1),Round(len0*e0),w1,h1,allpos))
      or (err1=0 and err0=0
      and (ok:=PicFind(mode,color,n,Scan0,Stride,sx,sy,sw,sh
      ,v,s1,s0,Round(len1*0.1),Round(len0*0.1),w1,h1,allpos)))
    {
      Loop, % ok
        pos:=NumGet(allpos, 4*(A_Index-1), "uint")
        , rx:=(pos&0xFFFF)+x, ry:=(pos>>16)+y
        , arr.Push( [rx,ry,w1,h1,comment] )
    }
  }
  SetBatchLines, %bch%
  return, arr.MaxIndex() ? arr:0
}

PicFind(mode, color, n, Scan0, Stride, sx, sy, sw, sh
  , ByRef text, ByRef s1, ByRef s0
  , err1, err0, w1, h1, ByRef allpos)
{
  static MyFunc
  if !MyFunc
  {
    x32:="5557565383EC508B8424800000002B84249C0000008B742"
    . "4788944241C83C001894424348B8424840000002B8424A0000"
    . "0008944242083C001894424188B44247C0FAF4424748D04B08"
    . "94424148B8424A000000085C00F8E7405000031ED31FF31F68"
    . "92C248BAC249000000031DB897C24048B84249C00000085C07"
    . "E548B8424880000008B8C24880000008B54240401D8039C249"
    . "C00000001D9895C2408EB1183C0018954B50083C60183C2043"
    . "9C1741C80383175EA8B9C248C00000083C0018914BB83C7018"
    . "3C20439C175E48B5C2408830424018B5424748B04240154240"
    . "4398424A0000000758989F839F7897C24100F4CC68944240C8"
    . "B6C246485ED0F850B0200008B7C241885FF0F8EB90400008B4"
    . "4241403442468034424708B5C24208B7C2470C744243000000"
    . "00089742414894424408B44247C894424388D4403018B5C241"
    . "08944244C8B84248C0000008D04988B5C2478894424448B842"
    . "4900000008D04B0894424488B44241C8D4418018B5C2468894"
    . "4242C8B44247001D8894424288B6C243485ED0F8E560100008"
    . "B442438C1E0108944243C8B442478894424188B44244089442"
    . "4248DB426000000008B4424240FB6580289C62B742428891C2"
    . "40FB658010FB600895C24048B5C240C8944240885DB0F84F50"
    . "200008B84249800000031DB894424208B84249400000089442"
    . "41CEB778D76008DBC27000000003B5C24147D5A8B842490000"
    . "0008B149801F20FB64C17020FB64417012B0C242B4424040FB"
    . "614172B54240889CDC1FD1F31E929E989C5C1FD1F31E829E88"
    . "9D5C1FD1F01C131EA29EA01CA3954246C7C10836C242001787"
    . "589F68DBC270000000083C3013B5C240C0F84640200003B5C2"
    . "4107D8D8B8C248C0000008B049901F00FB64C07020FB654070"
    . "12B0C242B5424040FB604072B44240889CDC1FD1F31E929E98"
    . "9D5C1FD1F31EA29EA89C5C1FD1F01D131E829E801C83B44246"
    . "C0F8E3FFFFFFF836C241C010F8934FFFFFF834424180183442"
    . "424048B4424183B44242C0F85CCFEFFFF83442438018B74247"
    . "48B442438017424403B44244C0F8583FEFFFF8B5C243083C45"
    . "089D85B5E5F5DC244008B4424688BBC248400000083C00169E"
    . "8E80300008B4424140344247089C38B842480000000C1E0028"
    . "9042431C085FF7E628974240489C789DE908DB426000000008"
    . "B8C248000000085C97E338B042489F18D1C060FB651020FB64"
    . "10169D22B01000069C04B02000001C20FB6016BC07201D039C"
    . "50F9F410383C10439CB75D583C7010374247439BC248400000"
    . "075B28B7424048B4424148B54241883C00385D20F8E1402000"
    . "08B5C24208944242489F58B44247C8B7C2478C744241400000"
    . "000894424188D4403018B9C248C000000894424288B4424108"
    . "D1C8389C6895C24208B5C241C8D5C3B01895C24108B5C240C8"
    . "B44243485C00F8EA20000008B442418C1E0108944241C8B442"
    . "4788904248B4424248944240C85DB0F84350100008B8424980"
    . "000008B9424940000008B4C240C034C2470894424088954240"
    . "431C0EB2C9039E87D1C8B9424900000008B3C8201CF803F007"
    . "40B836C240801782B8D74260083C00139D80F84E700000039F"
    . "07DD18B94248C0000008B3C8201CF803F0174C0836C2404017"
    . "9B9830424018344240C048B04243B4424100F8578FFFFFF834"
    . "42418018B7C24748B442418017C2424394424280F8537FFFFF"
    . "F8B5C2414E932FEFFFF8B4C24308B4424180B44243C8B9424A"
    . "40000008D590181FBFF03000089048A0F8F0DFEFFFF8B4C241"
    . "08B84248C00000085C974248B4C24448DB426000000008B108"
    . "3C00401F239C8C6441702FFC6441701FFC60417FF75E78B542"
    . "4148B8424900000008B4C244885D2741D8D7426008B1083C00"
    . "401F239C8C6441702FFC6441701FFC60417FF75E7895C2430E"
    . "96AFDFFFF8B4C24148B04240B44241C8B9424A40000008D790"
    . "181FFFF03000089048A7F3985F674618B4C24708B54240C897"
    . "C24048B84248C0000008B7C242001D1908B1083C00401CA39C"
    . "7C6020075F28B7C2404897C2414E9DCFEFFFF83C45089FB89D"
    . "85B5E5F5DC2440031DBE933FDFFFFC744240C0000000031F6C"
    . "744241000000000E911FBFFFF897C2414E9A7FEFFFF"
    x64:="4157415641554154555756534883EC588BAC24100100008"
    . "B8424D80000008BB424180100008BBC24C80000004D89CD898"
    . "C24A0000000899424A800000029E844898424B00000004C8BB"
    . "C24F00000008944240883C001488B9C24F8000000894424148"
    . "B8424E000000029F08944240C83C001894424048B8424D0000"
    . "0000FAF8424C000000085F68D04B88904240F8E120600004C8"
    . "9AC24B80000004C8BAC24E80000008D3CAD000000004531C94"
    . "531D24531E44531F64531DB0F1F800000000085ED7E444963D"
    . "3458D04394489C84C01EAEB164963CC4883C2014183C401890"
    . "48B83C0044139C0741C803A3175E54963CE4883C2014183C60"
    . "14189048F83C0044139C075E44101EB4183C20144038C24C00"
    . "000004439D675A74C8BAC24B80000004539E64489E5410F4DE"
    . "E448B9424A00000004585D20F857B020000448B4C24044585C"
    . "90F8E54050000486304248B7C240C4863B424C00000004C89B"
    . "C24F0000000C74424100000000048899C24F80000008944244"
    . "848894424208B8424D000000048897424408BB424C80000008"
    . "94424188D4407018944244C418D46FF498D4487044589E7488"
    . "9442438418D4424FF488D448304488944243048638424A8000"
    . "00048894424288B4424088D4430018944240C448B442414458"
    . "5C00F8E990100008B442418448B542448C1E0108944241C488"
    . "B44242048034424284D8D6405008B8424C8000000890424660"
    . "F1F44000085ED410FB65C2402410FB6742401410FB63C240F8"
    . "4570300008B8424080100004531C0894424088B84240001000"
    . "089442404E996000000660F1F8400000000004539F97D7B488"
    . "B8C24F8000000428B04814401D08D50024863D2410FB64C150"
    . "08D50014898410FB64405004863D2410FB654150029D94189C"
    . "929F841C1F91F29F24431C94429C94189D141C1F91F4431CA4"
    . "429CA4189C141C1F91F01D14431C84429C801C8398424B0000"
    . "0007C12836C2408010F88980000000F1F80000000004983C00"
    . "14439C50F8EA30200004539F04589C10F8D67FFFFFF488B8C2"
    . "4F0000000428B04814401D08D50024863D2410FB64C15008D5"
    . "0014898410FB64405004863D2410FB654150029D94189CB29F"
    . "841C1FB1F29F24431D94429D94189D341C1FB1F4431DA4429D"
    . "A4189C341C1FB1F01D14431D84429D801C83B8424B00000000"
    . "F8EFAFEFFFF836C2404010F89EFFEFFFF830424014983C4044"
    . "183C2048B04243944240C0F8596FEFFFF83442418018BBC24C"
    . "00000008B442418017C2448488B7C244048017C24203B44244"
    . "C0F8531FEFFFF8B4424104883C4585B5E5F5D415C415D415E4"
    . "15FC38B8424A80000008B8C24E0000000448D48014569C9E80"
    . "3000085C90F8E9C00000048638424C00000004C6314244531D"
    . "B44897424104489642418448BB424E0000000448BA424D8000"
    . "0004889C78B8424D80000004D01EA83E801488D34850400000"
    . "00F1F80000000004585E47E394E8D04164C89D10F1F40000FB"
    . "651020FB6410169D22B01000069C04B02000001C20FB6016BC"
    . "07201D04139C10F9F41034883C1044939C875D24183C301490"
    . "1FA4539DE75B6448B742410448B6424188B04248B54240483C"
    . "00385D20F8E180200008B7C240C894424108B8424D00000008"
    . "BB424C8000000C7042400000000894424048D4407018944240"
    . "C418D46FF4D8D5C87048B4424088D7C30018B44241485C00F8"
    . "E8A0000008B4424048BB424C8000000448B442410C1E010894"
    . "4240885ED0F8455010000448B942408010000448B8C2400010"
    . "00031C0EB2D904439E17D1B8B14834401C24863D241807C150"
    . "000740A4183EA01782F0F1F40004883C00139C50F8E1401000"
    . "04439F089C17DCD418B14874401C24863D241807C15000174B"
    . "B4183E90179B583C6014183C00439FE758D83442404018BB42"
    . "4C00000008B442404017424103944240C0F854CFFFFFF8B042"
    . "4E92AFEFFFF0F1F400048635424108B0C240B4C241C488BBC2"
    . "42001000089D083C001890C973DFF0300000F8FFFFDFFFF458"
    . "5F6488B8C24F00000004C8B4C243874380F1F80000000008B1"
    . "14883C1044401D24C39C9448D42024D63C043C6440500FF448"
    . "D42014863D24D63C043C6440500FF41C6441500FF75CF4585F"
    . "F488B8C24F80000004C8B4C243074318B114883C1044401D24"
    . "C39C9448D42024D63C043C6440500FF448D42014863D24D63C"
    . "043C6440500FF41C6441500FF75CF89442410E924FDFFFF0F1"
    . "F00486314248B4C24084C8B94242001000009F189D041890C9"
    . "283C0013DFF0300000F8F40FDFFFF4585F6741C4C89FA66908"
    . "B0A4883C2044401C14939D34863C941C6440D000075E983C60"
    . "14183C00489042439FE0F854AFEFFFFE9B8FEFFFF31C0E901F"
    . "DFFFF31ED4531E44531F6E979FAFFFF9090"
    MCode(MyFunc, A_PtrSize=8 ? x64:x32)
  }
  return, DllCall(&MyFunc, "int",mode
    , "uint",color, "int",n, "ptr",Scan0, "int",Stride
    , "int",sx, "int",sy, "int",sw, "int",sh
    , "AStr",text, "ptr",&s1, "ptr",&s0
    , "int",err1, "int",err0, "int",w1, "int",h1, "ptr",&allpos)
}

xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h)
{
  SysGet, zx, 76
  SysGet, zy, 77
  SysGet, zw, 78
  SysGet, zh, 79
  left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
  left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
  up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
  x:=left, y:=up, w:=right-left+1, h:=down-up+1
}

GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits)
{
  VarSetCapacity(bits,w*h*4,0), bpp:=32
  Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
  Ptr:=A_PtrSize ? "UPtr" : "UInt", PtrP:=Ptr . "*"
  win:=DllCall("GetDesktopWindow", Ptr)
  hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
  mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
  ;-------------------------
  VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
  NumPut(w, bi, 4, "int"), NumPut(-h, bi, 8, "int")
  NumPut(1, bi, 12, "short"), NumPut(bpp, bi, 14, "short")
  ;-------------------------
  if hBM:=DllCall("CreateDIBSection", Ptr,mDC, Ptr,&bi
    , "int",0, PtrP,ppvBits, Ptr,0, "int",0, Ptr)
  {
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr)
    DllCall("BitBlt", Ptr,mDC, "int",0, "int",0, "int",w, "int",h
      , Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000)
    DllCall("RtlMoveMemory", Ptr,Scan0, Ptr,ppvBits, Ptr,Stride*h)
    DllCall("SelectObject", Ptr,mDC, Ptr,oBM)
    DllCall("DeleteObject", Ptr,hBM)
  }
  DllCall("DeleteDC", Ptr,mDC)
  DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
}

MCode(ByRef code, hex)
{
  ListLines, Off
  bch:=A_BatchLines
  SetBatchLines, -1
  VarSetCapacity(code, len:=StrLen(hex)//2)
  Loop, % len
    NumPut("0x" SubStr(hex,2*A_Index-1,2),code,A_Index-1,"uchar")
  Ptr:=A_PtrSize ? "UPtr" : "UInt", PtrP:=Ptr . "*"
  DllCall("VirtualProtect",Ptr,&code, Ptr,len,"uint",0x40,PtrP,0)
  SetBatchLines, %bch%
  ListLines, On
}

base64tobit(s)
{
  ListLines, Off
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  StringCaseSense, On
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:=(i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,A_LoopField,v)
  }
  StringCaseSense, Off
  s:=SubStr(s,1,InStr(s,"1",0,0)-1)
  s:=RegExReplace(s,"[^01]+")
  ListLines, On
  return, s
}

bit2base64(s)
{
  ListLines, Off
  s:=RegExReplace(s,"[^01]+")
  s.=SubStr("100000",1,6-Mod(StrLen(s),6))
  s:=RegExReplace(s,".{6}","|$0")
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:="|" . (i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,v,A_LoopField)
  }
  ListLines, On
  return, s
}

ASCII(s)
{
  if RegExMatch(s,"\$(\d+)\.([\w+/]+)",r)
  {
    s:=RegExReplace(base64tobit(r2),".{" r1 "}","$0`n")
    s:=StrReplace(StrReplace(s,"0","_"),"1","0")
  }
  else s=
  return, s
}

; You can put the text library at the beginning of the script,
; and Use Pic(Text,1) to add the text library to Pic()'s Lib,
; Use Pic("comment1|comment2|...") to get text images from Lib

Pic(comments, add_to_Lib=0)
{
  static Lib:=[]
  if (add_to_Lib)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]+"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
    Lib[""]:=""
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}

PicN(number)
{
  return, Pic(Trim(RegExReplace(number,".","$0|"),"|"))
}

; Use PicX(Text) to automatically cut into multiple characters

PicX(Text)
{
  if !RegExMatch(Text,"\|([^$]+)\$(\d+)\.([\w+/]+)",r)
    return, Text
  w:=r2, v:=base64tobit(r3), Text:=""
  c:=StrLen(StrReplace(v,"0"))<=StrLen(v)//2 ? "1":"0"
  wz:=RegExReplace(v,".{" w "}","$0`n")
  SetFormat, IntegerFast, d
  While InStr(wz,c) {
    While !(wz~="m`n)^" c)
      wz:=RegExReplace(wz,"m`n)^.")
    i:=0
    While (wz~="m`n)^.{" i "}" c)
      i++
    v:=RegExReplace(wz,"m`n)^(.{" i "}).*","$1")
    wz:=RegExReplace(wz,"m`n)^.{" i "}")
    if v!=
      Text.="|" r1 "$" i "." bit2base64(v)
  }
  return, Text
}

; Reordering the objects returned from left to right,
; from top to bottom, ignore slight height difference

SortOK(ok, dy=10) {
  if !IsObject(ok)
    return, ok
  ok2:=[]
  For k,v in ok
  {
    x:=v.1+v.3//2, y:=v.2+v.4//2
    y:=A_Index>1 and Abs(y-lasty)<dy ? lasty : y, lasty:=y
    n:=y*150000+x, s:=A_Index=1 ? n : s "-" n, ok2[n]:=v
  }
  ok:=[]
  Sort, s, N D-
  For k,n in StrSplit(s,"-")
    ok.Push( ok2[n] )
  return, ok
}

FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=20)
{
  OCR:="", Right_X:=nX+nW-1
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    Loop, % ok.MaxIndex()
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      i:=A_Index, x:=ok[i].1, y:=ok[i].2
        , w:=ok[i].3, h:=ok[i].4, comment:=ok[i].5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<Left_X)
        Left_X:=x, Left_W:=w, Left_OCR:=comment
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and Left_X-nX-1>Interval ? "*":"") . Left_OCR
    ; Update nX and nW for next search
    nX:=Left_X+Left_W-1, nW:=Right_X-nX+1
  }
  Return, OCR
}


/***** C source code of machine code *****

int __attribute__((__stdcall__)) PicFind(
  int mode, int c, int n, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , char * text, int * s1, int * s0
  , int err1, int err0, int w1, int h1, int * allpos)
{
  int o, i, j, k, x, y, w, h, ok=0;
  int r, g, b, rr, gg, bb, e1, e0, len1, len0, max;
  w=sw-w1+1; h=sh-h1+1; k=sy*Stride+sx*4;
  // Generate Lookup Table
  o=len1=len0=0;
  for (y=0; y<h1; y++)
  {
    for (x=0; x<w1; x++)
    {
      j=y*Stride+x*4;
      if (text[o++]=='1')
        s1[len1++]=j;
      else
        s0[len0++]=j;
    }
  }
  max=len1>len0 ? len1 : len0;
  // Color Mode
  if (mode==0)
  {
    for (y=0; y<h; y++)
    {
      for (x=0; x<w; x++)
      {
        o=y*Stride+x*4+k; e1=err1; e0=err0;
        j=o+c; rr=Bmp[2+j]; gg=Bmp[1+j]; bb=Bmp[j];
        for (i=0; i<max; i++)
        {
          if (i<len1)
          {
            j=o+s1[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b>n && (--e1)<0) goto NoMatch1;
          }
          if (i<len0)
          {
            j=o+s0[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b<=n && (--e0)<0) goto NoMatch1;
          }
        }
        allpos[ok++]=(sy+y)<<16|(sx+x);
        if (ok>=1024) goto Return1;
        //Clear the image that has been found
        for (i=0; i<len1; i++)
          { j=o+s1[i]; Bmp[2+j]=0xFF; Bmp[1+j]=0xFF; Bmp[j]=0xFF; }
        for (i=0; i<len0; i++)
          { j=o+s0[i]; Bmp[2+j]=0xFF; Bmp[1+j]=0xFF; Bmp[j]=0xFF; }
        NoMatch1:
        continue;
      }
    }
    goto Return1;
  }
  // Gray Threshold Mode
  c=(c+1)*1000;
  for (y=0; y<sh; y++)
  {
    for (x=0; x<sw; x++)
    {
      o=y*Stride+x*4+k;
      Bmp[3+o]=Bmp[2+o]*299+Bmp[1+o]*587+Bmp[o]*114<c ? 1:0;
    }
  }
  k=k+3;
  for (y=0; y<h; y++)
  {
    for (x=0; x<w; x++)
    {
      o=y*Stride+x*4+k; e1=err1; e0=err0;
      for (i=0; i<max; i++)
      {
        if (i<len1 && Bmp[o+s1[i]]!=1 && (--e1)<0) goto NoMatch2;
        if (i<len0 && Bmp[o+s0[i]]!=0 && (--e0)<0) goto NoMatch2;
      }
      allpos[ok++]=(sy+y)<<16|(sx+x);
      if (ok>=1024) goto Return1;
      //Clear the image that has been found
      for (i=0; i<len1; i++) Bmp[o+s1[i]]=0;
      NoMatch2:
      continue;
    }
  }
  Return1:
  return ok;
}

*/


; Note: This function is used for combination lookup,
; for example, a 0-9 text library has been set up,
; then any ID number can be found.
; Use Pic(Text,1) and PicN(number) when using.
; Use PicX(Text) to automatically cut into multiple characters.
; Only grayscale threshold mode is currently supported.

FindText2(x,y,w,h,err1,err0,text,Interval=20)
{
  xywh2xywh(x,y,w,h,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  sx:=0, sy:=0, sw:=w, sh:=h
  arr:=[], info:=[], allw:=0, allv:=allcolor:=allcomment:=""
  if (err1=0 and err0=0)
    err1:=err0:=0.1
  Loop, Parse, text, |
  {
    v:=A_LoopField
    IfNotInString, v, $, Continue
    comment:="", e1:=err1, e0:=err0
    ; You Can Add Comment Text within The <>
    if RegExMatch(v,"<([^>]*)>",r)
      v:=StrReplace(v,r), comment:=Trim(r1)
    ; You can Add two fault-tolerant in the [], separated by commas
    if RegExMatch(v,"\[([^\]]*)]",r)
    {
      v:=StrReplace(v,r), r1.=","
      StringSplit, r, r1, `,
      e1:=r1, e0:=r2
    }
    StringSplit, r, v, $
    color:=r1, v:=r2
    if !InStr(color,"*")
      Continue
    StringSplit, r, v, .
    w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
    if (r0<2 or h1<1 or w1>sw or h1>sh or StrLen(v)!=w1*h1)
      Continue
    if (allcolor="")
      allcolor:=StrReplace(color,"*")
    StrReplace(v,"1","",len1), len0:=StrLen(v)-len1
    e1:=Round(len1*e1), e0:=Round(len0*e0)
    info.Push(StrLen(allv),w1,h1,len1,len0,e1,e0)
    allv.=v, allw+=w1, allcomment.=comment
  }
  if (allv="")
  {
    SetBatchLines, %bch%
    return, 0
  }
  num:=info.MaxIndex(), VarSetCapacity(in,num*4,0)
  Loop, % num
    NumPut(info[A_Index], in, 4*(A_Index-1), "int")
  VarSetCapacity(ss, sw*sh, 0), k:=StrLen(allv)*4
  VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
  VarSetCapacity(allpos, 1024*4, 0)
  offsetX:=Interval, offsetY:=5
  if (ok:=PicFind2(allcolor,offsetX,offsetY,Scan0,Stride
    ,sx,sy,sw,sh,ss,allv,s1,s0,in,num,allpos))
  {
    Loop, % ok
      pos:=NumGet(allpos, 4*(A_Index-1), "uint")
      , rx:=(pos&0xFFFF)+x, ry:=(pos>>16)+y
      , arr.Push( [rx,ry,allw,h1,allcomment] )
  }
  SetBatchLines, %bch%
  return, arr.MaxIndex() ? arr:0
}

PicFind2(color, offsetX, offsetY
  , Scan0, Stride, sx, sy, sw, sh
  , ByRef ss, ByRef text, ByRef s1, ByRef s0
  , ByRef in, num, ByRef allpos)
{
  static MyFunc
  if !MyFunc
  {
    x32:="5557565383EC788B8424C4000000C7442414000000008BB"
    . "C24BC00000085C00F8EBA0000008B4424148B9C24C00000008"
    . "B1C83895C24088B9C24C00000008B7483048B44830885C0897"
    . "42404894424107E778B74240831EDC704240000000089F28B4"
    . "4240485C07E4C8B4C24088D1C28896C240C89E829E9038C24B"
    . "4000000EB0D89049783C00183C20139C3741B803C013175ED8"
    . "BAC24B80000008944B50083C00183C60139C375E58B5C24040"
    . "15C24088B6C240C8304240103AC24A80000008B04243944241"
    . "0759883442414078B442414398424C40000000F8F46FFFFFF8"
    . "B84248C0000008B9C24A00000008BAC24AC000000C70424000"
    . "00000C74424040000000083C00169F8E80300008B8424A4000"
    . "0000FAF84249C0000008D14988B8424A80000008B9C249C000"
    . "000F7D88D0483894424088B8424A8000000C1E00285ED89442"
    . "40C7E7A89D58BB424A800000085F67E598B8C24980000008B5"
    . "C24048BB42498000000039C24B000000001E9036C240C01EE0"
    . "FB651020FB6410169D22B01000069C04B02000001C20FB6016"
    . "BC07201D039C70F9F0383C10483C30139CE75D38BB424A8000"
    . "0000174240483042401036C24088B0424398424AC000000758"
    . "88B9424C00000008B8424C00000008B9C24C00000008BB424C"
    . "00000008B52148B40048B5B0C8B76108954242C8B9424C0000"
    . "000895C2424897424288B5218895424308B9424A800000029C"
    . "239F30F4DF38B9C24C00000008954241C8934248BB424AC000"
    . "0002B7308897424680F88740300008B9C24B80000008B74242"
    . "483E801C744242000000000C744245C00000000C7442470000"
    . "00000894424608D1CB3895C24748B44241C85C00F88E500000"
    . "08B74245C8B8424A4000000C74424080000000001F0C1E0108"
    . "944246C89F02BB4249400000089F3BE000000000F49F389742"
    . "4580FAFB424A8000000897424548BB4249400000001C689742"
    . "4648D7426008B4424088B1C240344242085DB89C18944240C0"
    . "F8EAA0000008B7424308B54242C31C0038C24B00000008B5C2"
    . "4248B6C242889742404EB30908DB4260000000039C57E1C8BB"
    . "424BC0000008B3C8601CF803F00740B836C24040178268D742"
    . "60083C001390424745B39C37ED48BB424B80000008B3C8601C"
    . "F803F0174C383EA0179BE83442408018B4424083944241C0F8"
    . "D6BFFFFFF8344245C018BB424A80000008B44245C017424203"
    . "94424680F8DF1FEFFFF8B5C247083C47889D85B5E5F5DC2400"
    . "08B4424600344240883BC24C400000007894424340F8E93010"
    . "0008B8424C0000000C74424480700000083C020894424388B4"
    . "424388B9424A80000008B7424348B0029C28944244C8B84249"
    . "000000001F039C20F4EC289C38944245039F30F8C5CFFFFFF8"
    . "B4424388B5C24648B700C8B6808897424108B7010897424188"
    . "9C68B4014894424408B8424AC0000002B460439C30F4EC3894"
    . "424148B46FC8BB424B800000089442404C1E00201C6038424B"
    . "C000000894424448B4424588B7C2434037C24543B442414894"
    . "424040F8F8700000085ED897C240C7E258B9C24B00000008B5"
    . "4241831C001FB8B0C8601D9803901740583EA01784783C0013"
    . "9C575EA8B4C241085C97E7D8B9C24B0000000896C243C31C08"
    . "B4C24408B6C244401FBEB0983C00139442410745B8B5485000"
    . "1DA803A0074EC83E90179E78B6C243C6690834424040103BC2"
    . "4A80000008B442404394424140F8D79FFFFFF83442434018B4"
    . "42434394424500F8D4CFFFFFF83442408018B4424083944241"
    . "C0F8DC0FDFFFFE950FEFFFF8B4424348B5C244C83442448078"
    . "34424381C8D4418FF894424348B442448398424C40000000F8"
    . "F83FEFFFF8B7424708B442408038424A00000008B9424C8000"
    . "0000B44246C8D5E0181FBFF0300008904B20F8F1BFEFFFF8B5"
    . "4242485D27E278B74240C8B8424B00000008B9424B80000008"
    . "D0C308B7424748B0283C20401C839F2C6000075F2834424080"
    . "1895C24708B4424083944241C0F8D1EFDFFFFE9AEFDFFFF83C"
    . "47831DB89D85B5E5F5DC240009090909090909090909090"
    x64:="4157415641554154555756534881EC88000000488B84243"
    . "8010000488BAC2418010000898C24D00000008B8C244001000"
    . "0899424D800000044898424E00000004C898C24E80000004C8"
    . "BA4242001000085C94C8BBC24280100004C8BAC24300100004"
    . "889442408C7442404000000000F8EA1000000448BB42408010"
    . "0004889AC2418010000488B4424088B68088B308B780485ED7"
    . "E5C89F14189F24531DB31DB85FF7E444863D6468D0C1F4489D"
    . "84C01E2EB174C63C14883C20183C101438944850083C001413"
    . "9C1741C803A3175E44D63C24883C2014183C2014389048783C"
    . "0014139C175E401FE83C3014501F339DD75AE8344240407488"
    . "34424081C8B442404398424400100000F8F77FFFFFF488BAC2"
    . "4180100008B8424D00000008B9C24F800000031FF8B9424100"
    . "10000448D50018B8424000100000FAF8424F00000004569D2E"
    . "8030000448D1C988B8424080100008B9C24F0000000F7D8448"
    . "D34838B84240801000031DB85D2448D2485000000000F8E8F0"
    . "000004C89BC24280100004C89AC24300100008BB4240801000"
    . "0448BBC24100100004C8BAC24E800000085F67E494963C34C6"
    . "3CB4531C0498D4C05024901E90FB6110FB641FF69D22B01000"
    . "069C04B02000001C20FB641FE6BC07201D04139C2430F9F040"
    . "14983C0014883C1044439C67FCD4501E301F383C7014501F34"
    . "139FF75A84C8BBC24280100004C8BAC2430010000488B9C243"
    . "8010000488B8424380100008B5B148B5004448B700C8B40108"
    . "95C2414488B9C243801000089C68B5B18895C24188B9C24080"
    . "1000029D34139C6895C2408488B9C2438010000410F4DC6418"
    . "9C48B8424100100002B43088944246C0F8894030000418D46F"
    . "F4C89EFC7442410000000004589E548C744245800000000498"
    . "9EC498D4487044C89FDC7442474000000004589F74189F6488"
    . "94424788D42FF894424648B44240885C00F88DE000000488B5"
    . "C24588B8424000100004889BC24300100004489EF01D8C1E01"
    . "08944247089D82B8424E000000089C6B8000000000F49C631F"
    . "6894424604989F5488BB424300100000FAF842408010000894"
    . "424548B8424E000000001D8894424688B44241085FF44896C2"
    . "404428D1C280F8EA6000000448B4C2418448B44241431C0EB3"
    . "00F1F8400000000004139CE7E1B8B148601DA4863D241803C1"
    . "400740C4183E901782B660F1F4400004883C00139C77E68413"
    . "9C789C17ED18B54850001DA4863D241803C140174C14183E80"
    . "179BB4983C50144396C24087D854189FD4889F748834424580"
    . "18BB42408010000488B442458017424103944246C0F8DF6FEF"
    . "FFF8B4C247489C84881C4880000005B5E5F5D415C415D415E4"
    . "15FC38B4424640344240483BC2440010000078944241C0F8EA"
    . "D010000488B84243801000048896C24208BAC2408010000448"
    . "97C243CC74424340700000044897424404883C020897C24444"
    . "C896C24484989C74889742428418B0789EA8B5C241C29C2894"
    . "424388B8424D800000001D839C20F4EC289C68944245039DE0"
    . "F8CCF000000418B47148BBC2410010000412B7F04496377FC4"
    . "58B4F08458B6F0C894424308B442468458B771039F80F4EF84"
    . "88B44242048C1E6024C8D143048037424288B442460448B442"
    . "41C440344245439F84189C37F694585C94489C37E234489F13"
    . "1D2418B04924401C0489841803C0401740583E901783B4883C"
    . "2014139D17FE24585ED7E7D8B4C243031D2EB0E0F1F4400004"
    . "883C2014139D57E678B04964401C0489841803C040074E883E"
    . "90179E30F1F004183C3014101E84439DF7D978344241C018B4"
    . "4241C394424500F8D6FFFFFFF4C8B6C2448448B7C243C448B7"
    . "424408B7C2444488B6C2420488B7424284983C50144396C240"
    . "80F8DCAFDFFFFE940FEFFFF8B44241C8B7424384983C71C834"
    . "42434078D4430FF8944241C8B442434398424400100000F8FA"
    . "BFEFFFF448B7C243C448B7424408B7C24444C8B6C2448488B6"
    . "C2420488B74242848634424748B542404039424F80000004C8"
    . "B9C24480100000B5424708D480181F9FF030000418914830F8"
    . "FF4FDFFFF4585FF7E1D4C8B4424784889E88B104883C00401D"
    . "A4C39C04863D241C604140075EB4983C50144396C2408894C2"
    . "4740F8D1AFDFFFFE990FDFFFF31C9E9B3FDFFFF9090"
    MCode(MyFunc, A_PtrSize=8 ? x64:x32)
  }
  return, DllCall(&MyFunc, "int",color, "int",offsetX
    , "int",offsetY, "ptr",Scan0, "int",Stride
    , "int",sx, "int",sy, "int",sw, "int",sh
    , "ptr",&ss, "AStr",text, "ptr",&s1, "ptr",&s0
    , "ptr",&in, "int",num, "ptr",&allpos)
}

/***** C source code of machine code *****

int __attribute__((__stdcall__)) PicFind2(
  int c, int offsetX, int offsetY
  , unsigned char * Bmp, int Stride
  , int sx, int sy, int sw, int sh
  , char * ss, char * text, int * s1, int * s0
  , int * in, int num, int * allpos )
{
  int o, x, y, i, j, max, e1, e0, ok=0;
  int o1, x1, y1, w1, h1, sx1, sy1, len1, len0, err1, err0;
  int o2, x2, y2, w2, h2, sx2, sy2, len21, len20, err21, err20;
  // Generate Lookup Table
  for (i=0; i<num; i+=7)
  {
    o=o1=o2=in[i]; w1=in[i+1]; h1=in[i+2];
    for (y=0; y<h1; y++)
    {
      for (x=0; x<w1; x++)
      {
        j=y*sw+x;
        if (text[o++]=='1')
          s1[o1++]=j;
        else
          s0[o2++]=j;
      }
    }
  }
  // Gray Threshold Mode
  c=(c+1)*1000; o=sy*Stride+sx*4; j=Stride-4*sw; i=0;
  for (y=0; y<sh; y++, o+=j)
  {
    for (x=0; x<sw; x++, o+=4, i++)
      ss[i]=Bmp[2+o]*299+Bmp[1+o]*587+Bmp[o]*114<c ? 1:0;
  }
  // Start Lookup
  w1=in[1]; h1=in[2]; len1=in[3]; len0=in[4]; err1=in[5]; err0=in[6];
  sx1=sw-w1; sy1=sh-h1; max=len1>len0 ? len1 : len0;
  for (y=0; y<=sy1; y++)
  {
    for (x=0; x<=sx1; x++)
    {
      o=y*sw+x; e1=err1; e0=err0;
      for (j=0; j<max; j++)
      {
        if (j<len1 && ss[o+s1[j]]!=1 && (--e1)<0) goto NoMatch1;
        if (j<len0 && ss[o+s0[j]]!=0 && (--e0)<0) goto NoMatch1;
      }
      x1=x+w1-1; y1=y-offsetY; if (y1<0) y1=0;
      for (i=7; i<num; i+=7)
      {
        o2=in[i]; w2=in[i+1]; h2=in[i+2];
        len21=in[i+3]; len20=in[i+4]; err21=in[i+5]; err20=in[i+6];
        sx2=sw-w2; j=x1+offsetX; if (j<sx2) sx2=j;
        sy2=sh-h2; j=y+offsetY; if (j<sy2) sy2=j;
        for (x2=x1; x2<=sx2; x2++)
        {
          for (y2=y1; y2<=sy2; y2++)
          {
            o=y2*sw+x2; e1=err21; e0=err20;
            for (j=0; j<len21; j++)
              if (ss[o+s1[o2+j]]!=1 && (--e1)<0) goto NoMatch2;
            for (j=0; j<len20; j++)
              if (ss[o+s0[o2+j]]!=0 && (--e0)<0) goto NoMatch2;
            goto MatchOK;
            NoMatch2:
            continue;
          }
        }
        goto NoMatch1;
        MatchOK:
        x1=x2+w2-1;
      }
      allpos[ok++]=(sy+y)<<16|(sx+x);
      if (ok>=1024) goto Return1;
      //Clear the image that has been found
      for (j=0; j<len1; j++) ss[o+s1[j]]=0;
      NoMatch1:
      continue;
    }
  }
  Return1:
  return ok;
}

*/


;================= The End =================

;
Exit:
; gdi+ may now be shutdown on exiting the program
Gdip_Shutdown(pToken)
ExitApp
Return

Thank you.

Paul.
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

09 Oct 2018, 11:40

@paulpma, To draw boxes around searched area using GDIp_All, You can try this:

Code: Select all


#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
;#Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
#SingleInstance, Force
#Include <GDIp_All>

; Start gdi+
If !pToken := Gdip_Startup()
{
	MsgBox, 48, gdiplus error!, Gdiplus failed to start. Please ensure you have gdiplus on your system
	ExitApp
}
OnExit, Exit

Width := A_ScreenWidth, Height := A_ScreenHeight ; Set the width and height we want as our drawing area, to draw everything in. This will be the dimensions of our bitmap

NGDOBC := [181, 180, 57, 14] ;NG DOB Cords xywh

Gui, Boxes: -Caption +E0x80000 +LastFound +AlwaysOnTop +ToolWindow +OwnDialogs ; Create a layered window (+E0x80000 : must be used for UpdateLayeredWindow to work!) that is always on top (+AlwaysOnTop), has no taskbar entry or caption
Gui, Boxes: Show, x0 y0 NA ; Show the window
hwnd1 := WinExist() ; Get a handle to this window we have created in order to update it later


DrawBoxes:

hbm := CreateDIBSection(Width, Height) ; Create a gdi bitmap with width and height of what we are going to draw into it. This is the entire drawing area for everything
hdc := CreateCompatibleDC() ; Get a device context compatible with the screen
obm := SelectObject(hdc, hbm) ; Select the bitmap into the device context
G := Gdip_GraphicsFromHDC(hdc) ; Get a pointer to the graphics of the bitmap, for use with drawing functions
Gdip_SetSmoothingMode(G, 4) ; Set the smoothing mode to antialias = 4 to make shapes appear smother (only used for vector drawing and filling)

dobPen := Gdip_CreatePen(0x66161616, 1) 
Gdip_DrawRectangle(G, dobPen, NGDOBC[1], NGDOBC[2], NGDOBC[3], NGDOBC[4]) 
Gdip_DeletePen(dobPen)  

UpdateLayeredWindow(hwnd1, hdc, 0, 0, Width, Height) ; Update the specified window we have created (hwnd1) with a handle to our bitmap (hdc), specifying the x,y,w,h we want it positioned on our screen ; So this will position our gui at 

(0,0) with the Width and Height specified earlier

SelectObject(hdc, obm) ; Select the object back into the hdc
DeleteObject(hbm) ; Now the bitmap may be deleted
DeleteDC(hdc) ; Also the device context related to the bitmap may be deleted
Gdip_DeleteGraphics(G) ; The graphics may now be deleted

return


Exit:
; gdi+ may now be shutdown on exiting the program
Gdip_Shutdown(pToken)
ExitApp
Return


F1::

CoordMode, Mouse

MouseGetPos, x, y

NGDOBC[1]:=x-NGDOBC[3]//2, NGDOBC[2]:=y-NGDOBC[4]//2

Gosub, DrawBoxes

return


F2::

Text:=""
Text.="|<1>*155$4.009e8W802"
Text.="|<2>*156$4.00yF5gw02"
Text.="|<3>*156$6.000Sm2C3nS000U"
Text.="|<4>*127$6.26+GGWS27U"
Text.="|<5>*140$4.00yDY9w02"
Text.="|<6>*140$6.000SHynlHS000U"
Text.="|<7>*127$5.z8EW48EY"
Text.="|<8>*140$4.00uNONw02"
Text.="|<9>*156$6.000SnnT3nS000U"
Text.="|<0>*157$7.0001sgmNAaFk000E"

t1:=A_TickCount
;------------------------------
OCR:=FindTextOCR(NGDOBC[1], NGDOBC[2], NGDOBC[3], NGDOBC[4], 0, 0, Text)
;------------------------------
t1:=A_TickCount-t1
MsgBox, 4096, OCR, OCR Result: [%OCR%] in %t1% ms.
Return

paulpma
Posts: 12
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

09 Oct 2018, 21:24

Thank you feiyue for fast reply and you are genius. It works great.....Thank you again.

I was trying to make the box static on the screen, at certain coordinates and not to move. I think I should be able to figure it out in a few.... The coordinates for findtext() seem tricky. Once I figure it out, I should make a small tutorial on it. : :D

Thank you feiyue.
paulpma
Posts: 12
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

09 Oct 2018, 23:37

I am continuing to work with the function, but strange behaviour I have discovered. Using the following text:

Code: Select all

Text:=""
Text.="|<1>*160$4.009e8W802"
Text.="|<2>*160$4.00yF5gw02"
Text.="|<3>*160$6.000Sm2C3nS000U"
Text.="|<4>*160$6.00066+Gz22000U"
Text.="|<5>*160$4.00yDY9w02"
Text.="|<6>*160$6.000SHynlHS000U"
Text.="|<7>*160$5.z8EW48EY"
Text.="|<8>*160$4.00uNONw02"
Text.="|<9>*160$6.000SnnT3nS000U"
Text.="|<0>*160$5.003xtnbPk01"
I am trying to capture a date from image.
date.jpg
(1.84 KiB) Downloaded 101 times
OCR reads 01011356 instead of 01011956. I have done individual tests with "capture tool" for numbers 3 and 9 separately, and the tool finds them correctly. No errors via "capture tool" even if they are on the same image. Here is an example of date including #3,
date2.jpg
(1.21 KiB) Downloaded 101 times
Function finds 3 no problem, but it mistakes 9 for a #3 only during FindTextOCR; I have replicated this issue with other dates too. My both errors set to zero. Any ideas?
I have tried different color modes for #9, - Text.="|<9>*0x040100@0.50$6.000SnnT3GS000U" ; same result, comes out as #3
tried to modify #3 text as well and no luck.
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

10 Oct 2018, 23:04

@paulpma, You can try this:

Code: Select all


FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=20)
{
  OCR:="", RightX:=nX+nW-1
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    For k,v in ok
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      x:=v.1, y:=v.2, w:=v.3, h:=v.4, comment:=v.5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<LeftX)
        LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
      else if (x=LeftX)
      {
        Loop, 100
        {
          err:=A_Index/100
          if FindText(x, y, w, h, err, err, Text)
          {
            LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
            Break
          }
          if FindText(LeftX, LeftY, LeftW, LeftH, err, err, Text)
            Break
        }
      }
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and LeftX-nX-1>Interval ? "*":"") . LeftOCR
    ; Update nX and nW for next search
    nX:=LeftX+LeftW-1, nW:=RightX-nX+1
  }
  return, OCR
}


Return to “Scripts and Functions”

Who is online

Users browsing this forum: DuyMinh and 34 guests