FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

FindText - Capture screen image into text and then find it

14 May 2016, 12:10

I wrote a useful function to share with you.
It is used to find text or images on the screen.
Hope you like it ! :)

Detailed usage can refer to:
1、The introduction by ed1chandler.
2、The introduction by c4p (latest).

Code: Select all

/*
===========================================
  FindText - Capture screen image into text and then find it
  https://autohotkey.com/boards/viewtopic.php?f=6&t=17834

  Author  :  FeiYue
  Version :  6.1
  Date    :  2018-10-07

  Usage:
  1. Capture the image to text string.
  2. Test find the text string on full Screen.
  3. When test is successful, you may copy the code
     and paste it into your own script.
     Note: Copy the "FindText()" function and the following
     functions and paste it into your own script Just once.

  Note:
     After upgrading to v6.0, the search scope using WinAPI's
     upper left corner X, Y coordinates, and width, height.
     This will be better understood and used.

===========================================
  Introduction of function parameters:

  returnArray := FindText(
      X --> upper left corner X coordinates
    , Y --> upper left corner Y coordinates
    , W --> the search scope's Width
    , H --> the search scope's Height
    , Character "0" fault-tolerant in percentage --> 0.1=10%
    , Character "_" fault-tolerant in percentage --> 0.1=10%
    , text --> The Base64 encoding string for the text to find
    )

  The range used by AHK is determined by the upper left
  corner and the lower right corner: (x1, y1, x2, y2),
  it can be converted to: (x1, y1, x2-x1+1, y2-y1+1).

  The fault-tolerant parameters allow the loss of specific characters.

  Text parameters can be a lot of text to find, separated by "|".

  return is a array, contains the [X,Y,W,H,Comment] results of Each Find,
  if no image is found, the function returns 0.

===========================================
*/

#NoEnv
#SingleInstance Force
SetBatchLines, -1
CoordMode, Mouse
CoordMode, Pixel
CoordMode, ToolTip
SetWorkingDir, %A_ScriptDir%
Menu, Tray, Icon, Shell32.dll, 23
Menu, Tray, Add
Menu, Tray, Add, Main_Window
Menu, Tray, Default, Main_Window
Menu, Tray, Click, 1
; The capture range can be changed by adjusting the numbers
;----------------------------
  ww:=35, hh:=12
;----------------------------
nW:=2*ww+1, nH:=2*hh+1
Gosub, MakeCaptureWindow
Gosub, MakeMainWindow
Gosub, Load_ToolTip_Text
OnExit, savescr
Gosub, readscr
return


F12::    ; Hotkey --> Reload
SetTitleMatchMode, 2
SplitPath, A_ScriptName,,,, name
IfWinExist, %name%
{
  ControlSend, ahk_parent, {Ctrl Down}s{Ctrl Up}
  Sleep, 500
}
Reload
return


Load_ToolTip_Text:
ToolTip_Text=
(LTrim
Capture   = Initiate Image Capture Sequence
Test      = Test Results of Code
Copy      = Copy Code to Clipboard
AddFunc   = Additional FindText() in Copy
U         = Cut the Upper Edge by 1
U3        = Cut the Upper Edge by 3
L         = Cut the Left Edge by 1
L3        = Cut the Left Edge by 3
R         = Cut the Right Edge by 1
R3        = Cut the Right Edge by 3
D         = Cut the Lower Edge by 1
D3        = Cut the Lower Edge by 3
Auto      = Automatic Cutting Edge`r`nOnly after Color2Two or Gray2Two
Similar   = Adjust color similarity as Equivalent to The Selected Color
SelCol    = Selected Image Color which Determines Black or Pixel White Conversion (Hex of Color)
Gray      = Grayscale Threshold which Determines Black or White Pixel Conversion (0-255)
Color2Two = Converts Image Pixels from Color to Black or White
Gray2Two  = Converts Image Pixels from Grays to Black or White
UsePos    = Use position instead of color value to suit any color
Modify    = Allows for Pixel Cleanup of Black and White Image`r`nOnly After Gray2Two or Color2Two
Reset     = Reset to Original Captured Image
Comment   = Optional Comment used to Label Code ( Within <> )
SplitAdd  = Using Markup Segmentation to Generate Text Library
AllAdd    = Append Another FindText Search Text into Previously Generated Code
OK        = Create New FindText Code for Testing
Close     = Close the Window Don't Do Anything
)
return

readscr:
f=%A_Temp%\~scr.tmp
FileRead, s, %f%
GuiControl, Main:, scr, %s%
s=
return

savescr:
f=%A_Temp%\~scr.tmp
GuiControlGet, s, Main:, scr
FileDelete, %f%
FileAppend, %s%, %f%
ExitApp

Main_Window:
Gui, Main:Show, Center
return

MakeMainWindow:
Gui, Main:Default
Gui, +AlwaysOnTop
Gui, Margin, 15, 15
Gui, Color, DDEEFF
Gui, Font, s6 bold, Verdana
Gui, Add, Edit, xm w660 r25 vMyEdit -Wrap -VScroll
Gui, Font, s12 norm, Verdana
Gui, Add, Button, w220 gMainRun, Capture
Gui, Add, Button, x+0 wp gMainRun, Test
Gui, Add, Button, x+0 wp gMainRun Section, Copy
Gui, Font, s10
Gui, Add, Text, xm, Click Text String to See ASCII Search Text in the Above
Gui, Add, Checkbox, xs yp w220 r1 -Wrap Checked vAddFunc, Additional FindText() in Copy
Gui, Font, s12 cBlue, Verdana
Gui, Add, Edit, xm w660 h350 vscr Hwndhscr -Wrap HScroll
Gui, Show,, Capture Image To Text And Find Text Tool
;---------------------------------------
OnMessage(0x100, "EditEvents1")  ; WM_KEYDOWN
OnMessage(0x201, "EditEvents2")  ; WM_LBUTTONDOWN
OnMessage(0x200, "WM_MOUSEMOVE") ; Show ToolTip
return

EditEvents1()
{
  ListLines, Off
  if (A_Gui="Main") and (A_GuiControl="scr")
    SetTimer, ShowText, -100
}

EditEvents2()
{
  ListLines, Off
  if (A_Gui="Capture")
    WM_LBUTTONDOWN()
  else
    EditEvents1()
}

ShowText:
ListLines, Off
Critical
ControlGet, i, CurrentLine,,, ahk_id %hscr%
ControlGet, s, Line, %i%,, ahk_id %hscr%
s := ASCII(s)
GuiControl, Main:, MyEdit, % Trim(s,"`n")
return

MainRun:
k:=A_GuiControl
WinMinimize
Gui, Hide
DetectHiddenWindows, Off
Gui, +LastFound
WinWaitClose, % "ahk_id " WinExist()
if IsLabel(k)
  Gosub, %k%
Gui, Main:Show
GuiControl, Main:Focus, scr
return

Copy:
GuiControlGet, s,, scr
GuiControlGet, AddFunc
if AddFunc != 1
  s:=RegExReplace(s,"\n\K[\s;=]+ Copy The[\s\S]*")
Clipboard:=StrReplace(s,"`n","`r`n")
s=
return

Capture:
Gui, Mini:Default
Gui, +LastFound +AlwaysOnTop -Caption +ToolWindow +E0x08000000
WinSet, Transparent, 200
Gui, Color, Red
x:=nW+2, y:=nH+2, w:=nW+4, h:=nH+4
Gui, Show, Hide w%w% h%h%
WinSet, Region
  , 0-0 %w%-0 %w%-%h% 0-%h% 0-0 2-2 %x%-2 %x%-%y% 2-%y% 2-2
;------------------------------
Hotkey, $*RButton, _RButton_Off, On
ListLines, Off
oldx:=oldy:=""
Loop {
  MouseGetPos, x, y
  if (oldx=x and oldy=y)
    Continue
  oldx:=x, oldy:=y, px:=x, py:=y
  Gui, Show, % "NA x" (px-w//2) " y" (py-h//2)
  ToolTip, % "The Capture Position : " px "," py
    . "`nMove and Press RButton to start capture"
    . "`nMove and Release RButton to end capture"
  Sleep, 50
} Until GetKeyState("RButton","P")
oldx:=oldy:=""
Loop {
  MouseGetPos, x, y
  if (oldx=x and oldy=y)
    Continue
  oldx:=x, oldy:=y
  ToolTip, % "The Capture Position : " px "," py
    . "`nMove and Press RButton to start capture"
    . "`nMove and Release RButton to end capture"
  Sleep, 50
} Until !GetKeyState("RButton","P")
ToolTip
ListLines, On
Gui, Destroy
WinWaitClose
cors:=getc(px,py,ww,hh)
Hotkey, $*RButton, _RButton_Off, Off
Goto, ShowCaptureWindow
_RButton_Off:
return

ShowCaptureWindow:
cors.Event:="", cors.Result:=""
;--------------------------------
Gui, Capture:Default
k:=nW*nH+1
Loop, % nW
  GuiControl,, % C_[k++], 0
GuiControl,, SelCol
GuiControl,, Gray
GuiControl,, Modify, % Modify:=0
GuiControl,, UsePos, % UsePos:=0
GuiControl, Focus, Gray
Gosub, Reset
Gui, Show, Center
DetectHiddenWindows, Off
Gui, +LastFound
WinWaitClose, % "ahk_id " WinExist()
;--------------------------------
if InStr(cors.Event,"OK")
{
  if !A_IsCompiled
  {
    FileRead, fs, %A_ScriptFullPath%
    fs:=SubStr(fs,fs~="i)\n[;=]+ Copy The")
  }
  GuiControl, Main:, scr, % cors.Result "`n" fs
  cors.Result:=fs:=""
  return
}
if InStr(cors.Event,"Add")
  add(cors.Result, 0), cors.Result:=""
return

WM_LBUTTONDOWN()
{
  global
  ListLines, Off
  MouseGetPos,,,, mclass
  IfNotInString, mclass, progress
    return
  MouseGetPos,,,, mid, 2
  For k,v in C_
    if (v=mid)
    {
      if (k>nW*nH)
      {
        GuiControlGet, i, Capture:, %v%
        GuiControl, Capture:, %v%, % i ? 0:100
      }
      else if (Modify and bg!="")
      {
        c:=cc[k], cc[k]:=c="0" ? "_" : c="_" ? "0" : c
        c:=c="0" ? "White" : c="_" ? "Black" : WindowColor
        Gosub, SetColor
      }
      else
      {
        GuiControl, Capture:, SelCol, % cors[k]
        cors.Color:=cors[k]
      }
      return
    }
}

getc(px, py, ww, hh)
{
  xywh2xywh(px-ww,py-hh,2*ww+1,2*hh+1,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  cors:=[], k:=0, nW:=2*ww+1, nH:=2*hh+1
  ListLines, Off
  fmt:=A_FormatInteger
  SetFormat, IntegerFast, H
  Loop, %nH% {
    j:=py-hh-y+A_Index-1
    Loop, %nW% {
      i:=px-ww-x+A_Index-1, k++
      if (i>=0 and i<w and j>=0 and j<h)
        c:=NumGet(Scan0+0,i*4+j*Stride,"uint")
          , cors[k]:="0x" . SubStr(0x1000000|c,-5)
      else
        cors[k]:="0xFFFFFF"
    }
  }
  SetFormat, IntegerFast, %fmt%
  ListLines, On
  cors.LeftCut:=Abs(px-ww-x)
  cors.RightCut:=Abs(px+ww-(x+w-1))
  cors.UpCut:=Abs(py-hh-y)
  cors.DownCut:=Abs(py+hh-(y+h-1))
  SetBatchLines, %bch%
  return, cors
}

Test:
GuiControlGet, s, Main:, scr
s:="`n#NoEnv`nMenu, Tray, Click, 1`n"
  . "Gui, _ok_:Show, Hide, _ok_`n"
  . s "`nExitApp`n#SingleInstance off`n"
if (!A_IsCompiled) and InStr(s,"MCode(")
{
  Exec(s)
  DetectHiddenWindows, On
  WinWait, _ok_ ahk_class AutoHotkeyGUI,, 3
  WinWaitClose, _ok_ ahk_class AutoHotkeyGUI,, 3
}
else
{
  t1:=A_TickCount
  RegExMatch(s,"=""\K[^$\n]+\$\d+\.[\w+/]+",Text)
  ok:=FindText(0, 0, 150000, 150000, 0, 0, Text)
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
    . "Pos:`t" X ", " Y "`n`n"
    . "Result:`t" (ok ? "Success !":"Failed !"), 3
  MouseMove, X, Y
}
return

Exec(s)
{
  Ahk:=A_IsCompiled ? A_ScriptDir "\AutoHotkey.exe":A_AhkPath
  s:=RegExReplace(s, "\R", "`r`n")
  Try {
    oExec:=ComObjCreate("WScript.Shell").Exec(Ahk " /r *")
    oExec.StdIn.Write(s)
    oExec.StdIn.Close()
  }
  catch {
    f:=A_Temp "\~test1.tmp"
    s=`r`nFileDelete, %f%`r`n%s%
    FileDelete, %f%
    FileAppend, %s%, %f%
    Run, %Ahk% /r "%f%"
  }
}

MakeCaptureWindow:
WindowColor:="0xCCDDEE"
Gui, Capture:Default
Gui, +LastFound +AlwaysOnTop +ToolWindow
Gui, Margin, 15, 15
Gui, Color, %WindowColor%
Gui, Font, s14, Verdana
ListLines, Off
Gui, -Theme
w:=800//nW, h:=(A_ScreenHeight-300)//nH, w:=h<w ? h-1:w-1
Loop, % nW*(nH+1) {
  i:=A_Index, j:=i=1 ? "" : Mod(i,nW)=1 ? "xm y+1" : "x+1"
  j.=i>nW*nH ? " cRed BackgroundFFFFAA":""
  Gui, Add, Progress, w%w% h%w% %j%
}
WinGet, s, ControlListHwnd
C_:=StrSplit(s,"`n"), s:=""
Loop, % nW*(nH+1)
  Control, ExStyle, -0x20000,, % "ahk_id " C_[A_Index]
Gui, +Theme
ListLines, On
Gui, Add, Button, xm+95  w45 gUpCut Section, U
Gui, Add, Button, x+0    wp gUpCut3, U3
Gui, Add, Text,   xm+310 yp+6 Section, Color Similarity  0
Gui, Add, Slider
  , x+0 w150 vSimilar Page1 NoTicks ToolTip Center, 100
Gui, Add, Text,   x+0, 100
Gui, Add, Checkbox, x+15 gRun vUsePos, UsePos
Gui, Add, Button, xm     w45 gLeftCut, L
Gui, Add, Button, x+0    wp gLeftCut3, L3
Gui, Add, Button, x+15   w70 gRun, Auto
Gui, Add, Button, x+15   w45 gRightCut, R
Gui, Add, Button, x+0    wp gRightCut3, R3
Gui, Add, Text,   xs     w160 yp, Selected  Color
Gui, Add, Edit,   x+15   w140 vSelCol
Gui, Add, Button, x+15   w145 gRun, Color2Two
Gui, Add, Button, xm+95  w45 gDownCut, D
Gui, Add, Button, x+0    wp gDownCut3, D3
Gui, Add, Text,   xs     w160 yp, Gray Threshold
Gui, Add, Edit,   x+15   w140 vGray
Gui, Add, Button, x+15   w145 gRun Default, Gray2Two
Gui, Add, Checkbox, xm   y+21 gRun vModify, Modify
Gui, Add, Button, x+5    yp-6 gRun, Reset
Gui, Add, Text,   x+20   yp+6, Comment
Gui, Add, Edit,   x+5    w132 vComment
Gui, Add, Button, x+10   yp-6 gRun, SplitAdd
Gui, Add, Button, x+10   gRun, AllAdd
Gui, Add, Button, x+10   w80 gRun, OK
Gui, Add, Button, x+10   gCancel, Close
Gui, Show, Hide, Capture Image To Text
return

Run:
Critical
k:=A_GuiControl
Gui, +OwnDialogs
if IsLabel(k)
  Goto, %k%
return

Modify:
GuiControlGet, Modify
return

UsePos:
GuiControlGet, UsePos
return

SetColor:
c:=c="White" ? 0xFFFFFF : c="Black" ? 0x000000
  : ((c&0xFF)<<16)|(c&0xFF00)|((c&0xFF0000)>>16)
SendMessage, 0x2001, 0, c,, % "ahk_id " . C_[k]
return

Reset:
if !IsObject(cc)
  cc:=[], gc:=[], pp:=[]
left:=right:=up:=down:=k:=0, bg:=""
Loop, % nW*nH {
  cc[++k]:=1, c:=cors[k], gc[k]:=(((c>>16)&0xFF)*299
    +((c>>8)&0xFF)*587+(c&0xFF)*114)//1000
  Gosub, SetColor
}
Loop, % cors.LeftCut
  Gosub, LeftCut
Loop, % cors.RightCut
  Gosub, RightCut
Loop, % cors.UpCut
  Gosub, UpCut
Loop, % cors.DownCut
  Gosub, DownCut
return

Color2Two:
GuiControlGet, Similar
GuiControlGet, r,, SelCol
if r=
{
  MsgBox, 4096, Tip
    , `n  Please Select a Color First !  `n, 1
  return
}
Similar:=Round(Similar/100,2), n:=Floor(255*3*(1-Similar))
color:=r "@" Similar, k:=i:=0
rr:=(r>>16)&0xFF, gg:=(r>>8)&0xFF, bb:=r&0xFF
Loop, % nW*nH {
  if (cc[++k]="")
    Continue
  c:=cors[k], r:=(c>>16)&0xFF, g:=(c>>8)&0xFF, b:=c&0xFF
  if Abs(r-rr)+Abs(g-gg)+Abs(b-bb)<=n
    cc[k]:="0", c:="Black", i++
  else
    cc[k]:="_", c:="White", i--
  Gosub, SetColor
}
bg:=i>0 ? "0":"_"
return

Gray2Two:
GuiControl, Focus, Gray
GuiControlGet, Threshold,, Gray
if Threshold=
{
  Loop, 256
    pp[A_Index-1]:=0
  Loop, % nW*nH
    if (cc[A_Index]!="")
      pp[gc[A_Index]]++
  IP:=IS:=0
  Loop, 256
    k:=A_Index-1, IP+=k*pp[k], IS+=pp[k]
  NewThreshold:=Floor(IP/IS)
  Loop, 20 {
    Threshold:=NewThreshold
    IP1:=IS1:=0
    Loop, % Threshold+1
      k:=A_Index-1, IP1+=k*pp[k], IS1+=pp[k]
    IP2:=IP-IP1, IS2:=IS-IS1
    if (IS1!=0 and IS2!=0)
      NewThreshold:=Floor((IP1/IS1+IP2/IS2)/2)
    if (NewThreshold=Threshold)
      Break
  }
  GuiControl,, Gray, %Threshold%
}
color:="*" Threshold, k:=i:=0
Loop, % nW*nH {
  if (cc[++k]="")
    Continue
  if (gc[k]<Threshold+1)
    cc[k]:="0", c:="Black", i++
  else
    cc[k]:="_", c:="White", i--
  Gosub, SetColor
}
bg:=i>0 ? "0":"_"
return

gui_del:
cc[k]:="", c:=WindowColor
Gosub, SetColor
return

LeftCut3:
Loop, 3
  Gosub, LeftCut
return

LeftCut:
if (left+right>=nW)
  return
left++, k:=left
Loop, %nH% {
  Gosub, gui_del
  k+=nW
}
return

RightCut3:
Loop, 3
  Gosub, RightCut
return

RightCut:
if (left+right>=nW)
  return
right++, k:=nW+1-right
Loop, %nH% {
  Gosub, gui_del
  k+=nW
}
return

UpCut3:
Loop, 3
  Gosub, UpCut
return

UpCut:
if (up+down>=nH)
  return
up++, k:=(up-1)*nW
Loop, %nW% {
  k++
  Gosub, gui_del
}
return

DownCut3:
Loop, 3
  Gosub, DownCut
return

DownCut:
if (up+down>=nH)
  return
down++, k:=(nH-down)*nW
Loop, %nW% {
  k++
  Gosub, gui_del
}
return

getwz:
wz=
if bg=
  return
ListLines, Off
k:=0
Loop, %nH% {
  v=
  Loop, %nW%
    v.=cc[++k]
  wz.=v="" ? "" : v "`n"
}
ListLines, On
return

Auto:
Gosub, getwz
if wz=
{
  MsgBox, 4096, Tip
    , `nPlease Click Color2Two or Gray2Two First !, 1
  return
}
While InStr(wz,bg) {
  if (wz~="^" bg "+\n")
  {
    wz:=RegExReplace(wz,"^" bg "+\n")
    Gosub, UpCut
  }
  else if !(wz~="m`n)[^\n" bg "]$")
  {
    wz:=RegExReplace(wz,"m`n)" bg "$")
    Gosub, RightCut
  }
  else if (wz~="\n" bg "+\n$")
  {
    wz:=RegExReplace(wz,"\n\K" bg "+\n$")
    Gosub, DownCut
  }
  else if !(wz~="m`n)^[^\n" bg "]")
  {
    wz:=RegExReplace(wz,"m`n)^" bg)
    Gosub, LeftCut
  }
  else Break
}
wz=
return

OK:
AllAdd:
SplitAdd:
Gosub, getwz
if wz=
{
  MsgBox, 4096, Tip
    , `nPlease Click Color2Two or Gray2Two First !, 1
  return
}
if InStr(color,"@") and (UsePos)
{
  StringSplit, r, color, @
  k:=i:=j:=0
  Loop, % nW*nH {
    if (cc[++k]="")
      Continue
    i++
    if (cors[k]=r1)
    {
      j:=i
      Break
    }
  }
  if (j=0)
  {
    MsgBox, 4096, Tip
      , Please select the core color again !, 2
    return
  }
  color:="#" . j . "@" . r2
}
GuiControlGet, Comment
Gui, Hide
cors.Event:=A_ThisLabel
if A_ThisLabel=SplitAdd
{
  SetFormat, IntegerFast, d
  bg:=StrLen(StrReplace(wz,"_"))
    > StrLen(StrReplace(wz,"0")) ? "0":"_"
  s:="", k:=nW*nH+1+left, i:=0, w:=nW-left-right
  Loop, % w {
    i++
    GuiControlGet, j,, % C_[k++]
    if (j=0 and A_Index<w)
      Continue
    v:=RegExReplace(wz,"m`n)^(.{" i "}).*","$1")
    wz:=RegExReplace(wz,"m`n)^.{" i "}"), i:=0
    While InStr(v,bg) {
      if (v~="^" bg "+\n")
        v:=RegExReplace(v,"^" bg "+\n")
      else if !(v~="m`n)[^\n" bg "]$")
        v:=RegExReplace(v,"m`n)" bg "$")
      else if (v~="\n" bg "+\n$")
        v:=RegExReplace(v,"\n\K" bg "+\n$")
      else if !(v~="m`n)^[^\n" bg "]")
        v:=RegExReplace(v,"m`n)^" bg)
      else Break
    }
    if v!=
      s.=towz(color,v,SubStr(Comment,1,1))
    Comment:=SubStr(Comment,2)
  }
  cors.Result:=s
  return
}
s:=towz(color,wz,Comment)
if A_ThisLabel=AllAdd
{
  cors.Result:=s
  return
}
px1:=px-ww+left+(nW-left-right)//2
py1:=py-hh+up+(nH-up-down)//2
s:=StrReplace(s, "Text.=", "Text:=")
s=
(

t1:=A_TickCount
%s%
if (ok:=FindText(%px1%-150000//2, %py1%-150000//2, 150000, 150000, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, `%X`%, `%Y`%
}

MsgBox, 4096,, `% "Time:``t" (A_TickCount-t1) " ms``n``n"
  . "Pos:``t" X ", " Y "``n``n"
  . "Result:``t" (ok ? "Success !":"Failed !"), 3
MouseMove, X, Y

)
cors.Result:=s
return

towz(color,wz,comment="")
{
  SetFormat, IntegerFast, d
  wz:=StrReplace(StrReplace(wz,"0","1"),"_","0")
  wz:=(InStr(wz,"`n")-1) "." bit2base64(wz)
  return, "`nText.=""|<" comment ">" color "$" wz """`n"
}

add(s, rn=1)
{
  global hscr
  if (rn=1)
    s:="`n" s "`n"
  s:=RegExReplace(s,"\R","`r`n")
  ControlGet, i, CurrentCol,,, ahk_id %hscr%
  if i>1
    ControlSend,, {Home}{Down}, ahk_id %hscr%
  Control, EditPaste, %s%,, ahk_id %hscr%
}

WM_MOUSEMOVE()
{
  ListLines, Off
  static CurrControl, PrevControl
  CurrControl := A_GuiControl
  if (CurrControl!=PrevControl)
  {
    PrevControl := CurrControl
    ToolTip
    if CurrControl !=
      SetTimer, DisplayToolTip, -1000
  }
  return

  DisplayToolTip:
  ListLines, Off
  k:="ToolTip_Text"
  TT_:=RegExMatch(%k%, "m`n)^" CurrControl "\K\s*=.*", r)
    ? Trim(r,"`t =") : ""
  MouseGetPos,,, k
  WinGetClass, k, ahk_id %k%
  if k = AutoHotkeyGUI
  {
    ToolTip, %TT_%
    SetTimer, RemoveToolTip, -5000
  }
  return

  RemoveToolTip:
  ToolTip
  return
}


;===== Copy The Following Functions To Your Own Code Just once =====


; FindText() used to find images restored by Base64 text on screen.
; X is upper left corner X coordinates
; Y is upper left corner Y coordinates
; W is the search scope's Width
; H is the search scope's Height.
; err1 is the character "0" fault-tolerant in percentage.
; err0 is the character "_" fault-tolerant in percentage.
; Text can be a lot of text to find, separated by "|".
; ruturn is a array, contains the [X,Y,W,H,Comment] results of Each Find.

FindText(x, y, w, h, err1, err0, text)
{
  xywh2xywh(x,y,w,h,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  sx:=0, sy:=0, sw:=w, sh:=h, arr:=[]
  Loop, 2 {
  Loop, Parse, text, |
  {
    v:=A_LoopField
    IfNotInString, v, $, Continue
    comment:="", e1:=err1, e0:=err0
    ; You Can Add Comment Text within The <>
    if RegExMatch(v,"<([^>]*)>",r)
      v:=StrReplace(v,r), comment:=Trim(r1)
    ; You can Add two fault-tolerant in the [], separated by commas
    if RegExMatch(v,"\[([^\]]*)]",r)
    {
      v:=StrReplace(v,r), r1.=","
      StringSplit, r, r1, `,
      e1:=r1, e0:=r2
    }
    StringSplit, r, v, $
    color:=r1, v:=r2
    StringSplit, r, v, .
    w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
    if (r0<2 or h1<1 or w1>sw or h1>sh or StrLen(v)!=w1*h1)
      Continue
    ;--------------------------------------
    mode:=InStr(color,"*") ? 2 : !InStr(color,"#") ? 1 : 0
    color:=RegExReplace(color,"[*#]") . "@"
    StringSplit, r, color, @
    color:=mode=0 ? ((r1-1)//w1)*Stride+Mod(r1-1,w1)*4 : r1
    n:=Round(r2,2)+(!r2), n:=Floor(255*3*(1-n))
    StrReplace(v,"1","",len1), len0:=StrLen(v)-len1
    e1:=Round(len1*e1), e0:=Round(len0*e0)
    VarSetCapacity(ss, sw*sh, 0), k:=StrLen(v)*4
    VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
    VarSetCapacity(allpos, 1024*4, 0)
    ;--------------------------------------
    if (ok:=PicFind(mode,color,n,Scan0,Stride
      ,sx,sy,sw,sh,ss,v,s1,s0,e1,e0,w1,h1,allpos))
    {
      Loop, % ok
        pos:=NumGet(allpos, 4*(A_Index-1), "uint")
        , rx:=(pos&0xFFFF)+x, ry:=(pos>>16)+y
        , arr.Push( [rx,ry,w1,h1,comment] )
    }
  }
  if (err1=0 and err0=0 and !arr.MaxIndex())
    err1:=err0:=0.1
  else Break
  }
  SetBatchLines, %bch%
  return, arr.MaxIndex() ? arr:0
}

PicFind(mode, color, n, Scan0, Stride, sx, sy, sw, sh
  , ByRef ss, ByRef text, ByRef s1, ByRef s0
  , err1, err0, w1, h1, ByRef allpos)
{
  static MyFunc, Ptr:=A_PtrSize ? "UPtr" : "UInt"
  if !MyFunc
  {
    x32:="5557565383EC3C8B9C24900000008B7C245085DB0F8E530"
    . "60000C744241400000000C74424100000000031F6C744240C0"
    . "0000000C744240800000000C7442418000000008B4C24108BA"
    . "C248C0000008B5C24188B54241401CD89C829CB8B8C248C000"
    . "000035C247885C97E67892C2489F989D5895C2404EB1F8DB42"
    . "6000000008B9C248000000083C50483C0018914B383C601390"
    . "424742E8B7C240485C989EA0F45D0803C073175D78B5C24088"
    . "B7C247C83C50483C00139042489149F8D7B01897C240875D28"
    . "B9C248C000000015C241889CF8344240C018B4C246C8B44240"
    . "C014C24108B4C2460014C2414398424900000000F854BFFFFF"
    . "F8B44240839F00F4CC68944241085FF0F85DE0100008B44246"
    . "C034424642B84248C000000894424288B442468034424702B8"
    . "4249000000039442468894424380F8F280500008B4424608B7"
    . "C24640FAF442468897424148B74245CC744242C000000008D0"
    . "4B803442454894424348B442428394424640F8F4D0100008B4"
    . "42468C1E010894424308B442434894424248B4424648944241"
    . "8908B4424248B6C24100FB65C060289C72B7C245485ED891C2"
    . "40FB65C06010FB60406895C24048944240C0F84120300008B8"
    . "4248800000031DB894424208B8424840000008944241CEB748"
    . "DB426000000003B5C24147D5A8B8424800000008B149801FA0"
    . "FB64C16020FB64416012B0C242B4424040FB614162B54240C8"
    . "9CDC1FD1F31E929E989C5C1FD1F31E829E889D5C1FD1F01C13"
    . "1EA29EA01CA395424587C10836C242001787289F68DBC27000"
    . "0000083C3013B5C24100F84840200003B5C24087D8D8B4C247"
    . "C8B049901F80FB64C06020FB65406012B0C242B5424040FB60"
    . "4062B44240C89CDC1FD1F31E929E989D5C1FD1F31EA29EA89C"
    . "5C1FD1F01D131E829E801C83B4424580F8E42FFFFFF836C241"
    . "C010F8937FFFFFF834424180183442424048B4424183944242"
    . "80F8DCFFEFFFF83442468018B7C24608B442438017C24343B4"
    . "424680F8D8AFEFFFF8B5C242C83C43C89D85B5E5F5DC248008"
    . "B4424608B5C24640FAF4424688B54246CF7DA83FF018D04988"
    . "B5C24608D1C93895C24140F84680200008B7C24548B4C2470C"
    . "7042400000000C7442404000000008D570169FAE803000089F"
    . "B8B7C246CC1E70285C9897C240C7E738974241889C589DF908"
    . "B54246C85D27E4D8B4C245C8B5C24048B74245C035C247401E"
    . "9036C240C01EE0FB651020FB6410169D22B01000069C04B020"
    . "00001C20FB6016BC07201D039C70F970383C10483C30139F17"
    . "5D38B74246C0174240483042401036C24148B0424394424707"
    . "59A8B7424188B44246C2B84248C0000008944240C8B4424702"
    . "B842490000000894424280F88880200008B4424088B7C247C8"
    . "B5C2410C744241400000000C744241C00000000C7442418000"
    . "000008D3C8789C5897C242489F78B44240C85C00F889400000"
    . "08B44241C03442468C7042400000000C1E01089442420908B0"
    . "4240344241485DB89C1894424100F84BB0000008B842488000"
    . "0008B942484000000034C2474894424088954240431C0EB243"
    . "9F87D198B9424800000008B348201CE803E007408836C24080"
    . "178219083C00139D8747939E87DD88B54247C8B348201CE803"
    . "E0174CA836C24040179C3830424018B04243944240C7D83834"
    . "4241C018B4C246C8B44241C014C2414394424280F8D45FFFFF"
    . "F8B5C2418E90FFEFFFF8B7C242C8B4424300B4424188B8C249"
    . "40000008D5F0181FBFF0300008904B90F8FEAFDFFFF895C242"
    . "CE9AAFDFFFF908B4C24188B0424034424648B9424940000000"
    . "B4424208D710181FEFF03000089048A7F4C85ED0F845801000"
    . "08B4C24748B542410897424048B44247C8B74242401D18B108"
    . "3C00401CA39F0C6020075F2830424018B7424048B042439442"
    . "40C897424180F8DC7FEFFFFE93FFFFFFF669083C43C89F389D"
    . "85B5E5F5DC248008B5424548B5C2454C744241800000000C74"
    . "4241C00000000C1EA100FB6FA893C240FB6FF897C24040FB67"
    . "C2454897C240C8B7C246CC1E702897C24248B7C247085FF0F8"
    . "EF4FDFFFF897424288DB6000000008B5C246C85DB7E6E8B5C2"
    . "45C8B74241C8B6C245C0374247401C3034424248944242001C"
    . "50FB64B020FB653012B0C242B5424040FB6032B44240C89CFC"
    . "1FF1F31F929F989D7C1FF1F31FA29FA89C7C1FF1F01D131F82"
    . "9F801C8394424580F9D0683C30483C60139DD75BA8B74246C0"
    . "174241C8B4424208344241801034424148B7C2418397C24700"
    . "F8573FFFFFF8B742428E954FDFFFF31DBE973FCFFFFC744241"
    . "00000000031F6C744240800000000E982FAFFFF89742418E91"
    . "3FEFFFF909090"
    x64:="4157415641554154555756534883EC38448B94240001000"
    . "031F64531ED4531F68954241444898424900000004C89CD448"
    . "BA424B8000000488BBC24C80000004585D24C8BBC24E000000"
    . "00F8EEA0000004489A424B8000000488BAC24D800000031C04"
    . "48BA424F80000004889BC24C800000031DB31F64531ED4531F"
    . "6C7442404000000004C898C249800000089C766904585E47E6"
    . "54863542404458D1C1C89D848039424D00000004189F8EB1E0"
    . "F1F0083C0014D63D54183C0044183C5014883C2014139C3478"
    . "90C97742A85C94589C1440F45C8803A3175D783C0014D63D64"
    . "183C0044183C6014883C2014139C346894C950075D64401642"
    . "40483C601039C24B800000003BC24A000000039B4240001000"
    . "00F8578FFFFFF488BAC2498000000448BA424B80000004539E"
    . "E488BBC24C80000004489EE410F4DF685C90F854D0200008B8"
    . "424B0000000038424C00000004403A424A80000002B8424000"
    . "10000442BA424F8000000398424B00000008944242C4489642"
    . "4180F8F070600008B8424A00000008BBC24A80000000FAF842"
    . "4B00000004C89BC24E0000000C7442420000000004589F7458"
    . "9EE8D04B803442414894424288B442418398424A80000000F8"
    . "F910100008B8424B0000000448B6C2428C1E010894424248B8"
    . "424A800000089442404418D45024589EC442B64241485F6489"
    . "8440FB65C0500418D450148980FB65C05004963C50FB67C050"
    . "00F84A20300008B8424F00000004531C0894424088B8424E80"
    . "0000089442410E98D0000004539CE7E7B488B8C24E00000004"
    . "28B04814401E08D50024863D20FB64C15008D500148980FB64"
    . "405004863D20FB65415004429D94189C929F841C1F91F29DA4"
    . "431C94429C94189D141C1F91F4431CA4429CA4189C141C1F91"
    . "F01D14431C84429C801C8398424900000007C14836C2408010"
    . "F8898000000660F1F8400000000004983C0014439C60F8EF70"
    . "200004539C74589C10F8E67FFFFFF488B8C24D8000000428B0"
    . "4814401E08D50024863D20FB64C15008D500148980FB644050"
    . "04863D20FB65415004429D94189CA29F841C1FA1F29DA4431D"
    . "14429D14189D241C1FA1F4431D24429D24189C241C1FA1F01D"
    . "14431D04429D001C83B8424900000000F8EFCFEFFFF836C241"
    . "0010F89F1FEFFFF83442404014183C5048B442404394424180"
    . "F8D8DFEFFFF838424B0000000018BBC24A00000008B44242C0"
    . "17C24283B8424B00000000F8D3AFEFFFF8B4424204883C4385"
    . "B5E5F5D415C415D415E415FC38B8424A00000008B9C24A8000"
    . "0000FAF8424B00000008D04988B9C24A000000089C24489E0F"
    . "7D883F9018D0483894424040F8487020000448B542414448B8"
    . "424C000000031DB31C9428D04A5000000004183C2014569D2E"
    . "80300004585C00F8EA70000004489742410448BB424C000000"
    . "04189D344896C24084C89BC24E00000004189C7448B6C24048"
    . "974240489CE66904585E47E544963C34C63CE4531C0488D4C0"
    . "5024901F9662E0F1F8400000000000FB6110FB641FF69D22B0"
    . "1000069C04B02000001C20FB641FE6BC07201D04139C2430F9"
    . "704014983C0014883C1044539C47FCD4501FB4401E683C3014"
    . "501EB4139DE759C448B742410448B6C24088B7424044C8BBC2"
    . "4E00000004489E58B8424C00000002BAC24F80000002B84240"
    . "0010000894424200F88C8020000488B9C24D8000000418D46F"
    . "F896C240448C744240800000000C7442410000000004489A42"
    . "4B8000000488D448304488944241831C089C58B44240485C00"
    . "F88900000008B442408038424B00000004531D2C1E01089442"
    . "41485F64589D4468D4415000F84C1000000448B9C24F000000"
    . "0448B8C24E800000031C0EB300F1F8400000000004439E97D1"
    . "B418B14874401C24863D2803C1700740B4183EB0178290F1F4"
    . "400004883C00139C67E7D4439F089C17DD18B14834401C2486"
    . "3D2803C170174C24183E90179BC4983C20144395424047D854"
    . "8834424080103AC24B8000000488B442408394424200F8D48F"
    . "FFFFF8B442410E9CAFDFFFF48635424208B4C24240B4C24044"
    . "88BBC240801000089D083C001890C973DFF0300000F8FA2FDF"
    . "FFF89442420E95AFDFFFF48635424104403A424A8000000440"
    . "B642414488B8C240801000089D083C001448924913DFF03000"
    . "00F8F6BFDFFFF4585F6741D4C8B4C24184889DA8B0A4883C20"
    . "44401C14C39CA4863C9C6040F0075EB4983C20144395424048"
    . "94424100F8DCCFEFFFFE942FFFFFF8B4C2414448B8C24C0000"
    . "0004531D24889C889CB0FB6C4C1EB104189C30FB6C131C9418"
    . "9C0428D04A5000000004585C90FB6DB894424100F8E10FEFFF"
    . "F4889AC24980000008BAC2490000000448974240844896C241"
    . "44589D6897424184889BC24C80000004189D54C89BC24E0000"
    . "0004489DE4489C74189CF662E0F1F8400000000004585E47E7"
    . "5488B8C24980000004D63DF4C039C24C80000004963C54531C"
    . "94C8D440102410FB608410FB650FF410FB640FE29D929F2418"
    . "9CA29F841C1FA1F4431D14429D14189D241C1FA1F4431D2442"
    . "9D201CA89C1C1F91F31C829C801D039C5430F9D040B4983C10"
    . "14983C0044539CC7FB144036C24104501E74183C60144036C2"
    . "4044439B424C00000000F856FFFFFFF448B742408448B6C241"
    . "48B742418488BBC24C80000004C8BBC24E0000000E916FDFFF"
    . "F31C0E9F3FBFFFF9090909090"
    MCode(MyFunc, A_PtrSize=8 ? x64:x32)
  }
  return, DllCall(&MyFunc
    , "int",mode, "uint",color, "int",n, Ptr,Scan0
    , "int",Stride, "int",sx, "int",sy, "int",sw, "int",sh
    , Ptr,&ss, "AStr",text, Ptr,&s1, Ptr,&s0
    , "int",err1, "int",err0, "int",w1, "int",h1, Ptr,&allpos)
}

xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h)
{
  SysGet, zx, 76
  SysGet, zy, 77
  SysGet, zw, 78
  SysGet, zh, 79
  left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
  left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
  up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
  x:=left, y:=up, w:=right-left+1, h:=down-up+1
}

GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits)
{
  VarSetCapacity(bits,w*h*4,0), bpp:=32
  Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
  Ptr:=A_PtrSize ? "UPtr" : "UInt", PtrP:=Ptr . "*"
  win:=DllCall("GetDesktopWindow", Ptr)
  hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
  mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
  ;-------------------------
  VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
  NumPut(w, bi, 4, "int"), NumPut(-h, bi, 8, "int")
  NumPut(1, bi, 12, "short"), NumPut(bpp, bi, 14, "short")
  ;-------------------------
  if hBM:=DllCall("CreateDIBSection", Ptr,mDC, Ptr,&bi
    , "int",0, PtrP,ppvBits, Ptr,0, "int",0, Ptr)
  {
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr)
    DllCall("BitBlt", Ptr,mDC, "int",0, "int",0, "int",w, "int",h
      , Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000)
    DllCall("RtlMoveMemory", Ptr,Scan0, Ptr,ppvBits, Ptr,Stride*h)
    DllCall("SelectObject", Ptr,mDC, Ptr,oBM)
    DllCall("DeleteObject", Ptr,hBM)
  }
  DllCall("DeleteDC", Ptr,mDC)
  DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
}

MCode(ByRef code, hex)
{
  bch:=A_BatchLines
  SetBatchLines, -1
  VarSetCapacity(code, len:=StrLen(hex)//2)
  Loop, % len
    NumPut("0x" SubStr(hex,2*A_Index-1,2),code,A_Index-1,"uchar")
  Ptr:=A_PtrSize ? "UPtr" : "UInt", PtrP:=Ptr . "*"
  DllCall("VirtualProtect",Ptr,&code, Ptr,len,"uint",0x40,PtrP,0)
  SetBatchLines, %bch%
}

base64tobit(s)
{
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  StringCaseSense, On
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:=(i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,A_LoopField,v)
  }
  StringCaseSense, Off
  s:=SubStr(s,1,InStr(s,"1",0,0)-1)
  s:=RegExReplace(s,"[^01]+")
  return, s
}

bit2base64(s)
{
  s:=RegExReplace(s,"[^01]+")
  s.=SubStr("100000",1,6-Mod(StrLen(s),6))
  s:=RegExReplace(s,".{6}","|$0")
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:="|" . (i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,v,A_LoopField)
  }
  return, s
}

ASCII(s)
{
  if RegExMatch(s,"\$(\d+)\.([\w+/]+)",r)
  {
    s:=RegExReplace(base64tobit(r2),".{" r1 "}","$0`n")
    s:=StrReplace(StrReplace(s,"0","_"),"1","0")
  }
  else s=
  return, s
}

; You can put the text library at the beginning of the script,
; and Use Pic(Text,1) to add the text library to Pic()'s Lib,
; Use Pic("comment1|comment2|...") to get text images from Lib

Pic(comments, add_to_Lib=0)
{
  static Lib:=[]
  if (add_to_Lib)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]+"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
    Lib[""]:=""
  }
  else
  {
    Text:=""
    Loop, Parse, comments, |
      Text.="|" . Lib[Trim(A_LoopField)]
    return, Text
  }
}

PicN(number)
{
  return, Pic(Trim(RegExReplace(number,".","$0|"),"|"))
}

; Use PicX(Text) to automatically cut into multiple characters

PicX(Text)
{
  if !RegExMatch(Text,"\|([^$]+)\$(\d+)\.([\w+/]+)",r)
    return, Text
  w:=r2, v:=base64tobit(r3), Text:=""
  c:=StrLen(StrReplace(v,"0"))<=StrLen(v)//2 ? "1":"0"
  wz:=RegExReplace(v,".{" w "}","$0`n")
  SetFormat, IntegerFast, d
  While InStr(wz,c) {
    While !(wz~="m`n)^" c)
      wz:=RegExReplace(wz,"m`n)^.")
    i:=0
    While (wz~="m`n)^.{" i "}" c)
      i++
    v:=RegExReplace(wz,"m`n)^(.{" i "}).*","$1")
    wz:=RegExReplace(wz,"m`n)^.{" i "}")
    if v!=
      Text.="|" r1 "$" i "." bit2base64(v)
  }
  return, Text
}

FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=20)
{
  OCR:="", RightX:=nX+nW-1
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    For k,v in ok
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      x:=v.1, y:=v.2, w:=v.3, h:=v.4, comment:=v.5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<LeftX)
        LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
      else if (x=LeftX)
      {
        Loop, 100
        {
          err:=A_Index/100
          if FindText(x, y, w, h, err, err, Text)
          {
            LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
            Break
          }
          if FindText(LeftX, LeftY, LeftW, LeftH, err, err, Text)
            Break
        }
      }
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and LeftX-nX-1>Interval ? "*":"") . LeftOCR
    ; Update nX and nW for next search
    nX:=LeftX+LeftW-1, nW:=RightX-nX+1
  }
  return, OCR
}

; Reordering the objects returned from left to right,
; from top to bottom, ignore slight height difference

SortOK(ok, dy=10) {
  if !IsObject(ok)
    return, ok
  SetFormat, IntegerFast, d
  For k,v in ok
  {
    x:=v.1+v.3//2, y:=v.2+v.4//2
    y:=A_Index>1 and Abs(y-lasty)<dy ? lasty : y, lasty:=y
    n:=(y*150000+x) "." k, s:=A_Index=1 ? n : s "-" n
  }
  Sort, s, N D-
  ok2:=[]
  Loop, Parse, s, -
    ok2.Push( ok[(StrSplit(A_LoopField,".")[2])] )
  return, ok2
}

NearestOK(ok, px, py) {
  if !IsObject(ok)
    return, ok
  SetFormat, IntegerFast, d
  For k,v in ok
  {
    x:=v.1+v.3//2, y:=v.2+v.4//2
    n:=((x-px)**2+(y-py)**2) "." k
    s:=A_Index=1 ? n : s "-" n
  }
  Sort, s, N D-
  ok2:=[]
  Loop, Parse, s, -
    ok2.Push( ok[(StrSplit(A_LoopField,".")[2])] )
  return, ok2
}

; Note: This function is used for combination lookup,
; for example, a 0-9 text library has been set up,
; then any ID number can be found.
; Use Pic(Text,1) and PicN(number) when using.
; Use PicX(Text) to automatically cut into multiple characters.
; Color position mode is not supported.

FindText2(x, y, w, h, err1, err0, text, Interval=20)
{
  xywh2xywh(x,y,w,h,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  sx:=0, sy:=0, sw:=w, sh:=h, arr:=[]
  info:=[], allw:=-1, allv:=allcolor:=allcomment:=""
  if (err1=0 and err0=0)
    err1:=err0:=0.1
  Loop, Parse, text, |
  {
    v:=A_LoopField
    IfNotInString, v, $, Continue
    comment:="", e1:=err1, e0:=err0
    ; You Can Add Comment Text within The <>
    if RegExMatch(v,"<([^>]*)>",r)
      v:=StrReplace(v,r), comment:=Trim(r1)
    ; You can Add two fault-tolerant in the [], separated by commas
    if RegExMatch(v,"\[([^\]]*)]",r)
    {
      v:=StrReplace(v,r), r1.=","
      StringSplit, r, r1, `,
      e1:=r1, e0:=r2
    }
    StringSplit, r, v, $
    color:=r1, v:=r2
    IfInString, color, #, Continue
    StringSplit, r, v, .
    w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
    if (r0<2 or h1<1 or w1>sw or h1>sh or StrLen(v)!=w1*h1)
      Continue
    if (allcolor="")
    {
      mode:=InStr(color,"*") ? 1 : 0
      color:=StrReplace(color,"*") . "@"
      StringSplit, r, color, @
      allcolor:=r1, n:=Round(r2,2)+(!r2), n:=Floor(255*3*(1-n))
    }
    StrReplace(v,"1","",len1), len0:=StrLen(v)-len1
    e1:=Round(len1*e1), e0:=Round(len0*e0)
    info.Push(StrLen(allv),w1,h1,len1,len0,e1,e0)
    allv.=v, allw+=w1+1, allcomment.=comment
  }
  if (allv="")
  {
    SetBatchLines, %bch%
    return, 0
  }
  num:=info.MaxIndex(), VarSetCapacity(in,num*4,0)
  Loop, % num
    NumPut(info[A_Index], in, 4*(A_Index-1), "int")
  VarSetCapacity(ss, sw*sh, 0), k:=StrLen(allv)*4
  VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
  VarSetCapacity(allpos, 1024*4, 0)
  offsetX:=Interval, offsetY:=5
  ;--------------------------------------
  if (ok:=PicFind2(mode,allcolor,n,Scan0,Stride,sx,sy,sw,sh
    ,ss,allv,s1,s0,in,num,offsetX,offsetY,allpos))
  {
    Loop, % ok
      pos:=NumGet(allpos, 4*(A_Index-1), "uint")
      , rx:=(pos&0xFFFF)+x, ry:=(pos>>16)+y
      , arr.Push( [rx,ry,allw,h1,allcomment] )
  }
  SetBatchLines, %bch%
  return, arr.MaxIndex() ? arr:0
}

PicFind2(mode, color, n, Scan0, Stride, sx, sy, sw, sh
  , ByRef ss, ByRef text, ByRef s1, ByRef s0
  , ByRef in, num, offsetX, offsetY, ByRef allpos)
{
  static MyFunc, Ptr:=A_PtrSize ? "UPtr" : "UInt"
  if !MyFunc
  {
    x32:="5557565383EC788B8424C40000008BAC24BC000000C7442"
    . "4140000000085C00F8EB90000008BB424C00000008B4424148"
    . "B1C86895C24088B5C86048B448608895C240485C0894424107"
    . "E7D8B74240831FFC704240000000089F28DB6000000008B442"
    . "40485C07E4C8B4C24088D1C38897C240C89F829F9038C24B40"
    . "00000EB0E8944950083C00183C20139C3741A803C013175EC8"
    . "BBC24B80000008904B783C00183C60139C375E68B5C2404015"
    . "C24088B7C240C8304240103BC24A80000008B0424394424107"
    . "59883442414078B442414398424C40000000F8F47FFFFFF8B8"
    . "424A40000008B9C24A00000000FAF84249C0000008D3C988B8"
    . "424A80000008B9C249C000000F7D88D0483894424108B84248"
    . "C00000085C00F85F30400008B842490000000C744240C00000"
    . "000C744241400000000C1E8100FB6C08904248B84249000000"
    . "00FB6C4894424040FB6842490000000894424088B8424A8000"
    . "000C1E0028944241C8B8424AC00000085C00F8EB100000089A"
    . "C24BC0000008BAC24A800000085ED7E7E8B9C24980000008B7"
    . "424148BAC249800000003B424B000000001FB037C241C897C2"
    . "41801FD908D7426000FB64B020FB653012B0C242B5424040FB"
    . "6032B44240889CFC1FF1F31F929F989D7C1FF1F31FA29FA01D"
    . "19931D029D001C83B8424940000000F9E0683C30483C60139D"
    . "D75BB8B9C24A8000000015C24148B7C24188344240C01037C2"
    . "4108B44240C398424AC0000000F855DFFFFFF8BAC24BC00000"
    . "08B9424C00000008B9C24C00000008B8424C00000008B52148"
    . "B730C8B40048B5B10895424288B9424C000000089742424891"
    . "C248B52188954242C8B9424A800000029C239DE0F4DDE8BB42"
    . "4C00000008954241889DA8B9C24AC0000002B5E08895C24680"
    . "F88460400008B9C24B80000008B74242483E80189AC24BC000"
    . "000C744241C0000000089D5C744245C00000000C7442470000"
    . "000008D1CB389442460895C24748B44241885C00F88CC00000"
    . "08B74245C8B8424A4000000C74424080000000001F0C1E0108"
    . "944246C89F02BB424CC00000089F3BE000000000F49F389742"
    . "4540FAFB424A8000000897424588BB424CC00000001C689742"
    . "4648B4424080344241C85ED89C1894424200F8E9A0000008B7"
    . "4242C8B54242831C0038C24B00000008B5C242489742404EB2"
    . "63904247E1A8BB424BC0000008B3C8601CF803F007409836C2"
    . "404017823669083C00139C5745939C37ED68BB424B80000008"
    . "B3C8601CF803F0174C583EA0179C083442408018B442408394"
    . "424187D808344245C018B9C24A80000008B44245C015C241C3"
    . "94424680F8D0AFFFFFF8B5C247083C47889D85B5E5F5DC2480"
    . "066908B4424600344240883BC24C400000007894424300F8EA"
    . "50100008B8424C0000000C744244407000000896C244C83C02"
    . "0894424348B4424348B9424A80000008B7424308B0029C2894"
    . "424488B8424C800000001F039C20F4EC289C38944245039F30"
    . "F8C060100008B4424348B5C24648B700C8B68088974240C8B7"
    . "0108974241489C68B40148944243C8B8424AC0000002B46043"
    . "9C30F4EC3894424108B46FC8BB424B800000089442404C1E00"
    . "201C6038424BC000000894424408B4424548B7C2430037C245"
    . "83B442410894424040F8F8600000085ED7E258B9C24B000000"
    . "08B54241431C001FB8B0C8601D9803901740583EA01784A83C"
    . "00139C575EA8B4C240C85C90F8E870000008B9C24B00000008"
    . "96C243831C08B4C243C8B6C244001FBEB0983C0013944240C7"
    . "4658B54850001DA803A0074EC83E90179E78B6C24389083442"
    . "4040103BC24A80000008B442404394424100F8D7AFFFFFF834"
    . "42430018B442430394424500F8D4DFFFFFF83442408018B6C2"
    . "44C8B442408394424180F8DCCFDFFFFE947FEFFFF8DB426000"
    . "000008B4424308B5C24488344244407834424341C8D4418FF8"
    . "94424308B442444398424C40000000F8F79FEFFFF8B6C244C8"
    . "B7424708B442408038424A00000008B9424D00000000B44246"
    . "C8D5E0181FBFF0300008904B20F8F07FEFFFF8B54242485D27"
    . "E278B7424208B8424B00000008B9424B80000008D0C308B742"
    . "4748B0283C20401C839F2C6000075F28344240801895C24708"
    . "B442408394424180F8D1FFDFFFFE99AFDFFFF8D76008DBC270"
    . "00000008B8424900000008BB424AC000000C7042400000000C"
    . "74424040000000083C00169C0E803000089C38B8424A800000"
    . "0C1E00285F6894424080F8ED7FBFFFF89AC24BC00000089DD8"
    . "B9C24A800000085DB7E598B8C24980000008B5C24048BB4249"
    . "8000000039C24B000000001F9037C240801FE0FB651020FB64"
    . "10169D22B01000069C04B02000001C20FB6016BC07201D039C"
    . "50F970383C10483C30139F175D38B9C24A8000000015C24048"
    . "3042401037C24108B0424398424AC0000007588E94AFBFFFF8"
    . "3C47831DB89D85B5E5F5DC248009090"
    x64:="4157415641554154555756534881EC88000000448B94244"
    . "0010000488B842438010000898C24D00000008954241044898"
    . "424E00000004C898C24E80000004585D24C8BA424180100004"
    . "C8BB424200100004C8BBC2428010000488BBC2430010000488"
    . "9442408C7442404000000000F8EA4000000448BAC240801000"
    . "04C89A42418010000488B442408448B60088B308B68044585E"
    . "47E5D89F14189F24531DB31DB85ED7E444863D6468D4C1D004"
    . "489D84C01F2EB164C63C14883C20183C1014289048783C0014"
    . "139C1741C803A3175E54D63C24883C2014183C201438904878"
    . "3C0014139C175E401EE83C3014501EB4139DC75AD834424040"
    . "748834424081C8B442404398424400100000F8F74FFFFFF4C8"
    . "BA424180100008B8424000100008BB424F80000000FAF8424F"
    . "0000000448B8C24D00000008D2CB08B8424080100008BB424F"
    . "0000000F7D84585C9448D34860F85FA0400008B742410448B8"
    . "4241001000031C931D24889F089F3400FB6F60FB6C4C1EB104"
    . "189C58B8424080100000FB6DBC1E0024585C0894424040F8EE"
    . "40000004C89BC24280100004889BC2430010000448BBC24E00"
    . "000008BBC240801000044897424084C89A424180100004189D"
    . "64189CC85FF0F8E7C000000488B8C24E80000004D63DE4C039"
    . "C24180100004863C54531C94C8D440102660F1F440000410FB"
    . "608410FB650FF410FB640FE29D94429EA4189CA29F041C1FA1"
    . "F4431D14429D14189D241C1FA1F4431D24429D201CA89C1C1F"
    . "91F31C829C801D04439F8430F9E040B4983C1014983C004443"
    . "9CF7FAF036C24044101FE4183C401036C24084439A42410010"
    . "0000F8566FFFFFF4C8BA424180100004C8BBC2428010000488"
    . "BBC2430010000488B842438010000488BB42438010000448B7"
    . "00C448B68108B50048B40144489ED89442414488B842438010"
    . "0008B4018894424288B84240801000029D04539EE894424084"
    . "10F4DEE8B8424100100002B46088944246C0F882B040000418"
    . "D46FF4C89BC2428010000C74424100000000048C7442458000"
    . "00000C744247400000000498D4487044589F74589EE4189ED4"
    . "88BAC242801000048894424788D42FF894424608B44240885C"
    . "00F88D1000000488B5C24588B8424000100004889BC2430010"
    . "0004489EF01D8C1E0108944247089D82B84245001000089C6B"
    . "8000000000F49C631F6894424504989F5488BB424300100000"
    . "FAF842408010000894424548B84245001000001D8894424648"
    . "B44241085FF44896C2404428D1C280F8E99000000448B4C242"
    . "8448B44241431C0EB234139CE7E168B148601DA4863D241803"
    . "C140074074183E9017826904883C00139C77E684139C789C17"
    . "ED68B54850001DA4863D241803C140174C64183E80179C0498"
    . "3C50144396C24087D924189FD4889F74883442458018B9C240"
    . "8010000488B442458015C24103944246C0F8D03FFFFFF8B4C2"
    . "47489C84881C4880000005B5E5F5D415C415D415E415FC38B4"
    . "424600344240483BC2440010000070F8E9D010000488B8C243"
    . "801000048896C24188BAC240801000044897C2438488974242"
    . "0C74424300700000089C64883C120448974243C897C24484C8"
    . "96C2440895C24684989CF418B0789EA29C2894424348B84244"
    . "801000001F039C20F4EC239F08944244C0F8CBB000000418B4"
    . "7148BBC2410010000412B7F0449635FFC458B4F08458B6F0C8"
    . "944242C8B442464458B771039F80F4EF8488B44241848C1E30"
    . "24C8D141848035C24208B442454448D04068B44245039F8418"
    . "9C37F610F1F004585C97E234489F131D2418B04924401C0489"
    . "841803C0401740583E90178334883C2014139D17FE24585ED7"
    . "E738B4C242C31D2EB094883C2014139D57E628B04934401C04"
    . "89841803C040074E883E90179E34183C3014101E84439DF7DA"
    . "283C6013974244C7D834C8B6C2440448B7C2438448B74243C8"
    . "B7C2448488B6C2418488B7424204983C50144396C24080F8DE"
    . "FFDFFFFE958FEFFFF0F1F8400000000008B442434834424300"
    . "74983C71C8D7406FF8B442430398424400100000F8FC5FEFFF"
    . "F448B7C2438448B74243C8B7C24484C8B6C24408B5C2468488"
    . "B6C2418488B74242048634424748B542404039424F80000004"
    . "C8B9C24580100000B5424708D480181F9FF030000418914830"
    . "F8F08FEFFFF4585FF7E1D4C8B4424784889E88B104883C0040"
    . "1DA4C39C04863D241C604140075EB4983C50144396C2408894"
    . "C24740F8D3BFDFFFFE9A4FDFFFF0F1F4000448B5424108B942"
    . "41001000031DB8B84240801000031F6448B9C2408010000418"
    . "3C2014569D2E803000085D2448D2C85000000000F8EECFBFFF"
    . "F4585DB7E5A488B8C24E80000004863C54C63CE4531C04D01E"
    . "1488D4C01020F1F8400000000000FB6110FB641FF69D22B010"
    . "00069C04B02000001C20FB641FE6BC07201D04139C2430F970"
    . "4014983C0014883C1044539C37FCD4401ED4401DE83C301440"
    . "1F5399C24100100007592E979FBFFFF31C9E916FDFFFF9090"
    MCode(MyFunc, A_PtrSize=8 ? x64:x32)
  }
  return, DllCall(&MyFunc
    , "int",mode, "uint",color, "int",n, Ptr,Scan0
    , "int",Stride, "int",sx, "int",sy, "int",sw, "int",sh
    , Ptr,&ss, "AStr",text, Ptr,&s1, Ptr,&s0
    , Ptr,&in, "int",num, "int",offsetX, "int",offsetY, Ptr,&allpos)
}


/***** C source code of machine code *****

int __attribute__((__stdcall__)) PicFind(
  int mode, unsigned int c, int n, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , char * ss, char * text, int * s1, int * s0
  , int err1, int err0, int w1, int h1, int * allpos)
{
  int o, i, j, x, y, sx1, sy1, ok=0;
  int r, g, b, rr, gg, bb, e1, e0, len1, len0, max;
  // Generate Lookup Table
  o=len1=len0=0;
  for (y=0; y<h1; y++)
  {
    for (x=0; x<w1; x++)
    {
      i=mode==0 ? y*Stride+x*4 : y*sw+x;
      if (text[o++]=='1')
        s1[len1++]=i;
      else
        s0[len0++]=i;
    }
  }
  // Color Position Mode
  // This mode will not clear the image that has been found
  if (mode==0)
  {
    sx1=sx+sw-w1; sy1=sy+sh-h1; max=len1>len0 ? len1 : len0;
    for (y=sy; y<=sy1; y++)
    {
      for (x=sx; x<=sx1; x++)
      {
        o=y*Stride+x*4; e1=err1; e0=err0;
        j=o+c; rr=Bmp[2+j]; gg=Bmp[1+j]; bb=Bmp[j];
        for (i=0; i<max; i++)
        {
          if (i<len1)
          {
            j=o+s1[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b>n && (--e1)<0) goto NoMatch2;
          }
          if (i<len0)
          {
            j=o+s0[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b<=n && (--e0)<0) goto NoMatch2;
          }
        }
        allpos[ok++]=y<<16|x;
        if (ok>=1024) goto Return1;
        NoMatch2:
        continue;
      }
    }
    goto Return1;
  }
  // Generate Two Value Image
  o=sy*Stride+sx*4; j=Stride-4*sw; i=0;
  if (mode==1)  // Color Mode
  {
    rr=(c>>16)&0xFF; gg=(c>>8)&0xFF; bb=c&0xFF;
    for (y=0; y<sh; y++, o+=j)
      for (x=0; x<sw; x++, o+=4, i++)
      {
        r=Bmp[2+o]-rr; g=Bmp[1+o]-gg; b=Bmp[o]-bb;
        if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
        ss[i]=r+g+b<=n ? 1:0;
      }
  }
  else  // Gray Threshold Mode
  {
    c=(c+1)*1000;
    for (y=0; y<sh; y++, o+=j)
      for (x=0; x<sw; x++, o+=4, i++)
        ss[i]=Bmp[2+o]*299+Bmp[1+o]*587+Bmp[o]*114<c ? 1:0;
  }
  // Start Lookup
  sx1=sw-w1; sy1=sh-h1; max=len1>len0 ? len1 : len0;
  for (y=0; y<=sy1; y++)
  {
    for (x=0; x<=sx1; x++)
    {
      o=y*sw+x; e1=err1; e0=err0;
      for (i=0; i<max; i++)
      {
        if (i<len1 && ss[o+s1[i]]!=1 && (--e1)<0) goto NoMatch1;
        if (i<len0 && ss[o+s0[i]]!=0 && (--e0)<0) goto NoMatch1;
      }
      allpos[ok++]=(sy+y)<<16|(sx+x);
      if (ok>=1024) goto Return1;
      //Clear the image that has been found
      for (i=0; i<len1; i++) ss[o+s1[i]]=0;
      NoMatch1:
      continue;
    }
  }
  Return1:
  return ok;
}


int __attribute__((__stdcall__)) PicFind2(
  int mode, unsigned int c, int n, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , char * ss, char * text, int * s1, int * s0
  , int * in, int num, int offsetX, int offsetY, int * allpos )
{
  int o, i, j, x, y, r, g, b, rr, gg, bb, max, e1, e0, ok=0;
  int o1, x1, y1, w1, h1, sx1, sy1, len1, len0, err1, err0;
  int o2, x2, y2, w2, h2, sx2, sy2, len21, len20, err21, err20;
  // Generate Lookup Table
  for (j=0; j<num; j+=7)
  {
    o=o1=o2=in[j]; w1=in[j+1]; h1=in[j+2];
    for (y=0; y<h1; y++)
    {
      for (x=0; x<w1; x++)
      {
        i=y*sw+x;
        if (text[o++]=='1')
          s1[o1++]=i;
        else
          s0[o2++]=i;
      }
    }
  }
  // Generate Two Value Image
  o=sy*Stride+sx*4; j=Stride-4*sw; i=0;
  if (mode==0)  // Color Mode
  {
    rr=(c>>16)&0xFF; gg=(c>>8)&0xFF; bb=c&0xFF;
    for (y=0; y<sh; y++, o+=j)
      for (x=0; x<sw; x++, o+=4, i++)
      {
        r=Bmp[2+o]-rr; g=Bmp[1+o]-gg; b=Bmp[o]-bb;
        if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
        ss[i]=r+g+b<=n ? 1:0;
      }
  }
  else  // Gray Threshold Mode
  {
    c=(c+1)*1000;
    for (y=0; y<sh; y++, o+=j)
      for (x=0; x<sw; x++, o+=4, i++)
        ss[i]=Bmp[2+o]*299+Bmp[1+o]*587+Bmp[o]*114<c ? 1:0;
  }
  // Start Lookup
  w1=in[1]; h1=in[2]; len1=in[3]; len0=in[4]; err1=in[5]; err0=in[6];
  sx1=sw-w1; sy1=sh-h1; max=len1>len0 ? len1 : len0;
  for (y=0; y<=sy1; y++)
  {
    for (x=0; x<=sx1; x++)
    {
      o=y*sw+x; e1=err1; e0=err0;
      for (i=0; i<max; i++)
      {
        if (i<len1 && ss[o+s1[i]]!=1 && (--e1)<0) goto NoMatch1;
        if (i<len0 && ss[o+s0[i]]!=0 && (--e0)<0) goto NoMatch1;
      }
      x1=x+w1-1; y1=y-offsetY; if (y1<0) y1=0;
      for (j=7; j<num; j+=7)
      {
        o2=in[j]; w2=in[j+1]; h2=in[j+2];
        len21=in[j+3]; len20=in[j+4]; err21=in[j+5]; err20=in[j+6];
        sx2=sw-w2; i=x1+offsetX; if (i<sx2) sx2=i;
        sy2=sh-h2; i=y+offsetY; if (i<sy2) sy2=i;
        for (x2=x1; x2<=sx2; x2++)
        {
          for (y2=y1; y2<=sy2; y2++)
          {
            o1=y2*sw+x2; e1=err21; e0=err20;
            for (i=0; i<len21; i++)
              if (ss[o1+s1[o2+i]]!=1 && (--e1)<0) goto NoMatch2;
            for (i=0; i<len20; i++)
              if (ss[o1+s0[o2+i]]!=0 && (--e0)<0) goto NoMatch2;
            goto MatchOK;
            NoMatch2:
            continue;
          }
        }
        goto NoMatch1;
        MatchOK:
        x1=x2+w2-1;
      }
      allpos[ok++]=(sy+y)<<16|(sx+x);
      if (ok>=1024) goto Return1;
      //Clear the image that has been found
      for (i=0; i<len1; i++) ss[o+s1[i]]=0;
      NoMatch1:
      continue;
    }
  }
  Return1:
  return ok;
}

*/


;================= The End =================

;
Last edited by feiyue on 10 Oct 2018, 23:46, edited 174 times in total.
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

14 May 2016, 12:25

This function is similar to the AHK built-in command "ImageSearch",
but it can convert images into strings, written into the script without
saving as an external image, so it is easier to use than "ImageSearch". :beer:
Last edited by feiyue on 30 Jun 2017, 17:19, edited 37 times in total.
User avatar
kwstas13
Posts: 43
Joined: 19 Mar 2016, 06:26

Re: FindText - Catch screen image into text and then find it

14 May 2016, 12:28

wow nice this is cool script thank you!! :bravo: :D
Guest888

Re: FindText - Catch screen image into text and then find it

14 May 2016, 17:43

:thumbup: :thumbup: :thumbup:
This could be very useful for automatic operating. Thank you very much for sharing.
:bravo: :bravo: :bravo: :bravo:
guest3456
Posts: 2419
Joined: 09 Oct 2013, 10:31

Re: FindText - Catch screen image into text and then find it

14 May 2016, 19:16

very interesting image search / OCR example. thanks a lot. good job

User avatar
BGM
Posts: 419
Joined: 20 Nov 2013, 20:56
GitHub: bgmCoder
Contact:

Re: FindText - Catch screen image into text and then find it

14 May 2016, 21:36

__________________________________________00000_______________00_________
_________________________________________00___________________00_________
_________________________________________00___________________00_________
_________________________________________00___________________00_________
________00____00_____0000______00000____000000__00____00____00_________
________00____00____00___0____00___00____00_____00____00____00_________
________00____00____00________00___00____00_____00____00____00_________
________00____00____00_______00____000___00_____00____00____00_________
________00____00____0000_____00_____00___00_____00____00____00_________
________00____00_____0000____000000000___00_____00____00____00_________
________00____00_______000___00__________00_____00____00____00_________
________00____00________000__00__________00_____00____00____00_________
________00___000________000___00_________00_____00___000____00_________
________00__0000___00___00____000___00___00_____00__0000____00_________
_________0000_00____00000_______00000____00______0000_00____00_________
Guest

Re: FindText - Catch screen image into text and then find it

18 May 2016, 01:11

This is such an useful script. Your idea and approach are brilliant! I register to say thank you FeiYue.

For those who haven't tried: this is not for OCR. But image search, rather.
It stands out from the internal ImageSearch command in that no ImageFile is required, and runs much faster.
The FindText function takes less than half a second to seach a full 1680x1050 screen.
The fact that it can search for both text/character and graph/picture won my vote for killer script of the year!
I use this to automate all sort of web pages and traditional applications. Great work! :superhappy:
User avatar
jmeneses
Posts: 450
Joined: 28 Oct 2014, 11:09
Location: Catalan Republic

Re: FindText - Catch screen image into text and then find it

18 May 2016, 03:05

Hi feiyue, excellent job and thanks for sharing
:thumbup: :thumbup:
Donec Perficiam
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

19 May 2016, 09:43

In addition to searching for screen text and images,
This function can also be used for simple text recognition.
Here is an example of a simple identification Numbers.
FindTextOCR() function can also be used to identify simple verification code.

Code: Select all

F2::    ; To identify the number near the mouse

; For identification, we need to create a text library,
; Of course, the following text library is not strong enough,
; Perhaps in other computers, other screen resolutions,
; Other browser magnification, different fonts, need to regenerate.
; You can add the newly generated to the following existing text library,
; To enhance the generality of this text library.

Text:="|<0>*147$6.SnVVVVVnSU"
Text.="|<1>*147$5.9kV248Hw"
Text.="|<2>*148$6.yX11248EzU"
Text.="|<3>*149$6.yX13C11XyU"
Text.="|<4>*149$6.66+GmWz22U"
Text.="|<5>*149$6.TEES311XyU"
Text.="|<6>*148$6.CEUyXVVnSU"
Text.="|<7>*149$6.z1224A8MEU"
Text.="|<8>*148$6.SnVnSVVnSU"
Text.="|<9>*148$6.SnVVlT12QU"

CoordMode, Mouse
MouseGetPos, x, y
t1:=A_TickCount
;------------------------------
OCR:=FindTextOCR(x, y, 150, 20, 0.2, 0.1, Text)
;------------------------------
t1:=A_TickCount-t1
MsgBox, 4096, OCR, OCR Result: [%OCR%] in %t1% ms.
Return


FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=20)
{
  OCR:="", Right_X:=nX+nW-1
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    Loop, % ok.MaxIndex()
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      i:=A_Index, x:=ok[i].1, y:=ok[i].2
        , w:=ok[i].3, h:=ok[i].4, comment:=ok[i].5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<Left_X)
        Left_X:=x, Left_W:=w, Left_OCR:=comment
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and Left_X-nX-1>Interval ? "*":"") . Left_OCR
    ; Update nX and nW for next search
    nX:=Left_X+Left_W-1, nW:=Right_X-nX+1
  }
  Return, OCR
}


Last edited by feiyue on 22 Sep 2018, 13:39, edited 9 times in total.
ed1chandler
Posts: 15
Joined: 15 May 2015, 11:18

Re: FindText - Catch screen image into text and then find it

20 May 2016, 09:34

Want to make sure I'm understanding use case and usage:

1) Download script and run it. This pops up the "Catch Image To Text And Find Text Tool" box with "Catch", "Test", and "Copy" buttons.

2) Click the "Catch" button and the cursor is accompanied by a transparent red box.

3) Move the red box over the text you want to search for and click, then move the box more than 100 px in any direction to trigger capture.

4) This brings up the "Catch Image To Text" box with a magnified view of the captured text.

5) The lower left portion of that GUI has a series of "trimming" buttons, used to trim the edges of the captured region. I'm not sure what the abbreviations are supposed to stand for, but it appears that "LD" trims one pixel from the left edge, "LD3" trims three pixels, and the other buttons do the same for "up", "down", and "right." Use those buttons to trim the region down to what you really want to search for.

6) Click somewhere on the magnified capture to select a pixel with the color you want to use for matching. When you do, the text box next to the "Color2Two" button will populate with the hex value of the color corresponding with the pixel you chose.

It appears there are two modes: color and greyscale. If you want to use color, it appears to go something like this:
7a) After selecting a pixel (step 6), if you click the "Color2Two" button, it appears to convert the magnified image into a two-tone version where all pixels of the selected color are converted to black and all other pixels are converted to white.

And if you want to use greyscale, it goes something like this:
7b) After selecting a pixel (step 6), if you click the "Gray2Two" button, it appears to make a "best guess" at a threshold, then convert the magnified image into a two-tone black/white version where all pixels on one side of the threshold are black and all others are white. It also appears that you can change the threshold and click the button again if you want to adjust the program's guess.

8) Once you've got the two-tone image you want, click the "Ok" button. This returns a GUI with a "Text=..." block of text and a related function call. You can click the "Test" button to test that the search will actually find what you're looking for, then click the "Copy" button to copy the text block and the related function call to the clipboard. The "Exchange" button will negate the image and the "Load" button will reload the original capture.

9) Copy that block of text and the function call into the script from which the search will be run ... and be sure the script can "see" the FindText() function.

10) Modify and enjoy.

What I'm *not* quite following is how to modify the function parameters. The "w" and "h" parameters say that they're "offsets" but what does that *mean*? (i.e. What's the effect of changing them?) The default is 150, but the capture tool seeds these to 150000. What is this changing?
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

20 May 2016, 13:36

@ed1chandler, your description for use is very detailed, very good! :bravo:
Because my English is not very good, so the message is less in front,
thank you for your description. I add a few points:

1, LD meaning is the abbreviation of Left-Del, appear to be quite strange. :D

2, You can cut the edge first, then Make the image into black and white,
Can also be, first to make the image into black and white, and then cut the edge.
I recommend the way back, Because click "Auto" button can automatically cut edge.

(Slight correction)the Gray2Two button don't use the Selected color, will automatically
calculate a threshold, if you want to customize can manually enter the threshold.

3, the parameters of X, Y, W, H to determine a range:
the upper left corner (X-W, Y-H), the lower right corner of (X+W, Y+H),
So X, Y is the center point of the range, W, H is the distance between the center point
of the offset distance. The advantage is that the location of the search results is returned
Text / image center, and the parameters can be compared to know the relative text / image
as the origin of the offset.

4, In the default generated code, I use the offset 150000, So it's all around the screen,
it is full screen search. The actual use of the time required to select the appropriate scope,
such as use 150, in accordance with (3) to determine the scope of the search.

(Machine automatic translation)
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 14:01

I am not a programmer and I struggle to make these things work. Could you post an example of how you would run this code once it is generated?
I'm getting a "call to non-existing function" error
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

20 May 2016, 14:25

Useage:
1. Catch the image to text string.
2. Test find the text string on full Screen.
3. When test is successful, copy the code
and paste it into your own script.
4. Copy the "FindText()" function and the function
of the back, paste it into your own script.

You didn't do the fourth step, to copy the functions to your application.

Actual use of the time, you may also need to modify the scope of the search ( W, H offset),
the default (offset 150000) full screen to find a longer time and find the location may not be what you need.
Last edited by feiyue on 20 May 2016, 15:13, edited 1 time in total.
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 14:43

So I'm trying to write a standalone script that finds your name when I hit Win F3
How would I modify this?

#f3::

Text=
(
__000__________________________________
_00__________00________________________
_00____________________________________
0000___0000__00_00___00_00__00___0000__
_00___00__00_00__00__0__00__00__00__00_
_00___000000_00__00_00__00__00__000000_
_00___00_____00___0_00__00__00__00_____
_00___00___0_00___000___00__00__00___0_
_00____00000_00___000____00_00___00000_
__________________00___________________
__________________00___________________
)

if FindText(44,243,Text,"*177",150000,150000,X,Y)
{
CoordMode, Mouse
MouseMove, X, Y
}
return
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

20 May 2016, 15:04

Find this line at the end of my code,
Copy and paste the following into your script.

“ ;======== Copy The Next Functions To Your Code ======== ”
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 15:16

1-I clicked on catch and caught the text
2-I did the "test" and it worked fine
3=I selected "Copy" I did a ctrl-a to copy everything

I could not find
“ ;======== Copy The Next Functions To Your Code ======== ”
feiyue
Posts: 119
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

20 May 2016, 15:21

Find this line at the end of My source code,
Copy and paste the following into your script.

My source code on Post 1,that is:
Code: [Select all] [Expand] [Download]
------------------------------------------
……
……
;======== Copy The Next Functions To Your Code ========
------------------------------------------
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 15:59

I tried pasting this first at the beginning of the code then after the code. No luck
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 16:07

Here is my code:

; Note: parameters of the X,Y is the center of the coordinates,
; and the W,H is the offset distance to the center

FindText(x,y,Text,c,w=150,h=150,ByRef rx="",ByRef ry="")
{
xywh2xywh(x-w,y-h,2*w+1,2*h+1,x,y,w,h,c)
if (w<1 or h<1)
Return, 0
v:=Trim(RegExReplace(Text,"[^0_\n]+"),"`n") . "`n"
w2:=InStr(v,"`n")-1, h2:=StrLen(v)//(w2+1)
fmt:=A_FormatInteger
SetFormat, IntegerFast, d
re1:="[0_]{" (w2+0) "}\n", re2:=".{" (w-w2) "}"
SetFormat, IntegerFast, %fmt%
if (w2<1 or RegExReplace(v,re1)!="")
Return, 0
bch:=A_BatchLines
SetBatchLines, -1
if InStr(c,"-")
re:=StrReplace(v,"_","1"), c:=StrReplace(c,"-")
else
re:=StrReplace(StrReplace(v,"0","1"),"_","0")
; Starting from the 1 and 0 mutants can speed up the match
pos:=re~="10|01", re:=SubStr(re,pos+=!pos,-1)
GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits,c)
if !InStr(c,"*") ; Color Mode, it is very fast
ss:=StrGet(&bits,w*h,"utf-16")
else ; Gray Threshold Mode
{
; In order to speed up the speed,
; only to take the green as the gray value
ListLines, Off
Threshold:=StrReplace(c,"*"), i:=-3
VarSetCapacity(tmp,w*h+1,Asc("0")), k:=Asc("1")
Loop, % w*h
if NumGet(bits,i+=4,"uchar")<=Threshold
NumPut(k,tmp,A_Index,"char")
ListLines, On
ss:=StrGet(&tmp+1,w*h,0)
}
SetBatchLines, %bch%
if i:=(ss~=StrReplace(re,"`n",re2))
{
rx:=x+Mod(i-1,w)-Mod(pos-1,w2+1)+(w2-1)//2
ry:=y+(i-1)//w-(pos-1)//(w2+1)+(h2-1)//2
Return, 1
}
Return, 0
}

; Fixed the range to make it no more than screen borders

xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h,c="*")
{
; Use this rather than A_ScreenWidth etc can support multiple displays
SysGet, zx, 76
SysGet, zy, 77
SysGet, zw, 78
SysGet, zh, 79
left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
x:=left, y:=up, w:=right-left+1, h:=down-up+1
; Color Mode --> if Mod(w,2)!=0 --> (Stride:=((w*16+31)//32)*4)!=w*2
if !InStr(c,"*")
w:=(w//2)*2
}

GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits,c="*")
{
Ptr:=A_PtrSize ? "UPtr" : "UInt"
; Use this rather than GetDC(0) can support multiple displays
win:=DllCall("GetDesktopWindow", Ptr)
hDC:=DllCall("GetDC", Ptr,win, Ptr)
mDC1:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
hBM1:=DllCall("CreateCompatibleBitmap", Ptr,hDC, "int",w, "int",h, Ptr)
oBM1:=DllCall("SelectObject", Ptr,mDC1, Ptr,hBM1, Ptr)
DllCall("BitBlt", Ptr,mDC1, "int",0, "int",0, "int",w, "int",h
, Ptr,hDC, "int",x, "int",y, "uint",0xCC0020) ; SRCCOPY
if !InStr(c,"*") ; Color Mode
{
; Color c(RGB)-->COLORREF for WinApi=RGB(R,G,B)=BGR
BGR:=((c&0xFF)<<16)|(c&0xFF00)|((c&0xFF0000)>>16)
; All colors are converted to 1 or 0 utf-16 characters
c1:=16bppTo24bpp(Asc("1")), c0:=16bppTo24bpp(Asc("0"))
VarSetCapacity(Rect, 16)
NumPut(0, Rect, 0, "uint"), NumPut(0, Rect, 4, "uint")
NumPut(w, Rect, 8, "uint"), NumPut(h, Rect, 12, "uint")
mDC2:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
hBM2:=DllCall("CreateCompatibleBitmap", Ptr,hDC, "int",w, "int",h, Ptr)
oBM2:=DllCall("SelectObject", Ptr,mDC2, Ptr,hBM2, Ptr)
hBrush1:=DllCall("CreateSolidBrush", "uint",BGR^c1^c0, Ptr)
DllCall("FillRect", Ptr,mDC2, Ptr,&Rect, Ptr,hBrush1)
DllCall("DeleteObject", Ptr,hBrush1)
DllCall("msimg32.dll\TransparentBlt", Ptr,mDC2
, "int",0, "int",0, "int",w, "int",h, Ptr,mDC1
, "int",0, "int",0, "int",w, "int",h, "uint",BGR)
DllCall("BitBlt", Ptr,mDC2, "int",0, "int",0, "int",w, "int",h
, Ptr,mDC1, "int",0, "int",0, "uint",0x660046) ; SRCINVERT
; Color c-->BGR^(BGR^c1^c0)=c1^c0, the others-->X^X=0
hBrush2:=DllCall("CreateSolidBrush", "uint",c0, Ptr)
DllCall("FillRect", Ptr,mDC1, Ptr,&Rect, Ptr,hBrush2)
DllCall("DeleteObject", Ptr,hBrush2)
DllCall("BitBlt", Ptr,mDC1, "int",0, "int",0, "int",w, "int",h
, Ptr,mDC2, "int",0, "int",0, "uint",0x660046) ; SRCINVERT
; Color c-->(c1^c0)^c0=c1, the others-->0^c0=c0
DllCall("SelectObject", Ptr,mDC2, Ptr,oBM2)
DllCall("DeleteObject", Ptr,hBM2)
DllCall("DeleteDC", Ptr,mDC2)
}
DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
VarSetCapacity(bits, w*h*4, 0), bpp:=!InStr(c,"*") ? 16:32
VarSetCapacity(bi, 40, 0)
NumPut(40, bi, 0, "int"), NumPut(w, bi, 4, "int")
NumPut(-h, bi, 8, "int"), NumPut(1, bi, 12, "short")
NumPut(bpp, bi, 14, "short"), NumPut(0, bi, 16, "int")
DllCall("GetDIBits", Ptr,mDC1, Ptr,hBM1
, "int",0, "int",h, Ptr,&bits, Ptr,&bi, "int",0)
DllCall("SelectObject", Ptr,mDC1, Ptr,oBM1)
DllCall("DeleteObject", Ptr,hBM1)
DllCall("DeleteDC", Ptr,mDC1)
Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
}

16bppTo24bpp(c) {
B:=(c&0x1F)<<3, G:=((c>>5)&0x1F)<<3, R:=((c>>10)&0x1F)<<3
Return, (B<<16)|(G<<8)|R
}

;================= The End =================

Text=
(
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_00000__________0________0_____________________________________________
00___00__________________0_____________________________________________
0_____0__________________0_____________________________________________
0_____00_0___0__0___000__0__00____0000___000___000___000__000__000_____
0_____00_0___0__0__00____0_0_________0__00____00____0___0_0____0_______
0_____00_0___0__0__0_____000_______000__0_____0_____00000_00___00______
0_____0__0___0__0__0_____000______0__0__0_____0_____0______00___00_____
00___00__0___0__0__00____0_00_____0__0__00____00____00______00___00____
_00000____0000__0___000__0__00____0000___000___000___0000_000__000_____
______00_______________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
)

if FindText(81,134,Text,"*147",1500,1500,X,Y)
{
CoordMode, Mouse
MouseMove, X, Y
}
Mkonopko
Posts: 21
Joined: 25 Jan 2015, 07:30

Re: FindText - Catch screen image into text and then find it

20 May 2016, 19:37

Got it!!! This will be very useful to me. I'd like to recommend that you automatically post the additional code to the file.
Anyway many thanks for your help.

Marty Konopko

Return to “Scripts and Functions”

Who is online

Users browsing this forum: No registered users and 26 guests