ImageSearch Tips & Tricks

Helpful script writing tricks and HowTo's
Miguel7
Posts: 168
Joined: 08 Sep 2014, 07:06

ImageSearch Tips & Tricks

11 Sep 2014, 20:19

Overview

In all my work with AHK, one of the most mind-blowingly awesome, and at the same time intensely frustrating, functions I've ever used is ImageSearch. If you're trying to automate a program that doesn't use standard GUI controls, it can be a game-changer; most games fall into this category, as they usually draw the game onto a control (such C#'s System.Drawing, as Java's AWT, or HTML5's Canvas object) but it's not just limited to games. It's a big part of one of my projects at work, and at home I used it to set up hotkeys for Netflix.

But at times, it can be soooooo annoyingly hard to make it work! Don't get me wrong - AHK's developers are amazing just for creating it; I am not commenting on them in any way. They rock just for having developed AHK, my all-time favorite programming language. But ImageSearch does tend to be a colossal PITA (Pain In The And I'll let you fill in the blank) to work with. This is because in most cases, every little nano-detail has to be just so; otherwise it'll fail without much of a clue as to why.

But you can get ImageSearch working, and in this tutorial I hope to help you with some tips and tricks I've picked up from experience. And of course, if you have others you'd like to share, I'd be interested to hear them. So let's get started.

Tip #1: Play with your numbers

This one alone fixes the overwhelming majority of my ImageSearch problems. The documentation on ImageSearch lays it out like this:

Code: [Select all] [Download] (Script.ahk)GeSHi © Codebox Plus

ImageSearch, OutputVarX, OutputVarY, X1, Y1, X2, Y2, ImageFile

A few things to note here:
A: X2 and Y2 are essentially "absolute" coordinates; in other words, they're not width and height (unlike GUI functions). Sometimes I do dumb things like this:

Code: [Select all] [Download] (Script.ahk)GeSHi © Codebox Plus

ImageSearch, OutputVarX, OutputVarY, 400, 200, 50, 50, ImageFile.png

thinking like it's a GUI function and the other 2 coordinates are width and height. But it's actually a separate point, with another x and y, both of which should be greater than the first x and y. So to correct the example above:

Code: [Select all] [Download] (Script.ahk)GeSHi © Codebox Plus

ImageSearch, OutputVarX, OutputVarY, 400, 200, 450, 250, ImageFile.png

or if this helps, add x to width and y to height to get those coordinates.

B: Expanding the area being searched might change the results. In the example above, I made the mistake of making the search area the same size as the image I was searching for (50 by 50 pixels). This will mess it up almost every time. Instead, the search area should be bigger than the image.

Code: [Select all] [Download] (Script.ahk)GeSHi © Codebox Plus

ImageSearch, OutputVarX, OutputVarY, 300, 100, 600, 600, ImageFile.png

Now the first point (x1 and x2) has been moved up and to the left, while the second point (the bottom right corner of the area being searched, a.k.a. x2 and y2) has been moved down and to the right. So now the function is searching a bigger area, which may give it a better chance of the image being found. The more you mess with these numbers, the more you'll get an idea of where your search area needs to be.

Tip #2: Check for zoom-factor
By "zoom-factor" I mean any combination of the following:
- System settings (screen resolution, font sizes, magnification, transparency etc.)
- Application-specific zooming (like IE, Word etc. have zoom functionality)

I haven't had to deal with this one too much but I've read it can affect the search. Some of that is common sense (can't search an area in the 1000s on an 800x600 screen resolution etc.) and it's not really that common (unless you use a screen magnifier like I do, or share the script with people running a different OS or with different settings.

Tip #3: Find ways to check your coordinates
Sometimes when I don't get why ImageSearch seems to foul up for no good reason, I use things like MsgBox, MouseMove and other AHK functions to figure out exactly where my search area is. What this looks like will depend on your code, but basically the idea is to make sure your search area actually contains the image being searched for. Again, I can be a moron sometimes - and wondering why my ImageSearch fails when the image isn't even in that area to begin with is a typical Miguel move. :lol:

Tip #4: Make sure your image exists
Again, sometimes I can be an ID10T, and this is another goof I'm as guilty of as anyone. Make sure not only that the image exists, but also that it's in the location you're using in your code. Checking this is no biggy:

Code: [Select all] [Download] (Script.ahk)GeSHi © Codebox Plus

^i::
IfNotExist, ImageFile.png
MsgBox Error: Your file either doesn't exist or isn't in this location.
ImageSearch, OutputVarX, OutputVarY, 400, 200, 600, 600, ImageFile.png

Those first 2 lines say it all. :)

Tip #5: Try modifying the image
On some occasions I've found this helps. If your image is too big, or too complex for the *n property to help, maybe you need a smaller image. Think of "Where's Waldo" (a game where you have to pick out a specific guy out of a crowd): the people around Waldo are never the same way twice, so searching an exact picture of Waldo isn't going to work; but searching for his nose or the stripes on his shirt might.

Tip #6: Refer to the "Docs"
This sounds like a cop-out answer, but sometimes it helps. There's more to ImageSearch than what I've described here, so it might be helpful to check it out. Here's the link: http://ahkscript.org/docs/commands/ImageSearch.htm

Tip #7: If all else fails, "Ask for Help"
There are people on this forum whose mad skillz blow me away. If none of my suggestions help, someone on the forum can.

Tip #8: Check your variable names
This is another classic duh move. :lol: This one happened to me today: I copied a block of code from one script to another and sure enough, ImageSearch failed. Stupid little stuff like this is so easy to overlook; I crack jokes about being dumber than a bag of hammers, but really this kind of thing happens to everyone.

Tip #9: Try some different "Options"
I followed my own advice just now and referred to the docs - and I found some info I had never seen before. There are things in the "options" parameter that we can use to allow for some variation in the image being searched. You can specify things like transparent colors, width and height of the image (again, not to be confused with the X2 and Y2 parameters) and others. You might want to experiment with some of these.

Tip #10: Above all else, don't let it drive you nuts!
I think this was the one that took me the longest to learn. It's important to be patient and not let it get to you. No code is worth the aggravation. Step away from the computer for awhile, or if at work pick another project or function, and when you come back to it you'll have a fresh perspective. This has helped me in many projects, not just at work and not just in AHK. It's also the one I tend to forget the quickest - uh, duh! :lol:

Anyway, I hope these tips & tricks help you make your ImageSearches work, so you can move on with your script and get back to having fun with AHK! :)

Edit: - Today at work, one of the end users of my script pointed out a problem that was ImageSearch-related. The program I'm automating has a section that's divided into "cells" (like Excel), but some cells spanned multiple rows. My script was currently only able to work with the info one row at a time. The guy asked a very simple question that led to a discovery so cool I had to include it here. He asked, "can you have it look for the border?"

That got me thinking. I knew ImageSearch could do whole images, but I needed some way to search for a partial image, a pattern that would let my script know, "this cell spans two rows". Then it hit me: I took a screenshot of the area I wanted to search, replaced most of it with white, and did something like this:

Code: [Select all] [Download] GeSHi © Codebox Plus

CellSpans2Rows(){
ImageSearch, foundX, foundY, x1, y1, x2, y2, *TransWhite %A_ScriptDir%\oneRow.png
if (foundX == ""){
return true ;It didn't find a match for the image with a border, so it spans more than one row
}else{
return false ;it found a match, so the cell only occupies one row.
}
}

By setting the *TransWhite option in my search, the only thing the ImageSearch needed to find a match for was the black border (in the example above, a border as seen on cells that only span one row). The awesome thing about this is, ImageSearch is fast - faster than I ever thought it would be - so I could throw in a few more to get the exact position in the control.

So the lesson here is, if you're searching for an image that changes often, you can use a transparent color and the *TransColor option to search for locial patterns or partial matches. I used a simple cell border as an example, but the same idea would also work for an animated object in a game or whatever else you're using it for. Now I'm getting beyond just finding images - now we're talking about a script that "sees" things more like we do; very cool! :D
Last edited by Miguel7 on 12 Mar 2015, 22:09, edited 2 times in total.
User avatar
Almost_there
Posts: 404
Joined: 30 Sep 2014, 10:32

Re: ImageSearch Tips & Tricks

23 Oct 2014, 11:35

Nice post.

Also I can put in a suggestion, typical user error. I always think on coordinates where origo is placed in bottom lect corner. Most graphical programs works this way.
The screen/window coordinate system that AHK uses is turned upside down, the Y-axis start on the top.
OMG how frustrated I was until I some day figured it out - by using MouseMove, and withnessed that search area vas far away from where it was intended to search.
Miguel7
Posts: 168
Joined: 08 Sep 2014, 07:06

Re: ImageSearch Tips & Tricks

12 Mar 2015, 21:25

You know I never thought of that; in most 2D game design programs it's just like AHK, but I met a guy doing some kind of math graphing and he said it was "wrong" - I think math gurus do it more like what you're describing, but I've been used to 0, 0 being at the top left since long before I knew AHK.

Another good one for this is CoordMode, Pixel. I goofed on that one this week, telling it to click a button that was technically in another "window" (two embedded browsers in the same application at the same time - madness :lol:). So that can mess things up too.
User avatar
Blackholyman
Posts: 1179
Joined: 29 Sep 2013, 22:57
Facebook: socialjsz
Google: +Jszapp
Location: Denmark
Contact:

Re: ImageSearch Tips & Tricks

13 Mar 2015, 01:49

Its not AutoHotkey that's doing the coordinate system wrong, it's a MS windows thing

It's also one of the things seen in this Infographic
Image

Curtesy of: Jszapp.com
Miguel7
Posts: 168
Joined: 08 Sep 2014, 07:06

Re: ImageSearch Tips & Tricks

13 Mar 2015, 09:32

Well that makes sense. I've only ever programmed on Windows (and also the web, but JavaScript/CSS are like that too). Anything else just seems backwards to me. :lol:
fourst4r
Posts: 2
Joined: 31 May 2017, 00:27

Re: ImageSearch Tips & Tricks

31 May 2017, 00:55

Sorry for the bump, but I think it's important to include the fact that coordinates are relative to the active window, (new to AHK) this one stumped me for a while.
The reason I thought this was important to include is since this thread is the second result when you search "imagesearch ahk" on Google.
User avatar
boiler
Posts: 1616
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

31 May 2017, 06:43

You can change it to search relative to screen coordinates using the CoordMode command.

Return to “Tutorials”

Who is online

Users browsing this forum: No registered users and 4 guests