17 replies to this topic

[NoobSawce]

Extended Image Search Library (ImgSearch)

Credit/Acknowledgements
Large portions of this code were directly extracted/modified from AutoHotkey_L distribution v1.0.92.02 (https://github.com/L...os/AutoHotkey_L) - Many thanks to Lexikos

As such, all distributions of this library include the source used to build the files (in the Source folder), and have been duly commented with any an all changes from the original source, as required under the GNU General Public License v2. Furthermore, as a derivative work, all files in this distribution are subject to the same licensing terms as AutoHotkey_L, and a copy of these licensing terms have been included in the distribution for your reference.Overview

The overall purpose of this library is to provide a modified version of AHK's ImageSearch function that allows the caller to search an image on the clipboard, rather than the screenshot taken internally using GDI/GDI+. This library distribution consists of two parts:

[*:38wrmfil]A compiled DLL, which contains the modified version of the ImageSearch implementation from the AHK_L source (i.e. ImageSearchClip). In order to support different builds of AHK_L, four precompiled versions of this DLL have been provided (32-Bit Unicode, 32-Bit ANSI, 64-Bit Unicode, and 64-Bit ANSI). Please note that scripts should't need to worry about which version to call, as this is handled by the AHK wrapper script.
[*:38wrmfil]An AHK script that wraps the call to the native function. This wrapper function (_ImageSearchClip), handles the following tasks for the caller:

[*:38wrmfil]Determining the correct DLL to load and pre-loading it (for better performance).
[*:38wrmfil]Invoking the native function via DllCall
[*:38wrmfil]Translating the output from the native function to something that is consistent to the expected behavior of it's progenitor function, ImageSearch.[/list][/list]Please refer to the Released\Lib\_ImageSearchClip.ahk file for more information about the wrapper function, including the parameter list, call syntax, and some examples.
Compatibility

The DLL's are built to support Window 2000 and higher, and do not officially support Windows 95/98/Me (if anyone is even still using those). I'm not saying with absolute certainty that it won't work on those operating systems, but if you use it and have any issues, you're on your own. I won't be able to help you.

The wrapper script should be compatible with all flavors of AHK v1.0.48.05 or AHK_L v1.0.92.02 and higher. However, I was only able to test with AHK_L v1.0.92.02 (32 ANSI version), so a lot of the support I added is based on the documentation for the two versions (i.e. it should work fine, but as with life, nothing is 100%). The same goes for
different builds of AHK_L (Unicode/ANSI, 32/64-Bit). I guess what I'm saying is, I made a best effort to support as many versions as possible, but your mileage may vary.

Also, this version does not support precompiled scripts in any way. Like the blurb I made about Win9x/Me support, if it works for you, great. Otherwise, I won't be able to help (nor would I know where to begin).
Installation

To install the library, simply extract the contents of the Distributed folder to your AutoHotkey installation folder, which is usually C:\Program Files\AutoHotkey or C:\Program Files (x86)\AutoHotkey, depending on your version of Windows and which build of AHK you have installed. Assuming the default installation location, once installed, you should have following files/folders:

[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch32A.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch32W.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch64A.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch64W.DLL
[*:38wrmfil]%AHK Install Path%\Lib\_ImageSearchClip.ahk[/list]After installation, you can call _ImageSearchClip as you would any other built-in AHK or AHK_L function. Including the ImageSearchClip.ahk is not required, although you can still do so if you wish. You just have to ensure that you include it using the absolute path to the file, otherwise AHK will be unable to resolve the file location, producing an error when you load the script.
Change Log/Version History

[*:38wrmfil]Version 1.0.0.0

[*:38wrmfil]Initial public release.[/list]Download
Version 1.0.0.0 (Beta)

[NoobSawce]

Reserved for future use.

[flak]

Could you add option for haystack and needle to be both GDI bitmaps, not clipboard and file?

[NoobSawce]

It will require a more extensive refactoring of the ImageSearch code, but yes, it is definitely possible. I will add it to the feature request list, however the next version may be a few weeks. I want to see how well received the library is, and wait for possible bug reports from users. However, I can make sure to include it in the next version.

EDIT: I found this post by Lexikos that tells you how to load a file and put it on the clipboard (last post on the first page). It's only a few lines of code, and seems simple enough (compared to the refactor work involved) that I think I'll hold off on adding a file to file image search function for now. However, if there are really a large enough number of requests for it, I'll be sure to bump up the priority.

[MasterFocus]

Thanks for sharing.

[tic]

I know I haven't exactly made it easy to find, but image searching is already available in the gdi+ library, and loading to clipboard is as easy as

Gdip_SetBitmapToClipboard(pBitmap := Gdip_CreateBitmapFromFile(File))

[flak]

I just want function that can do image searching in memory, without messing with clipboard and loading image files, but can use transparent color and color variation.

[NoobSawce]

@flak - Technically speaking, the clipboard is memory. As far as a memory to memory search is concerned, I'm not sure I see much of a use case for that, now that I think about it. If neither of the images are on disk, then that would seem to imply they were both created on the fly, in which case I think a search might be redundant (since you could just mark the image location as you generate it). The other scenarioI could think of would be if your needle image was generated on the fly, but I would still question how accurate the search could be, given the potential color palette issues. However, like I said before, it's on my radar, and as the demand for such a feature increases, so will the priority on my todo list. My resources are extremely limited, so I have to pick and choose my updates for now; I hope you understand.

@tic - As far as searching using GDI+, my only issue is that something as performance intensive as a pixel-by-pixel search isn't some that I personally would want to do from a scripted language such as AHK (no offense intended). Especially considering the lack of full BLOB and Array support. However, if you know of a specific function in the GDI+ API I've overlooked, I'm open to using it (anything that can make my life simpler ). I guess I'm just too spoiled by native code for my own good. At the very least, the user has options, and who doesn't like more options?

EDIT: I know AHK_L has Array support, but since one of my design goals was to create something universal (i.e. AHK basic users), I had to take that into account when I was designing a solution. I personally use AHK_L, so array support isn't necessarily a major issue for me, although I still don't think I would want to use AHK_L arrays to do any kind of large binary search.

[tic]

Hey NoobSawce, yes I completely agree that anthing like an image search shouldn't be performed by an interpreted language like ahk as it is much too slow as soon as the images start to get large, that is why (and I apologise I still haven't cleaned it up, wrpped it and put it into the gdi+ voodoo section) I wrote it in machine code. It's posted nearish to the end of the gdi+ thread

[NoobSawce]

Cool beans, thanks for letting me know. I'll be sure to check it out.

[flak]

As far as a memory to memory search is concerned, I'm not sure I see much of a use case for that

When needle is loaded from disk, it is loaded each time the search is initiated. If you just load it once and keep there, it reduces overhead. tic's function works, and I hacked it to support transparency, but it still can't do color variation.

[NoobSawce]

What is the return code you are getting from _ImageSearchClip? Also, how are you getting an image onto the clipboard? I notice that the code is not sending a PrintScreen or Alt+PrintScreen, so I suspect the failure may be because there is no image on the clipboard at the time you are calling the function. Just to verify, you can add the *Dump option to get the function to write the image from the clipboard (if there is one) to %UserProfile%\ImageSearchClip.bmp; if no file is written, then that confirms that you don't have an image on the clipboard.

Also, there is no need to set the CoordMode prior to calling this function. It does not have access to the internal AutoHotkey settings, so you have to use the *S (screen) or *R (relative or window) options to specify the CoordMode to the function (the default is screen).

EDIT: Also, please put scripts inside [code][/code] tags. It makes the code easier to read.

EDIT2: You can take a look at my last post on this thread. It gives an example of how I am using this function to do an image search in a fullscreen game using the clipboard. You can remove the lines that call _LogInfo; that is a custom function I made to log information to debug my scripts. You can also look at the second example at the top of the _ImageSearchClip.ahk file.

[NoobSawce]

I just want function that can do image searching in memory, without messing with clipboard and loading image files, but can use transparent color and color variation.

Actually, loading the file is pretty fast and has fairly low overhead. The bulk of the function's time (and the main bottleneck) is the actual binary search (especially if you are searching a large area), which unfortunately, isn't avoidable. The second largest bottleneck is pulling the bits from the handle, but again, that will be the same regardless of whether you load the image yourself, or the function does it for you.

The only other possible issue is memory fragmentation, which is less likely to be an issue if needle is small (Win32 memory management is pretty good in this regard). Fragmentation in the haystack image can be avoided entirely if you are searching a static image by pre-loading it using GDI+ or LoadImage and pushing the handle onto the clipboard prior to doing the search. However, I don't see a huge use case for this, either, because searching the static haystack more than once (unless you have more than one needle, but even then only once per needle) seems kind of a waste.

To summarize, there really isn't much of a difference between you loading the image or having the function do it for you. The only possible benefits would be pre-loading a static needle to search dynamic images, but even that overhead is fairly tiny unless you have a very large needle, and even then, the binary search and pulling the bits (which caching wouldn't avoid) would have an exponentially larger overhead.

[Lucid_Method]

Hey NoobSawce, just wanted to encourage you to keep developing this library. Imagesearch has amazing potential for quick automation when it works well. I've done a fair amount of testing with the basic image search function as I've worked on a macro recorder. It works great when it works but I found there were a number of times where I was looking at the search needle on screen but it wasn't located by the search for whatever reason. I was looking into options for different approaches, I was hoping to combine several methods into 1 function to allow multiple verification points, at the same time building in a hotkey to recapture additional search images during the process - increasing the number of "views" of the needle we're searching for. In my case speed isn't as much of an issue, more improving accuracy in the success rate.

I played with the machine code function Tic mentioned above, it works great and it's really fast. I haven't put them head to head yet but when I do I'll throw your Dll in the mix and see which methods are more accurate. I'm hoping the basic search, your dll, and the machine code all trying to find the search needle will work - where if 2/3 finds it then it's success to click and continue.. I don't know the details behind how the image search code works but I welcome suggestions / improvements on the dll :-)