Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Extended ImageSearch Library (Beta)


  • Please log in to reply
17 replies to this topic
NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
Extended Image Search Library (ImgSearch)

Credit/Acknowledgements
Large portions of this code were directly extracted/modified from AutoHotkey_L distribution v1.0.92.02 (https://github.com/L...os/AutoHotkey_L) - Many thanks to Lexikos

As such, all distributions of this library include the source used to build the files (in the Source folder), and have been duly commented with any an all changes from the original source, as required under the GNU General Public License v2. Furthermore, as a derivative work, all files in this distribution are subject to the same licensing terms as AutoHotkey_L, and a copy of these licensing terms have been included in the distribution for your reference.Overview

The overall purpose of this library is to provide a modified version of AHK's ImageSearch function that allows the caller to search an image on the clipboard, rather than the screenshot taken internally using GDI/GDI+. This library distribution consists of two parts:

[*:38wrmfil]A compiled DLL, which contains the modified version of the ImageSearch implementation from the AHK_L source (i.e. ImageSearchClip). In order to support different builds of AHK_L, four precompiled versions of this DLL have been provided (32-Bit Unicode, 32-Bit ANSI, 64-Bit Unicode, and 64-Bit ANSI). Please note that scripts should't need to worry about which version to call, as this is handled by the AHK wrapper script.
[*:38wrmfil]An AHK script that wraps the call to the native function. This wrapper function (_ImageSearchClip), handles the following tasks for the caller:

[*:38wrmfil]Determining the correct DLL to load and pre-loading it (for better performance).
[*:38wrmfil]Invoking the native function via DllCall
[*:38wrmfil]Translating the output from the native function to something that is consistent to the expected behavior of it's progenitor function, ImageSearch.[/list][/list]Please refer to the Released\Lib\_ImageSearchClip.ahk file for more information about the wrapper function, including the parameter list, call syntax, and some examples.
Compatibility

The DLL's are built to support Window 2000 and higher, and do not officially support Windows 95/98/Me (if anyone is even still using those). I'm not saying with absolute certainty that it won't work on those operating systems, but if you use it and have any issues, you're on your own. I won't be able to help you.

The wrapper script should be compatible with all flavors of AHK v1.0.48.05 or AHK_L v1.0.92.02 and higher. However, I was only able to test with AHK_L v1.0.92.02 (32 ANSI version), so a lot of the support I added is based on the documentation for the two versions (i.e. it should work fine, but as with life, nothing is 100%). The same goes for
different builds of AHK_L (Unicode/ANSI, 32/64-Bit). I guess what I'm saying is, I made a best effort to support as many versions as possible, but your mileage may vary. :)

Also, this version does not support precompiled scripts in any way. Like the blurb I made about Win9x/Me support, if it works for you, great. Otherwise, I won't be able to help (nor would I know where to begin).
Installation

To install the library, simply extract the contents of the Distributed folder to your AutoHotkey installation folder, which is usually C:\Program Files\AutoHotkey or C:\Program Files (x86)\AutoHotkey, depending on your version of Windows and which build of AHK you have installed. Assuming the default installation location, once installed, you should have following files/folders:

[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch32A.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch32W.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch64A.DLL
[*:38wrmfil]%AHK Install Path%\Bin\ImgSearch64W.DLL
[*:38wrmfil]%AHK Install Path%\Lib\_ImageSearchClip.ahk[/list]After installation, you can call _ImageSearchClip as you would any other built-in AHK or AHK_L function. Including the ImageSearchClip.ahk is not required, although you can still do so if you wish. You just have to ensure that you include it using the absolute path to the file, otherwise AHK will be unable to resolve the file location, producing an error when you load the script.
Change Log/Version History

[*:38wrmfil]Version 1.0.0.0

[*:38wrmfil]Initial public release.[/list]Download
Version 1.0.0.0 (Beta)

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
Reserved for future use.

flak
  • Members
  • 283 posts
  • Last active: Jan 01 2012 06:20 PM
  • Joined: 02 Oct 2009
Could you add option for haystack and needle to be both GDI bitmaps, not clipboard and file?

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
It will require a more extensive refactoring of the ImageSearch code, but yes, it is definitely possible. I will add it to the feature request list, however the next version may be a few weeks. I want to see how well received the library is, and wait for possible bug reports from users. However, I can make sure to include it in the next version.

EDIT: I found this post by Lexikos that tells you how to load a file and put it on the clipboard (last post on the first page). It's only a few lines of code, and seems simple enough (compared to the refactor work involved) that I think I'll hold off on adding a file to file image search function for now. However, if there are really a large enough number of requests for it, I'll be sure to bump up the priority.

MasterFocus
  • Moderators
  • 4323 posts
  • Last active: Jan 28 2016 01:38 AM
  • Joined: 08 Apr 2009
Thanks for sharing.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Antonio Fran├ža -- git.io -- github.com -- ahk4.net -- sites.google.com -- ahkscript.org

Member of the AHK community since 08/Apr/2009. Moderator since mid-2012.


tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007
I know I haven't exactly made it easy to find, but image searching is already available in the gdi+ library, and loading to clipboard is as easy as

Gdip_SetBitmapToClipboard(pBitmap := Gdip_CreateBitmapFromFile(File))


flak
  • Members
  • 283 posts
  • Last active: Jan 01 2012 06:20 PM
  • Joined: 02 Oct 2009
I just want function that can do image searching in memory, without messing with clipboard and loading image files, but can use transparent color and color variation.

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
@flak - Technically speaking, the clipboard is memory. :) As far as a memory to memory search is concerned, I'm not sure I see much of a use case for that, now that I think about it. If neither of the images are on disk, then that would seem to imply they were both created on the fly, in which case I think a search might be redundant (since you could just mark the image location as you generate it). The other scenarioI could think of would be if your needle image was generated on the fly, but I would still question how accurate the search could be, given the potential color palette issues. However, like I said before, it's on my radar, and as the demand for such a feature increases, so will the priority on my todo list. My resources are extremely limited, so I have to pick and choose my updates for now; I hope you understand.

@tic - As far as searching using GDI+, my only issue is that something as performance intensive as a pixel-by-pixel search isn't some that I personally would want to do from a scripted language such as AHK (no offense intended). Especially considering the lack of full BLOB and Array support. However, if you know of a specific function in the GDI+ API I've overlooked, I'm open to using it (anything that can make my life simpler :)). I guess I'm just too spoiled by native code for my own good. At the very least, the user has options, and who doesn't like more options? :D

EDIT: I know AHK_L has Array support, but since one of my design goals was to create something universal (i.e. AHK basic users), I had to take that into account when I was designing a solution. I personally use AHK_L, so array support isn't necessarily a major issue for me, although I still don't think I would want to use AHK_L arrays to do any kind of large binary search.

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007
Hey NoobSawce, yes I completely agree that anthing like an image search shouldn't be performed by an interpreted language like ahk as it is much too slow as soon as the images start to get large, that is why (and I apologise I still haven't cleaned it up, wrpped it and put it into the gdi+ voodoo section) I wrote it in machine code. It's posted nearish to the end of the gdi+ thread

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
Cool beans, thanks for letting me know. I'll be sure to check it out. :)

flak
  • Members
  • 283 posts
  • Last active: Jan 01 2012 06:20 PM
  • Joined: 02 Oct 2009
As far as a memory to memory search is concerned, I'm not sure I see much of a use case for that
When needle is loaded from disk, it is loaded each time the search is initiated. If you just load it once and keep there, it reduces overhead. tic's function works, and I hacked it to support transparency, but it still can't do color variation.

b1tch
  • Guests
  • Last active:
  • Joined: --
here full my code. this code is translate imagesearch to _imagesearchclip function. but ahk script excution working well. but imagesearch is not working fullscreen and window mode both. what's the problem.

-------------------

#MaxThreadsPerHotkey 1
SetWorkingDir, %A_ScriptDir%
CoordMode, Pixel, Screen

3::

loop,1
{
;Serarch Start
_ImageSearchClip(,,6,1033,45,1074,*100 A.bmp)
if ErrorLevel=0
{
Send,9
break
}

_ImageSearchClip(,,6,1033,45,1074,*100 I.bmp)
if ErrorLevel=0
{
Send,{F10}
break
}
else
{
send,{f9}
break
}
}
return

------------------------------------

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011
What is the return code you are getting from _ImageSearchClip? Also, how are you getting an image onto the clipboard? I notice that the code is not sending a PrintScreen or Alt+PrintScreen, so I suspect the failure may be because there is no image on the clipboard at the time you are calling the function. Just to verify, you can add the *Dump option to get the function to write the image from the clipboard (if there is one) to %UserProfile%\ImageSearchClip.bmp; if no file is written, then that confirms that you don't have an image on the clipboard.

Also, there is no need to set the CoordMode prior to calling this function. It does not have access to the internal AutoHotkey settings, so you have to use the *S (screen) or *R (relative or window) options to specify the CoordMode to the function (the default is screen).

EDIT: Also, please put scripts inside [code][/code] tags. It makes the code easier to read. :)

EDIT2: You can take a look at my last post on this thread. It gives an example of how I am using this function to do an image search in a fullscreen game using the clipboard. You can remove the lines that call _LogInfo; that is a custom function I made to log information to debug my scripts. You can also look at the second example at the top of the _ImageSearchClip.ahk file.

NoobSawce
  • Members
  • 103 posts
  • Last active: Mar 19 2011 05:11 AM
  • Joined: 15 Feb 2011

I just want function that can do image searching in memory, without messing with clipboard and loading image files, but can use transparent color and color variation.

Actually, loading the file is pretty fast and has fairly low overhead. The bulk of the function's time (and the main bottleneck) is the actual binary search (especially if you are searching a large area), which unfortunately, isn't avoidable. The second largest bottleneck is pulling the bits from the handle, but again, that will be the same regardless of whether you load the image yourself, or the function does it for you.

The only other possible issue is memory fragmentation, which is less likely to be an issue if needle is small (Win32 memory management is pretty good in this regard). Fragmentation in the haystack image can be avoided entirely if you are searching a static image by pre-loading it using GDI+ or LoadImage and pushing the handle onto the clipboard prior to doing the search. However, I don't see a huge use case for this, either, because searching the static haystack more than once (unless you have more than one needle, but even then only once per needle) seems kind of a waste.

To summarize, there really isn't much of a difference between you loading the image or having the function do it for you. The only possible benefits would be pre-loading a static needle to search dynamic images, but even that overhead is fairly tiny unless you have a very large needle, and even then, the binary search and pulling the bits (which caching wouldn't avoid) would have an exponentially larger overhead.

Lucid_Method
  • Members
  • 147 posts
  • Last active: Dec 06 2014 08:39 AM
  • Joined: 19 Apr 2010
Hey NoobSawce, just wanted to encourage you to keep developing this library. Imagesearch has amazing potential for quick automation when it works well. I've done a fair amount of testing with the basic image search function as I've worked on a macro recorder. It works great when it works but I found there were a number of times where I was looking at the search needle on screen but it wasn't located by the search for whatever reason. I was looking into options for different approaches, I was hoping to combine several methods into 1 function to allow multiple verification points, at the same time building in a hotkey to recapture additional search images during the process - increasing the number of "views" of the needle we're searching for. In my case speed isn't as much of an issue, more improving accuracy in the success rate.

I played with the machine code function Tic mentioned above, it works great and it's really fast. I haven't put them head to head yet but when I do I'll throw your Dll in the mix and see which methods are more accurate. I'm hoping the basic search, your dll, and the machine code all trying to find the search needle will work - where if 2/3 finds it then it's success to click and continue.. I don't know the details behind how the image search code works but I welcome suggestions / improvements on the dll :-)
Posted Image
Macro Everything
Lucid_Method Index