ImagePut - A core library for images in AutoHotkey (Now supports HEIC & WEBP)

[kairushinjuu]

Hello,

I was wondering if you could help me with a problem I am having. I am trying to capture of the window of an application "LdPlayer" (Android emulator).
However it is only capture some of the window.

Code: Select all

^2::
{
    pic := ImagePutBuffer("LdPlayer - *")
    pic.Show(pic.pBitmap)
}

*If image isn't showing. https://imgur.com/Hex40w6

Any suggestions would be appreciated

[iseahound]

Looks like some scaling going on. There isn't much you can do except maybe use a screenshot instead. Most graphics engines won't expose their APIs due to hacking / cheating etc.

In case it helps, I've added a mini function to ImagePut that lets you get the coordinates for a screenshot of a window easier.

Code: Select all

Imageshow({screenshot: "AutoHotkey Dash"})

either of the two should work now. But your window must be on screen and not covered by any other applications. For the future this should be improved to make it DPI invariant and should remove the window shadows. And maybe restrict it to the client size as well.

2023-03-26 - Now uses WinGetClientPos instead!

EDIT: 2023-04-30 Removed Imageshow(["AutoHotkey Dash"]) from the latest version of ImagePut

[kairushinjuu]

Looks like some scaling going on. There isn't much you can do except maybe use a screenshot instead. Most graphics engines won't expose their APIs due to hacking / cheating etc.

In case it helps, I've added a mini function to ImagePut that lets you get the coordinates for a screenshot of a window easier.

Code: Select all

Imageshow({screenshot: "AutoHotkey Dash"})
Imageshow(["AutoHotkey Dash"])

either of the two should work now. But your window must be on screen and not covered by any other applications. For the future this should be improved to make it DPI invariant and should remove the window shadows. And maybe restrict it to the client size as well.

2023-03-26 - Now uses WinGetClientPos instead!

I figured it out, the reason it clips is because each monitor has a different scale if I set them to the same scale it works haha. but then cords are all out haha. Thanks for your help though!

[iseahound]

Glad to be of help! Here's a simple PixelSearch script using ImagePut for Bing Chat AI to find:

Code: Select all

; Include the ImagePut library.
#include ImagePut.ahk

; Create a bitmap buffer image object that captures the current virtual screen (all monitors)
screen := ImagePutBuffer(0)

; The color red
color := 0xFFFF0000

; Perform pixelsearch by calling the pixelsearch method of the bitmap buffer.
xy := screen.PixelSearch(color)

; xy is an array [x, y]. If the PixelSearch failed, the return value is false.
if (xy)
    MsgBox "Pixel was found at: " xy[1] ", " xy[2]
else
    MsgBox "A matching pixel of the same color was not found."
    
; Show the screen capture. Right click to close.
ImageShow(screen)

; You can also use screen.show()

[fiendhunter]

hi, is it possible to get ImagePutBuffer(x,y,w,h) as relative of active window. example: buf := ImagePutBuffer([X-227, 30, 217, 900],r) ; r for relative of the active window.

Code: Select all

f1::
coordmode,mouse,relative ; native coordmode is relative to an active window. 
mousegetpos X,Y ; example x=300, but imageputbuff start cropping from x=308
buf := ImagePutBuffer({window: "A", crop: [X-227, 30, 217, 900]}) ; it flashes (%90 rate) the screen and crop pure white image. When there is no flashing, the coordinates x (+8 pixels) and y (+31 pixels) are shifted according to "mousegetpos x,y" 
ImageShow(buf) ; show pure white "buf" or shifted x,y of image

I want to run more than 80-90 imagesearch for 1 frame (to increase speed of search.)

[iseahound]

Why not just use a screenshot instead?

Code: Select all

WinGetClientPos &X, &Y, &W, &H, "A"
im := ImagePutBuffer({screenshot: [x, y, w, h], crop: [100, 100, -100, -100]}) ; <- Modify xywh with relative coordinates. 
im.show()

Shorter version:

Code: Select all

im := ImagePutBuffer({screenshot: "A", crop: [100, 100, -100, -100]})
im.show()

The crop is ALWAYS relative to your input image. Let me know if ImageSearch is fast enough to do 80-90. I didn't test it much.

[fiendhunter]

Why not just use a screenshot instead?

Code: Select all

WinGetClientPos &X, &Y, &W, &H, "A"
im := ImagePutBuffer({screenshot: [x, y, w, h], crop: [100, 100, -100, -100]}) ; <- Modify xywh with relative coordinates. 
im.show

Shorter version:

Code: Select all

im := ImagePutBuffer({screenshot: "A", crop: [100, 100, -100, -100]})
im.show

The crop is ALWAYS relative to your input image. Let me know if ImageSearch is fast enough to do 80-90. I didn't test it much.

Im still using v1. The shorter version returns with error and first suggestion with "wingetpos" work nice, it crops without shifting or flashing. (EDIT: I update the Imageput.ahk it fixed the error.)

piece of code:

Code: Select all

F1::
ix := 300 ; example
gosub, activatebmp
WinGetPos, X, Y, W, H, A
buf := ImagePutBuffer({screenshot: [x, y, w, h], crop: [iX-234,35, 210,960]}) ; working flawless
if xy := buf.ImageSearch("HBITMAP:*%HTestImage%") ; in this line it wants an image file.
{
MouseMove, xy[1], xy[2]  
}else{
msgbox, not found
}
return


activatebmp:
;@Ahk2Exe-IgnoreBegin
HTestImage := LoadPicture("C:\Bitmaplib\TestImage.bmp")
return
;@Ahk2Exe-IgnoreEnd
;@Ahk2Exe-AddResource C:\Bitmaplib\TestImage.bmp
HTestImage:= LoadBitmapResource("TestImage.bmp")
return

How can I use HTestImage from resource?

[iseahound]

I am not to familiar with ahkexe, but here are some suggestions:

Code: Select all

F1::
ix := 300 ; example
gosub, activatebmp
WinGetPos, X, Y, W, H, A
buf := ImagePutBuffer({screenshot: [x, y, w, h], crop: [iX-234,35, 210,960]}) ; working flawless
target := ImagePutBuffer("C:\Bitmaplib\TestImage.bmp") ; will be faster if you cache your target image in a buffer too
; also try:
target2 := ImagePutBuffer("TestImage.bmp")
; and 
target3 := ImagePutBuffer(LoadBitmapResource("TestImage.bmp"))
if xy := buf.ImageSearch(target) ; in this line it wants an image file.
{
MouseMove, xy[1], xy[2]  
}else{
msgbox, not found
}
return


activatebmp:
;@Ahk2Exe-IgnoreBegin
HTestImage := LoadPicture("C:\Bitmaplib\TestImage.bmp")
return
;@Ahk2Exe-IgnoreEnd
;@Ahk2Exe-AddResource C:\Bitmaplib\TestImage.bmp
HTestImage:= LoadBitmapResource("TestImage.bmp")
return

Also does the imagesearch 80-90 times work well? It's untested on my end TBH

The shorter version returns with error

What do you mean? Can you show me? It seems to work fine here. (I added the parathesis back in im.show() for v1)

AddResource? - viewtopic.php?t=90580

[fiendhunter]

in v1, im := ImagePutBuffer({screenshot: "A", crop: [100, 100, -100, -100]}) was return with error. I update ImagePut.ahk from github. Now has no error.

Im now testing in .ahk I couldnt pass .exe yet. I need a working methot for both. @Ahk2Exe allows to put images in .exe file, so you can call images with variable example; ImageSearch, XE, YE, 31,35, 110,100, *TransWhite *2 HBITMAP:*%HTestImageFound% ; this method is slow, recapture screen for every search. As you mention, I need to put captured image in buffer and compare with other bitmaps.

Code: Select all

F1::
ix := 300 ; example
gosub, activatebmp
buf := ImagePutBuffer({screenshot: "A", crop: [iX-234,35, 210,960]})  ; working flawless
; ImageShow(buf) ; checked image, it gets right image from ImagePutBuffer.

; add and test 2 image. TestImageFound.bmp and TestImageNot.bmp

target := ImagePutBuffer("C:\Bitmaplib\TestImageFound.bmp") ; it returns not found and checked ImageShow(target) it shows image well.
target := ImagePutBuffer("C:\Bitmaplib\TestImageNot.bmp") ; it returns not found and checked ImageShow(target) it shows image well.
; --------------------------------

target := ImagePutBuffer("TestImageFound.bmp") ; it returns "Imagetype could not be identified" image file is not in workingdir.
target := ImagePutBuffer("TestImageNot.bmp") ; it returns "Imagetype could not be identified" image file is not in workingdir.
; --------------------------------

target := ImagePutBuffer(LoadBitmapResource("TestImageFound.bmp")) ; (in .ahk) returns "could not load bitmap" as expected. (in .exe) returns not found and checked ImageShow(target) it shows image well.
target := ImagePutBuffer(LoadBitmapResource("TestImageNot.bmp")) ; (in .ahk) returns "could not load bitmap" as expected. (in .exe) returns not found and checked ImageShow(target) it shows image well.
; --------------------------------

target := ImagePutBuffer(LoadPicture("C:\Bitmaplib\TestImageFound.bmp")) ; it returns found and checked ImageShow(target) it shows nothing.
target := ImagePutBuffer(LoadPicture("C:\Bitmaplib\TestImageNot.bmp")) ; it returns found and checked ImageShow(target) it shows nothing.
; --------------------------------

target := ImagePutBuffer(HTestImageFound) ; it returns found and checked ImageShow(target) it shows nothing.
target := ImagePutBuffer(HTestImageNot) ; it returns found and checked ImageShow(target) it shows nothing.



; --------------------------------
if xy := buf.ImageSearch(target) ; also tried if (xy := buf.ImageSearch(target))
{
msgbox, found
}else{
msgbox, not found
}
; --------------------------------

return


activatebmp:
;@Ahk2Exe-IgnoreBegin
HTestImage := LoadPicture("C:\Bitmaplib\TestImageFound.bmp")
HTestImage := LoadPicture("C:\Bitmaplib\TestImageNot.bmp")
return
;@Ahk2Exe-IgnoreEnd
;@Ahk2Exe-AddResource C:\Bitmaplib\TestImageFound.bmp
;@Ahk2Exe-AddResource C:\Bitmaplib\TestImageNot.bmp
HTestImage:= LoadBitmapResource("TestImageFound.bmp")
HTestImage:= LoadBitmapResource("TestImageNot.bmp")
return

I also add LoadBitmapResource, it may help.
LoadBitmapResource and LoadPicture returns in same variable and output same type. So, imagesearch can search HBITMAP:*%HTestImage% even in (.ahk) or (.exe)

Code: Select all

LoadBitmapResource(resName, shared := true)
{
	resNameType := "str"
	if resName is integer
		if resName between 0 and 0xFFFF
			resNameType := "ptr"
	if not ret := DllCall("LoadImage", "ptr", GetOwnModuleHandle(), resNameType, resName, "uint", 0, "int", 0, "int", 0, "uint", shared ? 0x8000 : 0, "ptr")
		throw Exception("Could not load bitmap!", -1, "resName: " resName " - ErrLvl: " ErrorLevel " - LastErr: " A_LastError)
	return ret
}

[iseahound]

Once again, I don't know the specifics of ahk's compiled resources. All you have to do is pass the hBitmap into ImageShow or ImagePutBuffer. In this case, it's your HTestImage. The image parameter accepts almost everything, you can pass a file, an hBitmap, a pBitmap, the clipboard, a screenshot, etc. Using the output of LoadImage works. Just try ImageShow(hTestImage)

Basically you should just do something like this:

Code: Select all

;@Ahk2Exe-IgnoreBegin
imageputwindow(LoadPicture("cats.jpg"))
; or
imageputwindow("cats.jpg")
;@Ahk2Exe-IgnoreEnd
;@Ahk2Exe-AddResource cats.jpg
if A_Compiled
imageputwindow(LoadBitmapResource("cats.jpg"))

[iseahound]

2023-04-30 16꞉16꞉02.png (463.17 KiB) Viewed 4547 times

Getting close to having Bing AI answer questions about this script. I think I need to revamp the documentation and include more code examples: It's having a hard time figuring out PixelSearch and ImageSearch and just guesses randomly.

[temp_2321]

hi, i'm new. your library is very easy to use. thank you for sharing.
do you have any plans to add variation option to imagesearch like Gdip_ImageSearch?
imagesearch works perfectly but only when all pixels in captured image are exactly same as all pixels in target image.

[boiler]

Gdip_ImageSearch doesn’t do any kind of fuzzy image matching either. It also is looking for exact pixel matches.

[temp_2321]

Gdip_ImageSearch doesn’t do any kind of fuzzy image matching either. It also is looking for exact pixel matches.

sorry im not a native english speaker so I think I misrepresented it.
i mean, Gdip_imageSearch has "variation" option which is helpful if the coloring of the image varies slightly.
so i was wondering if he had any plans to add this option to imagesearch in ImagePut.

[boiler]

I see. I thought you were comparing it to AHK’s ImageSearch, which also has a variation option, and that the Gdip version did something more in that regard.

[iseahound]

No actually you are the first person to request ImageSearch with variation / fuzzy matching.

[kairushinjuu]

Hello,
Any chance that youll be adding pixel variation on the imagesearch? The screen i am trying to search with has pixel shift implemented in it.

Thank you

[WarlordAkamu67]

I've been using ImagePut for a little while now and its helped me immensely. 10/10 would recommend. Thank you @iseahound!

i mean, Gdip_imageSearch has "variation" option which is helpful if the coloring of the image varies slightly.
so i was wondering if he had any plans to add this option to imagesearch in ImagePut.

and

Hello,
Any chance that youll be adding pixel variation on the imagesearch? The screen i am trying to search with has pixel shift implemented in it.

Thank you

This makes me the third- If possible, I would love to have this option as well ^.^

[iseahound]

Sure! I have some time on Labor Day weekend here in the US—you won't hear from me again until it's complete.

[iseahound]

hi, is it possible to get ImagePutBuffer(x,y,w,h) as relative of active window. example: buf := ImagePutBuffer([X-227, 30, 217, 900],r) ; r for relative of the active window.

The latest version allows you do to ImagePutBuffer([x, y, w, h, r]) where r is relative to the window.

Code: Select all

; Show the 300x300 top-left square of the active window.
ImagePutWindow([0, 0, 300, 300, "A"])

https://github.com/iseahound/ImagePut/blob/master/ImagePut.ahk