CGDipSnapShot - GDI replacement for PixelGetColor

Post your working scripts, libraries and tools
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

CGDipSnapShot - GDI replacement for PixelGetColor

22 Dec 2014, 12:16

What is it?
This is a class that makes using GDI to accelerate pixel detection nice and easy.

It works by introducing the concept of a "Snapshot". A Snapshot is a copy of the portion of the screen stored in memory that you can query using a command that works like AHK's PixelGetColor.

One advantage of this technique is that you are taking a copy of a portion of the screen at a single point in time, so you do not have to worry about the screen changing part way through making multiple pixel checks.
ie, it makes it easy to perform multiple checks on one "frame" of the stream of images.
Also, when you check a pixel color from the snapshot, the result is cached, meaning faster results if you check that pixel's color in the future - so no need to worry about storing local copies of pixel colors to speed up processing.

What are the features / How do I use it?

Using the class is a Snap (ahem).
Define the Snapshot area as a 200x200 block of the screen at coordinates 300,300:

Code: [Select all] [Download] GeSHi © Codebox Plus

#include <CGdipSnapshot>
snap := new CGdipSnapshot(300,300,200,200) ; x, y, width, height
Smile for the camera! Store the snapshot as a "variable".
snap.TakeSnapshot()

Some time later.... We can still inspect this snapshot, even if the screen has changed.
Sleep 999999

Pixel values can be read as Hex or r/g/b values.
You can use coordinates relative to the Screen or the Snapshot.

Code: [Select all] [Download] GeSHi © Codebox Plus

hex_value:= snap.PixelGetColor(300,300).rgb
= 0xFF0000
r_value := snap.SnapshotGetColor(0,0).r
= 255

Handy cached pixel arrays! Re-query pixels without subsequent DLL calls!

Code: [Select all] [Download] GeSHi © Codebox Plus

myval := snap.PixelScreen[300,300].rgb
myval := snap.PixelScreen[300,300].rgb ; no DLL call!
myval := snap.PixelSnap[0,0].rgb ; Same value, still no DLL call!

Pixels array elements are objects! Chain commands!
bool := snap.PixelScreen[300,300].Compare(snap.PixelScreen[0,0]) ; this will always be true, and only ever require one DLL call!

Use multiple snapshots at one time! Easily compare!

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

A handy function to let you easily display a snapshot in an AHK GUI, so you can see what your code is seeing!
Just create a Text box with attribute 0xE and store the HWND, then call the function:

Code: [Select all] [Download] GeSHi © Codebox Plus

Gui, Add, Text, 0xE x5 y5 w%w% h%h% hwndSnapshotPreview
snap.ShowSnapshot(SnapshotPreview)
Easily save snapshots to disk!
snap.SaveSnapshot("myfile.png")

Where can I get it?

Requires the gdip_all library from here

GitHub Page (Download from here)

Wiki - Full Documentation

Full example script
Last edited by evilC on 22 Jan 2015, 04:45, edited 22 times in total.
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 01:43

Feature ideas / plans

  • Optimization - any suggestions welcomed
  • system to allow users to draw / select a snapshot area somehow?
    Overlay wish move / size keys? Mouse only? Force fullscreen window integration?

Last edited by evilC on 22 Jan 2015, 04:58, edited 2 times in total.
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 02:08

awesome! having problems with PixelGetColor but now its fine :D
AutoHotkey & AutoHotkey_H v1.1.22.07
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 02:24

Big update.
ToRGB() is gone, R/G/B are now Dynamic Properties.

"Cached" method of reading added via PixelScreen[] and PixelSnap[] arrays.
Uses Dynamic Properties to ensure that DLL calls are only made when an unknown value (pixel) is requested.
Array elements are color objects with hex or r/g/b dynamic properties.
Array elements also have Compare() and Diff() methods to compare easily to other pixels.

Stuff considered internal is now prefixed with _

Implemented proper Get and Set for Snapshot coordinates (x,y,w,h). Setting these will properly release the old image, clear the pixel cache and flag that no snapshot is present - allowing easy movement of a snapshot window.

Soft wrote:awesome! having problems with PixelGetColor but now its fine :D

You mean you were having problems with the previous version and this solved it, or having problems with AHK's PixelGetColor and this solved it?

Also, sitting back and thinking about it, I am wondering if Compare(c1,c2) in the main class should be left as-is?

It does not logically belong in the snapshot, as it could be used to compare two non-snapshot pixels, but i was wary of polluting the global namespace.

However, thinking forward, snap1.Compare(snap2) should be syntax for comparing the whole of one snapshot for another.

Thoughts?
Last edited by evilC on 22 Jan 2015, 04:50, edited 1 time in total.
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 02:44

having problems with AHK's PixelGetColor and this solved it?
- PixelGetColor doen not work on my game but, this one worked.

snap1.Compare(snap2) +1
AutoHotkey & AutoHotkey_H v1.1.22.07
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 02:57

OK, snap1.Compare(snap2) style whole-snapshot comparing implemented.

The Compare, Diff and ToRGB functions are now global functions and called PixelCompare, PixelDiff and HexToRGB respectively, though you shouldn't need these, so it probably won't matter.

I thought I may as well make these changes quickly before any code is based on the lib, breaking compatibility.

Also, bear in mind that a whole snapshot compare may take a fair while for it to read all the pixels (Lots of DLL calls)
However, once it has read all the pixels, subsequent compares should be quick.
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 03:07

Soft wrote:- PixelGetColor doen not work on my game but, this one worked.

Try turning off Aero or UAC. But hey, this method is much nicer than that anyway, so who cares :P

If you can, run the game in windowed mode while writing the script and show the snapshot every time you take it to easily see what the script is looking at in real-time - it makes debugging so much easier ;)
Or get multiple monitors ;) You can use android phones etc for 2nd monitor too ;)
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 04:09

I just used the QPX benchmarker to write a benchmark script that compares this lib against PixelGetColor and to see if the Caching is doing it's job:

One Pixel with regular PixelGetColor
AHK_Pixel_Get_Color1
0.000538

Hmm, AHK caches too?
AHK_Pixel_Get_Color1
0.000055

Different Pixel
AHK_Pixel_Get_Color2
0.000049

AHK_Pixel_Get_Color2
0.000086

Take a 1x1 Snapshot with lib
GDI_Take_Snap_1x1
0.000166

Grab a pixel from the snapshot
GDI_PixelSnap
0.000080

Repeat same pixel grab, check cache
GDI_PixelSnap
0.000020

Re-take 1x1 Snapshot
GDI_Take_Snap_1x1
0.000144

First check of pixel again
GDI_PixelSnap
0.000048

Second check of a pixel again
GDI_PixelSnap
0.000019

Take a 10x10 Snapshot
GDI_Take_Snap_10x10
0.000145

Take a 100x100 Snapshot
GDI_Take_Snap_100x100
0.000150

Compare two 10x10 snapshots
GDI_Compare_Snap_10x10
0.006371

Repeat compare - should be nice cache hit
GDI_Compare_Snap_10x10
0.002203

Repeat again
GDI_Compare_Snap_10x10
0.002203

Clear cache - not a benchmark really
GDI_Reset_Snaps
0.000113

Same snaps, first compare after cache clear
GDI_Compare_Snap_10x10
0.006668

Repeat
GDI_Compare_Snap_10x10
0.002202

Looks like it is working :)
Last edited by evilC on 23 Jan 2015, 08:53, edited 1 time in total.
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 05:14

Nice :D
AutoHotkey & AutoHotkey_H v1.1.22.07
guest3456
Posts: 2077
Joined: 09 Oct 2013, 10:31

Re: CGDipSnapShot - GDI replacement for PixelGetColor

22 Jan 2015, 11:33

evilC wrote:It works by introducing the concept of a "Snapshot". A Snapshot is a copy of the portion of the screen stored in memory that you can query using a command that works like AHK's PixelGetColor.

One advantage of this technique is that you are taking a copy of a portion of the screen at a single point in time, so you do not have to worry about the screen changing part way through making multiple pixel checks.


I don't mean to rain on the parade, but your technique is no different than whats already available in the GDIP library, or also using PrintWindow() or BitBlt(). As far as I understand, there is nothing new, your class is just a wrapper for the existing GDIP functions

Handy cached pixel arrays! Re-query pixels without subsequent DLL calls!


There are still DllCalls being done through your class, ie GdipGetPixel. This "caching/snapshotting" if you want to call it that is the same as using PrintWindow and then GetPixel msdn functions. PrintWindow is noticably slow however. But if the DWM is turned on via Aero theme in Windows, then you can simply GetDC and then GetPixel for nearly instantaneous results, since the DWM has everything in memory already. But the GDIP library already manages all of this for you, if you care to check the code

Nonetheless, if someone wants to use class syntax instead of just calling the GDIP library directly, then I suppose this has value

User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 06:21

Jeez, someone got out of bed on the wrong side this morning, no wondered you stayed anon.

Yes, all my class was ever intended to be is a wrapper for GDI.
Did you say to Tic - "Hey, GDI is nothing new - this is just a wrapper for DLLCall"?

Yes, there are DLL calls done through my class, but if you request a pixel, instead of just blindly making a DLL call like GDI would, my class checks if you requested that pixel already since the snapshot was last taken, and if so just returns the previously read value, thus saving you a DLL call. AKA it caches results.

Secondly - you are a little presumptuous thinking that everyone knows windows coding and API calls. I have no idea what DWM, GetDC, GetPixel etc is, and I doubt many of the people using my library will either.
Hell, for many people, even basic GDIP is too complicated - they do not understand the difference between a pBitmap and an hBitmap, and frankly don't care. All they want is a way to detect pixel colours that actually works and is easy to understand.

As an exercise, why don't we write a basic script?
The user wants to:
Check a 10x10 area of the screen and if it changes make a sound.

here's the code using my lib:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus



10 lines / 8 statements - let's see your best in pure GDI.
Then, as an exercise, we can ask future posters which version they find easier to understand.
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 06:30

Oh, and don't forget that your code must build an array of the pixel values for the source image, so you only make one DLL call per pixel...
Plus start up and shut down GDI properly, and dispose of the images after use.
guest3456
Posts: 2077
Joined: 09 Oct 2013, 10:31

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 07:41

lol, by their very nature, wrappers are going to be easier to understand. you hand pick a 10 line example, forgetting that you've written 300 lines of code behind that to get it to work. post your full 300 lines and lets see how easy to understand it is

i'm not here to prove anything to you. if you have no idea what gdip is or the various msdn functions, thats fine. i'm just saying that you're 'snapshot' idea is nothing new and is already done in gdi/gdip. i suppose storing the pixels in your own ahk array is a form of cache that will be marginally faster though, but require double the memory.

User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 08:26

But that's the whole point!
I haven't "forgotten" the 300 lines behind the lib - that's the argument!
You say it serves no purpose, yet agree that it turns what would normally take 300 lines of code into 10 ?!?

What is the point of a wrapper library if it doesn't simplify things.

The whole point of the library is to make it easy for people for whom PixelGetColor is slow / not working, but do not want to have to learn GDI in order to do pixel detection scripts. Yes, snapshots are not "new" from a GDI perspective, but they are new from a stock AHK perspective.
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 08:33

guest3456 wrote:i suppose storing the pixels in your own ahk array is a form of cache that will be marginally faster though, but require double the memory.

Intended use is mainly games, where people will be monitoring a small part of the screen. A couple of 10x10 or 100x100 bitmaps are not going to make much of a dent in a modern gaming system's RAM.
The primary concern here is speed, not memory. GDI is already not fast enough as to remove all constraints on time, so anything that can be done to squeeze out extra performance is worth it. < 2x the RAM (Bitmap is ARGB, cache is RGB) for 3x the read performance for subsequent reads is a no-brainer. Furthermore, only pixels that are read are cached, so it is only ~2x the RAM if you read the whole snapshot.
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

23 Jan 2015, 08:47

guest3456 wrote:i suppose storing the pixels in your own ahk array is a form of cache

Wikipedia wrote:In computing, a cache (/ˈkæʃ/ kash)[1] is a component that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparatively faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparatively slower.
Check.

Furthermore, if the cached data becomes invalid (snapshot for that object is re-taken), the cache is cleared.

ONLY Pixels that were explicitly inspected by the user are cached (Not all Pixels).

How does this not fit the definition of a cache perfectly?

guest3456 wrote:that will be marginally faster though

Greater than 2x speed is not "marginal".

evilC wrote:Grab a pixel from the snapshot
GDI_PixelSnap
0.000080

Repeat same pixel grab, check cache
GDI_PixelSnap
0.000020

...

First check of pixel again
GDI_PixelSnap
0.000048

Second check of a pixel again
GDI_PixelSnap
0.000019

...

Compare two 10x10 snapshots
GDI_Compare_Snap_10x10
0.006371

Repeat compare - should be nice cache hit
GDI_Compare_Snap_10x10
0.002203
drbent15
Posts: 3
Joined: 28 Jan 2015, 10:47

Re: CGDipSnapShot - GDI replacement for PixelGetColor

28 Jan 2015, 11:19

Thanks for this, the compare is just what i was looking for. But I can't seem to get it to work, the return value is always "0'. I tried


#include <CGdipSnapshot>
snap1 := new CGDipSnapshot(1,1,10,10)
snap2 := new CGDipSnapshot(1,1,10,10)
snap1.TakeSnapshot()
Loop {
snap2.TakeSnapshot()
if(!snap1.Compare(snap2),0){
soundbeep
}
}

But no beep.

So I changed the code so to make sure the snaps where different so I save the snap as file.

#include <CGdipSnapshot>
snap1 := new CGDipSnapshot(1,1,10,10)
snap2 := new CGDipSnapshot(12,12,10,10)
snap1.TakeSnapshot()
snap1.SaveSnapshot("img1.bmp")

Loop {
snap2.TakeSnapshot()
snap2.SaveSnapshot("img2.bmp")
if(!snap1.Compare(snap2),0){
soundbeep
}
}

Please help, Thanks
User avatar
evilC
Posts: 3593
Joined: 27 Feb 2014, 12:30

Re: CGDipSnapShot - GDI replacement for PixelGetColor

28 Jan 2015, 11:27

To compare ONE pixel WITHIN a snapshot to another pixel in another shapshot:
snap1.PixelSnap[x,y].Compare(snap2.PixelSnap[x2,y2])

To compare ONE pixel WITHIN a snapshot to a hard-coded value (red in this example):
snap1.PixelSnap[x,y].Compare({r: 255, g: 0, b: 0})

To compare a WHOLE snapshot to another WHOLE snapshot:
snap1.Compare(snap2)
drbent15
Posts: 3
Joined: 28 Jan 2015, 10:47

Re: CGDipSnapShot - GDI replacement for PixelGetColor

28 Jan 2015, 18:39

Thanks for reply....

I'm trying to compare WHOLE snapshots, snap1 to snap2. When I run this code it displays a value of "0" even those I know that the snaps are different. Everything else works, the snaps are taken and the snaps are save in files. I don't know what i'm doing wrong or my setup my be the problem (win 8 pro 64bit) AHK 1.1.15.04 Unicode 32bit.


#include <CGdipSnapshot>
snap1 := new CGDipSnapshot(1,1,10,10)
snap2 := new CGDipSnapshot(200,200,10,10)

snap1.TakeSnapshot()
snap2.TakeSnapshot()

snap1.SaveSnapshot("img1.bmp")
snap2.SaveSnapshot("img2.bmp")

msgbox % snap1.Compare(snap2,0)
Last edited by drbent15 on 28 Jan 2015, 18:49, edited 1 time in total.

Return to “Scripts and Functions”

Who is online

Users browsing this forum: No registered users and 8 guests