FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » Today, 08:00

Hi JenJen,

Thank you for the thorough investigation of the issue you're having. I'm always appreciative if bugs can be found, and a clear explanation makes identifying them way easier.

I'm a little confused by the exact issue you're having however, although your sample code has given me an idea. If you are primarily using the last two parameters of the function (the byref FoundX and FoundY) to get your image information, you should be aware that those are ALWAYS relative to the screen regardless of the setting of "r". (This also applies to the returned value of the function.) So in your sample code, you’d need Coordmode, Mouse, Screen – either before MouseMove or globally at the start of your script – or else moving the window will have the effect you are talking about. (Coordinates relative to the window is the default for MouseMove.)

I know this isn’t very intuitive, but there was some reason I can’t recall now that caused me to originally code it like that. I was going to change it but never got around to it. At the very least I guess maybe I should make that quirk more clear in the documentation.

Anyway, please let me know if that doesn’t fix your issue. The "r" is really pretty simple and just uses a search area that matches the window coordinates, but even the simplest stuff breaks sometimes I guess.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by JenJen » Yesterday, 17:34

Hey berban,

I've been battling for a while now with this, and no matter how I try to define 'r' and give it an active window - it still uses the 'monitor' coordinates and apply them to X/Y.

An example of this is if you dont use fullscreen, and then search an image within your application. If you place your active window in the top right corner (I have 3 monitors though), it'll work fine. It'll get the coordinates on your main monitor and move the mouse to the image.

Issue comes if you move that non-fullsized active window somewhere else on your main monitor. Now, if you move your active window 300px to the right, the 900 coordinate becomes 1200, and obviously since your mouse is moving in the 'active' window, it'll move it to coordinate 1200, which is not where it's supposed to be (900).

I think the 'relative' function is broken, honestly. I've run out of ideas to test this, I've made it as simple as:

1. Open Application with an image in your want to click.
2.

Code: [Select all]GeSHi © Codebox Plus

F5::
{
FindClick(A_ScriptDir . "\images\icon.png", "r""Application"" n", x, y)
MouseMove, x, y
MsgBox, x:%x%, y:%y%
}

3. Move the active application (non full screen) around
4. Watch the mouse move all over the place.

Any help would be appreciated, thanks!

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 03 Sep 2018, 18:33

Well I can't walk you through the entire process but I did make a "Quick-Start Guide" chapter in the documentation that you should check out. You should also read the entire chapter on AutoHotkey Functions in the AutoHotkey documentation, as I also point out in the FindClick guide.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by WaterZephyre » 03 Sep 2018, 16:52

Can you tell people how to use this? It took me a good hour of searching to find out how to do it, I just started using ahk so I didn't know about the include command. Thank you for this program, it helps a lot.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by carno » 05 Aug 2018, 09:17

Thanks again. I tried both and they work great, but I now use only the second version:

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "e n2")
and

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "n2")

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 04 Aug 2018, 18:54

Yeah it’s

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "e n2")

(Maybe you know that already but just wanted to repeat it in case.)

Let me know if you have any other questions!

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by carno » 04 Aug 2018, 14:24

Hi, berban, thanks very much! I use your work in several of my apps and they all work great. I read your FindClick Documentation and it mentions option "n" but I was trying 2 (number of clicks) incorrectly like this:

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "e 2")
.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 03 Aug 2018, 07:38

Hi carno, sorry for the delay. You are right there is a simple way - use the "n" option.

FindClick documentation, page 9 wrote:n – Number of clicks
What to give: any integer
Description: The number of times to click on each image.


Use it like this:

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "e n2")


If you want to vary the amount of time between clicks, you can change that with the "Sleep" option.

Code: [Select all]GeSHi © Codebox Plus

FindClick(Image, "e n2 Sleep100") ; will sleep exactly 100 milliseconds like in your code


(By the way, "e" means click on all instances. If this is intentional that's fine, but if you only expect to ever find one you might want to take it out because it slows the script down a bit.)

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by carno » 31 Jul 2018, 00:57

I am using FindClick in the code below to double click on an Image. Is there a better option to put this in one line? I tried FindClick(Image, "e 2"), FindClick(Image, "2 e"), FindClick(Image, 2,"e"), etc., but none worked!

Code: [Select all]GeSHi © Codebox Plus

Image := "C:\Documents and Settings\User\My Documents\Someimage.png"
FindClick(Image, "e")
Sleep, 100
FindClick(Image, "e")
Return

I posted this here also but was of no help: https://autohotkey.com//boards/viewtopi ... =5&t=52965

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 25 Jun 2018, 18:38

Glad it works. Wish I had room for 3 monitors haha. Happy scripting! :D

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by daffy82 » 25 Jun 2018, 01:56

berban wrote:Hm interesting, well you could try a few things like adding "stay" so it doesn’t move away, or "n2" for doing a double click. But maybe the way you were originally doing it is more reliable. If it is clicking in the right spot and the window isn’t responding correctly that’s probably not the best way to approach it


Nice! With "stay1" and this: I got this part working!

Now I believe I can make the whole script! Thanks! :bravo: :dance:


EDIT: Now my script is complete. I have 3 monitors and this script does not give a damn where the dialog box is - it just works! Thanks to Berbans FindClick script. Man this will stay in my toolbox for life!

Again thanks alot! :D

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 22 Jun 2018, 08:55

Hm interesting, well you could try a few things like adding "stay" so it doesn’t move away, or "n2" for doing a double click. But maybe the way you were originally doing it is more reliable. If it is clicking in the right spot and the window isn’t responding correctly that’s probably not the best way to approach it

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by daffy82 » 22 Jun 2018, 06:06

berban wrote:Thanks for the helpful gifs! So I'm still not completely sure why it wouldn't work but my suggestion would be that for the ribbon one, I generally use the alt keys for those. For instance, in Microsoft Word, if I want to access the ribbon I just press Alt and the accelerator keys show up in little black tooltips:

Press "alt":
Image
Press "h" for home tab:
Image
Press "ff" for font (font face, I assume)

So if I want to change the font to Arial, it'd be something along the lines of

Code: [Select all]GeSHi © Codebox Plus

Send !hffArial{enter}

This is incredibly reliable in office apps. You can probably do something similar for the outlook one. Maybe alt+h+a to get to the archive button, then {down} or {down 2] to go to the correct option and then {enter} to select it.

Also by the way the red shading thing isn't a problem, when you click the archive button it creates a new menu window which goes over the red window from AutoHotkey.

Similarly, for the dropdown one I'd try using Send to press {tab} a bunch of times until you get focus to the right input, then send {down} to make the dropdown menu appear. It might be more reliable. Either that or use findclick to search for the little black dropdown arrow instead - in this case, the one you want is the top one so it'll be the one that gets clicked.



After you wrote I changed the script to TAB to enter the dialog box from the ribbon menu (genius idea btw :D) and I took some new pictures with FindClick and now the script finds the images! Hurray! Thanks alot!

Now I have another question (sorry)...

See this gif: https://i.imgur.com/WhMuhAF.gifv

As you can see the dropdown arrow turns blue'ish when i hover the mouse pointer over it. But the script points to the dropdown arrow SO fast that the dropdown arrow dos'nt get activated. I can see the mouse pointer is the right place but its too fast for the dropdown menu... Funny thing is that it works ca. 1/10 times.

I tried with but no difference. Im a beginner to AHK i should say. I think this one must be solvable quite easy but since im a newbie I dont know where to look. I know this has nothing to so with FindClick but you seem like a AHK guru so I thought Ill ask you :)

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 21 Jun 2018, 03:45

Thanks for the helpful gifs! So I'm still not completely sure why it wouldn't work but my suggestion would be that for the ribbon one, I generally use the alt keys for those. For instance, in Microsoft Word, if I want to access the ribbon I just press Alt and the accelerator keys show up in little black tooltips:

Press "alt":
Image
Press "h" for home tab:
Image
Press "ff" for font (font face, I assume)

So if I want to change the font to Arial, it'd be something along the lines of

Code: [Select all]GeSHi © Codebox Plus

Send !hffArial{enter}

This is incredibly reliable in office apps. You can probably do something similar for the outlook one. Maybe alt+h+a to get to the archive button, then {down} or {down 2] to go to the correct option and then {enter} to select it.

Also by the way the red shading thing isn't a problem, when you click the archive button it creates a new menu window which goes over the red window from AutoHotkey.

Similarly, for the dropdown one I'd try using Send to press {tab} a bunch of times until you get focus to the right input, then send {down} to make the dropdown menu appear. It might be more reliable. Either that or use findclick to search for the little black dropdown arrow instead - in this case, the one you want is the top one so it'll be the one that gets clicked.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by daffy82 » 21 Jun 2018, 00:38

berban wrote:Hi daffy82, sorry but I can't tell you based on what you've posted. Nothing in that code would prevent it from working correctly.

A few comments I could give you:
  1. k{click} and Center1 are not needed. The default behavior is to click at the center of the image.
  2. Try using the dx option to help diagnose what is wrong. FindClick("2archive", "dx") Run it and scroll through the items in the GUI to see if anything looks suspicious
  3. x300 means it'll click 300 pixels to the right of the image. Is that what you intended?
  4. It's possible that the first click changes the look of things onscreen - for instance, when you click on a menu it might cover up another part of the screen. Make sure that isn't happening here by making sure you take the screenshot images when the screen looks how it'll look while the script is running.



Hi berban.

Thanks for your reply and thanks for making this script. Its very nice and I love it!

1. ok
2. I will and post update if I find anything
3. Yes thats exactly what I need. Script looks for a text box (Company) and then moves 300px to the right and then clicks a dropdown arrorw - gif: https://i.imgur.com/3NQYB5B.gifv (this part works if I run it by itself)
4. The screen does change when the script clicks the first time... It clicks on an outlook ribbon. I made a gif so you can see what i mean: https://i.imgur.com/zISpdZP.gifv

Edit: result of the debugger https://justpaste.it/69e9a

Edit 2: I have 3 monitors so the debugger shows 3 Searches. When I click on a search the screen turns red. When the screen is red I click the ribbon manually and the popup menu is not red. I dunno if thats a problem? A gif showing this: https://i.imgur.com/hfmheYS.gifv

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 17 Jun 2018, 10:38

Hi daffy82, sorry but I can't tell you based on what you've posted. Nothing in that code would prevent it from working correctly.

A few comments I could give you:
  1. k{click} and Center1 are not needed. The default behavior is to click at the center of the image.
  2. Try using the dx option to help diagnose what is wrong. FindClick("2archive", "dx") Run it and scroll through the items in the GUI to see if anything looks suspicious
  3. x300 means it'll click 300 pixels to the right of the image. Is that what you intended?
  4. It's possible that the first click changes the look of things onscreen - for instance, when you click on a menu it might cover up another part of the screen. Make sure that isn't happening here by making sure you take the screenshot images when the screen looks how it'll look while the script is running.

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by daffy82 » 15 Jun 2018, 01:09

Question:

Why does my script only find the very first image? It does not click on 2nd or 3rd img:

Code: [Select all]GeSHi © Codebox Plus

#Include C:\Scripts\AHK\berbar_image_search.ahk
!t::
FindClick()
!k::
FindClick("ribbon", "k{Click}")
sleep 2000
FindClick("2archive", "Center1 k{Click}")
sleep 2000
FindClick("NewFile", "x300 k{Click}")
return

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by daffy82 » 14 Jun 2018, 07:21

Is it possible to find an image on screen and the click on that image but in the far right side of the picture? (I want the script to click the little arrow: https://i.imgur.com/duhGhZC.png to activate the dropdownmenu

Found the solution (offset)

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by berban » 29 May 2018, 14:08

Hi dice4321,

So the way AutoHotkey's ImageSearch command works, it basically scans the visible screen area for the image, which is essentially the same area that you can see with your two eyes. If the image is hidden behind another window and you can't physically see it, then this won't work. You'd need to use another method.

If, on the other hand, you just mean the window is not the ACTIVE window but still visible somewhere on the screen, then you can definitely use FindClick. You can either just search the entire screen (remove "r") or you can search ONLY the borders of that other window by defining it with the r option.

Code: [Select all]GeSHi © Codebox Plus

FindClick("image.png", "r""My window title""") ; searches relative to the window with the title "My Window Title", based on your TitleMatchMode settings
FindClick("image.png", "r" WinExist("My Window Title")) ; you can also use a HWND via WinExist. I prefer this way because you don't have to escape the quotation marks
FindClick("image.png", "r" WinExist("My Window Title") " mControlClick") ; use sendmode = controlclick to avoid activating that other window when a click is sent

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post by dice4321 » 28 May 2018, 19:16

The function has worked like magic so far thanks berban!

I have a question tho. I have the r option on, but for the function to work, it seems to be, the window always has to be on top else the function wont find the image, is there a way to find the image in a window behind other windows?

Top