Automate app with no (or hidden) controls?

Get help with using AutoHotkey and its commands and hotkeys
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Automate app with no (or hidden) controls?

14 Apr 2018, 01:49

Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected
John Mile

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 04:10

apoklyps3 wrote:Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected


jesswg might know a way via acc
Guest

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 04:35

Two options probably:

* you can send mouse clicks to specific coordinates to activate controls (relative to the Gui so check the CoordMode command). If the controls move around when resized you could try ImageSearch (which should be pretty fast if you can narrow it down to your Window position not your entire desktop)

* you may be able to send shortcuts, for example a "new" button may have the n underlined meaning sending Alt-n will press the new button
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 08:40

John Mile wrote:
apoklyps3 wrote:Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected


jesswg might know a way via acc


escuse my stupidity but what does "Acc" stand for?
User avatar
jeeswg
Posts: 4279
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 09:32

I would recommend that you start by trying out the AccViewer GUI window spy, to see the information that it can retrieve. See also the comments at the start of tutorial below:

jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=7&t=40590
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 10:55

are these still compatible with the latest version of autohotkey?
i have cread lib folder in as it wasn't event there.
placed acc.ahk and anchor.ahk
when i run AccViewer I get error at line 133
User avatar
jeeswg
Posts: 4279
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 12:13

- I just tested the scripts on the latest version of AutoHotkey (v1.1.28.02), and they appeared to work fine.
- Can you state the text of the error message, thanks.
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 14:47

err.png
err.png (4.49 KiB) Viewed 308 times
err.png
err.png (4.49 KiB) Viewed 308 times
swagfag
Posts: 494
Joined: 11 Jan 2017, 17:59

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 15:49

??? did u just copy paste the page source into an ahk file

https://github.com/Drugoy/Autohotkey-sc ... Viewer.ahk
User avatar
jeeswg
Posts: 4279
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Automate app with no (or hidden) controls?

14 Apr 2018, 17:42

I've provided a zip called AccPack.zip which you can download here:
Acc library (MSAA) and AccViewer download links - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=26201
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

15 Apr 2018, 05:46

getting somewhere :)
could any of the objects be hidden using this?
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

15 Apr 2018, 15:58

is it possible to hide objects within an application's gui using acc and autohotkey?
or only automate the app?
been reading the tutorials, but still can make sense of them so far. it's chinese to me lol
all I got so far is that Acc is winspy on steroids.
User avatar
jeeswg
Posts: 4279
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Automate app with no (or hidden) controls?

17 Apr 2018, 12:29

I don't know of Acc being able to hide controls.
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

17 Apr 2018, 16:43

Ok.
Thanks.
Much apreciated
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

21 Apr 2018, 04:06

going forward does Acc work with hidden windows and if so what would be the code to wait for a button (let's say it's object 1.3 in acc basic) to show up and then press it?
User avatar
jeeswg
Posts: 4279
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Automate app with no (or hidden) controls?

21 Apr 2018, 05:00

AFAIK Acc works on hidden windows. I'd use WinExist/WinWait with DetectHiddenWindows set to on to check/wait for a window's existence (I'd expect that any controls/GUI elements would also exist).
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

21 Apr 2018, 06:04

no controls except for the main window class Qt5QWindowIcon.
other than that just objects detected with Acc basic.
I have no ideea at all how to start with Acc ...
User avatar
Frosti
Posts: 50
Joined: 27 Oct 2017, 14:30
Facebook: They have no decency!
Google: They have no decency!
GitHub: Ixiko
Location: Somwhere between Mars and Earth
Contact:

Re: Automate app with no (or hidden) controls?

21 Apr 2018, 09:52

The manufacturer says about QT(http://doc.qt.io): "Qt is designed for developing applications and user interfaces once and deploying them across several desktop and mobile operating systems.".

At least I tried to read information from a QT based desktop app(an interface for a drug-database). The best usable data I got with the UISpy.exe from Microsoft.
More beautiful is Inspect.ahk. Both (the exe and the script) based on Microsofts UIAutomation framework. QT is a different framework. To develope a QT-Application for windows desktop,
they wrote that for some platforms there must be some adjustments. I am not a programmer. I only discovered Autohotkey last year and found that it's easy with it to automate large and complex workflows.
In the meantime, I am happy that the software on my work is not state-of-the-art. It's enough for me that the software consists of 4 different Microsoft frameworks. The first edition was in the year 1995.
One part - the newest additional part and unfortunately too - was devoloped in QT. As I said it's only a user-interface for a drug-database but in QT they need about, I stopped counting at 200+ visible and
hidden windows, to build it. QT is scriptable with Javascript, COM and more, but it must be enabled (it must be a part of your app). Look at the following spoiler.
With UIA, mostly you find unnamed Controls, but at least you always get a BoundingRectangle. You can collect all BoundingRectangles and use this positions for SearchImage. I think its the fastest way.
And moving of elements is no problem anymore. If you can identify controls, they have mostly a fixed axis (x oder y). So this axis I would use to identify groups. I've done a lot of interface automation since Sep.17
and learned that it's more efficient to build a database about all controls of the visible interface I want to control as somehow to go on to program.

But if you have a collection of 200+ windows and thousands of custom controls like me, you must be a masochist to try it further with Qt.

Spoiler


I do not want to deprive you all of the poente. The slogan of QT is: "The future is written with Qt!". No! It would be more correct: "In your future, it will never end to write with QT!

Hope, I can help you a little bit and did not scare you with my bad english.
apoklyps3
Posts: 89
Joined: 13 Feb 2016, 13:20

Re: Automate app with no (or hidden) controls?

22 Apr 2018, 16:19

Your english is more than ok, but I didn't get anything that could help me start with my script...

Return to “Ask For Help”

Who is online

Users browsing this forum: Bad husband, FanaticGuru, grimboto, joshatt, Spikea and 26 guests