Getting status of Metro apps

[tt_1111]

It is no problem to launch a Metro app in AHK; this topic is extensively handled in other threads of this forum.

My problem is: The AHK WindowSpy (and consequently AHK itself) doesn't seem to get any status info at least from specific Metro apps. E.g. Windows Defender (or Windows Security as it's called in v1809). The title is about the only information AHK can get about this app. No visible difference when updating or scanning is in progress / done.

Thus when I want a scheduled GUI update + scan of Windows Defender, AHK can call the program. I can program certain clicks to get the update going. Even though the difference is clearly visible in Defender's window, AHK doesn't seem to get any feedback whatsoever regarding changes within this window, thus I have to wait. Wait quite some time. Then some other clicks to initiate the actual scan. And again wait. A looooong time this time.

I know how to do a defender cmd scan in old DOS prompt, that's not the problem, too. My question and problem simply is: Is there any easy way to control and govern Windows Metro apps by AHK according to their actual state shown in the actually visible window?

It appears odd to me that there seemingly ain't no elegant way to do this. Even Metro apps produce some output a user can read and interact with. And I honestly don't believe this output is encrypted or something like pure pixel trash that forms a visible form only to the human eye, not to any internal mechanisms.

If I'm overlooking something (some hidden switch to detect Metro behaviour?) / doing something wrong I'd be glad for any hints.

[tt_1111]

The problem is still relevant after five years, and I still haven't found an easy answer anywhere on the web.

Thus I'm asking the question again: Is there a way for AHK to get the actual status of Windows Apps (as Metro Apps are called in the meantime) from the informations displayed in the app's window? It clearly is visible to the human eye, but apparently not to AHK ?!?

[boiler]

If the status is repeatable (i.e., something visually identical to a previous status), you could use ImageSearch to see when a particular status appears. You could try one of the various OCR (optical character recognition) packages, for which there are AHK wrappers, to grab the actual text. You could see if the status is revealed through "Accessible Viewer" info. More on that here, here, and here.

[tt_1111]

MSAA unfortunately has been discontinued in 2016; download neither of source code nor of executable isn't possible any more.

Regarding OCR on a modern app just running — of course it's possible, but in a fashion: Outch! Feels somehow like a shot from behind through the chest right into your eye (i.e. very complicated).

Thanks for the hint regarding ImageSearch. Requires some preparation — screenshot of status specific output text or button, but at least is a chance to get the job done without too much hassle.

[tt_1111]

Just wanted to give a short update after having tried it out.

The good news: ImageSearch works also with Metro (Modern) Apps. I've been able to run the complete procedure (calling the app, updating, scanning) via an AHK script.

It seems to be best to search the whole screen. You could activate Defender and run a WinGetPos beforehand, but the results of this call of course are always 'absolute' (measured from the screen's, not the actual window's upper left corner), thus a

Code: Select all

CoordMode, Pixel, Screen

would be required before starting ImageSearch.

As the Defender windows contain an 'Abort' button during pending actions (update, scan) this can be queried within a loop until the button isn't found any more. Thus extensive waiting till an action completes even under worst premises is dispensable, too. When the 'Abort' button can't be found any more you break the loop, maybe sleep another 1000 msec, and that's it.

Not so good: Every single state (window) needs its own screenshot. Which of course is language specific. And if the pix change only a little (e.g. within the scope of altering the system font — or a program update) the whole preparation (screenshots) needs to be done again.

Résumé: It works. It's arduous.

Any measure to simplify the task of interacting with modern apps would be highly appreciated.