9 replies to this topic

[Lexikos]

Active Accessibility 2.0 Software Development Kit

Microsoft Active Accessibility 2.0 SDK Tools include Accessible Event Watcher, Accessible Explorer, and Inspect Objects.

The Accessible Explorer allows you to browse all accessible objects, which will give you an idea of the capabilities of Active Accessibility.

What's extremely interesting: Accessible Explorer shows that practically every element of a web page is accessible. It also seems the entire UI of Firefox is accessible (menus, toolbars, menu items, etc.)

[Sean]

Microsoft Active Accessibility 2.0 SDK Tools include Accessible Event Watcher, Accessible Explorer, and Inspect Objects.

This is interesting. I did a quick try to mimic them, and it worked fine.
I think Accessible Explorer and Inspect Objects could be done with AHK alone. Accessible Event Watcher may be also done with the help of WinEventHook.dll.

[Lexikos]

When you say "AHK alone", do you mean "AHK alone", or "AHK alone with Active Accessibility" ? If you meant without AA, (how) can AHK detect the individual hyperlinks on web pages? (AA can.)

[Sean]

When you say "AHK alone", do you mean "AHK alone", or "AHK alone with Active Accessibility" ?

Of course with MSAA.
BTW, wineventhook.dll looks like designed to be used only for OutOfContext, i.e., no dll-injection, so Accessible Event Watcher can't be completely replaced with AHK.

[Lexikos]

I did a quick try to mimic them, and it worked fine.

So you managed to use MSAA from AHK? Could you post your/some sample code? I've been trying to use it with CoHelper, but am not having much success (probably due to my limited understanding of COM or CoHelper.)

[Sean]

Here is a sample code. It'll stop at level one, i.e., no recursion into children of children.
Please test it while firefox is running.

CoInitialize()
hModule := DllCall("LoadLibrary", "str", "oleacc")

hWnd := WinExist("ahk_class MozillaUIWindowClass")
DllCall("oleacc\AccessibleObjectFromWindow", "Uint", hWnd, "Uint", 0, "Uint", GUID4String(IID_IAccessible,"{618736E0-3C3D-11CF-810C-00AA00389B71}"), "UintP", pacc)

MsgBox, % "<Parent>`n"
	. "Children:`t" . (nCount:=AccessibleChildren(pacc, varChildren)) . "`n"
	. "Name:`t`t" . Invoke(pacc, "accName") . "`n"
	. "Role:`t`t" . GetRoleText(Invoke(pacc, "accRole")) . "`n"
	. "State:`t`t" . GetStateText(Invoke(pacc, "accState")) . "`n"
	. "Description:`t" . Invoke(pacc, "accDescription") . "`n"
	. "Value:`t`t" . Invoke(pacc, "accValue") . "`n"
	. "Help:`t`t" . Invoke(pacc, "accHelp") . "`n"
	. "Keyboard:`t" . Invoke(pacc, "accKeyboardShortcut") . "`n"
	. "DefAction:`t" . Invoke(pacc, "accDefaultAction") . "`n"
Loop,	%nCount%
	MsgBox, % "Children:`t" . Invoke(paccChild:=NumGet(varChildren,16*(A_Index-1),"Ushort")=9 ? NumGet(varChildren,16*(A_Index-1)+8) : Invoke(pacc, "accChild", NumGet(varChildren,16*(A_Index-1)+8)), "accChildCount") . "`n"
		. "Name:`t`t" . Invoke(paccChild, "accName") . "`n"
		. "Role:`t`t" . GetRoleText(Invoke(paccChild, "accRole")) . "`n"
		. "State:`t`t" . GetStateText(Invoke(paccChild, "accState")) . "`n"
		. "Description:`t" . Invoke(paccChild, "accDescription") . "`n"
		. "Value:`t`t" . Invoke(paccChild, "accValue") . "`n"
		. "Help:`t`t" . Invoke(paccChild, "accHelp") . "`n"
		. "Keyboard:`t" . Invoke(paccChild, "accKeyboardShortcut") . "`n"
		. "DefAction:`t" . Invoke(paccChild, "accDefaultAction") . "`n"
		, Release(paccChild)

Release(pacc)
DllCall("FreeLibrary", "Uint", hModule)
CoUninitialize()


AccessibleChildren(paccContainer, ByRef varChildren)
{
	cChildren := Invoke(paccContainer, "accChildCount")
	VarSetCapacity(varChildren, 16*cChildren, 0)
	DllCall("oleacc\AccessibleChildren", "Uint", paccContainer, "int", 0, "int", cChildren, "Uint", &varChildren, "intP", cObtained)
	Return	cObtained
}

GetRoleText(nRole)
{
	nSize := DllCall("oleacc\GetRoleTextA", "Uint", nRole, "Uint", 0, "Uint", 0)
	VarSetCapacity(sRole, nSize)
	DllCall("oleacc\GetRoleTextA", "Uint", nRole, "str", sRole, "Uint", nSize+1)
	Return	sRole
}

GetStateText(nState)
{
	nSize := DllCall("oleacc\GetStateTextA", "Uint", nState, "Uint", 0, "Uint", 0)
	VarSetCapacity(sState, nSize)
	DllCall("oleacc\GetStateTextA", "Uint", nState, "str", sState, "Uint", nSize+1)
	Return	sState
}

#Include CoHelper.ahk

PS. I deliberately omitted a few which are using VT_BYREF parameters, as have to use Invoke_() for these.

MsgBox, % Invoke_(pacc, "accHelpTopic", 0x4008, "") . ":`t" . _TEMP_VT_BYREF_1
Invoke_(pacc, "accLocation", 0x4003, 0, 0x4003, 0, 0x4003, 0, 0x4003, 0)
MsgBox, % "(l,t,w,h):`t" . "(" . _TEMP_VT_BYREF_1<<32>>32 . "," . _TEMP_VT_BYREF_2<<32>>32 . "," . _TEMP_VT_BYREF_3 . "," . _TEMP_VT_BYREF_4 . ")"

The accLocation returns Left, Top, Width, Height. OTOH, Access Explorer converts them to Left, Top, Right, Bottom.

[Lexikos]

Ah, I had been misinterpreting the MSDN definitions:

HRESULT get_accRole(
  VARIANT varID,
  VARIANT* pvarRole
);

I was trying something like:

Invoke_(pacc, "get_accRole", 3, childId, 0x4003, "")
role := _TEMP_VT_BYREF_2

where it should apparently be:

role := Invoke(pacc, "accRole")

I must admit that is more readable. Thanks.

[majkinetor]

So, this means that we can get text of basicly any object on the screen, the way some translators do (for instance Babylon) that let you click on any word on the screen to get translation - no matter where it is (title, button, browser...)

Am I right ?

[Lexikos]

It seems that way. You might be interested in reading: How to retrieve text under the cursor (mouse pointer).

[majkinetor]

Thx lexikos.