Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

(closed) Help with Acc Viewer Project


  • Please log in to reply
13 replies to this topic
jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009
Thread is now Inactive. Please refer to Accessible Info Viewer.

Would anyone like to help with or take over an Acc Viewer project I started? The premise was to create a window info tool that would also gather Accessible Information. I wanted to be able to create a navigable tree-view that would allow the user to gather info about relative Acc objects. Also, I wanted to be able to parse from the selected object back up the to window object & populate the structure back down to the selected object. I believe this would open the ability to easily get data from those controls that cannot be accessed via the control commands. As it stands right now, my AHK involvement has drastically dropped off, so I probably won't finish this project. Let me know if you want to help.

NOTE - this project requires a relatively good understanding of the IAccessible Interface - particularly for fixing the bugs.

Mickers
  • Members
  • 1239 posts
  • Last active: Sep 25 2015 03:03 PM
  • Joined: 11 Oct 2010
I would if I knew jack about Acc. :oops:

I've noticed you've not been around much lately. What are you up to?

sinkfaze
  • Moderators
  • 6367 posts
  • Last active: Nov 30 2018 08:50 PM
  • Joined: 18 Mar 2008
Why don't you release it as a collaborative project like tank did with the IE Web Recorder?

jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009
tank's IE Web Recorder was already complete - his project was to further develop & enhance the recorder. In my situation, the viewer is still needs bugs worked out. I will probably still post the source for anyone who may want to help, but I need to go back & add a lot of comments so it makes sense.

Additionally, there may need to be some collaborative though process & brainstorming that needs to occur outside of posting in the forum.

tank
  • Administrators
  • 4345 posts
  • AutoHotkey Foundation
  • Last active: May 02 2019 09:16 PM
  • Joined: 21 Dec 2007
I have been waiting for so long for you to post this
I didnt want to start over from scratch
I have a wretch of a schedule and workload but honestly the results of this solve so many problems. I will help when and if i can.
Never lose.
WIN or LEARN.

sinkfaze
  • Moderators
  • 6367 posts
  • Last active: Nov 30 2018 08:50 PM
  • Joined: 18 Mar 2008
I'm not an expert on Accessibility but I've worked with it a few times, I'm game to help as time permits.

tank
  • Administrators
  • 4345 posts
  • AutoHotkey Foundation
  • Last active: May 02 2019 09:16 PM
  • Joined: 21 Dec 2007
As to the bugs. no it was far from complete and remains so. but i just never needed it to do more so i lost interest.

At least share the source code with me and Sinkfaze?

but i think an open development thread is the best plan. Would you 2 have responded as you did if i hadnt posted such
Never lose.
WIN or LEARN.

maul.esel
  • Members
  • 790 posts
  • Last active: Jan 05 2013 09:26 PM
  • Joined: 28 Feb 2011
I would be interested in the source ;-) and willing to contribute from time to time. However, I already have a lot of projects I'm working more or less on, so I might help or I might not. I see there are already some experts interested in it, it seems it's in good hands. :)

Additionally, there may need to be some collaborative though process & brainstorming that needs to occur outside of posting in the forum.

github :!: github :!:    :lol::lol:
Join the discussion on The future of AutoHotkey
Posted Image Visit me on github Posted Image
Win7 HP SP1 64bit | AHK_L U 64bit

jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009
I posted a link to the source in the original post. Once I find the time/ambition to properly comment the source code, I'll start a thread in Scripts & Functions (unless someone else can properly take over this project).

The primary issue that I have not yet found a way to resolve/work around/bypass is what I asked in this post. I added a modified copy of the Acc_EnumChildren() function to the bottom of the source code. If utilized, this will show a MsgBox when the Acc_Children() call fails.

Lexikos
  • Administrators
  • 9844 posts
  • AutoHotkey Foundation
  • Last active:
  • Joined: 17 Oct 2006

The primary issue that I have not yet found a way to resolve/work around/bypass is what I asked in this post.

When you retrieve the parent object, query for the IAccessible interface.
Parent := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
Note that IAccessible::get_accParent() returns an IDispatch interface pointer. Because IAccessible derives from IDispatch, an object which implements IAccessible doesn't necessarily need a separate implementation of IDispatch: it can simply return an IAccessible pointer when an IDispatch pointer is requested. There's absolutely no guarantee that this will happen, so you need to query for the appropriate interface.

tinku99
  • Members
  • 560 posts
  • Last active: Feb 08 2015 12:54 AM
  • Joined: 03 Aug 2007
Looks like you might have to deal with a similar issue getting children as well as parents: <!-- m -->http://msdn.microsof... ... p/dd317975<!-- m -->
Here are modified versions of ACC_EnumChildren and GetAccPath to use IAccessible instead of IDispatch.
Acc_EnumChildren(Acc) {		 
 	Children := []		 
 	Loop, % Acc_Children(Acc, Acc.accChildCount, varChildren){
           i := (A_Index-1)*(A_PtrSize*2+8)+8
 	   child := NumGet(varChildren,i)
           aChild := NumGet(varChildren,i-8)=3?child:ComObj(9,child,1)
	   accChild := ComObj(9, ComObjQuery(aChild
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)	
	   Children.Insert(accChild ? accChild : aChild)
}
	return, {	Items:		Children
			,	_NewEnum:	Func("Acc_Enum_NewEnum")
			,	Next:		Func("Acc_Enum_Next")	
			,	base:		{__Get:Func("Acc_Enum_Get")}	}
}


GetAccPath(AccObj) {
/* returns:
	object
		AccObj:	Acc Object for Window/Control
		Path:	* Delimited List of child positions for each Acc Level down to the object
*/
	
	; return object if it already is the window object
        parent := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
	parent := ComObj(9, ComObjQuery(parent.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)		
	if Acc_WindowFromObject(parent) != Acc_WindowFromObject(AccObj)
		return {AccObj:AccObj}
	
	Levels := []
	; get the Window Acc Object
	AccWinObj := Acc_ObjectFromWindow(Acc_WindowFromObject(AccObj))
	AccWinLocation := GetAccLocation(AccWinObj)
	; Parse up until you reach the Window Acc Object
	Loop {
		AccObjLocation := GetAccLocation(AccObj)
		if (AccObjLocation = AccWinLocation)
			break
		Levels.Insert(1,AccObjLocation)
	;	AccObj := AccObj.accParent
                AccObj := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
	}
	AccObj := AccWinObj
	; Parse back down to identify the Child Path
	For LevNum, Location in Levels
		for child,type in Acc_EnumChildren(AccObj)
			if (type=9 and GetAccLocation(child)=Location) {
				ChildPath .= A_Index "*"
				AccObj := child
				break
			}
	return {AccObj:AccWinObj, Path:SubStr(ChildPath,1,-1)}
}
I had opened a related issue on github. Some relevant links:
IUIAutomation: <!-- m -->http://msdn.microsof... ... p/ee671406<!-- m -->
bewildr using IUIAutomation in ironruby: <!-- m -->http://www.natontest... ... -ironruby/<!-- m -->
<!-- m -->https://github.com/GraemeF/Fluid<!-- m -->
<!-- m -->http://www.autoitscr... ... utomation/<!-- m -->

jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009
Thanks for the solution Lexikos :D , & thanks for the example modifications tinku99. You guys make this seem so easy.

I suppose this means I get to reembark on this project :) :( :? :p


If anyone knows how to get the all ChildId's (to be used with get_accChild) of an object, that would be a great help. Acc_Children() can be used to get Child Element Id's, but not Child Object Id's, afaik.

Lexikos
  • Administrators
  • 9844 posts
  • AutoHotkey Foundation
  • Last active:
  • Joined: 17 Oct 2006

accChild := ComObj(9, ComObjQuery(aChild
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)   
      Children.Insert(accChild ? accChild : aChild)

I think you'll find that accChild will always be used, since ComObj() returns an object even if the pointer is NULL.

If anyone knows how to get the all ChildId's (to be used with get_accChild) of an object, that would be a great help.

Given that you don't have an ID of a child object in the first place, what use is get_accChild?

After reading the documentation for several IAccessible methods, I get the impression that only children which aren't objects have IDs. Even the documentation for get_accChild seems to confirm this (while possibly being contradictory):

Servers expose elements as either elements (child IDs) or full objects (IAccessible interface pointers).

Objects do not provide information about child objects; only child elements:

If a child is an element, get_accChild returns S_FALSE, and the parent will provide information for that child. If the child is a full object, get_accChild will return the IAccessible interface pointer and the parent will not provide information about that child.

So aside from get_accChild, there doesn't appear to be any reason for a child object to have an ID. If a child object has no ID, what use is get_accChild? If you rely only on MSDN, probably no use. However, MSDN only defines the interface; it is open to interpretation by the developer of each accessibility server. For instance, this bug report indicates Firefox uses child IDs from 1 to the number of children, and some of the examples in that thread rely on it.

jethrow
  • Moderators
  • 2854 posts
  • Last active: May 17 2017 01:57 AM
  • Joined: 24 May 2009

So aside from get_accChild, there doesn't appear to be any reason for a child object to have an ID. If a child object has no ID, what use is get_accChild? If you rely only on MSDN, probably no use. However, MSDN only defines the interface; it is open to interpretation by the developer of each accessibility server.

Thank you for looking into this. Based on this, I feel that the accChild method should not be encouraged as a standard way to access children.

Also, I wanted to be able to parse from the selected object back up the to window object & populate the structure back down to the selected object.

I think I finally accomplished this, with sinkfaze's help. Here is an example:
ComObjError(false)
F1:: ; access the object under then mouse, then re-access it from the window object
	PtObj := Acc_ObjectFromPoint(child)
	child_path := GetAccPath(PtObj, hwnd)
	WinObj := Acc_ObjectFromWindow(hwnd)
	
	ChildObj := Acc_GetChild(WinObj, child_path)
	
	MsgBox %	"--Window + Path--`nName:`t" ChildObj.accName(child) "`nValue:`t" ChildObj.accValue(child) "`nLocation:`t" Acc_Location(ChildObj, child).pos
			.	"`n`n`n`n"
			.	"--From Point--`nName:`t" PtObj.accName(child) "`nValue:`t" PtObj.accValue(child) "`nLocation:`t" Acc_Location(PtObj, child).pos
	return


Acc_GetChild(Acc, child_path) {
	Loop Parse, child_path, csv
		Acc := A_LoopField="P"? Acc_Parent(Acc):Acc_Children(Acc)[A_LoopField]
	return Acc
}
GetAccPath(Acc, byref hwnd="") {
	hwnd := Acc_WindowFromObject(Acc)
	WinObj := Acc_ObjectFromWindow(hwnd)
	while Acc_WindowFromObject(Parent:=Acc_Parent(WinObj)) = hwnd
		t1.="P,", WinObj:=Parent
	while Acc_WindowFromObject(Parent:=Acc_Parent(Acc)) = hwnd
		t2:=GetEnumIndex(Acc) "," t2, Acc:=Parent
	return SubStr(t1 t2,1,-1)
}
GetEnumIndex(Acc, ChildId=0) {
	if Not ChildId {
		ChildPos := Acc_Location(Acc).pos
		For Each, child in Acc_Children(Acc_Parent(Acc))
			if IsObject(child) and Acc_Location(child).pos=ChildPos
				return A_Index
	} else {
		ChildPos := Acc_Location(Acc,ChildId).pos
		For Each, child in Acc_Children(Acc)
			if Not IsObject(child) and Acc_Location(Acc,child).pos=ChildPos
				return A_Index
	}
}

The data should be the same, even though one was accessed via Acc_ObjectFromPoint & one via Acc_ObjectFromWindow + a child path. An exception may be if the object under the mouse doesn't expose much IAccessible information - such as a Google Chrome Webpage. Besides these types of exceptions, I would like to know if anyone identifies any IAccessible object that the above script provides differentiating data for.