Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

How to send to unseen controls in a Java app

Started by califauna , Jul 15 2013 01:44 AM

  • Please log in to reply
47 replies to this topic
califauna
  • Members
  • 23 posts
  • Last active: Jul 22 2013 11:59 AM
  • Joined: 02 May 2011

Hi all,

 

I am trying to automate controlling a java app  (Jajuk Advanced Jukebox - opening panels, entering text, clicking buttons/areas etc) but none of the window spy type apps can see any controls on the GUI (Ive tried every one I can find, Windows Spy, COntrol Viewer, Au3Info, Windows detective etc)

 

I have installed and run the Java  Ferret and Java Monkey apps (after installing access bridge 2.0.2)  and its giving a tree structure with an expanding list of panes and panels leading to elements in square brackets like [text] corresponding to where text / numbers are entered, and for example  a [scroll bar] item opening  to a [pushbutton] item. It also gives info like:

 

AccessibleActions info:
    Number of actions:  56
    Action 0 name: page-up.....etc

 

Java Ferret tells me things like parent windows, and 'descendants' like:

   

    Descendent 1 name:  expand
    Descendent 1 role:  label

 

and loads of other info.

 I dont know what half of this info refers to though, or the the next step.  i.e how to send messages to these elements.

 

Also, I have so far been unable to track messages sent to the GUI using Windows detective or UISpy , as none of the interaction with the GUI is being captured (the output window for Windows detective remains empty)

 

Id like to send to the panel and controls using windows messages and control send message if possible rather than resorting to controlling with mouse clicks and pixel colour recognition etc.

 

Can anyone offer some help?

 

Cheers.

 

Screen shots of apps mentioned.

 

http://sdrv.ms/12KyNzp

http://sdrv.ms/12Kz5pR

http://sdrv.ms/12KzdFK

http://sdrv.ms/12KzwAD


#1 - Posted 15 July 2013 - 01:44 AM
califauna
  • Members
  • 23 posts
  • Last active: Jul 22 2013 11:59 AM
  • Joined: 02 May 2011

Update : I now have Windows Detective working and showing me the messages sent and from the app and need to figure out which of the many windows messages I need to send, and how build the lparam and wparam parameters using the values in the captured messages.


#2 - Posted 17 July 2013 - 09:20 PM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Hi there!
 
You need to use the Java Access Bridge to go around in Java applications. Windows messages won't get you anywhere.
 
I have started to write an AHK wrapper for the JAB a while ago and while it's nowhere complete (text, table and many other functions are missing) it is sufficient to manipulate many applications.
 
You'll need the following:
The file "JavaAccessBridge.ahk"

Spoiler

 
The file "JControlWriter.ahk" which is a simple control browser. Press Ctrl-Alt-F10 or F11 while you are in a java window to display a list of controls.
Spoiler


And the file "JCWGUIStrings.ini" which contains the strings used by the control browser:
Spoiler

 

Here is an example code for controlling the SwingSet3 demo (http://download.java.../SwingSet3.jnlp):

Spoiler

Java applications can sometimes behave strangely. For instance when you use an action to call a function that opens a dialog, this call may not return until the dialog is closed again, effectively freezing your script. Use the "left click" action in those cases.
In any case: use at your own risk.

Cheers.


#3 - Posted 18 July 2013 - 11:36 PM
burton666
  • Members
  • 186 posts
  • Last active: Jul 05 2017 05:35 AM
  • Joined: 10 Aug 2012

Is the .ini file the only thing I need to edit to get this running? Could you provide some example of configuration?


#4 - Posted 17 January 2014 - 09:09 AM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Hi burton666!

 

I've edited my post above to fix a bug in JavaAccessBridge.ahk and include some example code for controlling a Java app.

If you save all four files into the same directory it should work.

 

The .ini just contains the strings for the gui in JControlWriter.ahk.

 

Elgin


#5 - Posted 17 January 2014 - 09:34 PM
burton666
  • Members
  • 186 posts
  • Last active: Jul 05 2017 05:35 AM
  • Joined: 10 Aug 2012

I want to send controls and retrieve text froma java application (Oracle applications). Is there some way to list all the elements like show source on a webpage?


#6 - Posted 21 January 2014 - 01:00 PM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

To get a flat list of all element you can use the functions:

 

GetVisibleChildrenFromTree(vmID, ac)
GetAllChildrenFromTree(vmID, ac)

 

to get all or only the visible gui elements. See JControlWriter.ahk on how to use them.

 

If you want to go through the element tree yourself you can do that with:
GetAccessibleContextInfo(vmID, ac) ; gets you the number of child elements among other things

GetAccessibleChildFromContext(vmID, ac, index) ; returns the ac-value of the child element
 

Getting text from an element is possible but currently not implemented in JavaAccessBridge.ahk.

The dll functions you need for that are:

getAccessibleTextInfo ; to retrieve the number of chars and

getAccessibleTextRange ; to get the actual text.

The definitions (from Delphi) are in the comments in JavaAccessBridge.ahk or on Oracle's webserver along with the rest of the Java access bridge documentation.

I'm a bit short on time these days but maybe I can put up some code for that next week...


#7 - Posted 22 January 2014 - 08:32 PM
GodlyCheese
  • Members
  • 719 posts
  • Last active: Nov 11 2014 07:12 PM
  • Joined: 30 Aug 2012

Would these functions work for reading object data in a java-based browser game? If not, then do you know of anything that might?


#8 - Posted 22 January 2014 - 09:58 PM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

These functions only read out gui elements that support Java accessibility. Games usually custom draw their gui and don't care about supporting accessibility.

You can try if you can get something out of it but I'd say your chances to get more than an empty frame are very low.

 

For games you're usually stuck with pixel search and ocr :-(


#9 - Posted 23 January 2014 - 09:32 AM
carry
  • Members
  • 1 posts
  • Last active: May 19 2014 05:38 AM
  • Joined: 19 May 2014
Elgin, thanks for posting your JavaAccessBridge wrapper code. It works brilliantly and is much easier than trying to do the same job in C++.
 
One comment about the 'JavaAccessBridge.ahk' code: On my installation (AHK Unicode 64-bit v1.1.15.00, Win7) I needed to change code in 'GetAccessibleActions' and 'DoAccessibleActions' from 'offset:=(A_Index-1)....' to 'offset:=(A_Index-3)....'.
I don't understand the code well enough to understand why this works, but it does for me.
 
It's also worth mentioning that you can use JAB to execute a 'ControlClick' style Java button press that even works on a minimized/hidden window. With something like:
 
Actret:=GetAccessibleActions(vmID, value)
DoAccessibleActions(vmID, value, Actret[A_Index])
 
This would replace calling 'MouseClickJControl' which uses AHK's 'MouseClick' function.
 
Again, thank you very much for sharing this.

#10 - Posted 19 May 2014 - 05:34 AM
rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

hello Elgin

 

 JAB Text functions

1.

I tried the AccessibleContext of controls with the  Treeview run, and it work ok.

even it does not show correctly in the tree veiw form , in JControlWriter.ahk.

2.

I tried to implement  getting/setting text values from AccessibleContext

with no success,

I saw the functions/dll are not implemented.in  JavaAccessBridge.ahk

how I get the AccessibleText from AccessibleContext  ?

3.

can somebody add the implementation in AHK of the text Functions, it will help

getAccessibleTextInfo

getAccessibleTextRange 

and how to set new values to a control

I think it's few lines of code.


#11 - Posted 17 September 2014 - 12:25 PM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Hi rani,

 

I can't seem to edit my post up there, so I include the latest version of JavaAccessBridge.ahk here. The text functions have found their way in there in the meantime. Setting text with SetTextContents does not work but that seems to be a problem of Java or the access bridge itself. You can select a text and paste over it to work around this.

I'm not aware of any working way to set values of control directly through the access bridge.

; The function JavaControlDoAction searches for the <occurrence>-instance of a Java-control that fits in <name>, <role> and <description> and performs <action> <count>-times
; actions can be all actions provided by the control itself or:
; - focus: attempts to focus the control and left clicks it if focusing fails
; - left/double/right/middle click; wheel up/down: performs the respective mouse action in the center of the control

JavaControlDoAction(hwnd=0, name="", role="", description="", occurrence="", action="", times=1, parentcontext=0)
{
	global JABVariables
	rval:=JavaControlGet(hwnd, name, role, description, occurrence, parentcontext)
	If (IsObject(rval))
	{
		Info:=getAccessibleContextInfo(rval["vmid"], rval["ac"])
		Loop, %times%
		{
			if (action="focus")
			{
				If (Instr(Info["States"],"focusable"))
				{
					failure:= RequestFocus(rval["vmid"], rval["ac"])
					if (failure<>0)
					{
						MouseClickJControl(Info)
					}
					return, 0
				}
				else return, -2
			}
			else if (action="left click")
			{
				MouseClickJControl(Info)
			}
			else if (action="double click")
			{
				MouseClickJControl(Info,"left",2)
			}
			else if (action="right click")
			{
				MouseClickJControl(Info,"right")
			}
			else if (action="middle click")
			{
				MouseClickJControl(Info,"middle")
			}
			else if (action="wheel up")
			{
				MouseClickJControl(Info,"wheel up")
			}
			else if (action="wheel down")
			{
				MouseClickJControl(Info,"wheel down")
			}
			else
			{
				DoAccessibleActions(rval["vmid"], rval["ac"], action)
			}
		}
		If (times>A_Index)
			Sleep, 50
		return, 0
	}
	else
	 return, rval
}

JavaControlGet(hwnd=0, name="", role="", description="", occurrence="", parentcontext=0)
{
	global JABVariables
	if (!JABVariables["JABInitialised"])
		InitJavaAccessBridge()
	if (JABVariables["JABInitialised"])
	{
		If (IsObject(parentcontext))
		{
			ac:=parentcontext["ac"]
			vmid:=parentcontext["vmid"]
		}
		else
		{
			if (hwnd=0)
			{
				hwnd:=WinExist("A")
				if (!IsJavaWindow(hwnd))
				{
					ControlGetFocus, currctrl, ahk_id %hwnd%
					ControlGet, currwin, Hwnd, , %currctrl%, ahk_id %hwnd%
				}
				else currwin:=hwnd
			}
			else currwin:=hwnd
			vmID:=0
			ac:=0
			if (IsJavaWindow(currwin))
			{
				getAccessibleContextFromHWND(currwin, vmID, ac)
			}
		}
		If (ac<>0)
		{
			Loop, 2
			{
				Info:=
				If (JABVariables["CachedParentAc"]=ac)
				{
					Children:=JABVariables["CachedChildren"]
				}
				else
				{
					Children:=GetVisibleChildrenFromTree(vmID, ac)
					JABVariables["CachedChildren"] :=Children
					JABVariables["CachedParentAc"] :=ac
				}
				occurcnt:=0
				For index, value in Children
				{
					Info:=getAccessibleContextInfo(vmID, value)
					if ((name="" or RegExMatch(Info["Name"],name))
						and (role="" or role=Info["Role"])
						and (description="" or RegExMatch(Info["Description"],description)))
					{
						occurcnt++
						if (Occurrence=occurcnt or Occurrence=0 or Occurrence="")  
						{
							return, CreateContext(vmid, value)
						}
					}
				}
				JABVariables["CachedParentAc"] :=0
			}
			return, -3
		}
	}
	else
	{
		return, -1
	}
}

CreateContext(vmid, ac)
{
	rval:=Object()
	rval["vmid"]:=vmid
	rval["ac"]:=ac
	rval["AccessMode"]:="JAB"
	return, rval
}

SplitContext(context, byref vmid, byref ac)
{
	If (IsObject(context))
	{
		vmid:=context["vmid"]
		ac:=context["ac"]
		return, 0
	}
	else
		return, -1
}

; used to retrieve an object with all visible children of the input control
GetVisibleChildrenFromTree(vmID, ac)
{
	global JABVariables
	Children:=Object()
	if (JABVariables["JABInitialised"])
	{
		RecurseVisibleChildren(vmID, ac, Children)
	}
	Return, Children
}

RecurseVisibleChildren(vmID, ac, byref Children)
{
	global JABVariables
	Info:=getAccessibleContextInfo(vmID,ac)
	If (Instr(Info["States"],"visible"))
	{
		If (Children.MaxIndex()="")
			Children[1]:=ac
		else
			Children[Children.MaxIndex()+1]:=ac
		Loop, % Info["Children count"]
		{
			rac:=GetAccessibleChildFromContext(vmID, ac, A_Index-1)
		RecurseVisibleChildren(vmID, rac, Children)
		}
	}
}

GetControlTree(vmID, ac, Invisible=0)
{
	global JABVariables
	RetObj:=Object()
	Info:=getAccessibleContextInfo(vmID,ac)
	If (Instr(Info["States"],"visible") or invisible)
	{
		RetObj["ac"]:=ac
		RetObj["Children"]:=Object()
		Loop, % Info["Children count"]
		{
			rac:=GetAccessibleChildFromContext(vmID, ac, A_Index-1)
			RetObj["Children"][A_Index-1]:=GetControlTree(vmID, rac, Invisible)
		}
	}
	return, RetObj
}


; used to retrieve an object with all children that are likely to be controlled 
GetControllableChildrenFromTree(vmID, ac)
{
	global JABVariables
	Children:=Object()
	if (JABVariables["JABInitialised"])
	{
		RecurseControllableChildren(vmID, ac, Children)
	}
	Return, Children
}

RecurseControllableChildren(vmID, ac, byref Children)
{
	global JABVariables
	Info:=getAccessibleContextInfo(vmID,ac)
	If (Instr(Info["States"],"visible"))
	{
		blockedroles:=JABVariables["BlockedRoles"]
		role:=Info["Role"]
		If role not in %blockedroles%
			Children.Insert(ac)
		;~ If (Children.MaxIndex()="")
			;~ Children[1]:=ac
		;~ else
			;~ Children[Children.MaxIndex()+1]:=ac
		Loop, % Info["Children count"]
		{
			rac:=GetAccessibleChildFromContext(vmID, ac, A_Index-1)
		RecurseControllableChildren(vmID, rac, Children)
		}
	}
}


; used to retrieve an object with all children of the input control
GetAllChildrenFromTree(vmID, ac)
{
	global JABVariables
	Children:=Object()
	if (JABVariables["JABInitialised"])
	{
		RecurseAllChildren(vmID, ac, Children)
	}
	Return, Children
}

RecurseAllChildren(vmID, ac, byref Children)
{
	global JABVariables
	Info:=getAccessibleContextInfo(vmID,ac)
	If (Children.MaxIndex()="")
		Children[1]:=ac
	else
		Children[Children.MaxIndex()+1]:=ac
	Loop, % Info["Children count"]
	{
		rac:=GetAccessibleChildFromContext(vmID, ac, A_Index-1)
		RecurseAllChildren(vmID, rac, Children)
	}
}

; performs mouse clicks in the center of the specified control
MouseClickJControl(byref Info, action="left", count=1)
{
		 xp:=Floor(Info["X"]+Info["Width"]/2)
		 yp:=Floor(Info["Y"]+Info["Height"]/2)
		 CoordMode, Mouse, Screen
		 SetMouseDelay, 0
		 SetDefaultMouseSpeed, 0
		 BlockInput On
		 MouseGetPos, MouseX, MouseY
		 ;~ sleep, 50
		 MouseClick, %action%, %xp%, %yp%, %count%
		 If (action<>"right")
		 {
			sleep, 100
			MouseMove, %MouseX%, %MouseY%
		}
		 BlockInput Off
		 ;~ msgbox, %action%
}

; Java Access Bridge functions; see Access Bridge documentation for details

; Initialises the bridge access
InitJavaAccessBridge(ForceLegacy=0)
{
	global JABVariables
	JABVariables:= Object()
	JABVariables["JABInitialised"]:=False
	JABVariables["JAB_DLLVersion"] :=""
	JABVariables["JAB_DLL"] :=""
	JABVariables["acType"] :="Int64"
	JABVariables["acPType"] :="Int64*"
	JABVariables["acSize"] :=8

	JABVariables["MAX_BUFFER_SIZE"] := 10240
	JABVariables["MAX_STRING_SIZE"] := 1024
	JABVariables["SHORT_STRING_SIZE"] := 256

	JABVariables["ACCESSIBLE_ALERT"] := "alert"
	JABVariables["ACCESSIBLE_COLUMN_HEADER"] := "column header"
	JABVariables["ACCESSIBLE_CANVAS"] := "canvas"
	JABVariables["ACCESSIBLE_COMBO_BOX"] := "combo box"
	JABVariables["ACCESSIBLE_DESKTOP_ICON"] := "desktop icon"
	JABVariables["ACCESSIBLE_INTERNAL_FRAME"] := "internal frame"
	JABVariables["ACCESSIBLE_DESKTOP_PANE"] := "desktop pane"
	JABVariables["ACCESSIBLE_OPTION_PANE"] := "option pane"
	JABVariables["ACCESSIBLE_WINDOW"] := "window"
	JABVariables["ACCESSIBLE_FRAME"] := "frame"
	JABVariables["ACCESSIBLE_DIALOG"] := "dialog"
	JABVariables["ACCESSIBLE_COLOR_CHOOSER"] := "color chooser"
	JABVariables["ACCESSIBLE_DIRECTORY_PANE"] := "directory pane"
	JABVariables["ACCESSIBLE_FILE_CHOOSER"] := "file chooser"
	JABVariables["ACCESSIBLE_FILLER"] := "filler"
	JABVariables["ACCESSIBLE_HYPERLINK"] := "hyperlink"
	JABVariables["ACCESSIBLE_ICON"] := "icon"
	JABVariables["ACCESSIBLE_LABEL"] := "label"
	JABVariables["ACCESSIBLE_ROOT_PANE"] := "root pane"
	JABVariables["ACCESSIBLE_GLASS_PANE"] := "glass pane"
	JABVariables["ACCESSIBLE_LAYERED_PANE"] := "layered pane"
	JABVariables["ACCESSIBLE_LIST"] := "list"
	JABVariables["ACCESSIBLE_LIST_ITEM"] := "list item"
	JABVariables["ACCESSIBLE_MENU_BAR"] := "menu bar"
	JABVariables["ACCESSIBLE_POPUP_MENU"] := "popup menu"
	JABVariables["ACCESSIBLE_MENU"] := "menu"
	JABVariables["ACCESSIBLE_MENU_ITEM"] := "menu item"
	JABVariables["ACCESSIBLE_SEPARATOR"] := "separator"
	JABVariables["ACCESSIBLE_PAGE_TAB_LIST"] := "page tab list"
	JABVariables["ACCESSIBLE_PAGE_TAB"] := "page tab"
	JABVariables["ACCESSIBLE_PANEL"] := "panel"
	JABVariables["ACCESSIBLE_PROGRESS_BAR"] := "progress bar"
	JABVariables["ACCESSIBLE_PASSWORD_TEXT"] := "password text"
	JABVariables["ACCESSIBLE_PUSH_BUTTON"] := "push button"
	JABVariables["ACCESSIBLE_TOGGLE_BUTTON"] := "toggle button"
	JABVariables["ACCESSIBLE_CHECK_BOX"] := "check box"
	JABVariables["ACCESSIBLE_RADIO_BUTTON"] := "radio button"
	JABVariables["ACCESSIBLE_ROW_HEADER"] := "row header"
	JABVariables["ACCESSIBLE_SCROLL_PANE"] := "scroll pane"
	JABVariables["ACCESSIBLE_SCROLL_BAR"] := "scroll bar"
	JABVariables["ACCESSIBLE_VIEWPORT"] := "viewport"
	JABVariables["ACCESSIBLE_SLIDER"] := "slider"
	JABVariables["ACCESSIBLE_SPLIT_PANE"] := "split pane"
	JABVariables["ACCESSIBLE_TABLE"] := "table"
	JABVariables["ACCESSIBLE_TEXT"] := "text"
	JABVariables["ACCESSIBLE_TREE"] := "tree"
	JABVariables["ACCESSIBLE_TOOL_BAR"] := "tool bar"
	JABVariables["ACCESSIBLE_TOOL_TIP"] := "tool tip"
	JABVariables["ACCESSIBLE_AWT_COMPONENT"] := "awt component"
	JABVariables["ACCESSIBLE_SWING_COMPONENT"] := "swing component"
	JABVariables["ACCESSIBLE_UNKNOWN"] := "unknown"
	JABVariables["ACCESSIBLE_STATUS_BAR"] := "status bar"
	JABVariables["ACCESSIBLE_DATE_EDITOR"] := "date editor"
	JABVariables["ACCESSIBLE_SPIN_BOX"] := "spin box"
	JABVariables["ACCESSIBLE_FONT_CHOOSER"] := "font chooser"
	JABVariables["ACCESSIBLE_GROUP_BOX"] := "group box"
	JABVariables["ACCESSIBLE_HEADER"] := "header"
	JABVariables["ACCESSIBLE_FOOTER"] := "footer"
	JABVariables["ACCESSIBLE_PARAGRAPH"] := "paragraph"
	JABVariables["ACCESSIBLE_RULER"] := "ruler"
	JABVariables["ACCESSIBLE_EDITBAR"] := "editbar"
	JABVariables["PROGRESS_MONITOR"] := "progress monitor"
	JABVariables["CachedParentAc"] :=0
	JABVariables["CachedChildren"] :=
	JABVariables["BlockedRoles"] :=	"panel,filler,root pane,layered pane,menu bar,tool bar,separator,split pane,viewport,scroll bar,scroll pane"
	JABVariables["HandledDefaultActions"] := "focus|left click|double click|right click|middle click|wheel up|wheel down"
	if (ForceLegacy=1)
	{
		JABVariables["JAB_DLLVersion"]:="WindowsAccessBridge"
		JABVariables["JAB_DLL"]:=DllCall("LoadLibrary", "Str", JABVariables["JAB_DLLVersion"] ".dll")
		JABVariables["acType"]:="Int"
		JABVariables["acPType"]:="Int*"
		JABVariables["acSize"]:=4
	}
	else
	if (A_PtrSize=8)
	{
		JABVariables["JAB_DLLVersion"]:="WindowsAccessBridge-64"
		JABVariables["JAB_DLL"]:=DllCall("LoadLibrary", "Str", JABVariables["JAB_DLLVersion"] ".dll")
	}
	else
	{
		JABVariables["JAB_DLLVersion"]:="WindowsAccessBridge-32"
		JABVariables["JAB_DLL"]:=DllCall("LoadLibrary", "Str", JABVariables["JAB_DLLVersion"] ".dll")
		if (JABVariables["JAB_DLL"]=0)
		{
			JABVariables["JAB_DLLVersion"]:="WindowsAccessBridge"
			JABVariables["JAB_DLL"]:=DllCall("LoadLibrary", "Str", JABVariables["JAB_DLLVersion"] ".dll")
			JABVariables["acType"]:="Int"
			JABVariables["acPType"]:="Int*"
			JABVariables["acSize"]:=4
		}
	}
	; it is necessary to preload the DLL
	; otherwise none of the calls suceed

	; start up the access bridge
	JABVariables["JABInitialised"]:=DllCall(JABVariables["JAB_DLLVersion"] "\Windows_run", "Cdecl Int")
	; it is necessary to give the application a few message cycles time before calling access bridge function
	; otherwise all calls will fail
	Sleep, 200 ; minimum 100 for all machines?
	Return, JABVariables["JABInitialised"]
}

; shuts down the access bridge
ExitJavaAccessBridge()
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall("FreeLibrary", "Ptr", JABVariables["JAB_DLL"])
	}
}

IsJavaWindow(hwnd)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\isJavaWindow", "Int", hWnd, "Cdecl Int")
	}
	else 
	{
		Return 0
	}
}

; returns JAB version information as object containing the keys: VMversion, bridgeJavaClassVersion, bridgeJavaDLLVersion, bridgeWinDLLVersion
GetVersionInfo(vmID)
{
	global JABVariables
	Info:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(TempInfo, 2048,0)
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getVersionInfo", "Int", vmID, "UInt", &TempInfo, "Cdecl Int"))
		{
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2
				jver:=Chr(NumGet(TempInfo,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			Info["VMversion"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=((A_Index-1)*2)+JABVariables["SHORT_STRING_SIZE"]*2
				jver:=Chr(NumGet(&TempInfo,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			Info["bridgeJavaClassVersion"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["SHORT_STRING_SIZE"]*4
				jver:=Chr(NumGet(&TempInfo,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			Info["bridgeJavaDLLVersion"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["SHORT_STRING_SIZE"]*6
				jver:=Chr(NumGet(&TempInfo,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			Info["bridgeWinDLLVersion"]:=verstr
		}
	}
	Return Info
}

ReleaseJavaObject(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\ReleaseJavaObject", "Int", vmID, JABVariables["acType"], ac, "Cdecl")
	}
}

IsSameObject(vmID, ac1, ac2)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\isSameObject", "Int", vmID, JABVariables["acType"], ac1, JABVariables["acType"], ac2, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}

; retrieves the root element from a window
GetAccessibleContextFromHWND(hwnd, ByRef vmID, ByRef ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleContextFromHWND", "Int", hWnd, "Int*", vmID, JABVariables["acPType"], ac, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}

GetHWNDFromAccessibleContext(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getHWNDFromAccessibleContext", "Int", vmID, JABVariables["acType"], ac, "Cdecl UInt")
	}
	else
	{
		Return 0
	}
}

GetAccessibleContextAtHWND(hwnd, x, y)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		GetAccessibleContextFromHWND(hwnd, vmID, Winac)
		return, FindContextAt(vmID, Winac, x, y)
	}
	else
	{
		Return 0
	}
}

FindContextAt(vmID, ac, x, y)
{
	global JABVariables
	Info:=getAccessibleContextInfo(vmID,ac)
	If (Instr(Info["States"],"visible"))
	{
		resac:=0
		Loop, % Info["Children count"]
		{
			rac:=GetAccessibleChildFromContext(vmID, ac, A_Index-1)
			resact:=FindContextAt(vmID, rac, x, y)
			If (resact<>0)
				resac:=resact
		}
		If ((resac=0) and (x>=Info["x"]) and (x<=(Info["x"]+Info["Width"])) and (y>=Info["y"]) and (y<=(Info["y"]+Info["Height"])))
		{
			return, ac
		}
		return, resac
	}
	else
		return, 0
}


GetAccessibleContextAt(vmID, acParent, x, y, ByRef ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleContextAt", "Int", vmID, JABVariables["acType"], acParent, "Int", x, "Int", y, JABVariables["acPType"], ac, "Cdecl Int")
		;~ Return DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleContextAt", "Int", vmID, "Int", acParent, "Int", x, "Int", y, "Ptr", ac, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}

GetAccessibleContextWithFocus(hwnd, ByRef vmID, ByRef ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		vmid:=0
		ac:=0
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleContextWithFocus", "Int", hwnd, "Int*", vmID, JABVariables["acPType"], ac, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}

GetActiveDescendent(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getActiveDescendent", "Int", vmID, JABVariables["acType"], ac, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}


GetObjectDepth(vmID, ac)
{
	global JABVariables
	depth:=0
	if (JABVariables["JABInitialised"])
	{
		depth:=DllCall(JABVariables["JAB_DLLVersion"] "\getObjectDepth", "Int", vmID, JABVariables["acType"], ac, "Cdecl Int")
	}
	Return depth
}

GetAccessibleParentFromContext(vmID, ac)
{
	global JABVariables
	acparent:=0
	if (JABVariables["JABInitialised"])
	{
		acparent:=DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleParentFromContext", "Int", vmID, JABVariables["acType"], ac, "Cdecl "JABVariables["acType"])  
	}
	Return, acparent
}

GetTopLevelObject(vmID, ac)
{
	global JABVariables
	acparent:=0
	if (JABVariables["JABInitialised"])
	{
		acparent:=DllCall(JABVariables["JAB_DLLVersion"] "\getTopLevelObject", "Int", vmID, JABVariables["acType"], ac, "Cdecl "JABVariables["acType"])  
	}
	Return, acparent
}



GetAccessibleChildFromContext(vmID, ac, index)
{
	global JABVariables
	acchild:=0
	if (JABVariables["JABInitialised"])
	{
		acchild:=DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleChildFromContext", "Int", vmID, JABVariables["acType"], ac, "Int", index, "Cdecl "JABVariables["acType"])  
	}
	Return, acchild
}

; retrieves information about a certain element as an object with the keys: 
; Name, Description, Role_local, Role, States_local, States, Index in parent, 
; Children count, X, Y, Width, Height, Accessible component, Accessible action, 
; Accessible selection, Accessible text, Accessible value interface, 
; Accessible action interface, Accessible component interface, 
; Accessible selection interface, Accessible table interface, 
; Accessible text interface, Accessible hypertext interface
GetAccessibleContextInfo(vmID, ac)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Info, 6188,0)
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleContextInfo", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Info, "Cdecl Int"))
		{
			verstr:=""
			Loop, % JABVariables["MAX_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.= jver
			}
			TempInfo["Name"]:=verstr
			verstr:=""
			Loop, % JABVariables["MAX_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["MAX_STRING_SIZE"]*2
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			TempInfo["Description"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["MAX_STRING_SIZE"]*4
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr:=verstr jver
			}
			TempInfo["Role_local"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*2
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			TempInfo["Role"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*4
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			TempInfo["States_local"]:=verstr
			verstr:=""
			Loop, % JABVariables["SHORT_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2+JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*6
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.=jver
			}
			TempInfo["States"]:=verstr
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Index in parent"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+4
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Children count"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+8
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["X"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+12
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Y"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+16
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Width"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+20
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Height"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+24
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Accessible component"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+28
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Accessible action"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+32
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Accessible selection"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+36
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Accessible text"]:=jver
			offset:=JABVariables["MAX_STRING_SIZE"]*4+JABVariables["SHORT_STRING_SIZE"]*8+40
			jver:=NumGet(&Info,offset,"Int")
			TempInfo["Accessible value interface"]:=jver & 1
			TempInfo["Accessible action interface"]:=jver & 2
			TempInfo["Accessible component interface"]:=jver & 4
			TempInfo["Accessible selection interface"]:=jver & 8
			TempInfo["Accessible table interface"]:=jver & 16
			TempInfo["Accessible text interface"]:=jver & 32
			TempInfo["Accessible hypertext interface"]:=jver & 64
		}
		else
		{
			msgbox, Error in GetAccessibleContextInfo vmID: %vmID% ac: %ac%
		}
	}
	Return TempInfo
}

GetVisibleChildrenCount(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		Return DllCall(JABVariables["JAB_DLLVersion"] "\getVisibleChildrenCount", "Int", vmID, JABVariables["acType"], ac, "Cdecl Int")
	}
	else
	{
		Return 0
	}
}

; this function seems to be unreliable under Win7 64bit
; works fine under WinXP
GetVisibleChildren(vmID, ac)
{
	global JABVariables
	Children:=Object()
	if (JABVariables["JABInitialised"])
	{
		NumChild:=getVisibleChildrenCount(vmID, ac)
		StartChild:=0
		cnt:=0
		VarSetCapacity(TempChildren, 257*JABVariables["acSize"],0)
		Loop
		{
			if (DllCall(JABVariables["JAB_DLLVersion"] "\getVisibleChildren", "Int", vmID, JABVariables["acType"], ac, "Int", StartChild, "Ptr", &TempChildren, "Cdecl Int"))
			{
				retchild:=NumGet(&TempChildren,0,"Int")
				str:=retchild ";;"
				Loop, %retchild%
				{
					Children[++cnt]:=Numget(&TempChildren, JABVariables["acSize"]*(A_Index), JABVariables["acType"])
				}
				StartChild:=StartChild+retchild
			}
			else break
		} Until StartChild>=NumChild
	}
	Return Children
}

GetAccessibleActions(vmID, ac)
{
	global JABVariables
	Actret:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Actions, 256*256*2+A_PtrSize,0)
		if DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleActions", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Actions, "Cdecl Int")
		{
			retact:=NumGet(&Actions,0,"Int")
			Loop, % retact
			{
				verstr:=""
				lind:=A_Index
				Loop, % JABVariables["SHORT_STRING_SIZE"]
				{
					offset:=(A_Index-1)*2+JABVariables["SHORT_STRING_SIZE"]*2*(lind-1)+A_PtrSize
					jver:=Chr(NumGet(&Actions,offset,"UChar"))
					if (jver=Chr(0))
					{
						break
					}
					verstr.= jver
				}
				Actret[A_Index]:=verstr
			}
		}
	}
	Return Actret  
}

DoAccessibleActions(vmID, ac, ByRef actionsToDo)
; actionsToDo : comma separated list of actions
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Actions, 256*256*2+A_PtrSize,0)
		Loop, Parse, actionsToDo, `,, %A_Space%
		{
			NumPut(A_Index,&Actions,0,"Int")
			lind:=A_Index
			Loop, Parse, A_LoopField
			{
				offset:=(A_Index-1)*2+JABVariables["SHORT_STRING_SIZE"]*2*(lind-1)+A_PtrSize
				NumPut(Asc(A_LoopField),&Actions,offset,"UChar")
			}
		}
		failure:=0
		DllCall( JABVariables["JAB_DLLVersion"] "\doAccessibleActions", "Int", vmID , JABVariables["acType"], ac , "Ptr", &Actions , "Int", failure, "Cdecl Int")
		return, failure
	}
}

RequestFocus(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		return DllCall(JABVariables["JAB_DLLVersion"] "\requestFocus", "Int", vmID, JABVariables["acType"], ac, "Cdecl Int")
	}
}

; retrieves information about a certain text element as an object with the keys: 
; CharCount, CaretIndex, IndexAtPoint
GetAccessibleTextInfo(vmID, ac, x=0, y=0)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Info, 12,0)
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextInfo", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Info, "Int", x, "Int", y, "Cdecl Int"))
		{
			jver:=NumGet(&Info,0,"Int")
			TempInfo["CharCount"]:=jver
			jver:=NumGet(&Info,4,"Int")
			TempInfo["CaretIndex"]:=jver
			jver:=NumGet(&Info,8,"Int")
			TempInfo["IndexAtPoint"]:=jver
		}
		else
		{
			TempInfo:=
			Return, TempInfo
		}
	}
	Return TempInfo
}

Is64bit(vmID, ac)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		If (A_Is64bitOS)
		{
			; find out if the JVM is running at 32 or 64 bit to circumvent a bug with the adress length in 32 bit JVMs (JRE 6+7)
			tlo:=GetTopLevelObject(vmID, ac)
			Awin:=GetHWNDFromAccessibleContext(vmID, tlo)
			currwin:=WinExist("A")
      WinGet, ProcID, PID, ahk_id %Awin%
			phandle:=DllCall("OpenProcess", "uint", 0x1000, "uint", 0, "uint", ProcID)
			isWow64process:=0
			DllCall("IsWow64Process", "uint", phandle, "int*", isWow64process)
			if (isWow64process)
			{
				Return, 0
			}
			else
			{
				Return, 1
			}
	  }
		else
		{
			Return, 0
		}	
	}
	Return, 0
}

; retrieves the currently selected text and its start and end index as an object with the keys: 
; SelectionStartIndex, SelectionEndIndex, SelectedText
GetAccessibleTextSelectionInfo(vmID, ac)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Info, JABVariables["MAX_STRING_SIZE"]*2+8,0)
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextSelectionInfo", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Info, "Cdecl Int"))
		{
			verstr:=""
			Loop, % JABVariables["MAX_STRING_SIZE"]
			{
				offset:=(A_Index-1)*2
				jver:=Chr(NumGet(&Info,offset,"UChar"))
				if (jver=Chr(0))
				{
					break
				}
				verstr.= jver
			}
			TempInfo["SelectedText"]:=verstr
			jver:=NumGet(&Info,0,"Int")
			TempInfo["SelectionStartIndex"]:=jver
			jver:=NumGet(&Info,4,"Int")
			TempInfo["SelectionEndIndex"]:=jver
			Return, TempInfo
		}
	}
	Return, 0
}

SelectTextRange(vmID, ac, startIndex, endIndex)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
  	; find out if the JVM is running at 32 or 64 bit to circumvent a bug with the adress length in 32 bit JVMs (JRE 6+7)
		if (Is64bit(vmID, ac))
		{
			Return, DllCall(JABVariables["JAB_DLLVersion"] "\selectTextRange", "Int", vmID, JABVariables["acType"], ac, "Int", startIndex, "Int", endIndex, "Cdecl Int")
		}
		else
		{
			Return, DllCall(JABVariables["JAB_DLLVersion"] "\selectTextRange", "Int", vmID, "Ptr", ac, "Int", startIndex, "Int", endIndex, "Cdecl Int")
		}
	}
	Return, 0
}


SetCaretPosition(vmID, ac, position)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
  	; find out if the JVM is running at 32 or 64 bit to circumvent a bug with the adress length in 32 bit JVMs (JRE 6+7)
		if (Is64bit(vmID, ac))
		{
			Return, DllCall(JABVariables["JAB_DLLVersion"] "\setCaretPosition", "Int", vmID, JABVariables["acType"], ac, "Int", position, "Cdecl Int")
		}
		else
		{
			Return, DllCall(JABVariables["JAB_DLLVersion"] "\setCaretPosition", "Int", vmID, "Ptr", ac, "Int", position, "Cdecl Int")
		}
	}
	Return, 0
}

; retrieves the caret location as an object with the keys: 
; Index, X, Y, Width, Height
GetCaretLocation(vmID, ac)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Info, 16,0)
		Index:=0
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getCaretLocation", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Info, "Int", Index, "Cdecl Int"))
		{
			jver:=NumGet(&Info,0,"Int")
			TempInfo["X"]:=jver
			jver:=NumGet(&Info,4,"Int")
			TempInfo["Y"]:=jver
			jver:=NumGet(&Info,8,"Int")
			TempInfo["Width"]:=jver
			jver:=NumGet(&Info,12,"Int")
			TempInfo["Height"]:=jver
			TempInfo["Index"]:=Index
		}
		else
		{
			TempInfo:=
			Return, TempInfo
		}
	}
	Return TempInfo
}

; retrieves the location of position Index as an object with the keys: 
; X, Y, Width, Height
GetAccessibleTextRect(vmID, ac, Index)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		VarSetCapacity(Info, 16,0)
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextRect", "Int", vmID, JABVariables["acType"], ac, "Ptr", &Info, "Int", Index, "Cdecl Int"))
		{
			jver:=NumGet(&Info,0,"Int")
			TempInfo["X"]:=jver
			jver:=NumGet(&Info,4,"Int")
			TempInfo["Y"]:=jver
			jver:=NumGet(&Info,8,"Int")
			TempInfo["Width"]:=jver
			jver:=NumGet(&Info,12,"Int")
			TempInfo["Height"]:=jver
		}
		else
		{
			TempInfo:=
			Return, TempInfo
		}
	}
	Return TempInfo
}

; retrieves the start and end index of the line conaining Index as an object with the keys: 
; StartPos, EndPos
GetAccessibleTextLineBounds(vmID, ac, Index)
{
	global JABVariables
	TempInfo:=Object()
	if (JABVariables["JABInitialised"])
	{
		StartPos:=0
		EndPos:=0
		if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextLineBounds", "Int", vmID, JABVariables["acType"], ac, "Int*", &StartPos, "Int*", &EndPos, "Cdecl Int"))
		{
			TempInfo["StartPos"]:=StartPos
			TempInfo["EndPos"]:=EndPos
		}
		else
		{
			TempInfo:=
			Return, TempInfo
		}
	}
	Return TempInfo
}

; retrieves text between start and end index
GetAccessibleTextRange(vmID, ac, startc=0, endc=0)
{
	global JABVariables
	TempStr:=""
	if (JABVariables["JABInitialised"])
	{
		maxlen:=10000 ; arbitrary value, larger values tend to fail sometimes
		TInfo:=GetAccessibleTextInfo(vmID, ac)
		If IsObject(TInfo)
		{
			If (startc<0)
			{
				startc:=0
			}
			cnt:=startc
			If (endc=0 or endc>TInfo["CharCount"])
			{
				endc:=TInfo["CharCount"]
			}
			If (endc>0)
			{
				Loop,
				{
					if (cnt+maxlen>endc)
					{
						cnt2:=endc-1
					}
					else 
					{
						cnt2:=cnt+maxlen
					}
          len:=maxlen+1
					VarSetCapacity(Txt, len*2,0)
					if (DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextRange", "Int", vmID, JABVariables["acType"], ac, "Int", cnt, "Int", cnt2, "Ptr", &Txt, "Int", len, "Cdecl Int"))
					{
						NumPut(0,Txt,(cnt2-cnt+1)*2, "UChar")
            if (cnt2>cnt) and (NumGet(Txt,0,"UChar")=0) ; occasionally the first call fails at the end of the text
						{
              DllCall(JABVariables["JAB_DLLVersion"] "\getAccessibleTextRange", "Int", vmID, JABVariables["acType"], ac, "Int", cnt, "Int", cnt2, "Ptr", &Txt, "Int", len, "Cdecl Int")
							NumPut(0,Txt,(cnt2-cnt+1)*2, "UChar")
						}
					  Loop, % maxlen+1
						{
							offset:=(A_Index-1)*2
							jver:=Chr(NumGet(&Txt,offset,"UChar"))
							if (jver=Chr(0))
							{
								break
							}	
							TempStr.=jver
						}
					}
          if (cnt>=cnt2)
					{
						cnt++
					}
					else 
					{
					  cnt:=cnt2+1
					}
          if (cnt>=endc-1)
						break
				}
			}
		}
	}
	Return TempStr	
}


SetTextContents(vmID, ac, ntext)
{
    global JABVariables
    if (JABVariables["JABInitialised"])
    {
        len := StrLen(ntext)
        VarSetCapacity(TempStr, len * 2 + 1, 0)
        Loop, Parse, ntext
        {
            offset:=(A_Index-1)*2
            NumPut(Asc(A_LoopField),TempStr,offset,"UChar")
        }
        Return, DllCall(JABVariables["JAB_DLLVersion"] "\setTextContents", "Int", vmID, JABVariables["acType"], ac, "UInt", &TempStr, "Cdecl Int")
    }
}


; callback set routines
; see access bridge documentation for definitions of callback funtions
;
; usage:
; 
; if InitJavaAccessBridge()
; {
;   Address := RegisterCallback("FocusGained","CDecl")
;   setFocusGainedFP(Address)
; }
; ...
; Return
; 
; FocusGained(vmID, event, source)
; {
;   DoSomething(vmID, source)
;   ReleaseJavaObject(vmID,event)
;   ReleaseJavaObject(vmID,source)
; }


setFocusGainedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setFocusGainedFP", "UInt", fp, "Cdecl")
	}
}

setFocusLostFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setFocusLostFP", "UInt", fp, "Cdecl")
	}
}

setJavaShutdownFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setJavaShutdownFP", "UInt", fp, "Cdecl")
	}
}

setCaretUpdateFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setCaretUpdateFP", "UInt", fp, "Cdecl")
	}
}

setMouseClickedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMouseClickedFP", "UInt", fp, "Cdecl")
	}
}

setMouseEnteredFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMouseEnteredFP", "UInt", fp, "Cdecl")
	}
}

setMouseExitedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMouseExitedFP", "UInt", fp, "Cdecl")
	}
}

setMousePressedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMousePressedFP", "UInt", fp, "Cdecl")
	}
}

setMouseReleasedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMouseReleasedFP", "UInt", fp, "Cdecl")
	}
}

setMenuCanceledFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMenuCanceledFP", "UInt", fp, "Cdecl")
	}
}

setMenuDeselectedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMenuDeselectedFP", "UInt", fp, "Cdecl")
	}
}

setMenuSelectedFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setMenuSelectedFP", "UInt", fp, "Cdecl")
	}
}

setPopupMenuCanceledFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPopupMenuCanceledFP", "UInt", fp, "Cdecl")
	}
}

setPopupMenuWillBecomeInvisibleFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPopupMenuWillBecomeInvisibleFP", "UInt", fp, "Cdecl")
	}
}

setPopupMenuWillBecomeVisibleFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPopupMenuWillBecomeVisibleFP", "UInt", fp, "Cdecl")
	}
}

setPropertyNameChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyNameChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyDescriptionChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyDescriptionChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyStateChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyStateChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyValueChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyValueChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertySelectionChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertySelectionChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyTextChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyTextChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyCaretChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyCaretChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyVisibleDataChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyVisibleDataChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyChildChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyChildChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyActiveDescendentChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyActiveDescendentChangeFP", "UInt", fp, "Cdecl")
	}
}

setPropertyTableModelChangeFP(fp)
{
	global JABVariables
	if (JABVariables["JABInitialised"])
	{
		DllCall(JABVariables["JAB_DLLVersion"] "\setPropertyTableModelChangeFP", "UInt", fp, "Cdecl")
	}
}


/*
functions that have not been ported yet

{ exported user functions of the Java Access Bridge }

{ AccessibleTable }
function getAccessibleTableInfo(vmID: longint; ac: AccessibleContext; tableinfo: PAccessibleTableInfo):JBool;
function getAccessibleTableCellInfo(vmID: longint; at: AccessibleTable; row: jint; column: jint; tableCellInfo: PAccessibleTableCellInfo):JBool;
function getAccessibleTableRowHeader(vmID: longint; acParent: AccessibleContext; tableInfo: PAccessibleTableInfo):JBool;
function getAccessibleTableColumnHeader(vmID: longint; acParent: AccessibleContext; tableinfo: PAccessibleTableInfo):JBool;
function getAccessibleTableRowDescription(vmID: longint; acParent: AccessibleContext; row: jint):AccessibleContext;
function getAccessibleTableColumnDescription(vmID: longint; acParent: AccessibleContext; column: jint):AccessibleContext;
function getAccessibleTableRowSelectionCount(vmID: longint; table: AccessibleTable):jint;
function isAccessibleTableRowSelected(vmID: longint; table: AccessibleTable; row: jint):Jbool;
function getAccessibleTableRowSelections(vmID: longint; table: AccessibleTable; count: jint; selections: pjint):JBool;
function getAccessibleTableColumnSelectionCount(vmID: longint; table: AccessibleTable):Jint;
function isAccessibleTableColumnSelected(vmID: longint; table: AccessibleTable; column: jint):JBool;
function getAccessibleTableColumnSelections(vmID: longint; table: AccessibleTable; count: jint; selections: pjint):JBool;
function getAccessibleTableRow(vmID: longint; table: AccessibleTable; index: jint):jint;
function getAccessibleTableColumn(vmID: longint; table: AccessibleTable; index: jint):jint;
function getAccessibleTableIndex(vmID: longint; table: AccessibleTable; row: jint; column: jint):jint;

{ AccessibleRelationSet }
function getAccessibleRelationSet(vmID: longint; ac: AccessibleContext; relationSetInfo: PAccessibleRelationSetInfo):JBool;

{ AccessibleHypertext }
function getAccessibleHypertext(vmID: longint; ac: AccessibleContext; hypertextInfo: PAccessibleHypertextInfo):JBool;
function activateAccessibleHyperlink(vmID: longint; ac: AccessibleContext; aH: AccessibleHyperlink):JBool;
function getAccessibleHyperlinkCount(vmID: longint; ac: AccessibleContext):Jint;
function getAccessibleHypertextExt(vmID: longint; ac: AccessibleContext; nStartIndex: jint; hypertextInfo: PAccessibleHypertextInfo):JBool;
function getAccessibleHypertextLinkIndex(vmID: longint; ah: AccessibleHypertext; nIndex: jint):jint;
function getAccessibleHyperlink(vmID: longint; ah: AccessibleHypertext; nIndex: jint; hyperlinkInfo: PAccessibleHyperlinkInfo):JBool;

{ Accessible KeyBindings, Icons and Actions }
function getAccessibleKeyBindings(vmID: longint; ac: AccessibleContext; keyBindings: PAccessibleKeyBindings):JBool;
function getAccessibleIcons(vmID: longint; ac: AccessibleContext; icons: PAccessibleIcons):JBool;

{ AccessibleText }
function GetCurrentAccessibleValueFromContext(vmID: longint; av: AccessibleValue; value: pwidechar; len: jshort):JBool;
function GetMaximumAccessibleValueFromContext(vmID: longint; av: AccessibleValue; value: pwidechar; len: jshort):JBool;
function GetMinimumAccessibleValueFromContext(vmID: longint; av: AccessibleValue; value: pwidechar; len: jshort):JBool;

procedure AddAccessibleSelectionFromContext(vmID: longint; acsel: AccessibleSelection; i: Jint);
procedure ClearAccessibleSelectionFromContext(vmID: longint; acsel: AccessibleSelection);
function GetAccessibleSelectionFromContext(vmID: longint; acsel: AccessibleSelection; i: jint):JObject;
function GetAccessibleSelectionCountFromContext(vmID: longint; acsel: AccessibleSelection):Jint;
function IsAccessibleChildSelectedFromContext(vmID: longint; acsel: AccessibleSelection; i: Jint):Jbool;
procedure RemoveAccessibleSelectionFromContext(vmID: longint; acsel: AccessibleSelection; i: Jint);
procedure SelectAllAccessibleSelectionFromContext(vmID: longint; acsel: AccessibleSelection);

{ Utility methods }

function getParentWithRole(vmID: longint; ac: AccessibleContext; role: pwidechar):AccessibleContext;
function getParentWithRoleElseRoot(vmID: longint; ac: AccessibleContext; role: pwidechar):AccessibleContext;
function getVirtualAccessibleName(vmID: longint; ac: AccessibleContext; name: pwidechar; len: integer):JBool;
function getTextAttributesInRange(vmID: longint; ac: AccessibleContext; startIndex: integer; endIndex: integer; attributes: PAccessibleTextAttributesInfo; len: Jshort): JBool;
function getEventsWaiting():Jint;


All unported functions in Windowsaccessbridge32.dll

activateAccessibleHyperlink
addAccessibleSelectionFromContext
clearAccessibleSelectionFromContext
getAccessibleHyperlink
getAccessibleHyperlinkCount
getAccessibleHypertext
getAccessibleHypertextExt
getAccessibleHypertextLinkIndex
getAccessibleIcons
getAccessibleKeyBindings
getAccessibleRelationSet
getAccessibleSelectionCountFromContext
getAccessibleSelectionFromContext
getAccessibleTableCellInfo
getAccessibleTableColumn
getAccessibleTableColumnDescription
getAccessibleTableColumnHeader
getAccessibleTableColumnSelectionCount
getAccessibleTableColumnSelections
getAccessibleTableIndex
getAccessibleTableInfo
getAccessibleTableRow
getAccessibleTableRowDescription
getAccessibleTableRowHeader
getAccessibleTableRowSelectionCount
getAccessibleTableRowSelections
getAccessibleTextAttributes
getAccessibleTextItems
getCurrentAccessibleValueFromContext
getEventsWaiting
getMaximumAccessibleValueFromContext
getMinimumAccessibleValueFromContext
getParentWithRole
getParentWithRoleElseRoot
getTextAttributesInRange
getVirtualAccessibleName
isAccessibleChildSelectedFromContext
isAccessibleTableColumnSelected
isAccessibleTableRowSelected
removeAccessibleSelectionFromContex
selectAllAccessibleSelectionFromContext


*/

Edited by Elgin, 25 September 2014 - 07:53 AM.

#12 - Posted 23 September 2014 - 04:07 PM
rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

hello Elgin,

thanks for reply ,

I am checking your latest text-value corrections.


#13 - Posted 23 September 2014 - 07:01 PM
rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

Hi ELgin,

SetTextContents

 

1. getting the text values work ok, and is actually very fast.

 

2.

 as you mentioned, the SetTextContents is not working.

 from the struct definitions, I tried to correct to Ptr or UChar, but with no success.

 seems, my knowledge about is not good enough.

 

3. I tried to select All on a control:

SelectTextRange(vmID, ac, startIndex, endIndex)
clipboard =
clipboard = %ntext%
ClipWait
send,  ^v
the SelectTextRange did not work(make blue the textbox)
I set indexs:startIndex=1 endIndex=len(text)
with no success.
 
so instead of SelectTextRange I made dblClick on the textbox (make blue range)
and then the 'new' setText work.
but ofcourse this is not the best elegant way.
will there be some solution to SetTextContents ?
 
4. cells in Tables.
 from tree Controls traverse, I see that on each cell it give's it's own ac (AccessibleContext)
 on scrolling, it will generate new ac's on new rows?
 or there is other approach to cells in table ?
 
your new JavaAccessBridge.ahk , gave a lot of knowledge, to the JAB handling
your elegant coding, shows that you are high professional with the internals of struct and memory usage,
that normally programmers don't deal with it.

#14 - Posted 24 September 2014 - 09:08 AM
Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Hi rani,

 

thank you for bringing this topic up again! After finding no solution for this back when I wrote the code I now googled again and found a recent post with working C# code which made me see my error :-)

 

SetTextContents is working now.

 

SelectTextRange works fine for me in this code:

#Include JavaAccessBridge.ahk
InitJavaAccessBridge()
sleep, 2000
hwnd:=Winexist("A")
GetAccessibleContextWithFocus(hwnd, vmID, ac)
SetTextContents(vmID, ac, "Some more text.")
sleep, 200
Info:=GetAccessibleTextInfo(vmID, ac)
SelectTextRange(vmID, ac, 0, Info["CharCount"])
msgbox, % info["charcount"]
ExitJavaAccessBridge()

All text gets selected here.

 

On tables: they have an extra bunch of functions for tables which I haven't coded in AHK as I have no use for them yet. See all the AccessibleTable functions in the comment section of of the bridge file. How your application publishes the cells on scrolling I can't tell you. I've only played around with fixed cell tables when I first tried it out years ago with Delphi.

 

 

your elegant coding, shows that you are high professional with the internals of struct and memory usage,
that normally programmers don't deal with it.

 

 

Thanks for the flowers. I'm just a humble amateur who started coding in the times of the C64 when that was still fairly normal ;-)


#15 - Posted 25 September 2014 - 08:37 AM
Back to Ask for Help
  1. AutoHotkey Community
  2. → AutoHotkey
  3. → Ask for Help