Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

How to send to unseen controls in a Java app


  • Please log in to reply
47 replies to this topic
rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

Hi Elgin,

 

setTextContents

 

sorry for my last post I update it:

I saw your edit on JavaAccessBridge.ahk

and copy to my script

I saw your correction, in numput with "UCHAR"

I played with it myself ,tried to adjust the struct definitions , but could not find the solution.

 

thanks again.



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Are you running the latest Java version (JRE7 Update 67) with access bridge 2.0.4?

 

I just tried it on my spare machine here which had an older Java version and it only worked in 64bit there but not in 32bit Java. After upgrading it works in both versions for me as on my main machine.



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

Hi Eligin

 

1. as far as I know it's jre6 32bit

    for some reason, the user don't want to upgrade to jre7.

 

2. the setTextContects is still not working,after I copied your edited JavaAccesBridge.ahk on this site.

 

3. the method ...withFocus (vmID,ac) from your example, gives each time other ac, (vmID remain same) ?

   so I can get text with older ac (from Tree traverse) and the new ac from ... withFoucs,

    is it correct behaviour ?



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

I use access bridge 2.0.2

you mentioned :

access bridge 2.0.4

do I have to download the 2.0.4 ?

from what site ?



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Re 2.0.4 and 1, 2: since a couple of releases in JRE7 the access bridge is part of the JRE. 2.0.4 gets installed by the latest JRE release. I have not tried to get it into JRE6 yet. Maybe you can install JRE7 to a different folder and then manually move the JAB files into JRE6 to see if something starts to work.

Out of the box the current code definitely does not work with JRE6 32bit.

 

Re 3: withFocus always creates a new ac. This is correct. The acs from the static element tree stay valid until the element is removed or your app calls ReleaseObject. The vmID refers to the underlying Java virtual machine. It will even by the same across several applications when they're running under the same Java process.



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

JAB

 

1. we cannot change the  jre6 32 bits.

2. i can stay with the dbl-mouseclick and send the text from clipboard,meanwhile,

    for setText - update  field.

    can the sendMessage in AHK  , do the work for write to control ?

    hopefully I'll not find some other surprises , that are not working.

 

3. can the JAB handle automation also , with SAP GUI ?

 

4. do you know maybe some better tool to interact with Java Forms ,from AHK ?

   maybe using  C# -> JAVA ?

    my knowledge in Java is poor(almost zero)

    in C# I can handle some utility programs.



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

2: I personally use the clipboard and sending ctrl-v ;-)

 

3: only if your SAP GUI runs in Java...ours does not. For the normal Windows GUIs there's the default AHK commands, MSAA and UIA.

 

4: C# is no better than AHK in that respect since you have to put all the DLL calls in by hand, too. For C/C++ you at least have the pre-made headers.

The hands-down best tool to interact with Java forms so you can do almost everything is Java...



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

4: C# is no better than AHK in that respect since you have to put all the DLL calls in by hand, too. For C/C++ you at least have the pre-made headers.

The hands-down best tool to interact with Java forms so you can do almost everything is Java...

Java UI-automation

with what tool in Java you can do it ?

how I can activate such a program ?

 

in AHK forum, there was a post, that somebody interact with Java program via TCP <-> AHK



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Java UI-automation

with what tool in Java you can do it ?

how I can activate such a program ?

 

I was referring to changing the actual source code of the Java application you need to behave differently...sorry if that raised any false hopes...

You do have the same commands as in the Java access bridge available from Java, though:

http://docs.oracle.c...ge-summary.html

if that's of any help.

 

If you have the Java sources you have several ways of communicating with AHK or other Windows programs

- with JNI calling a custom dll e.g. to trigger Windows messages or whatever you can think of

- with JACOB to access COM objects from Java

- TCP as you mentioned

 

It really depends on what you need to do...which I don't know ;-)



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

hi Elgin

 

AHK UI-Automation.

 

1. I use AHK mostly for UI automation

    on window forms(not JAVA)  the AHK not always recognize all controls(fileds) in a form.

     in many cases the AHK 'see' the control, but is not able to get it's text value(..known problem)

     is there some Accessible tool that can help AHK to get text and set text on window forms ?

 

2. there also in some applications in windows forms,that AHK cannot see even the control,(maybe old dos or else)

    as above the standard methdos like controlGetText, and control get ID under mouse, is not working correctly,

    maybe these controls are not 'standard' MS GUI ?

 

3. I tried also the ACC accessiblity interface, but itterating on controls, it also didnt give the correct results.

 

   so ,to sumerize, the question is, how AHK can detect('see') non standard window form controls ?

   I asked few times the question in AHK site, without any solution.

  I sure, that developers, face many times such a problem, using AHK.



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

The default AHK control commands work only with some very specific GUI elements. Through Windows' assistive technologies MSAA and UIAutomation you can get into more controls and their contents but still not to everything that can appear in an application. For using those two with AHK there are some threads in the forums.

 

It is possible in Windows (or Java for that matter) to program custom controls. For these the programmer needs to define how they should appear/behave to assistive technology. Doing so is usually not a big deal (setting a few values) but unfortunately a lot of programmers have never heard about assistive technology or care and thus don't do that...

That leads to controls that can be used mostly only by mouse and otherwise behave like an image to screen readers, speech recognition and automation tools.

 

The Windows SDK contains two tools that allow you to check if a certain control was properly implemented and can be used though MSAA or UIA or not:

UISpy.exe

Inspect.exe

 

If those don't show anything useful for a control, the only thing you're left with is image search and OCRing the screen (unless the application has its own API ;-).



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

I tried UIspy, not good not showing  , just standard controls.

 

the inspect I did not try. I'll try it.

 

MSAA and UIAutomation

I tried the MS ACC is it as MSAA ?I took some code samples from AHK forums, with same success as UI spy.

 

assistive technology.

actually for UI automation we need 3 things : control ID,controlText Get/Set

is there some code sample or link how to " program custom controls" ?

so it can custom in window for java forms either ?

 

screen Ocr

I used some screen 'ocr'

it gave good resluts, for all kinds of forms (even JAVA applications)

but for some reason, not for Oracle application Form(??) , therefore we have to handle the JAB.

do you know on some free screen 'ocr' ?



Elgin
  • Members
  • 79 posts
  • Last active: Dec 18 2015 08:07 PM
  • Joined: 29 Jul 2011

Inspect should give you the same as UISpy, so I wouldn't expect any magic ;-)

 

MSAA: http://en.wikipedia....e_Accessibility

I'm not sure what you refer to with MS ACC...

 

For Java I know of this tutorial: http://docs.oracle.c...isc/access.html

Google will probably get you more examples.

 

OCR: I only know of this free one: http://sourceforge.net/projects/jocr/

Somewhere there's a post on this forum on how to use it with AHK.

If your source image is not black text on very light background you may have to play with the colors/contrast a bit to get decent results which can be done with tic's GDI+ library.



rani
  • Members
  • 217 posts
  • Last active: Jul 21 2016 12:53 PM
  • Joined: 18 Mar 2008

'OCR'

I mean Screen 'OCR' (not image),

read from screen fonts(that gives good accuracy)

 

all other MS UI you gave, gives same results, and cannot catch non standard MS-GUI

 

I see there is no some general tool that can catch any visible control on a form (from html,java,non MS control...),

it's surprise, because all these controls are actually under MS-windows, no connect what application made it's visible.

and the user, actually see same controls.



Cyberfred95
  • Members
  • 3 posts
  • Last active: Apr 12 2015 02:38 PM
  • Joined: 07 Oct 2014

Hi,

I need to automate tasks on a java Application, Oracle E-business suite in fact. The application is generating multiple sub screens during task process and I need to identify which sub screen is active to send to it the good sequence of key. Can you please give me exemple on how I can do this with the library post on this topic ?

I try multiple think but was not able to achieve any recognition of the java application itself by Autohotkey.

thanks for your help

regards