24 replies to this topic

[cire]

Hello,

I'm using OmegaT, an opensource Computer Aided Translation tool, written in Java.

When Autohotkey is running, I can't get ô (an o with accent circonflexe "^" on top of it) in OmegaT and I get a simple "o" instead.
E.g. ô in "rôle" (which I normally obtain by typing the two letters ^ and o one after the other).
Within OmegaT, I get "role" (without the ^).

I haven't defined any hotstrings starting with "`^o" !

Can someone give me some hints on where to search or how to solve this problem?
Thanks in advance.

Here are the configuration details of my PC:
Operating system = Windows 2000
keyboard layout = azerty
keyboard locale = French (France)
Java version = Java 2 Standard Edition Version 1.5.0 (build 1.5.0_06-b05)

And herebelow an extract from the key history :
Window: D:\Home\bin\hotkeys.ahk - AutoHotkey v1.0.43.09
Keybd hook: yes
Mouse hook: yes
Enabled Timers: 0 of 0 ()
Interrupted threads: 0
Paused threads: 0 of 0 (0 layers)
Modifiers (GetKeyState() now) =
Modifiers (Hook's Logical) =
Modifiers (Hook's Physical) =
Prefix key is down: no

NOTE: To disable the key history shown below, add the line "#KeyHistory 0" anywhere in the script. The same method can be used to change the size of the history buffer. For example: #KeyHistory 100 (Default is 40, Max is 500)

The oldest are listed first. VK=Virtual Key, SC=Scan Code, Elapsed=Seconds since the previous event. Types: h=Hook Hotkey, s=Suppressed (blocked), i=Ignored because it was generated by an AHK script, a=Artificial, #=Disabled via #IfWinActive/Exist.

VK SC Type Up/Dn Elapsed Key Window
-------------------------------------------------------------------------------------------------------------
01 000 u 0.13 LButton OmegaT-1.6.0_RC8 :: 2Translate :: I225 Project Charter v2.1.ott
01 000 d 1.02 LButton
01 000 u 0.13 LButton
DD 01A u 1.64 ACCENT CIRCONFLEXE
DD 01A i d 0.48 ACCENT CIRCONFLEXE
DD 01A i u 0.00 ACCENT CIRCONFLEXE
4F 018 d 0.00 O
4F 018 u 0.17 O

[AT]

Hello Cire,

I had the same issue when I was making a script to send text to a program.

As the text was in Portuguese (BR), i found out this solution in the Forum:

o={alt down}{numpad0}{numpad2}{numpad4}{numpad4}{alt up} ;The var o, receives the value needed to send ^o correctly, use Character Map in Windows, to find out the missing letters and their keystroke (ex.: ALT+0244)

Send r%o%le ; send r^ole correctly!

Maybe it helps you!

[PhiLho]

I can confirm that, and I would file this in the Bug Report section.
I tested with two Java applications, one running with Java 1.5, another with Java 1.4.
When my Permanent script is running, I can't type ô in these applications (but it is OK with Firefox and SciTE, for example). When I exit the script, I can type this letter.
I narrowed the problem to a hotstring definition: with :*:²u::{Left 7} active (and other similar definitions), I get the problem. Without it but with other hotkeys, no problem.
I guess it is a problem with a hook, and it is probably hard to correct, but the problem must be known by users.
WinXP Pro SP2, French locale.

[Chris]

Thanks for reporting this. My initial guess is that it has something to do with ToAsciiEx() disrupting dead keys. ToAsciiEx() is the built-in OS function that converts a virtual key code with its scan code into a visible character; however, is has some bad side effects that are difficult to work around.

Although I've no idea how to fix it yet, I definitely want to. Here are the things I could use help with:

1) The key history excerpt posted above is strange because it says you released ACCENT CIRCONFLEXE without ever having pressed it down; that is, it shows a "u" event only with no preceding "d" event. Perhaps you could re-confirm that result; if it's accurate but occurs only in Java (not in other apps), there might be no way to fix it (it could even be a limitation of Java, though that is far from certain).

2) It would be good to have confirmation that this issue occurs when ANY script with ANY hotstring is running (which I suspect is true). Perhaps you could try a one-line script with the following in it, ensuring it's the ONLY script running:
::btw::by the way

3) Finally, it would be ideal if I could reproduce this on my own system via a simple Java app available for download -- preferably a small one that doesn't make a lot of system/registry changes. Perhaps someone could suggest one.

Thanks.

[PhiLho]

I initially pasted a key history, but looking OK, I removed it. I thought also to provide a link to a simple Java application, but since I don't know if you have ^ as dead key, I don't know if you can easily test this, so I didn't.
Also I forgot to mention I tested with versions

1) Key history in SciTE, without active hotstring, but with #InstallKeybdHook:

DD  01A	 	d	0.14	ACCENT CIRCONFLEXE	
DD  01A	 	u	0.08	ACCENT CIRCONFLEXE	
4F  018	 	d	0.14	O              	
4F  018	 	u	0.06	O              	

Note that sometime, I get:

DD  01A	 	d	0.06	ACCENT CIRCONFLEXE	
4F  018	 	d	0.14	O              	
DD  01A	 	u	0.03	ACCENT CIRCONFLEXE	
4F  018	 	u	0.08	O              	

as if the ^ took some time to go down...

2) Idem with ::btw::BTW!

DD  01A	s	d	0.08	ACCENT CIRCONFLEXE	
DD  01A	 	u	0.09	ACCENT CIRCONFLEXE	
4F  018	 	d	0.13	O              	
DD  01A	i	d	0.00	ACCENT CIRCONFLEXE	
DD  01A	i	u	0.00	ACCENT CIRCONFLEXE	
4F  018	 	u	0.06	O              	

3) In a Java applet: http://htmltea.sourc...editortest.html

DD  01A	s	d	0.13	ACCENT CIRCONFLEXE	
DD  01A	 	u	0.09	ACCENT CIRCONFLEXE	
4F  018	 	d	0.16	O              	
DD  01A	i	d	0.00	ACCENT CIRCONFLEXE	
DD  01A	i	u	0.00	ACCENT CIRCONFLEXE	
4F  018	 	u	0.08	O              	

ie. it is the same, but the o is left unaccented.

Sometime (quite often, actually), with the hotstring on, I get also this sequence (in any application):

DD  01A	s	d	1.06	ACCENT CIRCONFLEXE	
4F  018	 	d	0.06	O              	
DD  01A	i	d	0.00	ACCENT CIRCONFLEXE	
DD  01A	i	u	0.00	ACCENT CIRCONFLEXE	
DD  01A	 	u	0.03	ACCENT CIRCONFLEXE	
4F  018	 	u	0.05	O              	

HTH.

[Chris]

It's possible (though somewhat unlikely) that this issue was fixed in v1.0.44.03. In any case, I can't reproduce it on my system using the following steps:

1) Have the following keyboard layout available via Control Panel -> Regional:
Language = French (France)
Keyboard = French (that's how XP displays it anyway)

2) Visit a Java app like http://htmltea.sourc...editortest.html
(I tested it with both Java 1.4.1 and 1.5)

3) Ensure French layout is active and that at least one script containing hotstrings is running.

4) In the Java app, press ^ (which is '[' key on English keyboards) followed by the letter ‘o’. The character ô appears.

I suspect this is still an issue, but it’s hard for me to analyze and debug it without reproducing it. Therefore, if anyone can still reproduce this with a Java app (preferably one that’s easily accessible), please post here.

Thanks.

[numEric]

Chris, I could reproduce the problem with v1.0.44.07 using the same test procedure:

When a script with no hotstring is running:

[color=red]DD  01A	 	d	2.73	ACCENT CIRCONFLEXE[/color]
[color=orange]DD  01A	 	u	0.06	ACCENT CIRCONFLEXE[/color]	
[color=black]41  010	 	d	0.00	A              	[/color]
[color=darkblue]41  010	 	u	0.11	A              	[/color]
[color=red]DD  01A	 	d	0.16	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.08	ACCENT CIRCONFLEXE[/color]	
[color=black]45  012	 	d	0.08	E              	[/color]
[color=darkblue]45  012	 	u	0.09	E              	[/color]
[color=red]DD  01A	 	d	0.11	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.13	ACCENT CIRCONFLEXE[/color]	
[color=black]49  017	 	d	0.20	I              	[/color]
[color=darkblue]49  017	 	u	0.05	I              	[/color]
[color=red]DD  01A	 	d	0.23	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.09	ACCENT CIRCONFLEXE[/color]	
[color=black]4F  018	 	d	0.13	O              	[/color]
[color=darkblue]4F  018	 	u	0.08	O              	[/color]
[color=red]DD  01A	 	d	0.17	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.09	ACCENT CIRCONFLEXE[/color]	
[color=black]55  016	 	d	0.23	U              	[/color]
[color=darkblue]55  016	 	u	0.05	U              	[/color]

When a script containing a hotstring is running:

[color=red]DD  01A	s	d	1.27	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.09	ACCENT CIRCONFLEXE[/color]	
[color=black]41  010	 	d	0.00	A              	[/color]
[color=red]DD  01A	i	d	0.00	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	i	u	0.00	ACCENT CIRCONFLEXE[/color]	
[color=darkblue]41  010	 	u	0.09	A              	[/color]
[color=red]DD  01A	s	d	0.11	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.11	ACCENT CIRCONFLEXE[/color]	
[color=black]45  012	 	d	0.02	E              	[/color]
[color=red]DD  01A	i	d	0.00	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	i	u	0.00	ACCENT CIRCONFLEXE[/color]	
[color=darkblue]45  012	 	u	0.09	E              	[/color]
[color=red]DD  01A	s	d	0.13	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.09	ACCENT CIRCONFLEXE[/color]	
[color=black]49  017	 	d	0.20	I              	[/color]
[color=red]DD  01A	i	d	0.00	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	i	u	0.00	ACCENT CIRCONFLEXE[/color]	
[color=darkblue]49  017	 	u	0.08	I              	[/color]
[color=red]DD  01A	s	d	0.17	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.11	ACCENT CIRCONFLEXE[/color]	
[color=black]4F  018	 	d	0.11	O              	[/color]
[color=red]DD  01A	i	d	0.00	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	i	u	0.00	ACCENT CIRCONFLEXE[/color]	
[color=darkblue]4F  018	 	u	0.06	O              	[/color]
[color=red]DD  01A	s	d	0.19	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	 	u	0.13	ACCENT CIRCONFLEXE[/color]	
[color=black]55  016	 	d	0.19	U              	[/color]
[color=red]DD  01A	i	d	0.00	ACCENT CIRCONFLEXE[/color]	
[color=orange]DD  01A	i	u	0.00	ACCENT CIRCONFLEXE[/color]	
[color=darkblue]55  016	 	u	0.05	U              	[/color]

The cause seems obvious: when hotstrings are defined, AutoHotkey generates an unwanted repetition of dead-key events between the press and release of the next key pressed by user. (Java apps don't seem to appreciate such key sequences...! Maybe they require the last dead-key release to occur before a letter key is pressed, in order to interpret the sequence as a modified character?)

I noticed that it also occurs with modifiers obtained via a key combination, such as "`" (AltGr + 7 on a french keyboard -- shown as "è" in the key history):

[color=black]A2  01D	 	d	1.61	CTRL           	
A5  138	 	d	0.00	ALT DROITE     	
37  008	s	d	0.13	è              	
37  008	 	u	0.06	è              	
A2  01D	 	u	0.03	CTRL           	
A5  138	 	u	0.00	ALT DROITE     	
49  017	 	d	0.14	I              	[/color][color=red]
A2  01D	i	d	0.00	CTRL           	
A5  138	i	d	0.00	ALT DROITE     	
37  008	i	d	0.00	è              	
37  008	i	u	0.00	è              	
A2  01D	i	u	0.00	CTRL           	
A5  138	i	u	0.00	ALT DROITE     	[/color]
[color=black]49  017	 	u	0.05	I              	[/color]

(The unwanted key events in red were generated by AutoHotkey.)

[Chris]

Chris, I could reproduce the problem with v1.0.44.07 using the same test procedure.

I wonder what differences would explain why it doesn't occur on my system. Regardless, I'll continue to seek a solution.

when hotstrings are defined, AutoHotkey generates an unwanted repetition of dead-key events between the press and release of the next key pressed by user. (Java apps don't seem to appreciate such key sequences...!

It must generate such events; otherwise, hotstrings would disrupt dead keys in all applications. This is because the OS's ToAsciiEx() function interferes with pending dead keys the moment you call it. The only way I know of to undo this interference is to resend the dead key. This API issue is extremely obscure; hours of web searching and MSDN research turned up very little of use (beyond the fact that it's a known limitation of the OS).

Maybe they require the last dead-key release to occur before a letter key is pressed, in order to interpret the sequence as a modified character?)

Interesting. Or perhaps it's simply a timing issue, where Java needs more time between keystrokes.

Thanks for the detailed post. I'll ponder and experiment some more.

[numEric]

I wonder what differences would explain why it doesn't occur on my system. Regardless, I'll continue to seek a solution.

If you have tested with the same virtual-key and scan codes as I posted, then I have no idea what it could be...

when hotstrings are defined, AutoHotkey generates an unwanted repetition of dead-key events between the press and release of the next key pressed by user. (Java apps don't seem to appreciate such key sequences...!

It must generate such events; otherwise, hotstrings would disrupt dead keys in all applications. This is because the OS's ToAsciiEx() function interferes with pending dead keys the moment you call it.

Ok, now I understand why AutoHotkey's keyboard hook suppresses dead-key-down events when there are hotstrings in use. (Btw, suppressing the subsequents dead-key-up events as well would make sense to me!)
A good workaround could be to explicitly set the system's dead-key flag to the correct state after calling ToAsciiEx(), but this might be particularly difficult to achieve without a specific API function for that purpose! :roll:

Maybe they require the last dead-key release to occur before a letter key is pressed, in order to interpret the sequence as a modified character?)

Interesting. Or perhaps it's simply a timing issue, where Java needs more time between keystrokes.

I have examined the key events sent by AutoHotkey's hook using Microsoft's Keyboard Diagnostics utility (MSKey.exe). (I ensured that MSKey's hook was referenced after AutoHotkey's in the hook chain by starting MSKey before launching the script.)
I noticed that AutoHotkey's hook doesn't actually send the events in the order they appear in the key history: it always sends the dead-key events (down, up) before the letter-key events, which is the right behavior. So my remark above doesn't make sense...

I did further testing using the SendInput command, and found out that, while most applications need a {dead key down}{letter key down} sequence to produce a character with a diacritic, Java apps react on a {dead key up}{letter key down} sequence! (In both cases, other key events involving the same keys seem to have no effect, regardless of their position in the sequence.)
Example:

SendInput, {vkDDsc01A Up}{vk4Fsc018 Down}	; works with Java apps only

SendInput, {vkDDsc01A Down}{vk4Fsc018 Down}	; works with most non-Java apps

Another test shows that, under normal usage (i.e. when all key events (down + up) are sent, unlike in the previous example), sending key events to a Java app is more reliable with a 1 ms delay between each event (at least on my system!).

SendDelay := 1	; on my system, this test fails if SendDelay = 0

SendInput, {vkDDsc01A}
Sleep, %SendDelay%
SendInput, {vk41sc010}
Sleep, %SendDelay%
SendInput, {vkDDsc01A}
Sleep, %SendDelay%
SendInput, {vk45sc012}
Sleep, %SendDelay%
SendInput, {vkDDsc01A}
Sleep, %SendDelay%
SendInput, {vk49sc017}
Sleep, %SendDelay%
SendInput, {vkDDsc01A}
Sleep, %SendDelay%
SendInput, {vk4Fsc018}
Sleep, %SendDelay%
SendInput, {vkDDsc01A}
Sleep, %SendDelay%
SendInput, {vk55sc016}

[Chris]

..suppressing the subsequents dead-key-up events as well would make sense to me!

Interesting. Based on your comment below, maybe this would be enough to entirely solve the issue in Java.

I did further testing using the SendInput command, and found out that, while most applications need a {dead key down}{letter key down} sequence to produce a character with a diacritic, Java apps react on a {dead key up}{letter key down} sequence!

Great discovery! This could be the info I need to solve it.

A good workaround could be to explicitly set the system's dead-key flag to the correct state after calling ToAsciiEx(), but this might be particularly difficult to achieve without a specific API function for that purpose! :roll:

Yes, and it might be almost as good to find out if there are any pending dead keys in the buffer so that the call to ToAsciiEx could be avoided. But I haven't found any calls to do it (maybe some kind of device/driver call could do it, but an area of the API that's unfamiliar to me).

I noticed that AutoHotkey's hook doesn't actually send the events in the order they appear in the key history: it always sends the dead-key events (down, up) before the letter-key events, which is the right behavior.

I think that's caused by the keyboard hook procedure calling itself recursively so to speak (when it simulates keystrokes), which the primitive key-history feature isn't designed to handle.

[Chris]

I've made a longshot attempt to fix this by suppressing/blocking the up-events of dead keys to match that fact that their down-events are blocked.

If anyone has time, please test the following version, perhaps by using the numbered steps in my post above: http://www.autohotke... ... ressed.zip (the issue doesn't occur on my system, so I can't test it).

Thanks.

[PhiLho]

I am sorry to write that this version doesn't bring any improvement while typing in the applet... Well, actually, I saw a â once, but couldn't get another...

Is there any test or data dump that I can do to help you?

[Chris]

Just testing it was enough, thanks. However, I'd appreciate it if at least one other person who had this issue could test the above version (in case its a system-specific issue).

[PhiLho]

As long as it is documented somewhere, I can live with this problem...
It would have been nicer to solve this as long as it disrupts nothing.
I don't know if it is feasible, but if the Suspend Hotkeys menu item could also "suspend" this problem (removing hook or whatever) as long as it is ticked, it would allow to input such text in a Java application without stopping the permanent script I have, then resuming it painlessly.

[Chris]

I believe "Suspend" would work around this problem because hotstrings are disabled during Suspend (if anyone finds out otherwise, please let me know).

Here's another modified version:
http://www.autohotke... ... leep20.zip

It sleeps for 20 ms after reinserting the dead key but before allowing the completion key like "u" to go through. However, even if it solves it, it might be best not to use this fix because hooks aren't supposed to sleep or do other long delays (there would likely be side effects or performance problems).

Another possible solution is to have the hook substitute a new, simulated "u" for the physical "u". However, that is a longshot and even if it worked, it might cause side-effects in DirectX applications.

If anyone wants them, here are the technical details of the dead key problem: Calling ToAsciiEx() on the dead key itself doesn't disrupt anything. The disruption occurs on the next key (for which the dead key is pending): ToAsciiEx() consumes previous/pending dead key, which causes the active window's call of ToAsciiEx() to fail to see a dead key. So unless the program reinserts the dead key after the call to ToAsciiEx but before allowing the dead key's successor key to pass through to the active window, that window would see u instead of û.

Thanks for your help.