GUI: piano keyboard

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

26 Apr 2018, 14:48

Oh wow, what a beautiful GUI and thanks for conversion!
I noticed:
- at start the preselected instrument misses for one notch.
- note velocity is used as volume
- instrument is set every time the note is played
- only four channels and no possibility to set its distinctive instrument
- if not existing device is specified, at start non is selected
- responsiveness of the GUI keyboard is dependent to the current cycle of the 50ms timer
- relatively high latency (not of much importance for manual play)

Here is my attempt to address these observations, except for the last one inherent due to complex graphic.

P.s.:
Esc: exit, F1-F12: select channel 1-12, Shift+F3-F6: select channel 13-16
Selecting list box item with mouse or arrows sets instrument for current channel
Note, channel 10 is reserved for percussions

Edit: Just noticed, I didn't upload latest version. Not essential, but gives better feedback what was chosen.
Edit2: Still one more change for better feedback regarding channel 10.
Edit3: For new script see attachement below.
Last edited by rommmcek on 27 Apr 2018, 17:38, edited 1 time in total.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

27 Apr 2018, 05:01

For some reason information on using OpenGL is scarce. I tried getting rid of the 50ms timer myself but results are not consistent across all machines; while on this XP machine everything works fine with a subclassed WindowProc, on another XP machine with a very old AMD Duron CPU any attempt at resizing the keyboard window results in a CPU overload (100% usage) and the script hangs. Your script doesn't redraw the keyboard window on resize.

In a local version I had already added all 16 channels and provided feedback about Percussions only when channel 10 was selected. There are a few other improvements as well.
I also noted the Velocity vs Volume issue but had not yet time to analyze all the MIDI features, which are also kinda hard to find, MSDN being of no use in this regard.

Unfortunately I have no device that supports GM2, it would've been an extra challenge to support all instruments and percussion sets offered by that extension.

My current goal is to have the virtual keyboard display exactly the characters found on user's physical keyboard, which proves to be another challenge. I haven't yet found a way - if there is one - to build a set of Unicode characters and feed them to the OpenGL functions to be displayed on top of the keys. Currently only latin characters can be displayed and only those in the range A-Z, which doesn't even cover my Romanian layout.

Funny thing is while looking up information on the web I have found a few ancient applications built for Windows 3.1 and 9x that contain more functions, such as dynamic pitch change using a virtual pitch wheel and a few others. This script we work on is very basic compared to what MIDI can really do.

Here's what I have at the moment:
Klavier hero MOD 1.2.2.7z
(8.46 KiB) Downloaded 109 times
v1.2.2:
- externalized instrument list to an ini file for possible future translations
- added save/load window positions between sessions
- added first phase of a keyboard layout choice system (not working yet)
- added detailed info on currently selected MIDi device
- option to hide virtual keyboard by right-click on its titlebar
- keyboard window repaint is now done in a subclassed window procedure instead of a timer (may hang the script, see above comments)
- instrument list is nicely aligned by custom-sized tabs
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

27 Apr 2018, 17:59

- I forgot to place refresh timer in to 2GuiSize:, cause I never resized Gui.
- for low performance CPU we can use -100 in place of -0 (after Gui 2, Show: and in 2GuiSize:). This is better then 100, cause there is a chance it will act occasionally as -0.
- or I believe still better, we can use e.g. SetBatchLines, 20ms in UpdateOpenGL:

- About MIDI info try https://www.midi.org/

- GM2, me neither.

- You try to support all the characters, I got rid of them.

- I tried to implement pitch. But after each use I had to reselect instrument manually (I failed programmatically), so I dropped it for now.

bye!

Edit: Wheel (Pitch change) applied. Very basic though, relies on Key repeat. While playing a note hit & hold Space bar.
Edit2: For script see attachement below.
Last edited by rommmcek on 28 Apr 2018, 15:04, edited 1 time in total.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

28 Apr 2018, 01:59

Personally I hate timers with a passion, trying to keep their usage to a minimum, mostly only when a function needs to return quickly but still has possibly lengthy things to do. The WindowProc subclassing usually works but apparently not in all situations. Have you tried this version, did it work for you without hanging?

I've been at midi.org and they're a-holes just like others, requiring registration just to be able to download the damn specs. To hell with them, there are other sources, it just needs time and focus. I did gather bits and pieces, they just need to be put together.

GM2 is not big deal theoretically, most important thing in my opinion is the ability to switch banks, which gives a much richer set of instruments. I may attempt a "blind" implementation in the hope that someone someday may have the chance to test and provide feedback.

The user needs some visual cue as to how the keys are organized, we can't leave them in the dark. And it's best that the characters in the GUI correspond perfectly to those marked on the physical keyboard. Currently it automatically detects the current software layout which may or may not correspond to the physical markings. Moreover, certain physical layouts may have keys missing or differently arranged (notably laptops/notebooks) and the user may believe it is a bug in the script when they can't reach certain notes, so they need some sort of a key map.
However, Unicode is difficult to work with in conjunction with OpenGL.

Pitch wheel may be best implemented on mouse drag or hover as I've seen in a few very old applications. Keys may be used too, but do keep in mind pitch bend should work both up and down, usually +/- 2 semitones around the central note, so a couple keys should be assigned to this function (maybe Plus/Minus on NumPad, Ins/Del, Home/End, PgUp/PgDn). There are other effects that work this way too.

There's still something missing in the refresh routine in your version (tried both UR1 and UR2): when keyboard GUI is (partially) covered by another window or moved outside the screen and then revealed, it doesn't redraw. And on resize UR2 takes CPU at 100% on this machine.
Pitch bend does work.
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

28 Apr 2018, 15:02

- UR3 has no timers and very low CPU-usage for resize, it'll refresh when being activated. Can't do it better, since I do not experience any refresh problems (except for compiled script which does not refresh at all, i.e. never for no version, be it my or yours), besides I lack experience!

- I noticed at midi.org are .+ss-.+oles (read RegEx signs respectfully!), but I look just here

- Currently I'm not interested at GM2

- As mentioned above, wheel is very basic. An example how it works.

- For key disposition reference I suggest external info, besides I don't think current layout is the best.
- have a look at this one. Great idea, I guess (..., but sorry, site has very high CPU-usage).

Edit: In the UpdateOpenGL: of UR3 there is SetBatchLines, 100ms, but should be SetBatchLines, 1ms. That brought me to the idea of one more try.
Edit2: For script see attachement below.
Last edited by rommmcek on 30 Apr 2018, 23:47, edited 1 time in total.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

29 Apr 2018, 04:06

Ah, the curse of the timer! :) I finally made my version to work without hanging on the Duron machine by using a timer in the WindowProc for the GUI update. It now has a short lag in the refresh but unless I find a better solution it'll stay that way. Frankly I don't fancy this OpenGL at all but AHK doesn't have any better native way (that I know of) of drawing and resizing a complex element such as the keyboard.

I already have a solution for the key map reference but it strictly depends on a certain font that is not native to any Windows version, for a few symbols: Arial Unicode MS. The native Arial font may contain those symbols too but only in newer Windows such as 10 and maybe 8.x, definitely not in XP, however I'm not sure about the range of Unicode groups required for the actual characters.
There's Segoe UI Symbol too, but again it's not native to all OS versions and at least on my XP it doesn't display a few of my national characters.
Another solution would be to use images (icon, bitmap) for the symbol keys but that would complicate things and may not fit as well in the design.
Here's a preview:
20180429084546.png
20180429084546.png (13.25 KiB) Viewed 4427 times
I know the info on midi.org you linked to, it's incomplete, lacks detailed explanations and examples.

There's one small bug in UR3 and UR4: when switching channels and then reverting to previous channel, the instrument selected in the list will be one less. In example, if I had instrument 100 selected in channel 1, then switched to channel 5, then back to channel 1, at that point it will select instrument 99 instead of 100.
Another bug related to the refresh/redraw: keyboard GUI may remain glued to the mouse cursor after a titlebar click, without a mouse button being held down; a second click is required to detach it. Steps to reproduce (at least on my XP machine):
- grab the keyboard GUI by the titlebar, drag it partly outside the screen, bring it back and release it (a portion will be blanked out needing repaint)
- focus the main GUI by clicking its titlebar (not any other element!)
- click and release the keyboard GUI titlebar, then move the mouse - the keyboard would stick to the cursor
- click keyboard GUI titlebar again to detach it

Haven't checked out the virtualpiano site, wanted to reply here first in case the browser would choke or crash. Will edit later if necessary.

In the mean time here's a new version with a few fixes:
Klavier hero MOD 1.2.4.7z
(9.61 KiB) Downloaded 79 times
v1.2.4:

- fixed hang with 100% CPU usage on keyboard GUI resize on low-end machines (i.e. my 800MHz AMD Duron)
- improved mouse click responsiveness (added listening to double-clicks)
- added tray tip
- fixed unnecessary instrument list loading (eliminates flickering on channel change)
- attempted improvement of repaint timing
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

30 Apr 2018, 00:31

Your latest version works awesome, except loading at start is a bit slow
- but there must be a typo in EnableDraw: label, GoSub should be GoTo, otherwise repositioning at the start don't work properly, besides script crashes when Keyboard is hidden via ComboBox1 and then reshown.

I have been able to reproduce all the bugs you reported (see pic in the attachement).
- I wonder if my two versions work on your "slow" computer? (one descendant form your the other form my script)
- otherwise I'll adopt your solutions, but they are pretty extensive and I didn't have time yet.
Attachments
Klavier heros.zip
(21 KiB) Downloaded 75 times
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

30 Apr 2018, 11:47

Thank you for testing.

The slow start is because the script loads all keyboad layouts known to the sistem in order to get their HKLs and add them to the layout list. Currently this is not completely accurate and it is not the complete solution for the visual cues but since I added it to the script I let it stay.

Yesterday I found out about that silent crash but only after posting v1.2.4. Indeed there was a problem in EnableDraw which has been fixed. Now it works correctly.

I haven't yet tested your latest versions on the Duron machine but I suspect they'd work. Currently I have another machine running and three of them simultaneousy would be too much for my electricity bill but I'll try them on first occasion.

However, testing on this XP machine revealed a small issue in UR5: when hiding and showing the keyboard it will retain size but not position so it will always be displayed at the top of the screen although it may have been dragged at the bottom before being hidden.
The refresh is now alright and it doesn't hang onto the cursor anymore.

Here's a new version with fixes and improvements (or bugs, who knows):
Klavier hero MOD 1.2.5.7z
(10.09 KiB) Downloaded 72 times
v1.2.5:
- fixed silent crash on switching Hidden => OpenGL keyboard (regression since v1.2.2)
- fixed bad main window coordinates in ini file when window was minimized at script exit
- added automatic save/restore of current MIDI device's volume
- volume and velocity are now modified in real-time
- added mouse wheel control over all selectable controls (not only in THIS script, so be careful where you hover and steer!)
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

30 Apr 2018, 23:46

Made just a quick test, didn't found what causes my reshow positional flaw, but found your script saves position of the Virtual keyboard only after resizing, but not after moving the Gui.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

01 May 2018, 08:43

Looks like we both suffered from a similar issue. :)
Hiding the keyboard actually destroys the window so if the current X & Y coordinates are not retrieved before that it'll use the default ones.
AHK screws up badly when it comes to Window vs Client, and worse than that if a window is minimized WinGetPos will return bogus values (-32000 for x/y, 0 for w/h). I had to resort to a completely separate system to retrieve and set the correct sizes of the windows.
Klavier hero MOD 1.2.6.7z
(10.4 KiB) Downloaded 76 times
v1.2.6:
- changed retrieval/setting coordinates system for both windows
- fixed mouse wheel behavior to avoid unwanted interaction with other windows not belonging to this script
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

01 May 2018, 23:34

This time you got all wrong!
- first run is o.k., but in the center of the screen (does not obey settings)
- after hiding the keyboard reshow is mis-formed & misplaced, sometimes out of the screen with no possibility to bring it back
- second run is awful, main Gui nowhere to find probably out of the screen with no possibility to see it, keyboard the same as in first run after being hidden
- in such conditions was difficult to test further, but midi seems to work o.k.

P.s.: After minimizing the Gui Ahk sets Errorlevel in GuiSize label. That is not considered as misbehavior cause you can use:

Code: Select all

GuiSize:
If Errorlevel
   return
...
as first two lines of the label.
Attachments
VK misformed&misplaced.gif
VK misformed&misplaced.gif (4.39 KiB) Viewed 4341 times
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

02 May 2018, 00:27

There was a chance that might happen. I suspect it's related to the WINDOWPLACEMENT structure used in GetWndPos() and SetWndPos().

Can anyone confirm that it works correctly under x86? If it does then it's the alignment I added in the structure when it may not have been necessary.
On x64 this can be tested by replacing all occurences of A_PtrSize with the value of four (4) in both functions mentioned above. If it works that way, I'll fix the code. Unfortunately I don't have access to an x64 machine for testing so feedback is important.

There is no GuiSize for main GUI because that one is not resizable. Even if it were, moving the main GUI around and then minimizing it would lose the current coordinates anyway on exit. The pair GetWindowPlacement()/SetWindowPlacement() should work correctly in any situation, provided the coordinates are correctly set/retrieved.

So please someone test the script:
1. under 32-bit AHK as is
2. under 64-bit AHK after replacing A_PtrSize with 4 in GetWndPos() and SetWndPos()
and report back with the results. Thank you.
Oh and please check the values for all Main… and Kbd… keys in the ini file and fix them if they're off or just delete them all completely.
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

02 May 2018, 03:01

Changing A_PtrSize with 4 fixes the problem! Pretty cool, debugging just theoretically!
One little inconsistency remains though. When Keyboard is hidden via ComoBox4 and then Klavier hero is closed, at next run Keyboard starts up hidden, but is correctly positioned when shown. This does not occur if Keyboard was hidden via right click on Title bar.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

02 May 2018, 03:21

Oh, I'm too good for this century! :P :lol:
Just kiddin'. I'm glad it's an easy fix, I was torn whether I should add that alignment and decided to do it, if only to avoid any memory access violation or whatever error may be thrown when reading outside a buffer. I'll remove the alignment for the next version.

The inconsistency happens because DrawMode is not modified acordingly in that situation. The fix is easy:

Code: Select all

2GuiClose:
GuiControl, 1:Choose, DrawMode, 1
DrawMode--  ; add this to fix the inconsistency
Thank you for the prompt testing and for reporting the issues. I'll post a new version sometime later after trying a few tricks with that OpenGL.
Part of my AHK work can be found here.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

02 May 2018, 14:44

Hopefully this new version won't be a flop. I fixed the issues reported by rommmcek above and finally finished implementing layout selection, complete with preview window (which can be hidden on titlebar right-click just like the virtual keyboard).
Enjoy! :wave:
Klavier hero MOD 1.3.0.7z
(12.12 KiB) Downloaded 68 times
v1.3.0:
- roughly finished implementing layout selection (requires Arial Unicode MS or another very good/complete Unicode font)
- added preview window for the selected layout (just because I could :-D )
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

03 May 2018, 00:40

Works fine, a bit complicated, but very skillful coding.

- didn't get this: did you fix mouse wheel or disable it (for me doesn't work and I miss it)
- kb layout Gui is centered every time when reshown (I applied simple fix adopted form my version)
- limited interaction for Vk Gui with user keyboard (no Gui menu for move, size...)
- right click on Vk title bar goes trough invoking underneath win. menu or even activating it (pretty disturbing)
Attachments
Klavier heros 2.zip
(16.46 KiB) Downloaded 63 times
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

03 May 2018, 02:55

Glad it works. :) Yep, a bit complicated, I know, maybe because I'm trying to keep everything as much as possible in a "classic" syntax. At some point I may want to retrofit this script to AHK 1.0 for my 98SE, if I ever get it working again. Or maybe my patterns of thinking are weird. Oh well… :)

To the issues now:
1. Doesn't the wheel work at all now? Strange, it should work on hovering controls (except for the layouts list, couldn't figure out why yet). There is however a situation I'm not certain of whether I should allow wheel interaction or not, and decided not to; maybe that's where the problem lays. Please edit MWheel() and remove the zero after Return in

Code: Select all

	If (hA != hWnd)
		Return 0  ; remove the zero here
and see if it now works. With that modification the focused control will take wheel action even when it's not hovered and I wanted to avoid that because it's too easy to inadvertently modify a setting. Will check that function more thoroughly, maybe there is a good solution to that.

2. The preview window implementation is not fully complete, I did intend to apply the same load/save technique to it too and will do that in a future version.

3. The VK GUI was meant to be as distraction-free as possible, that's why I took out all titlebar elements. Besides, I'm a mouse guy and find it easier to drag/size using the mouse rather than keyboard interaction. But if it's needed I'll try to put together a few hotkeys (although I'm not a fan of them :) ) for moving/resizing the VK. However, that may become a problem if later on other functions will be implemented that would require key combinations - too many hotkeys may create confusion and/or unwanted behavior. Will see.

4. I noticed that click-through behavior too a couple times but only in certain applications. I'm afraid there's not much to be done since the context menu usually fires on Rbutton-up, not Rbutton-down, and hiding the window leaves the user with the right button pressed for a short time. However, I consider this a flaw in the other applications that process a right-click without them being the active/focused window (one example being Metapad which I'm using daily).
Problem is, the script doesn't seem to respond to WM_NCRBUTTONUP unless it's a double-right click - at least it does so on my XP. I've been trying to fix it to no avail. And strangest thing is, the preview window doesn't exhibit that behavior. I tried "breaking" the preview window by giving it the very same properties as the VK but it won't. No idea what makes them behave differently under apparently the same conditions.
At this point I can only delay the GUI hiding in the hope that the user will not hold the right button down for too long. If anybody has a better idea - except for removing the right-click feature - please chime in.
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

03 May 2018, 03:22

@1: On Wni10 x64 works perfect as it should (whether on control or not, or on back ground Win - not active)
@4: Delay would do great. (what about a temporary Gui underneath the cursor for the duration of delay time - could be transparent?)

Just figured it out, my "magic" line could be more condensed:

Code: Select all

Gui, 2:Show, % (!sr?"":"x" KbdX " y" KbdY sr:="") " w" KbdW " h" KbdH, Virtual keyboard ; for UR6
Gui, 3:Show, % !sl?"":"x" 380 " y" 360 sl:="", Preview layout: %Layout% ; for 1.30
Edit: This seem to fix the problem @4:

Code: Select all

HideKeys(wP, lP, msg, hwnd)
{
Global
MouseGetPos, mx, my
ToolTip, Hiding the "`n" Virutal keyboard, % mx-1, my-1
If (hwnd=hWnd2)
	GoSub 2GuiClose
Else If (hwnd=hPreview)
	GoSub 3GuiClose
ToolTip
}
Last edited by rommmcek on 03 May 2018, 03:38, edited 1 time in total.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: GUI: piano keyboard

03 May 2018, 03:37

OK, I'll leave that wheel fix in until a better solution is found.
Please try the following, see if the context menu issue gets improved:

Code: Select all

HideKeys(wP, lP, msg, hwnd)
{
Global
SetFormat, Integer, H
hwnd+=0
SetFormat, Integer, D
If hwnd not in %hWnd2%,%hPreview%
	   Return
SendMessage, 0xA5, %wP%, %lP%,, ahk_id %hwnd%
If (hwnd=hWnd2)
	   SetTimer, 2GuiClose, -100 ; value may vary upwards for a longer delay
Else If (hwnd=hPreview)
   	SetTimer, 3GuiClose, -10 ; this delay may not even be necessary, it can probably be set to '-0'
Return 0
}
A thought crossed my mind that the click goes through only for the VK window because that window eventually gets destroyed after hiding, while the preview is simply hidden, so theoretically the system could forward the right button state to the window underneath even if inactive.
I was already planning to replace this window destroy/recreate cycle with simple hide/show so the issue may go away completely if that was the cause.
Part of my AHK work can be found here.
User avatar
rommmcek
Posts: 1470
Joined: 15 Aug 2014, 15:18

Re: GUI: piano keyboard

03 May 2018, 03:52

Both my and your version work for short right click.
For my version if Gui would be applied, maybe Rbutton up would obey.

Edit: Yes Rbutton obey even for ToolTip, so

Code: Select all

KeyWait, Rbutton
ToolTip
does the job, providing the user does not move the mouse outside of ToolTip while still pressing Rbutton.
Last edited by rommmcek on 03 May 2018, 04:20, edited 1 time in total.

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: Google [Bot], OrangeCat, RussF and 127 guests