Did something change with the Zip handling?

Post AHK_H specific scripts & libraries and discuss the usage and development of HotKeyIt's fork/branch
User avatar
Maasq
Posts: 10
Joined: 09 Jun 2017, 14:36
Contact:

Did something change with the Zip handling?

10 Feb 2018, 18:11

As the title says - did something change in the Zip coding part of AHK_H between v1.1.24 and 1.1.27.7?

My application is preparing a filename.mod file for use in another program - Fantasy Grounds. This is just a renamed filename.zip.
Inside the zip there are a couple of folders of images (renamed and copied there by my application) and 2 XML files generated by my application.

Using the same code for both, 1.1.24 produces a mod file that opens, and 1.1.27 produces one that doesn't open. I have switched versions several times to be sure, as I thought I was going mad. I have used Notepad++'s 'compare' on the XML files and they are identical.

I can open both versions in 7Zip and access the files within, so it isn't broken as such.

Has anything changed with how a zip is encoded by default between the versions?

Maasq
Maasq
Author of Engineer Suite (Written in AHK)
www.masq.net
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

16 Feb 2018, 15:42

See this commit: Changed zip version to 4.5
And there were more other commits related to zip in versions after 1.1.24.5.
I wish I had an exe version of 1.1.24-something. maybe it'd work to compile a usable exe in XP.
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 04:55

Can you show your script and upload an example including the files you try to zip.
I just tried zipping text and jpg files and it works just fine.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 05:06

@HotkeyIt: While you're here, may I ask: is there any place one can get older versions of AHK_H for testing purposes? I've been bashing my head against the wall trying to compile some script to a working executable and so far failed with 1.1.25+.
Is there any known incompatibility between AHK_H and XP, any difference in bin vs exe so that scripts run with the exe but not when compiled with the bin?

[EDIT]
As not to be completely off-topic I just looked inside a compiled script and tried to extract some of the zipped resources in RCDATA as well as some others in LIB from another compiled script built by someone else. Files extracted (using Resource Hacker 4.5.30) from both executables appeared as corrupt archives to Explorer, Total Commander packer plug-ins, and 7-zip 9.31 alpha.
Compression was not used on either compiled files. Therefore I can confirm that the zip operation in AHK_H is faulty at least under XP-SP3 x86, which leads to inability to use compiled scripts on such systems.
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 08:48

You can find older versions here: https://github.com/HotKeyIt/ahkdll-v1-r ... its/master
I will try to get my XP Virtual machine set up and check what is wrong.

Compiled script does not use standard zip files, so you can't extract and open it.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 09:25

Thank you so much for the link, GitHub is so weird to me, can't wrap my head around its usage.
Ah, so the PK header in those resources is misleading. Well, that explains it, thank you.
I'll start performing some tests, hopefully something good will come out of it.
Is there any chance to lose the CRT dependency in AHK_H itself and inherently in compiled scripts? It's annoying to drag arround extra files for fear some system may not have them installed, and distribution of mixed x86/x64 compiled scripts gets a bit complicated.

One more question: are there changes in compiler's command line arguments compared to old AHK? I tried the /in argument and got an error "D:\in not found". I mean, when filey type association is set in registry and there's a Compile Script option in context menu, choosing that option leads to that error.
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 09:53

Unfortunately CRT can't be changed :(
Command line arguments should work if you compile Ahk2Exe, otherwise you will need to change them, see here.

Code: Select all

Ahk2Exe.exe Ahk2Exe.ahk /in "C:\Program Files\AutoHotkey\Compiler\Ahk2Exe.ahk" /out "C:\Program Files\AutoHotkey\Compiler\Ahk2ExeH.exe" /bin "C:\Program Files\AutoHotkey\Compiler\Ahk2Exe.exe"
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 10:13

For now I only have AutoHotkey.exe copied over from Win32w and renamed to Ahk2Exe.exe.
Thing is, my AHK_H Unicode scripts have a different extension (.ah1u), registered in the registry, while AHK_L is still installed and associated with .ahk. As such, launching an AHK_H script that still has an .ahk extension would actually run under AHK_L. This is a mess. :)

So, for compiling the correct Ahk2Exe I should use the code above and simply change the .ahk extension to .ah1u, is that right?
And then, context menu compiling, using that Ahk2Exe would run correctly when only the /in "<script_name>" argument is provided, right?
I'm asking just to be sure I do things right for future tests. Thank you for your help and patience.

[EDIT] Nevermind, I managed to compile Ahk2Exe in a few builds.

Finally found out which build exactly stopped responding on XP: it's this one.Something in the commits to 1.1.24.04-H006 made AHK_H fail to run compiled scripts on XP. I've compared by contents all ahk files in Lib, Compiler, Compiler/Lib between builds 005 and 006, and they are identical, so the problem is definitely in the code.
In Dependency Walker, the last call before modules start being detached is to get procedure address for IsWow64Process() in kernel, which returns a valid handle. From then on it's all downhill.

Code: Select all

GetProcAddress(0x7C800000 [KERNEL32.DLL], "IsWow64Process") called from "AHK2EXEH.EXE" at address 0x004B8577 and returned 0x7C8305B1 by thread 1.
DllMain(0x74720000, DLL_PROCESS_DETACH, 0x00000001) in "MSCTF.DLL" called by thread 1.
DllMain(0x74720000, DLL_PROCESS_DETACH, 0x00000001) in "MSCTF.DLL" returned 1 (0x1) by thread 1.
DllMain(0x5AD70000, DLL_PROCESS_DETACH, 0x00000001) in "UXTHEME.DLL" called by thread 1.
DllMain(0x5AD70000, DLL_PROCESS_DETACH, 0x00000001) in "UXTHEME.DLL" returned 1 (0x1) by thread 1.
DllMain(0x76390000, DLL_PROCESS_DETACH, 0x00000001) in "IMM32.DLL" called by thread 1.
DllMain(0x76390000, DLL_PROCESS_DETACH, 0x00000001) in "IMM32.DLL" returned 1 (0x1) by thread 1.
DllMain(0x78AA0000, DLL_PROCESS_DETACH, 0x00000001) in "MSVCR100.DLL" called by thread 1.
DllMain(0x78AA0000, DLL_PROCESS_DETACH, 0x00000001) in "MSVCR100.DLL" returned 1 (0x1) by thread 1.
[…]
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

17 Feb 2018, 18:55

It must be due to the code protection and anti debugger improvements, would be good to know what exactly is causing the problem. Unfortunately I have no xp currently.
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 02:46

Unfortunately I don't have an IDE either to try and modify/compile the code for testing, otherwise I would've done it already. And my C/C++ skills are very low anyway. If you know of any method/tool that could get us closer to the issue please do tell and I'll try it. Even a verbose debug version of the .bin could be of use, I could fire up DbgView and see where messages stop.

The search for the culprit could be narrowed down by the fact that only the .bin is at fault - the exe runs the scripts fine. So it's either in the code or in the building options. I've seen a Win2K flag in the preprocessor options introduces precisely in this build, it may or may not have to do with the issue.

The other recent problem with crashes upon restart when threads are present has to be tracked down too; I didn't get to it yet, but it's somewhere between 1.1.27.06 and 1.1.27.07.
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 05:44

Can you try to compile using .exe instead of .bin and see if the same problem persist?
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 06:20

Yes, I had already did that, forgot to mention it. Behavior is the same. So maybe the compilation sequence is bad? But as mentioned before, all related AHK scripts have been compared between versions and are identical bit by bit, so the possibility of a script error/bug is null. Internal operation must be faulty in regard to compiling.
Part of my AHK work can be found here.
HotKeyIt
Posts: 2364
Joined: 29 Sep 2013, 18:35
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 08:38

It is not the compiling part but code protection part that only executes when compiled script is includede. Some of the code seems to be not compatible with xp. I will see if I can get my xp on virtual machine running and see if I can fix that.
User avatar
Maasq
Posts: 10
Joined: 09 Jun 2017, 14:36
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 09:01

HotKeyIt wrote:Can you show your script and upload an example including the files you try to zip.
I just tried zipping text and jpg files and it works just fine.
Hi HotKeyIt. The script is huge, and it writes XML to a file then saves it. There's no issue with the actual compression as I can open it and unzip with 7-zip. I am checking with the Fantasy Grounds people to see if their application uses an older version of PKUNZIP - Drugwash may have hit the nail on the head.

Cheers for replying

Maasq
Maasq
Author of Engineer Suite (Written in AHK)
www.masq.net
User avatar
Drugwash
Posts: 850
Joined: 29 May 2014, 21:07
Location: Ploieşti, Romania
Contact:

Re: Did something change with the Zip handling?

18 Feb 2018, 13:43

HotKeyIt wrote:It is not the compiling part but code protection part that only executes when compiled script is included. Some of the code seems to be not compatible with xp. I will see if I can get my xp on virtual machine running and see if I can fix that.
I've tested NtSetInformationThread() in a script (under AHK_L and AHK_H) and it never returns true, neither compiled nor uncompiled.
I also tested ZwSetInformationThread() with the same result. Apparently a return value of zero indicats success. How unlike M$!
However, CheckRemoteDebuggerPresent() does work and indeed returns correct true value in BOOL parameter when run under a debugger.
Also IsDebuggerPresent() returns correct true value under a debugger (Dependency Walker was used for this purpose).

The very same testing script, compiled with AHK_H 1.1.24.00-H001 (which used to compile working scripts) bails out in DW, while when compiled in AHK_L 1.1.27.7 it works correctly and shows the debugger present. This seems to suggest there are other anti-debugging measures implemented in versions before the offending 1.1.24.04-006.

Now, considering how scripts may be poorly written, may have different/unwanted behavior in compiled vs uncompiled versions, shouldn't there be a switch somewhere in the compiler that would completely disable those anti-debugging measures, so the user/developer could freely test their own compiled scripts in a debugger?
But let's fix the bugs before feature requests.

[EDIT]
After further analysis I noticed you're calling RtlQueryPerformanceFrequency() and RtlQueryPerformanceCounter() which do not exist anywhere on an XP system but - according to Geoff Chappell - they are simply "forwarded from KERNELBASE function QueryPerformanceFrequency/QueryPerformanceCounter in 6.1 and higher". There are however their old counterparts QueryPerformanceFrequency() and QueryPerformanceCounter() which have been around in kernel32 since Windows 95.
This may be the whole issue.
Part of my AHK work can be found here.

Return to “AutoHotkey_H”

Who is online

Users browsing this forum: No registered users and 25 guests