Test build - Obj.Count(), OnError(), long paths, experimental switch-case

Community news and information about new or upcoming versions of AutoHotkey
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Test build - Obj.Count(), OnError(), long paths, experimental switch-case

20 Apr 2018, 17:48

v1.1.28.02-13+g52f9e5d

For v1.1.29.00:
  • Added Obj.Count() and ObjCount(Obj).
  • Added ObjGetBase(Obj) and ObjSetBase(Obj, BaseObj).
  • Added ObjRawGet(Obj, Key).
  • Added OnError(Func [, AddRemove]).
  • Removed the following limits for custom combination hotkeys: 1) max 50 unique prefix keys handled by VK for each suffix key, 2) max 16 unique prefix keys handled by SC for each suffix key.
  • Improved Send {Text} to avoid toggling CapsLock or waiting for Win+L when "{Text}" is at the beginning of the parameter or immediately following "{Blind}".
OnError specifies a function to run automatically when an unhandled error occurs. It could be used to suppress the standard error dialog (by returning true) and optionally show a custom one, or perform additional error-handling such as writing to a log file. See the included documentation for details.

No facility is given to intercept warnings because the need is less (warnings are optional), and most warnings are shown before the script starts executing (before it could call a hypothetical OnWarn()).

Also included in the test build:
  • Incomplete support for long path awareness.
  • Experimental switch-case statement.
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Test build - incomplete long path support

20 Apr 2018, 18:18

This test build includes heavy revision of Loop Files:
  • Fixes A_Loop variables and the loop itself failing or behaving incorrectly if the working directory is changed during the loop.
  • Fixes a significant performance issue with A_LoopFileLongPath (and A_LoopFileShortPath is also optimized).
  • Improves support for long paths.
It also includes improved support for long paths in other commands and in general (e.g. the script itself can have a long path).

What is MAX_PATH?

MAX_PATH is a constant defined in the Windows SDK, meaning 260.

Programs are affected by two kinds of path length limitations:

  1. The low-level APIs which provide file system access impose a limit of MAX_PATH characters, so 259 characters plus a null terminator. This is purportedly because a fixed-size buffer of MAX_PATH characters is used to hold the normalized path. Normalization converts relative paths to absolute paths, replaces '/' with '\', removes redundant (repeated) slashes, trailing spaces, and so on.

  2. Code for dealing with paths within the program often use a fixed-size buffer of MAX_PATH characters.
Further explanation/technical details

Working with long paths

Windows 10 v1607+ has "long path awareness", which basically removes limitation no. 1. This must be enabled by setting LongPathsEnabled to 1 under the following registry key:

Code: [Select all] [Download] GeSHi © Codebox Plus

HKLM\SYSTEM\CurrentControlSet\Control\FileSystem
However, the setting only affects applications which opt-in by including a specific value in their manifest (which is usually an embedded resource). The test build includes this.

Without long path awareness, using a long path requires the \\?\ prefix, as in \\?\C:\. This effectively skips normalization and passes the path directly to the file system driver. Relative paths cannot be used, and failing to normalize the path correctly may give surprising results (for instance, '/' in a filename). However, GetFullPathName can be used to normalize the path.

Either way, whatever code is handling the path must also support long paths. Some commands pass the path directly to the underlying API, so already worked with \\?\. Some commands have been updated in this test build to support long paths (when long path awareness is enabled or \\?\ is used). See "known limitations" below.

Both methods only work with the Unicode APIs. ANSI versions of AutoHotkey use the ANSI APIs, and therefore do not support long paths.


Loop Files and MAX_PATH

Previously, Loop Files was limited to paths up to MAX_PATH-1 characters. If A_LoopFilePath would exceed this, the file was skipped. However, if a relative or short path was given, the file's absolute/long-name path could exceed MAX_PATH. This is because limitation no. 1 only applied to the normalized search pattern (full_path\*.*), while limitation no. 2 only applied to the path as returned by A_LoopFilePath. Scripts were able to use this to work past the limit, though in those cases A_LoopFileLongPath was blank.

Now, if a directory is able to be searched by the API (that is, if FindFirstFile succeeds), all files in that directory are returned. On ANSI builds (or if long path awareness is disabled/absent and the \\?\ prefix is not used), FindFirstFile fails if the full path of the directory plus the search pattern exceeds MAX_PATH-1. For recursion into sub-directories, the search pattern "*" is used (in previous versions it was "*.*"). In theory (in unusual cases), A_LoopFilePath may be up to 513 characters, and A_LoopFileLongPath up to about 8K (due to expansion of short names).


Known limitations

ANSI versions of AutoHotkey use the ANSI APIs, and therefore do not support long paths.

Most notably: FileCopy and FileMove need to be updated. Fixing FileCopyDir, FileMoveDir and FileRemoveDir would require re-implemeting them from scratch, which carries the risk of changing behaviour in unexpected ways (like handling of file permissions or associated files).

It does not appear to be possible to run an executable with a full path which exceeds MAX_PATH. That being the case, it would not be possible to fully test any changes aimed at supporting longer executable paths. Therefore, MAX_PATH limits have been left in place for the following:
  • ahk_exe
  • The default script's path, which is based on the current executable's path.
  • Retrieval of the AutoHotkey installation directory, which is used by A_AhkPath in compiled scripts and may be used to launch Window Spy or the help file.
  • WinGet ProcessPath.
  • WinGet ProcessName (this theoretically isn't actually a limit, since it is applied only to the name portion, and NTFS only supports names up to 255 chars).

FileRecycle is artificially limited to MAX_PATH. It relies on SHFileOperation, which testing indicates does not work with long paths. When tested with a long path, it failed. When tested with the corresponding short (8.3) path, the file was recycled but Recycle Bin showed the name of one of the parent directories instead of the file's name.

FileMoveDir allows long paths if the R (rename) option is used. FileRemoveDir allows long paths if the Recurse? parameter is false (in which case the directory must be empty). Otherwise, FileCopyDir, FileMoveDir and FileRemoveDir are artificially limited to MAX_PATH. They rely on SHFileOperation, which testing indicates does not support long paths.

FileCopy and FileMove could support long paths, but are currently limited to MAX_PATH. Updating them is non-trivial due to their handling of recursion into sub-directories and wildcards in the destination path. For single files, the CopyFile or MoveFile function can be called via DllCall.

FileSelectFile: If the RootDir\Filename parameter exceeds MAX_PATH-1, it is passed to GetShortPathName. Without this, any path longer than MAX_PATH-1 would be ignored, presumably because the dialog, as part of the shell, does not support long paths. Surprisingly, although Windows 10 long path awareness does not allow the dialog to accept a long path as input, it does affect whether the long path is used in the address bar and returned filenames.

FileSelectFolder's StartingFolder is truncated to MAX_PATH*2+4 characters, which allows for two MAX_PATH paths (root directory and initial directory), one space/tab and one asterisk (and for reasons unknown, another 2 characters). The dialog itself does not support long paths, but (on some OS versions) automatically utilizes 8.3 short names while displaying long names.

FileGetShortcut and FileCreateShortcut are limited to MAX_PATH for each component. The shell likely also imposes the same limits.

DllCall is limited to a "DllFile\Function" of MAX_PATH*2, and Function is additionally limited to MAX_PATH on Unicode builds. It does not appear to be possible to load a DLL from a long path on Windows 10 v1709.

Drive Label: SetVolumeLabel can't seem to make use of long paths.

DriveGet Capacity and DriveSpaceFree: testing showed GetDiskFreeSpaceEx does not support long paths even with \\?\.

DriveGet's remaining sub-commands have no artificial MAX_PATH restriction, but testing showed that of the APIs it uses, only GetDriveType supports long paths.

SetWorkingDir and A_WorkingDir support long paths only when Windows 10 long path awareness is enabled. The \\?\ prefix cannot be used. If the working directory exceeds MAX_PATH, it becomes impossible to launch programs with Run. These limitations are imposed by the OS.

SoundPlay: research indicates that the underlying API limits paths to 127 chars.

The following are limited to MAX_PATH due to the underlying API, and will realistically
never have long paths anyway: A_WinDir, A_Temp.

A_MyDocuments, ProgramFiles, A_ProgramFiles, A_Programs, A_AppData, A_Desktop, A_StartMenu, A_Startup and Common variants: SHGetFolderPath is limited to MAX_PATH. SHGetKnownFolderPath could be used instead, but it's probably a moot point, since it seems the shell still does not support long paths. Even if it did, giving these special folders long paths would be ridiculous.

Long #Include paths shown in error messages may be truncated arbitrarily.
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Test build - experimental switch-case

20 Apr 2018, 18:26

Switch

In 2012 I wrote a basic implementation of switch-case to help weigh up cost vs. benefit of adding it to the language. The result wasn't really favourable, so I shelved it. There have been a few more requests for (and imitations of) switch-case, so I chose to complete the experiment, though I have not decided whether to integrate it into the release branch.

"Switch: Executes one case from a list of mutually exclusive candidates."

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


Syntax is similar to C or C-like languages, but without break or implicit fall-through, which would be a common source of errors. Each case takes between 1 and 20 comma-delimited expressions. If the switch value is omitted, the first "truthy" (non-zero, non-empty) case is executed, like a more compact if-else-if ladder. Switch false is almost the inverse, but requires the case to be zero and not empty.

Currently the switch cannot be written like Switch(value), so any existing functions with that name will still work.

See the included documentation for more details.

Historical note
User avatar
joedf
Posts: 6240
Joined: 29 Sep 2013, 17:08
Facebook: J0EDF
Google: +joedf
GitHub: joedf
Location: Canada, Quebec
Contact:

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

20 Apr 2018, 21:15

Looking forward to the switch! :+1:
I remember making pseudo switch methods but not the best...
User avatar
nnnik
Posts: 2791
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

21 Apr 2018, 00:09

Thanks for these updates.
Recommends AHK Studio
User avatar
Flipeador
Posts: 655
Joined: 15 Nov 2014, 21:31
GitHub: Flipeador
Location: Argentina
Contact:

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

21 Apr 2018, 07:32

Good job lexikos, as always. :bravo:
:wave:
Windws 1♂ Pro 64-Bits I make scripts for AHKv2 (my v2 compiler) & WIN_7+ Spanish Argentina SublimeText 3 & AHKv2
Helgef
Posts: 2901
Joined: 17 Jul 2016, 01:02
Contact:

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

21 Apr 2018, 14:07

Hello.
I like switch for control flow benefits over else if, requires fall-through, I hope that will be added. I guess having non-constant expressions will be convenient, but doesn't favour performance, another possible benefit of switch. Anyways, it gives convenient and readable syntax as it is now, very nice :thumbup: .
Spoiler

Allowing labels inside the block, but outside a case is a bit wierd.

ObjXXXBase, this is most useful :thumbup:.

Cheers and thanks for your efforts.
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

22 Apr 2018, 04:48

I have uploaded a new build which fixes:
  • Switch crashing, as reported by Helgef.
  • Loop Files, .., D reporting the directory's path incorrectly (an old bug).
  • Loop Files, C:* failing to iterate (a new bug).
  • Loop Files, C:, D failing to iterate (it previously iterated but returned an incorrect path).
On a related note (for those who don't know about it),

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


Helgef wrote:I like switch for control flow benefits over else if, requires fall-through, I hope that will be added.
As I said, there is no fall-through because it would be a common source of errors.

If there is implicit fall-through, there must be an explicit end to every case not intended to fall through. Aside from being a source of errors (when the user forgets break or uses it intending to break a loop), this makes the switch longer, reducing one of its advantages. It also means that one-line cases are impossible unless there is an exception for them (like one-line hotkeys, but adding an implicit break instead of return).

Numerous other languages do not allow fall-through.

If you want to share code between cases, use the standard constructs provided for code reuse: a subroutine or function, or goto if you must. If multiple case values should execute the exact same code, list them on one case statement.

I guess having non-constant expressions will be convenient, but doesn't favour performance, another possible benefit of switch.
Restricting the cases to constant expressions would greatly reduce flexibility and would not help performance (except maybe by consequence of reducing code size). If a switch with only constant case expressions can be optimized, it can be done regardless of whether other switch statements contain non-constant case expressions.

Allowing labels inside the block, but outside a case is a bit wierd.
Labels are not statements and do not interfere with the structure of the code. You can place a label above the first case, but you cannot place any statements between that label and first case. The only benefit of prohibiting it would be to avoid confusion when a user (who hasn't read the documentation, or has forgotten it) intends to "goto" the first case from a subsequent case, but places the label in the wrong place (above the case). It would not cover labels placed above any other case (which I think would be more likely), since jumping to the end of a block is permitted.
Switch wrote:As all cases are enclosed in the same block, a label defined in one case can be the target of Goto from another case. However, if a label is placed immediately above Case or Default, it targets the end of the previous case, not the beginning of the next one.

In other words, detecting it would increase code size marginally and be of benefit only in a very specific, probably very rare situation.
Helgef
Posts: 2901
Joined: 17 Jul 2016, 01:02
Contact:

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

22 Apr 2018, 06:14

Thanks for the fixes.
As I said, there is no fall-through because it would be a common source of errors.

Syntax is similar to C or C-like languages, but without break or implicit fall-through, which would be a common source of errors

I see, I read it as the absence of fall through would be a common source of errors, and it might as well be the case for those familiar with switch from languages where fall through is enabled. Of course, the manual is clear so no excuses there, however it could clearly state that fall through was enabled and I do not think that would cause much errors, it is a pretty simple concept. And it gives the switch an actual useful distinction from else if, disregarding syntax preferences. On the other hand, if fall through was enabled, one would have to consider if the case value expression which is fallen through to, should be exectued or not, in any case that could cause confusion I suppose. But clear documentation gives no room for such excuses (could cause confusion).
Numerous other languages do not allow fall-through.
Further down, from that link,
In some cases languages provide optional fallthrough. For example, Perl does not fall through by default, but a case may explicitly do so using a continue keyword. This prevents unintentional fallthrough but allows it when desired.
I like that, maybe a fall or next keyword could cause a fall through (continue and break are better to reserve for loops).
If a switch with only constant case expressions can be optimized, it can be done regardless of whether other switch statements contain non-constant case expressions.
Very good point.

About labels, it is fine. Overall, it is fine :thumbup: .

Cheers.
hobboy
Posts: 41
Joined: 05 Jan 2016, 09:59

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

22 Apr 2018, 07:02

Wow all of these changes sound fantastic, they address lots of common issues for me. Thanks!
jmeneses
Posts: 417
Joined: 28 Oct 2014, 11:09

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

03 May 2018, 12:21

Hi
This script puts together 4 CSV's in an XLSX, but It does not work for me with this version :crazy: .
I believe that it fails "for k, file in files {" is possible? :cry:

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus



Version: v1.1.28.02-10+g1a41623 64-bit / Unicode
Type: WIN32_NT
OS: 10.0.17134

Thanks community
burque505
Posts: 388
Joined: 22 Jan 2017, 19:37

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

03 May 2018, 18:28

Hola, @jmenses!

Tinc el mateix problema amb Win7 64-bit, Version: v1.1.28.02-10+g1a41623 64-bit / Unicode, i no l'he pogut resoldre.
Amb 1.1.28.02 AHK_L U64, cap problema.

It goes straight to "Done!" without ever opening Excel in my case.
I believe you are absolutely right about where it's happening.
If if put a msgbox in the loop like this:

Code: [Select all] [Download] GeSHi © Codebox Plus

for k, file in files {
fileCSV := dir "\" file
msgbox %fileCSV%
SplitPath, fileCSV, , dir, , noext

the msgbox never shows in the test build, but does in every pass through the loop with 1.1.28.02.


Salutacions,
burque505
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

04 May 2018, 23:10

jmeneses wrote:
burque505 wrote:

If MsgBox does not show, the for-loop is not iterating, which is almost certainly because files is empty.

I do not have Excel. Please see if you can narrow down the cause and post a more specific script to reproduce the problem without Excel.

I have already tried this, which works for me:

Code: [Select all] [Download] GeSHi © Codebox Plus

FileSelectFile, files, M3 , % "C:\Temp", % "Escull CSV's" , (*.*)
MsgBox % files
files:=StrSplit(files,"`n")
dir:=files.RemoveAt(1)
for k, file in files
s .= k "=" file "`n"
MsgBox % s
User avatar
Ragnar
Posts: 170
Joined: 30 Sep 2013, 15:25

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

05 May 2018, 02:50

#NoEnv is causing this issue (for whatever reason).
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

05 May 2018, 04:49

Thanks. It is fixed in the latest test build (see top post).

The problem was with for-in, switch and throw expressions which are just a single variable reference. For-in aborted because it received a variable, not an object. Switch and throw incorrectly delayed evaluation of the variable. #NoEnv affected the outcome because somevar is compiled as a variable reference when #NoEnv is present, and as a dynamic reference (which could yield either a variable or an environment string) when #NoEnv is absent.
burque505
Posts: 388
Joined: 22 Jan 2017, 19:37

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

05 May 2018, 17:19

Confirming it's fixed for Win7 6-bit also, latest build (May 5, 2018). Thanks!
jmeneses
Posts: 417
Joined: 28 Oct 2014, 11:09

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

06 May 2018, 13:47

It works wel :thumbup:
For my the switch-case statement is perfect, i never liked the < ELSE IF >
thanks to all !!!
lexikos
Posts: 5896
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: Test build - Obj.Count(), OnError(), long paths, experimental switch-case

12 May 2018, 20:28

I have uploaded a new test build with the following additional improvements:
  • Removed the following limits for custom combination hotkeys: 1) max 50 unique prefix keys handled by VK for each suffix key, 2) max 16 unique prefix keys handled by SC for each suffix key.
  • Improved Send {Text} to avoid toggling CapsLock or waiting for Win+L when "{Text}" is at the beginning of the parameter or immediately following "{Blind}".

Return to “Announcements”

Who is online

Users browsing this forum: No registered users and 2 guests