This tutorial might be of interest to
- those who want to automate the (un)installation, updating, and (re)configuration of software
- those extending editors (e.g. Emacs) that can use the standard streams (stdin, stdout, and stderr) to communicate with other programs
Preparation
You will need a way to edit the PE header of a Windows executable. You can use a hex editor like Frhed or a specialized tool like LordPE.
You will need Lexikos' RegisterSyncCallback to handle console control events.
Explanation
The Beginning
Code: Select all
#MaxMem 4095
#NoEnv
#NoTrayIcon
#SingleInstance Off
AutoTrim Off
CoordMode Caret, Client
CoordMode Menu, Client
CoordMode Mouse, Client
CoordMode Pixel, Client
CoordMode ToolTip, Client
ListLines Off
SendMode Input
SetBatchLines -1
SetFormat FloatFast, 0.6
SetFormat IntegerFast, D
SetTitleMatchMode 2
; Debugging
#Warn
ListLines On
The "Debugging" section can be commented out when you are not debugging.
The settings I have not drawn attention to so far are what they are planned to be in v2 and are improvements over v1's default behavior.
In v2, you will probably need to set the working directory to A_InitialWorkingDir. This variable does not exist in v1. v1 respects the initial working directory, but v2 currently sets it to A_ScriptDir, which is incorrect for command-line programs.
Connecting to the Standard Streams
Code: Select all
StdIn := FileOpen("*", "r `n")
StdOut := FileOpen("*", "w `n")
StdErr := FileOpen("**", "w `n")
GetLine()
{
local
global StdIn
return RTrim(StdIn.ReadLine(), "`n")
}
Show(String)
{
local
global StdOut
StdOut.Write(String)
StdOut.Read(0) ; Flush the write buffer.
}
ShowError(String)
{
local
global StdErr
StdErr.Write(String)
StdErr.Read(0) ; Flush the write buffer.
}
Processing Command-Line Arguments
There is considerable variation in how programs process command-line arguments. I attempt to explain an acceptable way, not the only way.
This section is long and includes many digressions. Rest assured that it is all relevant. I believe it is the easiest to understand when explained in this way.
Algorithms that follow use a specification for your command-line program. An example is below.
Code: Select all
COMMAND := "DoStuff"
VERSION := "1.2.3"
USAGE_PATTERNS := ["-?"
,"[-optimize <level>] [-out <output file>] <input file>"
,"-version"]
OPTIONS := {"?": {"Arg": ""
,"Csv": false
,"Desc": "show this help and exit"}
,"optimize": {"Arg": "level"
,"Csv": false
,"Desc": "set optimization 'off', 'on', or 'unstable' [default: 'off']"}
,"out": {"Arg": "output file"
,"Csv": false
,"Desc": "set the output file [default: <input filename with extension 'stf'>]"}
,"version": {"Arg": ""
,"Csv": false
,"Desc": "show the version and exit"}}
OPERANDS := ["input file"]
VERSION must contain the version of your program. Please use the Semantic Versioning format.
USAGE_PATTERNS must contain the patterns of arguments that can be used together. "-?" and "-version" must be present. Arguments surrounded by square brackets ([]) are optional. Arguments surrounded by angle brackets (<>) are to be substituted by actual values.
OPTIONS must contain the names of your options (variables that dictate the specifics of your program's operation) and a specification of their option-argument and purpose. "Arg" is the name of the option-argument for options that require an option-argument and "" otherwise. "Csv" is true for an option-argument in comma-separated value format and false otherwise. "Desc" contains the human-readable description of the option. Defaults are depicted as they are in the code above.
OPERANDS must contain the names of your operands (variables that are operated on by your program). "?" and "*" are processed differently. Operands following "?" are optional operands. Other operands are required. "*" is an operand that contains any arguments that are not contained by other operands. "*" can appear at the beginning, in the middle, or at the end of the operands. "?" and "*" must appear once, if at all. The names "?" and "*" were inspired by regular expression notation.
You must not name your parameters (options and operands) after any special Object keys (like base), methods (like Clone), or meta-functions (like __Get). That would make it impossible to reliably set and get their values.
The algorithms that use this specification do not check its contents for errors. Be careful when filling out yours!
The main procedure provides a framework for understanding the rest of this section.
Code: Select all
Main(Args)
{
local
global COMMAND
try
{
Exec(ValidateCliInput(ParseArgs(Args)))
Result := 0
}
catch Ex
{
ShowError( Format("{:L}: {}", COMMAND, Ex["Message"])
. (Ex["Extra"] <> "" ? "`n" . Ex["Extra"] : ""))
Result := 1
}
exitapp Result
}
AutoHotkey command-line programs are persistent, and therefore require the use of exitapp instead of exit, because RegisterSyncCallback uses Gui and OnMessage.
Programs should follow the convention of returning an exit status of 0 on success or 1 on error. This makes it easier to detect errors in shell scripts.
The main procedure will be called this way.
Code: Select all
Main(A_Args)
Code: Select all
ParseArgs(Args)
{
local
global COMMAND, OPTIONS, OPERANDS
ArgsIndex := 1
Opts := {}
ReadOpts := true
while (ReadOpts and ArgsIndex <= Args.Length())
{
FirstChar := SubStr(Args[ArgsIndex], 1, 1)
if FirstChar in -,/
{
if (Args[ArgsIndex] == "--")
{
++ArgsIndex
ReadOpts := false
}
else
{
Opt := Format("{:L}", SubStr(Args[ArgsIndex], 2))
if (not OPTIONS.HasKey(Opt))
{
throw Exception(Args[ArgsIndex] . " is not a valid option",
,Format("Try '{:L} -?' for more information.", COMMAND))
}
if (OPTIONS[Opt]["Arg"] == "")
{
Opts[Opt] := true
++ArgsIndex
}
else
{
if (ArgsIndex + 1 <= Args.Length())
{
if (OPTIONS[Opt]["Csv"])
{
Arg := []
loop parse, % Args[ArgsIndex + 1], CSV
{
Arg[A_Index] := A_LoopField
}
}
else
{
Arg := Args[ArgsIndex + 1]
}
if (Opts.HasKey(Opt))
{
if (OPTIONS[Opt]["Csv"])
{
if (Opts[Opt].Length() == Arg.Length())
{
loop % Opts[Opt].Length()
{
if (not Opts[Opt][A_Index] == Arg[A_Index])
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
}
else
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
else
{
if (not Opts[Opt] == Arg)
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
}
Opts[Opt] := Arg
ArgsIndex += 2
}
else
{
throw Exception(Args[ArgsIndex] . " requires an argument")
}
}
}
}
else
{
ReadOpts := false
}
}
Opds := {}
if (not (Opts.HasKey("?") or Opts.HasKey("version")))
{
OtherOpds := 0
for _, Opd in OPERANDS
{
if Opd not in ?,*
{
++OtherOpds
}
}
StarLength := Args.Length() - (ArgsIndex - 1) - OtherOpds
OptionalOpd := false
parseargs_read_opds:
for _, Opd in OPERANDS
{
if (Opd == "?")
{
OptionalOpd := true
}
else if (Opd == "*")
{
Star := []
loop %StarLength%
{
Star[A_Index] := Args[ArgsIndex]
++ArgsIndex
}
Opds[Opd] := Star
}
else
{
if (ArgsIndex <= Args.Length())
{
Opds[Opd] := Args[ArgsIndex]
++ArgsIndex
}
else
{
if (not OptionalOpd)
{
throw Exception("<" . Opd . "> is a required operand")
}
break parseargs_read_opds
}
}
}
if (ArgsIndex <= Args.Length())
{
throw Exception(Args[ArgsIndex] . " is an unexpected operand")
}
}
return {"Options": Opts, "Operands": Opds}
}
The value associated with an option that does not require an argument is true.
The value associated with an option that requires a CSV argument is an array.
It is a syntax error to specify an option more than once with different arguments because it is not obvious what should be done in that situation. Arguments must use the same letter case to be considered the same because sometimes letter case matters.
-- is a special option that is used to delimit the end of the options. It prevents operands that begin with - or / from being mistaken for options. It is sometimes used in a security context to separate trusted options from untrusted operands. If you are using it that way, make certain you can trust the options! Otherwise, it could be parsed as an option-argument.
Operands are not parsed if the help or version option is used.
The value associated with the * operand is an array.
The * operand behaves as consistently as possible, but a corner case might be surprising. If an optional operand’s argument was omitted, there is no key-value pair corresponding to it in the data structure ParseArgs returns, but there will be a key-value pair corresponding to * in the data structure ParseArgs returns unless an optional operand’s argument was omitted before it! This is consistent with how the * operand behaves when it is the last operand and there are no arguments left to fill it. An empty array should be equivalent semantically to a nonexistent key-value pair anyway.
Code: Select all
ValidateCliInput(CliInput)
{
local
; See the explanation below.
return CliInput
}
ValidateCliInput should check that the combination of options is valid, option values are valid, and the relationship between option values is valid (e.g. if one must be less than another, that this is so). Sometimes, ValidateCliInput can perform limited checks on operands, but remember that operands are not always parsed. If a check fails, ValidateCliInput should throw an Exception with a helpful message.
Some checks cannot be performed by ValidateCliInput because doing so would cause TOCTTOU (time of check to time of use) defects. For example, if it checked that a file the program is about to process exists, the file might be moved or deleted before it was opened.
Code: Select all
Exec(CliInput)
{
local
global COMMAND, VERSION
if (CliInput["Options"].HasKey("?"))
{
ShowCliHelp()
}
else if (CliInput["Options"].HasKey("version"))
{
Show(COMMAND . " " . VERSION)
}
else
{
; See the explanation below.
}
}
Programs should follow the convention of showing help if the help option is used or showing the version if the version option is used. Those options are checked for in that order and before any others. Any other options are ignored when those options are used.
Input should be credible when it reaches Exec, but it must handle some errors to avoid introducing TOCTTOU defects. Most of these errors involve using files, directories, and sockets. They should be handled with EAFP (it is easier to ask forgiveness than it is to get permission). In other words, try it and throw an Exception if it fails.
Code: Select all
ShowCliHelp()
{
local
global COMMAND, USAGE_PATTERNS, OPTIONS
Help := "Usage:`n"
for _, UsagePattern in USAGE_PATTERNS
{
Help .= Format(" {:L} {}`n", COMMAND, UsagePattern)
}
Help .= "`n"
Help .= "Options:`n"
ColumnWidth := 0
for Opt, Props in OPTIONS
{
CurrentWidth := 1 + StrLen(Opt) + (Props["Arg"] <> "" ? StrLen(Props["Arg"]) + 3 : 0)
ColumnWidth := CurrentWidth > ColumnWidth ? CurrentWidth : ColumnWidth
}
FormatStr := " {:-" . ColumnWidth . "} {}`n"
for Opt, Props in OPTIONS
{
Help .= Format(FormatStr, "-" . Opt . (Props["Arg"] <> "" ? " <" . Props["Arg"] . ">" : ""), Props["Desc"])
}
Help := RTrim(Help, "`n")
Show(Help)
}
Help should always be shown on stdout. This makes it easy to redirect and filter help messages. Users often want to do that to find an option they cannot remember the name for.
Handling Console Control Events
If you need your program to do something other than exit when Ctrl+C or Ctrl+Break is pressed or the console window is closed, you will need to write code to handle console control events.
Code: Select all
#Include RegisterSyncCallback.ahk
CtrlEvent := ""
HandlerRoutine(dwCtrlType)
{
local
global CtrlEvent
if (dwCtrlType == 0)
{
CtrlEvent := CtrlEvent <> "Ctrl+Break" ? "Ctrl+C" : CtrlEvent
Handled := true
}
else if (dwCtrlType == 1)
{
CtrlEvent := "Ctrl+Break"
Handled := true
}
else
{
; See the explanation below.
Handled := false
}
return Handled
}
DllCall("SetConsoleCtrlHandler", "ptr", RegisterSyncCallback("HandlerRoutine"), "int", 1)
PollForConsoleCtrlEvents()
{
; This should be called inside long-running loops.
;
; Do not forget to reset CtrlEvent to "" when handling the exceptions!
local
global CtrlEvent
if (CtrlEvent == "Ctrl+C")
{
throw Exception("Ctrl+C was pressed")
}
else if (CtrlEvent == "Ctrl+Break")
{
throw Exception("Ctrl+Break was pressed")
}
}
There are three console control events your process might receive:
- CTRL_C_EVENT (0) -- Ctrl+C was pressed to terminate your process or to terminate the algorithm your process is running
- CTRL_BREAK_EVENT (1) -- Ctrl+Break was pressed to terminate your process or to terminate the algorithm your process is running and show debugging information
- CTRL_CLOSE_EVENT (2) -- the console window was closed to terminate your process
Be aware that CTRL_CLOSE_EVENT is different from the other events in that your process will be terminated as soon as HandlerRoutine returns, no matter what it returns! Also, if HandlerRoutine takes more than 5 to 20 seconds (depending on Windows version) to handle CTRL_CLOSE_EVENT, your process will be terminated anyway! So HandlerRoutine should return false when handling CTRL_CLOSE_EVENT and handle it quickly. It must contain or call code to handle that event because the main thread will not get a chance to run further.
If you want your program to terminate the algorithm it is running when Ctrl+C or Ctrl+Break is pressed, you will need to set CtrlEvent to "Ctrl+C" for CTRL_C_EVENT or "Ctrl+Break" for CTRL_BREAK_EVENT in HandlerRoutine and call PollForConsoleCtrlEvents in your main thread. HandlerRoutine should not set CtrlEvent to "Ctrl+C" if it is already set to "Ctrl+Break". That implies only CTRL_BREAK_EVENT can clobber CTRL_C_EVENT, which should be acceptable to your users and unlikely. No events can be lost because HandlerRoutine should never reset CtrlEvent. CTRL_CLOSE_EVENT cannot be clobbered or lost because of its nature. Exception handling provides a good way to 'back out' of your algorithm, and you can catch and rethrow exceptions to perform cleanup or undo procedures in a telescoping fashion.
Avoid designing your program in a way that could result in data corruption if its process was terminated without having run its cleanup procedure. If power were interrupted, your process was forcibly terminated (e.g. by Task Manager), or similar situations arose, your cleanup procedure would not run.
Compiling and Editing the PE Header
AutoHotkey normally refuses to produce command-line programs, but it can be forced to with some effort.
You must compile your program. Otherwise, there is no PE header to edit.
You must edit the PE header to change the Subsystem field from WINDOWS_GUI (2) to WINDOWS_CUI (3). Otherwise, your program will be unable to attach to the console.
The Subsystem field is a 16-bit integer stored in little-endian order at offset 372 (0x174). That information is useful to those using a hex editor to change the field.
Advice
Know that CON, CONIN$, CONOUT$, CONERR$, NUL, wildcards, piping, and redirection exist. Use them. Do not reinvent them.
Consider the design of related command-line programs when designing yours. This should make your program easier to use with those programs. Adopt their good ideas. Avoid their bad ideas. This is how progress is made.
Consider the Microsoft Command Line Standard when designing your program, but be aware that even Microsoft’s programs do not follow it. That is why I suggest considering the design of related programs too.
It might be worthwhile to consider the POSIX and docopt standards when designing your program even though they are not Windows standards. They have some good ideas, like the -- option, that solve problems that exist but have no conventional solution on Windows.
Avoid accidental complexity. Keep the number of options small. Avoid using CSV options when possible. Avoid using optional operands when possible. If you must write a variadic command-line program, position the variadic operand (*) as the last operand when possible because other positions often confuse users.
The Complete Template
Code: Select all
#Include RegisterSyncCallback.ahk
#MaxMem 4095
#NoEnv
#NoTrayIcon
#SingleInstance Off
AutoTrim Off
CoordMode Caret, Client
CoordMode Menu, Client
CoordMode Mouse, Client
CoordMode Pixel, Client
CoordMode ToolTip, Client
ListLines Off
SendMode Input
SetBatchLines -1
SetFormat FloatFast, 0.6
SetFormat IntegerFast, D
SetTitleMatchMode 2
; Debugging
#Warn
ListLines On
StdIn := FileOpen("*", "r `n")
StdOut := FileOpen("*", "w `n")
StdErr := FileOpen("**", "w `n")
GetLine()
{
local
global StdIn
return RTrim(StdIn.ReadLine(), "`n")
}
Show(String)
{
local
global StdOut
StdOut.Write(String)
StdOut.Read(0) ; Flush the write buffer.
}
ShowError(String)
{
local
global StdErr
StdErr.Write(String)
StdErr.Read(0) ; Flush the write buffer.
}
COMMAND := ; <your code>
VERSION := ; <your code>
USAGE_PATTERNS := ; <your code>
OPTIONS := ; <your code>
OPERANDS := ; <your code>
Main(Args)
{
local
global COMMAND
try
{
Exec(ValidateCliInput(ParseArgs(Args)))
Result := 0
}
catch Ex
{
ShowError( Format("{:L}: {}", COMMAND, Ex["Message"])
. (Ex["Extra"] <> "" ? "`n" . Ex["Extra"] : ""))
Result := 1
}
exitapp Result
}
ParseArgs(Args)
{
local
global COMMAND, OPTIONS, OPERANDS
ArgsIndex := 1
Opts := {}
ReadOpts := true
while (ReadOpts and ArgsIndex <= Args.Length())
{
FirstChar := SubStr(Args[ArgsIndex], 1, 1)
if FirstChar in -,/
{
if (Args[ArgsIndex] == "--")
{
++ArgsIndex
ReadOpts := false
}
else
{
Opt := Format("{:L}", SubStr(Args[ArgsIndex], 2))
if (not OPTIONS.HasKey(Opt))
{
throw Exception(Args[ArgsIndex] . " is not a valid option",
,Format("Try '{:L} -?' for more information.", COMMAND))
}
if (OPTIONS[Opt]["Arg"] == "")
{
Opts[Opt] := true
++ArgsIndex
}
else
{
if (ArgsIndex + 1 <= Args.Length())
{
if (OPTIONS[Opt]["Csv"])
{
Arg := []
loop parse, % Args[ArgsIndex + 1], CSV
{
Arg[A_Index] := A_LoopField
}
}
else
{
Arg := Args[ArgsIndex + 1]
}
if (Opts.HasKey(Opt))
{
if (OPTIONS[Opt]["Csv"])
{
if (Opts[Opt].Length() == Arg.Length())
{
loop % Opts[Opt].Length()
{
if (not Opts[Opt][A_Index] == Arg[A_Index])
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
}
else
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
else
{
if (not Opts[Opt] == Arg)
{
throw Exception(Args[ArgsIndex] . " was specified more than once with different arguments")
}
}
}
Opts[Opt] := Arg
ArgsIndex += 2
}
else
{
throw Exception(Args[ArgsIndex] . " requires an argument")
}
}
}
}
else
{
ReadOpts := false
}
}
Opds := {}
if (not (Opts.HasKey("?") or Opts.HasKey("version")))
{
OtherOpds := 0
for _, Opd in OPERANDS
{
if Opd not in ?,*
{
++OtherOpds
}
}
StarLength := Args.Length() - (ArgsIndex - 1) - OtherOpds
OptionalOpd := false
parseargs_read_opds:
for _, Opd in OPERANDS
{
if (Opd == "?")
{
OptionalOpd := true
}
else if (Opd == "*")
{
Star := []
loop %StarLength%
{
Star[A_Index] := Args[ArgsIndex]
++ArgsIndex
}
Opds[Opd] := Star
}
else
{
if (ArgsIndex <= Args.Length())
{
Opds[Opd] := Args[ArgsIndex]
++ArgsIndex
}
else
{
if (not OptionalOpd)
{
throw Exception("<" . Opd . "> is a required operand")
}
break parseargs_read_opds
}
}
}
if (ArgsIndex <= Args.Length())
{
throw Exception(Args[ArgsIndex] . " is an unexpected operand")
}
}
return {"Options": Opts, "Operands": Opds}
}
ValidateCliInput(CliInput)
{
local
; <your code>
return CliInput
}
Exec(CliInput)
{
local
global COMMAND, VERSION
if (CliInput["Options"].HasKey("?"))
{
ShowCliHelp()
}
else if (CliInput["Options"].HasKey("version"))
{
Show(COMMAND . " " . VERSION)
}
else
{
; <your code>
}
}
ShowCliHelp()
{
local
global COMMAND, USAGE_PATTERNS, OPTIONS
Help := "Usage:`n"
for _, UsagePattern in USAGE_PATTERNS
{
Help .= Format(" {:L} {}`n", COMMAND, UsagePattern)
}
Help .= "`n"
Help .= "Options:`n"
ColumnWidth := 0
for Opt, Props in OPTIONS
{
CurrentWidth := 1 + StrLen(Opt) + (Props["Arg"] <> "" ? StrLen(Props["Arg"]) + 3 : 0)
ColumnWidth := CurrentWidth > ColumnWidth ? CurrentWidth : ColumnWidth
}
FormatStr := " {:-" . ColumnWidth . "} {}`n"
for Opt, Props in OPTIONS
{
Help .= Format(FormatStr, "-" . Opt . (Props["Arg"] <> "" ? " <" . Props["Arg"] . ">" : ""), Props["Desc"])
}
Help := RTrim(Help, "`n")
Show(Help)
}
CtrlEvent := ""
HandlerRoutine(dwCtrlType)
{
local
global CtrlEvent
if (dwCtrlType == 0)
{
CtrlEvent := CtrlEvent <> "Ctrl+Break" ? "Ctrl+C" : CtrlEvent
Handled := true
}
else if (dwCtrlType == 1)
{
CtrlEvent := "Ctrl+Break"
Handled := true
}
else
{
; <your code>
Handled := false
}
return Handled
}
DllCall("SetConsoleCtrlHandler", "ptr", RegisterSyncCallback("HandlerRoutine"), "int", 1)
PollForConsoleCtrlEvents()
{
; This should be called inside long-running loops.
;
; Do not forget to reset CtrlEvent to "" when handling the exceptions!
local
global CtrlEvent
if (CtrlEvent == "Ctrl+C")
{
throw Exception("Ctrl+C was pressed")
}
else if (CtrlEvent == "Ctrl+Break")
{
throw Exception("Ctrl+Break was pressed")
}
}
Main(A_Args)
Conclusion
I wrote this for several reasons:
- to save others the time and effort it took me to learn how to do this
- to encourage the AutoHotkey developers to improve support for writing command-line programs
- to thank the community for helping me
