jeeswg's benchmark tests

[jeeswg]

modulo v. compare and reset (compare and reset is faster)

Code: Select all

;modulo v. compare and reset
vCount := 5, vIndex := 0
vQPC1 := QPC()
Loop, 1000000
{
	;vIndex := Mod(A_Index-1, vCount)+1
	vIndex := (vIndex = vCount) ? 1 : vIndex+1
	;MsgBox, % vIndex
}
vQPC2 := QPC()
Clipboard := vQPC2-vQPC1
MsgBox, % Clipboard
return

;337.366168 ;mod
;357.299131
;358.314360

;272.308958 ;compare and reset
;267.904642
;270.292184

[jeeswg]

with/without obj.SetCapacity() (SetCapacity() can reduce the time needed by 37%)
I try a small number of pushes (100), multiple times

Code: Select all

q:: ;benchmark tests - obj.SetCapacity()
oQPC := []
vNumLoop := 10000
vNum := 100

oArray := ""
vQPC := QPC()
Loop, % vNumLoop
{
	oArray := {}
	oArray.SetCapacity(vNum)
	Loop, % vNum
		oArray.Push("abcdefghijklmnopqrstuvwxyz")
}
oQPC.Push(QPC()-vQPC)

oArray := ""
vQPC := QPC()
Loop, % vNumLoop
{
	oArray := {}
	Loop, % vNum
		oArray.Push("abcdefghijklmnopqrstuvwxyz")
}
oQPC.Push(QPC()-vQPC)

Clipboard .= "`r`n;" oQPC.1 "`t" oQPC.2
MsgBox, % Clipboard
return

;results
;SetCapacity used	SetCapacity not used
;360.029217	553.583494
;328.246638	515.111692
;341.136715	557.868070
;330.608522	528.977227
;358.573085	580.349369

;take averages and divide:
;343.718835/547.177970 = 0.628166

[jeeswg]

- I wanted to be able to check if a string was ASCII. If ASCII, save a file without a BOM, else, save a file as UTF-8 with a BOM.
- Loop can be slow, and RegEx can be slow, so it was a battle of two 'slownesses'. RegEx was much faster in these examples.

Code: Select all

q:: ;compare Loop v. RegEx to determine if text is ASCII
oQPC := []

vNum := 1000
vLen := 10000
vText := ""
VarSetCapacity(vText, vLen*2)
;vText .= Chr(8730) ;non-ASCII is first character (square root)
Loop, % vLen-1
	vText .= "a"
vText .= Chr(8730) ;non-ASCII is last character (square root)

vQPC := QPC()
Loop, % vNum
{
	Loop, Parse, vText
		if (Ord(A_LoopField) > 127)
			break
}
oQPC.Push(QPC()-vQPC)

vQPC := QPC()
Loop, % vNum
	RegExMatch(vText, "[^[:ascii:]]")
oQPC.Push(QPC()-vQPC)

Clipboard .= "`r`n;" oQPC.1 "`t" oQPC.2
MsgBox, % Clipboard
return

;Ord	RegExMatch [non-ASCII is first character]
;18.860856	0.866409
;20.594957	0.824927
;17.477852	0.316030
;11.557107	0.514885
;18.875824	0.827921

;Ord	RegExMatch [non-ASCII is last character]
;1570.211616	230.783343
;1549.919432	226.551368
;1565.157279	229.756140
;1573.236349	230.733309
;1540.161855	233.202531

[jeeswg]

- I was interested as to whether VarSetCapacity was much slower for: (a) larger capacities, and (b) using fill bytes.
- I would say that the cost is pretty low.

Code: Select all

q:: ;compare VarSetCapacity - 2K bytes v. 2M bytes
oQPC := []

vNum := 20000

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000*2)
}
oQPC.Push(QPC()-vQPC)

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000000*2)
}
oQPC.Push(QPC()-vQPC)

Clipboard .= "`r`n;" oQPC.1 "`t" oQPC.2
MsgBox, % Clipboard
return

;2KB	2MB
;16.560981	260.412729
;17.314492	343.399812
;16.707664	296.506824
;17.734439	295.749465
;16.543020	330.927117

Code: Select all

q:: ;compare VarSetCapacity - fill with zero bytes on/off
oQPC := []

vNum := 1000
vSize := 1000000*2

vNum := 1000000
vSize := 4

vNum := 1000000
vSize := 10000

vNum := 4000000
vSize := 1000

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	;VarSetCapacity(vOutput, 0)
	VarSetCapacity(vOutput, vSize)
}
oQPC.Push(QPC()-vQPC)

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	;VarSetCapacity(vOutput, 0)
	VarSetCapacity(vOutput, vSize, 0)
}
oQPC.Push(QPC()-vQPC)

Clipboard .= "`r`n;" oQPC.1 "`t" oQPC.2
MsgBox, % Clipboard
return

;889.000457	1016.502394
;877.666153	1017.984184
;887.575971	1014.010934
;879.429761	1015.356305
;878.179327	1018.168499

[nnnik]

2 things:
Don't trigger your benchmarks with a Hotkey and please let AutoHotkey do an unmeasured testrun before all serious runs.
e.g.:

Code: Select all

oQPC := []

vNum := 20000
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000*2)
}

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000*2)
}
oQPC.Push(QPC()-vQPC)

[jeeswg]

- Point taken, but what's wrong with a hotkey?
- Do you mean ignore the first result? I already do that.
- I usually alter the tests until a difference is clear, but yes, I'll create stand-alone scripts in future. I've created a new folder just now, this will make it easier to manage and share my benchmark tests in future. (I will port old benchmark tests there also.)
- Would you not suggest to use a separate script for each version of the test?
- My intention would be to output results using stdout.
- If you were so inclined, you could create a zip with what you consider the perfect setup, with an example benchmark test. Cheers.

[nnnik]

No I dont mean to ignore the first result - I'm not sure that will work.
The only way that you can be sure that it will work is by setting AHKs priority making the thread uninterupteable and reducing the amount of external influences. (e.g. a Hotkey would be an external influence)

Code: Select all

#NoEnv
#MaxHotkeysPerInterval 99000000
#HotkeyInterval 99000000
#KeyHistory 0
#SingleInstance force
ListLines Off
Process, Priority, , H
SetBatchLines, -1
SetKeyDelay, -1, -1
SetMouseDelay, -1
SetDefaultMouseSpeed, 0
SetWinDelay, -1
SetControlDelay, -1
SendMode Input
SetWorkingDir %A_ScriptDir%
Critical

oQPC := []

vNum := 20000
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000*2)
}

vQPC := QPC()
Loop, % vNum
{
	vOutput := ""
	VarSetCapacity(vOutput, 1000*2)
}
oQPC.Push(QPC()-vQPC)