jeeswg's benchmark tests

Helpful script writing tricks and HowTo's
User avatar
nnnik
Posts: 2506
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: jeeswg's benchmark tests

19 Jan 2018, 14:58

Thank you - this is the result of one of me wanting to find out some of Helgefs remarks.
I could always help you splitting the topics if you need that. Same goes for editing the first post if you want to add new stuff.
Recommends AHK Studio
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

19 Jan 2018, 15:29

- The most practical solution, I think, everything considered, is this:
- We move this thread to a different forum, Scripts and Functions perhaps.
- It's too early to try to make this a 'tidy' thread, the posts will be being changed and added to for some time to come.
- At some future date, I could think about consolidating them in one place, in a perfected form.
- I will start a new tutorial, soon, explicitly regarding how to speed up scripts. It would be a one-post wonder, that I would continue updating from time to time. It would summarise everything that I'd discovered from doing benchmark tests.
- To maintain some order in this thread, the OP will list/link to each benchmark test.
Helgef
Posts: 2674
Joined: 17 Jul 2016, 01:02
Contact:

Re: jeeswg's benchmark tests

20 Jan 2018, 14:16

Your test is well formatted nnnik :thumbup:, note that it is ms not µs and the

Code: [Select all] [Download] (Untitled.txt)GeSHi © Codebox Plus

String from a variable, variable from an object - function with byRef:

line is used twice, the last time appears to be a mistake.

Cheers.
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

20 Jan 2018, 14:40

STRINGS: SUBSTR: CROP FIRST CHARACTER V. CROP LAST CHARACTER

- SubStr(vText, 2) v. SubStr(vText, 1, -1)
In the example below, cropping the last character was roughly 7 times faster.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus


- I remembered seeing a nice QPC function when reading through threads, turns out it wasn't this thread though. A function by Helgef based on a function by wolf_II.
[QPC function, returns milliseconds]
Anagrams - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=19&t=34240&p=158464#p158464
[QPC function, returns seconds]
Code Puzzle Thread - Page 3 - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=17&t=25683&p=186646#p186646
- I have added my version of the function to 'ASSUME ALWAYS' in the OP. It returns milliseconds. It's basically the same as Helgef's.
Last edited by jeeswg on 01 Feb 2018, 10:34, edited 1 time in total.
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

01 Feb 2018, 00:35

Here are 7 benchmark tests. Obvious mistakes/having potentially already done these tests earlier in the thread, are concerns.

[classic dichotomy]
benchmark tests - <= v. <
< slightly faster

[curious]
benchmark tests - || v. else
|| slightly faster

[classic dichotomy]
benchmark tests - for loop v. loop (on array)
loop slightly faster

[classic dichotomy]
benchmark tests - assign key, don't care about value (value as string/number)
'0'/'1'/(blank) roughly the same, 'a' takes about 40% longer

[curious][expected to take longer, but not this much longer]
benchmark tests - get file paths (A_LoopFileFullPath v. A_LoopFileLongPath)
A_LoopFileLongPath took around 65 times longer

[classic AHK knowledge]
benchmark tests - read lines (FileRead / FileReadLine / A_LoopReadLine / .ReadLine())
FileRead fastest
A_LoopReadLine slightly slower
.ReadLine() around 5 times slower than FileRead
FileReadLine around 350 times slower than .ReadLine() (so around 1750 times slower than FileRead)

[classic AHK knowledge]
benchmark tests - append text (with/without prior VarSetCapacity)
if don't prepare capacity, around 8 times slower (this is very much dependent on how many times expansion/copying is necessary)

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

Last edited by jeeswg on 01 Feb 2018, 10:35, edited 1 time in total.
User avatar
nnnik
Posts: 2506
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: jeeswg's benchmark tests

01 Feb 2018, 03:50

From my tests the 1st test is always slightly slower than the following tests.
Recommends AHK Studio
User avatar
nnnik
Posts: 2506
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: jeeswg's benchmark tests

01 Feb 2018, 04:11

After switching the order of some tests I got opposite results see:
< vs <=
( BTW both < and <= should be just as fast as the other since they are both a single CPU instruction )

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

Recommends AHK Studio
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

01 Feb 2018, 04:53

Thanks nnnik, I had been concerned about whether order mattered. Does anyone have any ideas?
User avatar
nnnik
Posts: 2506
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: jeeswg's benchmark tests

01 Feb 2018, 05:06

Rather than leaving it to writing manual tests all the time we should probably write a benchmark library - that handles the tests correctly
Recommends AHK Studio
Helgef
Posts: 2674
Joined: 17 Jul 2016, 01:02
Contact:

Re: jeeswg's benchmark tests

01 Feb 2018, 05:35

for loop v. loop (on array)
loop slightly faster

For me for-loop is faster, as expected.

Cheers.


note: this function returns seconds, the original function returns milliseconds

Note: Your function returns milliseconds.
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

01 Feb 2018, 10:27

- I had thought that the for loop might be faster. Why did we get different results (did you perform the tests twice, swapping the order, did you perform the operation more times)? Why are earlier tests faster within the same script run, should we have a delay after a hotkey is executed or a dummy test?
- I've tried to incorporate every piece of advice I've been given. So getting incorrect results at this stage is really facepalm.
- Btw how do x64/x32 compare generally?
- Also I've been wondering about 'CPU' benchmark tests, or other measures, i.e. the method that takes less of a toll on the system.
- Yes, Helgef, it's milliseconds. I've now corrected it everywhere in this thread. [@nnnik: could you edit your post to make it say 'benchmark tests (ms)', thanks.]
QueryPerformanceCounter MSDN page:
Retrieves the current value of the performance counter, which is a high resolution (<1us) time stamp that can be used for time-interval measurements.
...
A pointer to a variable that receives the current performance-counter value, in counts.
QueryPerformanceFrequency MSDN page:
A pointer to a variable that receives the current performance-counter frequency, in counts per second.

Code: [Select all] [Download] GeSHi © Codebox Plus

;frequency = count / time
;count = frequency * time
;time = count / frequency
DllCall("kernel32\QueryPerformanceFrequency", Int64P,vFreq) ;count per sec
DllCall("kernel32\QueryPerformanceCounter", Int64P,vCount) ;count
MsgBox, % (vCount / vFreq) ;seconds
MsgBox, % (vCount / vFreq) * 1000 ;milliseconds
User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

17 Feb 2018, 07:25

Here's an interesting one, object count keys: clone+delete v. for loop v. delete. See results at the bottom. Cheers.

Code: [Select all] [Expand] [Download] GeSHi © Codebox Plus

User avatar
jeeswg
Posts: 3591
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's benchmark tests

Today, 07:00

- I'd be interested in any benchmark tests re. optimising DllCall results, e.g. specifying dll name or not, specifying .dll or not, specifying W/A or not, using Int or "Int" and anything else. I haven't been able to get any clear-cut results so far.
- The documentation does mention about using LoadLibrary and using a function address (for dlls that aren't pre-loaded).
DllCall
https://autohotkey.com/docs/commands/DllCall.htm#load
- Also, any good example dlls/functions for testing would be helpful, it's hard to find the best ones for testing.

- Also, doing a benchmark test, with the exact same code being tested twice in a row, with one being faster than the other, is concerning.
User avatar
nnnik
Posts: 2506
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: jeeswg's benchmark tests

Today, 07:35

jeeswg wrote:- Also, doing a benchmark test, with the exact same code being tested twice in a row, with one being faster than the other, is concerning.

Thats due to the way modern CPUs work - the thing you are looking for is called Branch Prediction.

Also after testing it myself it seems that using a pointer isn't faster than using the dll functions name.
Recommends AHK Studio

Return to “Tutorials”

Who is online

Users browsing this forum: No registered users and 4 guests