tinyBench - Code Benchmarking

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
Masonjar13
Posts: 1555
Joined: 20 Jul 2014, 10:16
Location: Не Россия
Contact:

tinyBench - Code Benchmarking

29 Jul 2016, 01:54

My take on a benchmarking script, aptly named "tinyBench." Simply follow the instructions of the comments.
Old code
Class version:

Code: Select all

#singleInstance ignore
#include tinyBenchClass.ahk
setBatchLines -1
setWorkingDir,% a_scriptDir

tb:=new tinyBench("strI",100) ; file name, iteration count

; code setup - if any static variables are needed, set them here
; userInfo will be logged in the output
iv:=randStr(100000,100000,1234)
tb.userInfo:="StrLen: " . strLen(iv)

loop % tb.iterations {
    tb.start()
    ; code to be benchmarked
    strI(iv)
    
    ; end code
    tb.end()
}
tb.output()
msgbox,4,Finished,% "Benchmark for " . tb.name . " completed! Open log now?"
ifMsgBox Yes
    run % tb.file
exitApp

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/strI.ahk
strI(str){
    static rev:=a_isUnicode?"_wcsrev":"_strrev"
    dllCall("msvcrt.dll\" . rev,"Ptr",&str,"CDECL")
    return str
}

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/randStr.ahk
/* Modes

 - Append numbers to combine
1=lowercase
2=uppercase
3=digits
4=symbols

example: randStr(1,5,124)

test:
rr:=[1,2,3,4,12,13,14,23,24,34,123,124,134,1234]
for i,a in rr
    msgbox % a "`n" randStr(10,20,a)
*/
randStr(lowerBound,upperBound,mode=1){
    if(!isDigit(lowerBound)||!isDigit(upperBound)||!isDigit(mode))
        return -1
    loop % rand(lowerBound,upperBound){
        t:=""
        if(strLen(mode)=1)
            t:=mode
        else{
            while(!ifContains(mode,t))
                t:=rand(1,4)
        }
        if(t=1)
            str.=chr(rand(97,122))
        else if(t=2)
            str.=chr(rand(65,90))
        else if(t=3)
            str.=rand(0,9)
        else if(t=4){
            i:=rand(1,4)
            str.=i=1?chr(rand(33,47)):i=2?chr(rand(58,64)):i=3?chr(rand(91,96)):chr(rand(123,126))
        }
    }
    return str
}

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/isDigit.ahk
isDigit(in){
    if in is digit
        return 1
}

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/rand.ahk
rand(lowerBound,upperBound){
    random,rand,% lowerBound,% upperBound
    return rand
}

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/ifContains.ahk
ifContains(haystack,needle){
    if haystack contains %needle%
        return 1
}

Code: Select all

class tinyBench{

    __new(name,iterations:=""){
        dllCall("QueryPerformanceFrequency","Int64*",f),this.freq:=f
        this.userInfo:="<UserInfo>"
        this.name:=name
        if(iterations)
            this.iterations(iterations)
        this.timings:=object()
    }
    
    iterations(iterationCnt){
        this.timings.setCapacity(this.iterations:=iterationCnt)
    }
    
    start(){
        this.ts:=a_tickCount,dllCall("QueryPerformanceCounter","Int64*",s),this.s:=s
    }
    
    end(){
        dllCall("QueryPerformanceCounter","Int64*",e),te:=a_tickcount
        this.timings.push({µs: ((e-this.s)*1000000)/this.freq
                            ,ms: ((e-this.s)*1000)/this.freq
                            ,mst: te-this.ts})
    }
    
    output(){
        for i,a in this.timings{
            µsA+=a.µs
            µsL:=!µsL?a.µs:a.µs<µsL?a.µs:µsL
            µsH:=a.µs>usH?a.µs:µsH
            
            msA+=a.ms
            msL:=!msL?a.ms:a.ms<msL?a.ms:msL
            msH:=a.ms>msH?a.ms:msH
            
            mstA+=a.mst
            mstL:=!mstL?a.mst:a.mst<mstL?a.mst:mstL
            mstH:=a.mst>mstH?a.mst:mstH
        }
        µsA/=this.timings.length()
        msA/=this.timings.length()
        mstA/=this.timings.length()
        fileAppend,% this.userInfo
        . "`nIterations: " . this.timings.length()
        . "`nQPC (µs)`n    • Average Time Elapsed: " . commaFormat(µsA)
        . "`n    • Lowest Time Elapsed: " . commaFormat(µsL)
        . "`n    • Highest Time Elapsed: " . commaFormat(µsH)
        . "`nQPC (ms)`n    • Average Time Elapsed: " . commaFormat(msA)
        . "`n    • Lowest Time Elapsed: " . commaFormat(msL)
        . "`n    • Highest Time Elapsed: " . commaFormat(msH)
        . "`nTickCount (ms)`n    • Average Time Elapsed: " . commaFormat(mstA)
        . "`n    • Lowest Time Elapsed: " . commaFormat(mstL)
        . "`n    • Highest Time Elapsed: " . commaFormat(mstH)
        . "`n`n",% this.file:=this.name . "_tinyBench.txt"
    }
    
}

; https://github.com/Masonjar13/AHK-Library/blob/master/Lib/commaFormat.ahk
commaFormat(num){
    num:=strSplit(num,.)
    nnum:=num[1]
    numlen:=strlen(nnum)
    loop,parse,nnum
        xnum.=!mod(numlen-a_index,3) && a_index != numlen ? a_loopfield "," : a_loopfield
    return num[2]?xnum "." num[2]:xnum
}
The class update adds ms for UPC, as well as comma formatting.
Last edited by Masonjar13 on 29 Jul 2016, 04:47, edited 4 times in total.
OS: Windows 10 Pro | Editor: Notepad++
My Personal Function Library | Old Build - New Build
Helgef
Posts: 4709
Joined: 17 Jul 2016, 01:02
Contact:

Re: tinyBench - Code Benchmarking

29 Jul 2016, 03:36

Nice script,

Code: Select all

timings.SetCapacity(iterations)
speeds up the loop's self time.

If you floor divide to averages beacuse of precision limitations, then you probably want to floor divide the high/low too. The precision of the counter is in the f variable, you could round to closest 1/f, I guess, but I haven't put a whole lot of thought into the matter and might be totally off here, would like to hear the reasoning behind the //= though. But, this is probably not very important for most code measurments of AHK scripts anyways.
User avatar
Masonjar13
Posts: 1555
Joined: 20 Jul 2014, 10:16
Location: Не Россия
Contact:

Re: tinyBench - Code Benchmarking

29 Jul 2016, 04:21

Helgef wrote:Nice script,

Code: Select all

timings.SetCapacity(iterations)
speeds up the loop's self time.
While the object handling time doesn't really matter, since it'll know how many there will be, I don't see a reason why not.
Helgef wrote:you probably want to floor divide the high/low too.
Divide it by what? No operation is being made on them. If you mean to just floor() them, I didn't do that, because I wanted to retain the precision.
Helgef wrote:would like to hear the reasoning behind the //= though.
I had this set before I implemented the high/low, before I thought about implementing them rather. It doesn't really matter for the average, as it's just that, an average. Taking the floor off and dividing normally doesn't seem to make any difference, strangely.
Helgef wrote:The precision of the counter is in the f variable
f holds the frequency of the ticks. QPC ticks are "arbitrary" (not really) and are translated to humanized time when divided by the frequency. In this case, µs (microseconds) are what we're looking for. Based on the math, QPC should be accurate to 10 ps (picoseconds), if divided immediately by the frequency. The precision is a combination of both the ticks and frequency; the frequency is not the precision of, but rather a component of.

I hope that covers everything for you :)
OS: Windows 10 Pro | Editor: Notepad++
My Personal Function Library | Old Build - New Build
Helgef
Posts: 4709
Joined: 17 Jul 2016, 01:02
Contact:

Re: tinyBench - Code Benchmarking

29 Jul 2016, 04:54

Masonjar13 wrote: If you mean to just floor() them
Yes I mean just floor()
Masonjar13 wrote:I hope that covers everything for you :)
It's ok.
User avatar
Masonjar13
Posts: 1555
Joined: 20 Jul 2014, 10:16
Location: Не Россия
Contact:

Re: tinyBench - Code Benchmarking

29 Jul 2016, 20:49

The comments in the code will guide you. It does require to be edited, it doesn't work as-is.
OS: Windows 10 Pro | Editor: Notepad++
My Personal Function Library | Old Build - New Build
User avatar
Masonjar13
Posts: 1555
Joined: 20 Jul 2014, 10:16
Location: Не Россия
Contact:

Re: tinyBench - Code Benchmarking

17 Nov 2017, 22:58

Decided to update this, for fun really. Code in OP.

By the way, if anyone could enlighten me on how to put this.var directly in the DLL call, let me know :)
OS: Windows 10 Pro | Editor: Notepad++
My Personal Function Library | Old Build - New Build
Helgef
Posts: 4709
Joined: 17 Jul 2016, 01:02
Contact:

Re: tinyBench - Code Benchmarking

18 Nov 2017, 03:19

Hello :wave:.
Thanks for sharing, very neat output :thumbup:. There are a few functions missing, like, ifcontains, isDigit and rand. :arrow: Masonjar13/AHK-Library.
By the way, if anyone could enlighten me on how to put this.var directly in the DLL call, let me know
dllcall(..., "int64*", this.var) :D. Jokes aside, you have already notice that it doesn't yield the desired result, as per the documentation,
Append an asterisk (with optional preceding space) to any of the above types to cause the address of the argument to be passed rather than the value itself (the called function must be designed to accept it). Since the value of such an argument might be modified by the function, whenever a naked variable is passed as the argument, that variable's contents will be updated.
So, this.var, is not a naked variable, it is an expression which yields a value, hence, the dll-function will be passed a pointer to an int64, pointing to this value. Now, if this.var is blank, ahk passes a pointer to a 0. The dll-function will write the frequency to that location, but you do not have the pointer, so you cannot retrieve the value. So your solution is the best you can do, imo. If we didn't have the feature of type* for dllcall, we'd have to do something like this,

Code: Select all

this.setCapacity("freq",8)
dllCall("QueryPerformanceFrequency","Ptr", this.getAddress("freq"))
this.freq := numget(this.getAddress("freq"),"int64")
or in v2, due to better number handling,

Code: Select all

dllCall("QueryPerformanceFrequency","Ptr", &(pf:=0))
this.freq := pf
:thumbup:
On a side note, I'd use a static variable for the frequency, eg,

Code: Select all

class tinyBench {
	; ...
	static freq := tinyBench.getFreq()
	getFreq(){
		local f
		dllCall("QueryPerformanceFrequency", "int64*",f)
		return f
	}
}
Cheers.
User avatar
Masonjar13
Posts: 1555
Joined: 20 Jul 2014, 10:16
Location: Не Россия
Contact:

Re: tinyBench - Code Benchmarking

18 Nov 2017, 05:14

Helgef wrote:There are a few functions missing
Typical mistake of mine :roll: ; I'm so dependent on my std lib.. I've edited to included the necessary functions (I believe).

Thanks for the insight! Interesting syntax for v2, I haven't used it myself yet.

You're entirely right about making it static, considering there's 0 chance of it changing. I'll change that when I get back to my main computer (to keep it synonymous) :thumbup:

Your input is most appreciated!
OS: Windows 10 Pro | Editor: Notepad++
My Personal Function Library | Old Build - New Build

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: Auntiejack56 and 132 guests