Why is there such a huge difference in numerical calculations?

Discuss the future of the AutoHotkey language
arcticir
Posts: 543
Joined: 17 Nov 2013, 11:32

Why is there such a huge difference in numerical calculations?

11 Aug 2018, 03:19

Code: Select all

MsgBox (0-0.25) "`n" (0-0.2)
;-0.25
;-0.20000000000000001
User avatar
nnnik
Posts: 3564
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Why is there such a huge difference in numerical calculations?

11 Aug 2018, 05:09

Numbers in our PC are represented as finite binary numbers. As finite numbers they have a maximum precision and length.
Integers represent all numbers that are not floating point values in the range from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807.
Doubles are a bit more difficult to describe. What I can tell you is that there is a float that perfectly describes the number 0.25 and it's value is 2**-2.
0.2 can not be represented like that. That seems confusing but keep in mind that our PC uses the binary system rather than the decimal system.
Recommends AHK Studio
lexikos
Posts: 6207
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: Why is there such a huge difference in numerical calculations?

11 Aug 2018, 20:33

0.00000000000000001 is not a "huge" difference. It is very small. :P

AutoHotkey v1 hides this imprecision from you by default, by showing only 6 decimal places. This leads to confusion like:

Code: Select all

MsgBox % 0.3 - 0.1  ; 0.200000
MsgBox % 0.2 + 0    ; 0.200000
MsgBox % 0.3 - 0.1 = 0.2 + 0  ; False!
This topic has been covered multiple times on these forums.
arcticir
Posts: 543
Joined: 17 Nov 2013, 11:32

Re: Why is there such a huge difference in numerical calculations?

12 Aug 2018, 00:56

Sorry, I should search the forum.

Thank you two.

Return to “AutoHotkey v2 Development”

Who is online

Users browsing this forum: No registered users and 2 guests