Page 1 of 1
easy calculation goes wrong
Posted: Sun Apr 28, 2024 10:00 am
by Horst
Hello
This line is calculating 806.38+24.54 and the result is 830.93 and thats wrong , look at screenshot.
Code: Select all
aMwstCodes [nCnt][04] := aMwstCodes [nCnt][04] + fSteuer
i changed all to decimal variables
Code: Select all
fSteuerTmp := fBetrag - fBetrag / (100+fProzent) * 100
fSteuer += fSteuerTmp
fSteuerTotal := aMwstCodes [nCnt][04]
fSteuerTotal := fSteuerTotal +fSteuer
aMwstCodes [nCnt][04] := fSteuerTotal
but the same result, all other calculations are ok.
any idea ?
Horst
Re: easy calculation goes wrong
Posted: Sun Apr 28, 2024 11:42 am
by Horst
Hello
After my post i sleep a houre, think has to do something with the setdecimal or something else in teh settings. Will read the help now.
Re: easy calculation goes wrong
Posted: Sun Apr 28, 2024 7:44 pm
by g.bunzel@domonet.de
Horst,
ich verwende diese Einstellung und hatte noch nie ein Problem mit Rundungsfehlern:
SetFloatDelta (0.0001)
Um bei solchen Berechnung mit möglichst wenig Nachkommastellen zu arbeiten, würde ich erst die Multiplikation und dann die Division machen - damit die Werte der Variablen nicht so klein sind und viele Nachkommastellen haben, die dann beim Runden abgeschnitten werden.
Damit alle Werte der Zeile als FLOAT behandelt werden, sollte diese Zeile so aussehen:
fSteuerTmp := fBetrag - fBetrag * 100.0 / (100.0 + fProzent)
HTH
Gerhard
Re: easy calculation goes wrong
Posted: Mon Apr 29, 2024 7:56 am
by robert
Guys,
This problem is caused by the fact that certain floating point numbers cannot exactly be represented.
If you want 100% accuracy you should either calculate with integers (in cents) or with the Decimal type.
Robert
Re: easy calculation goes wrong
Posted: Mon Apr 29, 2024 8:28 am
by FFF
But Horst wrote, that he changed to decimal with no avail?
Re: easy calculation goes wrong
Posted: Thu May 02, 2024 4:20 pm
by Horst
Hello
It was my fault
i changed to SetFloatDelta (0.0001) then i saw more decimal numbers 806.389 + 24.544 is 830.93
Wired, it was the only one in more then 300 calculations.
Murphys law
Horst