xsharp.eu • easy calculation goes wrong
Page 1 of 1

easy calculation goes wrong

Posted: Sun Apr 28, 2024 10:00 am
by Horst
Hello
This line is calculating 806.38+24.54 and the result is 830.93 and thats wrong , look at screenshot.

Code: Select all

aMwstCodes [nCnt][04] := aMwstCodes [nCnt][04]  + fSteuer  
i changed all to decimal variables

Code: Select all

fSteuerTmp  := fBetrag - fBetrag / (100+fProzent) * 100 
fSteuer	   += fSteuerTmp           
fSteuerTotal := aMwstCodes [nCnt][04]                   
fSteuerTotal := fSteuerTotal +fSteuer      
aMwstCodes [nCnt][04] := fSteuerTotal  
but the same result, all other calculations are ok.
any idea ?

Horst

Re: easy calculation goes wrong

Posted: Sun Apr 28, 2024 11:42 am
by Horst
Hello
After my post i sleep a houre, think has to do something with the setdecimal or something else in teh settings. Will read the help now.

Re: easy calculation goes wrong

Posted: Sun Apr 28, 2024 7:44 pm
by g.bunzel@domonet.de
Horst,

ich verwende diese Einstellung und hatte noch nie ein Problem mit Rundungsfehlern:
SetFloatDelta (0.0001)

Um bei solchen Berechnung mit möglichst wenig Nachkommastellen zu arbeiten, würde ich erst die Multiplikation und dann die Division machen - damit die Werte der Variablen nicht so klein sind und viele Nachkommastellen haben, die dann beim Runden abgeschnitten werden.
Damit alle Werte der Zeile als FLOAT behandelt werden, sollte diese Zeile so aussehen:

fSteuerTmp := fBetrag - fBetrag * 100.0 / (100.0 + fProzent)


HTH

Gerhard

Re: easy calculation goes wrong

Posted: Mon Apr 29, 2024 7:56 am
by robert
Guys,
This problem is caused by the fact that certain floating point numbers cannot exactly be represented.
If you want 100% accuracy you should either calculate with integers (in cents) or with the Decimal type.

Robert

Re: easy calculation goes wrong

Posted: Mon Apr 29, 2024 8:28 am
by FFF
But Horst wrote, that he changed to decimal with no avail?

Re: easy calculation goes wrong

Posted: Thu May 02, 2024 4:20 pm
by Horst
Hello

It was my fault

i changed to SetFloatDelta (0.0001) then i saw more decimal numbers 806.389 + 24.544 is 830.93
Wired, it was the only one in more then 300 calculations.
Murphys law :-)

Horst