Float, Real8, or Decimal?

Public support forum for peer to peer support with related to the Visual Objects and Vulcan.NET products
User avatar
ArneOrtlinghaus
Posts: 412
Joined: Tue Nov 10, 2015 7:48 am
Location: Italy

Float, Real8, or Decimal?

Post by ArneOrtlinghaus »

I have now verified what is the setfloatdelta in our programs without having specified it.
It gives:
0,0000000000001

Now I can again think about what I have written some minutes before and the consequences...

... Fortunately in our programs we should have already made more stable most of the floating comparisons using the following operations.
function NumEQU (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob zwei Werte bis auf Rundungen gleich sind
//g Math
//g General
if Abs(nWert1) < 1.E3
return (abs(nWert2-nWert1) < (1.E-11))
elseif Abs(nWert1) < 1.E6
return (abs(nWert2-nWert1) < (1.E-8))
elseif Abs(nWert1) < 1.E9
return (abs(nWert2-nWert1) < (1.E-5))
else
return (abs(nWert2-nWert1) < (1.E-3))
endif

function NumGEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen größer gleich als ein anderer ist
//g Math
//g General
return NumEQU (nWert1,nWert2) .or. nWert1 > nWert2

function NumGTR (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen größer als ein anderer ist
//g Math
//g General
return !NumEQU (nWert1,nWert2) .and. nWert1 > nWert2

function NumLEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen kleiner gleich ein anderer ist
//g Math
//g General
return NumEQU (nWert1,nWert2) .or. nWert1 < nWert2

function NumLSS (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen kleiner als ein anderer ist
//g Math
//g General
return !NumEQU (nWert1,nWert2) .and. nWert1 < nWert2

function NumNEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob zwei Werte bis auf Rundungen ungleich sind
//g Math
//g General
return !NumEQU (nWert1,nWert2)
User avatar
Chris
Posts: 4855
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

Float, Real8, or Decimal?

Post by Chris »

Hi Arne,

The problem is the precision of the real8 data type with decimal numbers, if you try this in any language:

LOCAL r1,r2 AS REAL8
r1 := 0.1
r2 := 0.3
r1 := r1 * 3.0
? r1 == r2 // FALSE!

this will return FALSE in all languages, including VO, X#, c# etc, because in binary format the value 0.1 cannot be exactly represented, so when doing math on it, you just go further away from the actual value.

VO attempted to overcome this inherent limitation of floating types by introducing SetFloatDelta(), which does work, but up to a point only, before needing to specify a larger value for SetFloatDelta(). In .Net this was fixed by introducing the System.Decimal data type, which can accurately represent decimal numbers, so you do not have to worry at all about the above anymore. Only problem is that it is slower than REAL8, which might become a problem if your calculations are extremely extensive.

Chris
Chris Pyrgas

XSharp Development Team
chris(at)xsharp.eu
User avatar
Otto
Posts: 174
Joined: Wed Sep 30, 2015 6:22 pm

Float, Real8, or Decimal?

Post by Otto »

ArneOrtlinghaus wrote:Hi Chris,
I didn't even know that there is this setfloatdelta after 20 years of VO programming....
same here... oh brother... would have saved us a lot of trouble.
User avatar
wriedmann
Posts: 3752
Joined: Mon Nov 02, 2015 5:07 pm
Location: Italy

Float, Real8, or Decimal?

Post by wriedmann »

Hi Arne, hi Otto,

using SetFloatDelta() is a relatively easy possibility to keep the problems of the floating point mess as low as possible.

I'm very happy that such a thing like the Decimal datatype exists in .NET!

Wolfgang
P.S. my first programming experience was Cobol, and there is absolutely no problem with decimal variables if you don't use binary datatypes.
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
User avatar
lumberjack
Posts: 726
Joined: Fri Sep 25, 2015 3:11 pm
Location: South Africa

Float, Real8, or Decimal?

Post by lumberjack »

Otto wrote:
ArneOrtlinghaus wrote: I didn't even know that there is this setfloatdelta after 20 years of VO programming....
same here... oh brother... would have saved us a lot of trouble.
It makes me think back to this in the c.l.c.vo NG...

Code: Select all

Willie Moore 	
8/31/07
Hey,
Here is a small tip and function in you ever run into any VB code that you need to translate to VO. in VB, int returns the integer part of the number ignoring the fractional part. But, if the number is negative, it take the abs of the fraction and if it is greater than 0, it returns the next negative number less than the passed number. So, if the number is int(-8.01) -n VB returns -9. (http://msdn2.microsoft.com/en-us/library/xh29swte(VS.71).aspx)
If you have to convert VO code from VB, here is a function to handle it.
Regards,
Willie

FUNCTION vbInt(nVar AS FLOAT) AS LONG PASCAL
   LOCAL nResult AS LONG
   LOCAL fFrac   AS FLOAT
   IF nVar >= 0
       nResult := INT(nVar)
   ELSE        
      fFrac := Frac(nVar)
      IF Abs(fFrac) > 0    
          nResult := INT(nVar) - 1
      ELSE
         nResult := INT(nVar)
     ENDIF
   ENDIF
   RETURN nResult

Code: Select all

Johan Nel
8/31/07
Hi Willie,
Yes you right, VB and VO handle negative integers differently.
You could also just use:
FUNCTION vbInt(nVar)
  RETURN Floor(nVar)

Code: Select all

Willie Moore 	
8/31/07
Johan,
Thanks. Dont think I have ever used floor. Makes the function ever more simple.
Regards,
Willie
______________________
Johan Nel
Boshof, South Africa
User avatar
Otto
Posts: 174
Joined: Wed Sep 30, 2015 6:22 pm

Float, Real8, or Decimal?

Post by Otto »

Seems like SetFloatDelta doesn't work for Real8 in XSharp (VO).
In VO it works for both Real8 the same way as for Float. :(
User avatar
robert
Posts: 4483
Joined: Fri Aug 21, 2015 10:57 am
Location: Netherlands

Float, Real8, or Decimal?

Post by robert »

Otto,
Thanks for the report.
To make this work we will have to change the compiler to generate different code for Real8 comparisons, a bit like we are doing now for string comparisons (/vo13) and we will have to add comparison routines to the runtime.
This is not very difficult but will take some time. Most likely we would add a new compiler option (vo17) for this.
How important is this for you ?

Robert
XSharp Development Team
The Netherlands
robert@xsharp.eu
User avatar
Chris
Posts: 4855
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

Float, Real8, or Decimal?

Post by Chris »

Hi Otto,

Are you sure? I just tried this in VO and it returns FALSE for REAL8 and TRUE (as expected) for FLOAT:

Code: Select all

LOCAL r1,r2 AS REAL8
LOCAL f1,f2 AS FLOAT
SetFloatDelta(0.1)
r1 := 0.01
r2 := 0.02
f1 := r1
f2 := r2
? r1 == r2 // FALSE
? f1 == f2 // TRUE
Chris Pyrgas

XSharp Development Team
chris(at)xsharp.eu
User avatar
Otto
Posts: 174
Joined: Wed Sep 30, 2015 6:22 pm

Float, Real8, or Decimal?

Post by Otto »

I've included the VO version.
But I compare the Real8 with a literal 0.0
Could it be that the 0.0 is seen as a float and that therefore the left hand side (real8) is converted into a float as wel?
Attachments
Termapp.zip
(797 Bytes) Downloaded 143 times
User avatar
Otto
Posts: 174
Joined: Wed Sep 30, 2015 6:22 pm

Float, Real8, or Decimal?

Post by Otto »

About the importance:
I'm looking for solutions for a lot of comparisons in my code between literal zero's and real8's.

If I can figure a structural solution for this in another way that could be ok. I am aware of the trouble in general with floating point comparison...

I'll come back on this tomorrow.
Post Reply