Comparing Floating Point Numbers

Written by Rod Wing

 

Transforming coordinates is a common task in many AutoCAD and MicroStation VBA programs. Coordinates in both applications are stored as double precision floating point numbers. Due to the nature of floating point numbers you can run into situations where values that should be equal are not because of rounding issues.

 

The Problem

In the example macro below some transformations are performed on two X coordinates and then the values are compared.

Public Sub myMacro()
    Dim X1 As Double
    Dim X2 As Double
    '
    ' Perform some transformations on the X values
    '     move, scale, rotate, etc.
    '
    If X1 = X2 Then
        MsgBox "Values are equal", vbInformation, "MyMacro"
    Else
        MsgBox "Values are different" _
                    & vbCrLf & "X1 = " & CStr(X1) _
                    & vbCrLf & "X2 = " & CStr(X2) _
                    , vbInformation, "MyMacro"
    End If
End Sub

Comparing Floating Point Numbers

You will notice that the values are the same for 12 places after the decimal point before they vary. If you only need to be accurate to four places after the decimal point then these values are equal. So how can you tell VBA to allow these numbers to be equal?

 

 

The Solution

The easiest way to get around this problem is to define a Double constant to the level of precision you need. In the example below the constant ZERO has been defined for four places after the decimal. To compare for equality you then take the absolute value of the difference. If that value is less than your constant the values are equal.

 

Public Const ZERO As Double = 0.0001
Public Sub myMacro()
    Dim X1 As Double
    Dim X2 As Double
    '
    ' Perform some transformations on the X values
    '
    If Abs(X1 - X2) < ZERO Then
        MsgBox "Values are equal", vbInformation, "MyMacro"
    Else
        MsgBox "Values are different" _
                    & vbCrLf & "X1 = " & CStr(X1) _
                    & vbCrLf & "X2 = " & CStr(X2) _
                    , vbInformation, "MyMacro"
    End If
End Sub

Comparing Floating Point Numbers

This comparison would be a good candidate for its own function. Something like this perhaps?

 

Public Function DoubleEqual(dValuel As Double, dValue2 As Double , Optional dLimit As Double = .0001) As Boolean
    DoubleEqual = (Abs(dValue1 - dValue2) < dLimit)
End Function

Last Month’s VBA Tip: Error Handling

Don’t want to miss out on other great information? Subscribe to this blog or our monthly eNewsletter now!

Learn More ◊ Contact us today ◊ Newsletter ◊

EnvisionCAD Group EnvisionCAD YouTube Channel   

 

EnvisionCAD

EnvisionCAD

Since 1996, EnvisionCAD has been a nationally recognized leader in the configuration, customization, implementation, training & support for CAD software solutions. Our individualized approach has benefited private engineering firms and government agencies alike. Basic or advanced, we can help you get the most from your CAD technology.

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *