Your Privacy Matters: We use our own and third-party cookies to improve your experience on our website. By continuing to use the website we understand that you accept their use. Cookie Policy
455
weird rounding or calculation error for a simple formula
posted

I have a calculated column in a grid, the formula is simple: D = A - B  - C  

(D being the calculated column)

When the result should add up to zero, I get instead something like this: -2.8421709430404E-14

Is this a bug in the calc manager?

For example these numbers: 217.28 - 37.08 - 180.2 should return 0 but instead return the small scientific number above

EDIT: I am using version 2013.1

Parents
No Data
Reply
  • 9298
    posted

    Talal,

    I would have to know more about what you are doing to figure out what is causing this weird result that you are getting.  Take a look at the sample application that I have attached and see if I am doing something very different from what you are doing. 

    What is the DataType of the columns that you are working with?

    CalculatingApp.zip
Children