
×InformationFix Windows 10 Update Issues
Resolve Windows 10 related issues for your HP computers or printers by HP Windows 10 Support Center

×InformationFix Windows 10 Update Issues
Resolve Windows 10 related issues for your HP computers or printers by HP Windows 10 Support Center
 HP Community
 >
 Other Products
 >
 Calculators
 >
 Calculating with decimals Error
 Mark Topic as New
 Mark Topic as Read
 Float this Topic for Current User
 Bookmark
 Subscribe
 Mute
 Printer Friendly Page
Create an account on the HP Community to personalize your profile and ask a question
Solved!
Calculating with decimals Error
03222018 09:18 PM  edited 03222018 09:18 PM
Hello,
I entered:
13.6+6.420 and press the enter key in CAS mode
and it shows 1.13686837722E13
However, I entered same in Home mode shows 0.
I tried several decimals making 0 in Home mode, always answer is 0.
But in the CAS mode shows smoetimes nonzero answers.
In the CAS mode, 55.8+22.177.9 is 0
but 55.8+22.278 is 4.54747350886E13
Is this just me or is this a bug in the CAS Mode??
This is my HP Prime info:
Software Version : 2018 02 12 (13441)
Hardware Version : C
CAS Version 1.4.9
Operating System : V0.050.639
Thank you.
Solved! Go to Solution.
03222018 11:00 PM  edited 03222018 11:00 PM
> Is this just me or is this a bug in the CAS Mode??
Neither. It's due to the fact that CAS mode uses 48bit binary floating point numbers, unlike Home mode which uses 12digit BCD floating point numbers.
Therefore, when you type 22.2 into Home, you get exactly 22.2, but in CAS it *displays* as 22.2 but internally is the closest that can be obtained with 48 bits over a power of 2 (in this case, the closest possible value is 195273265093017/2^43, which is exactly equal to 22.199999999999846522769075818359851837158203125, which as you can see is just a tiny bit less than 22.2). Since numbers are displayed rounded to 12 digits (or less, according to the Number Format you set in Home Settings), it is displayed as 22.2 but internally it's really the strangelooking value above. This is not a bug; all binary floatingpoint systems (including almost all computer math software) act that way.
You may ask, Why on earth does CAS use such a strange system, when Home is exact? Answer: Home's BCD system isn't infallible either. For example 3*(1/3) > not 1! This is not a bug. All BCD systems act that way.
My favorite instantlygoofy result in CAS is: 1.9.1 > not zero!
To avoid needing to learn all of the above, just remember to do all your floatingpoint realnumber numbercrunching approximate math in Home, and only use CAS for either EXACT math (when the ENTIRE input contains no decimal points anywhere) and/or SYMBOLIC math (e.g. algebraic expression manipulations, equation solving, and anything else that uses variables which do not have a value already stored in them).
I hope I haven't just confused you further. If I have, then just remember to use Home for anything with a decimal point in it, and CAS for everything else.
Disclaimer: I don't work for HP. I'm just another happy HP Prime user.
03232018 12:40 AM
Hello,
As Joe explained, all non exact computation systems (which is any system that will yeild decimals) will suffer of this type of issues.
Depending on the specifics of the system implementation, the artefacts will be different. Joe pointed the (1/3)*3 in decimal. You pointed the ones in the CAS (48 bit binary floating point reals). Because we are used to the decimal system's artefacts, but not to the binary ones, we find the 2nd strange and the first ones normal, but this is only due to the fact that we are used to the first ones and not the 2nd ones. (by the way, try 1/3 shift enter *3 enter in cas and you get 1!)
Excel (and all PC systems) will suffer from similar ones. Try entering =10099.90.1 in excel and you will get: 5.6..E15
Some systems try to hide this from you by rounding things, It looks good at first sight, but causes issues later on when you have a result that is trully 'small' and that the system rounds it down to 0 by mistake!
Wellcome to the wonderfull world of "applied mathematics"!
Cyrille
Didn't find what you were looking for? Ask the community