Showing results for 
Search instead for 
Did you mean: 
New member
1 0 0 0
Message 1 of 3
Flag Post

Rounding too early

HP Recommended
HP prime graphic calculator

Hi, I am a highschool student using this graphing calculator for my geometry class. This calculator is rounding numbers too soon in a string of numbers and that is causing problems. For example, when I tried to find the volume of a cone with a radius of 14 and a height of 14.3527000944, the calculator automatically rounded the 14.3527000944 to a 14.3527001 giving me the wrong answer and costing me valuable test points. Please help me to find a way to make the calculator more accurate. I tried using the floating setting for numbers like my geometry teacher recommended but that just rounded the answer up to 5 decimal places instead of 8 decimal places, and left the 14.3527001 unaffected.

0 Kudos
Level 8
622 609 117 207
Message 2 of 3
Flag Post
HP Recommended

Suggestion: set the Number Format (in Home Settings) to Standard, which shows ALL the internal digits.  That way you can see exactly what the value in the calculator really is (in Home view, of course).  You used Floating 8 mode, which shows at most 9 significant digits.  Rounded 8 mode and Fixed 8 mode show at most 8 digits after the decimal point.  But all these modes hide digits which are actually there, which is often a Bad Thing when doing real math.  To see all the digits, use Standard mode, whose philosophy is "What You See Is Exactly What You Have".


Disclaimer: I don't work for HP. I'm just a high school math teacher who has enjoyed using HP calculators since 1976.

0 Kudos
Level 8
Level 8
747 744 121 187
Message 3 of 3
Flag Post
HP Recommended

Note that no digits are acutally "lost" at all. That is purely a display setting and could not have impacted your final result.


How did you "lose points"? It wouldn't have been from rounding unless you teacher was grading on 12 digits or something????


Please put more details so you can get the help you want. What did you enter? What did the result say? What was your process?


Although I work for the HP calculator group as a head developer of the HP Prime, the views and opinions I post here are my own.
0 Kudos
† The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation