
×InformationFix Windows 10 Update Issues
Resolve Windows 10 related issues for your HP computers or printers by HP Windows 10 Support Center

×InformationFix Windows 10 Update Issues
Resolve Windows 10 related issues for your HP computers or printers by HP Windows 10 Support Center
 HP Community
 >
 Other Products
 >
 Calculators
 >
 HP Prime  Math error
 Mark Topic as New
 Mark Topic as Read
 Float this Topic for Current User
 Bookmark
 Subscribe
 Mute
 Printer Friendly Page
Create an account on the HP Community to personalize your profile and ask a question
Solved!
HP Prime  Math error
12042017 07:51 AM
When I put the equation 3/(1/3) = 9.00000000001
When I put the pentium FDIV bug test:
(1234/4321)*(4321/1234) = .9999999999
(Thank you Professor Niicely)
Why is this error happening and is there a fix?
Solved! Go to Solution.
12042017 08:54 AM
This is not an error.
This is the expected result when using the approximate numerical domain of the HOME environment.
In the exact CAS (Computer Algebraic System) environment, you will get the results I think you expect, i.e. 9 and 1.
All numerical calculators are approximate, however Some numerical calculators may *seem* to get the exact answers.
See my explanation here:
https://h30434.www3.hp.com/t5/Calculators/sinpinotequaling0/mp/6402852#M8604
_________________________________________________________
calculator enthusiast
12042017 09:30 AM
Further on the Pentium FDIV bug:
https://en.wikipedia.org/wiki/Pentium_FDIV_bug
Quote:
"The presence of the bug can be checked manually by performing the following calculation in any application that uses native floating point numbers, including the Windows Calculator or Microsoft Excel in Windows 95/98.
The correct value is:
4195835/3145727=1.333820449136241002
When converted to the hexadecimal value used by the processor, 4,195,835 = 0x4005FB and 3,145,727 = 0x2FFFFF. The '5' in 0x4005 triggers the fault in the FPU control logic. As a result, the value returned by a flawed Pentium processor is incorrect at or beyond four digits:
4195835/3145727=1.333739068902037589 "
The HP Prime gives the answer of
4195835/3145727=1.33382044914
which is 100% correct to the calculator's 12 digits of precision.
_________________________________________________________
calculator enthusiast
12042017 10:05 AM
Let's look at (1234/4321)*(4321/1234) a bit further:
1234/4321 = 0.285582041194168016662809534829900485998611432538764174959...
but rounded to the calculator's 12 digits = 0.285582041194
4321/1234 = 3.501620745542949756888168557536466774716369529983792544570...
but rounded to the calculator's 12 digits = 3.50162074554
Now 0.285582041194*3.50162074554 = 0.99999999999856927177476
rounded to the calculator's 12 digits = 0.999999999999
So the calculator is working perfectly within it's 12digit capability.
Perhaps it would be nice if the calculator had more digits, but how much is enough? and at what point does a calculator become a computer?
_________________________________________________________
calculator enthusiast
12042017 10:16 AM
Thank you for the logic and I understand that PI needs some approximation as it is irrational number.
I appricate that CAS provides the true result, So I am not concerned there is a "Chip" error.
I also see the results are true when run with a fixed number format.
I still need a bit more if you could?
As there is still some kind of rounding when the format is "Fixed"
(Also on the TI so they must stop extending and round the number somewhere)
Does anyone know where HP's program extend with "Standard" number formats?
20 places, 50.... (Or do I not really get it).
12052017 03:08 AM
Hi,
In HOME the Prime is 12 digits. That is the maximum number of digits it will retain for any number. If you use FIX, SCI or ENG notation, only the display is the number is affected, it will still retain all 12 digits. (The only functions that "loose" digits are e.g. "integer part", "fractional part", "round", "truncate" etc.).
However, you may read somewhere that "internally, functions are 15 digits". So what that means is that e.g. the function SIN will calculate to 15 digits, then round the result to 12 digits when it passes out of the function.
This "internally to 15 digits" does not extend outside individual functions,
for example consider a simple formula: SIN(num1)*COS(num2)
once the SIN and COS functions have completed, the two 12 digit results will be multiplied.
However, in CAS things are slightly different. For example it can handle almost infinite long integers (limited by available memory I believe).
For example in CAS do 69! and you'll get a very long number ( in HOME you'll get 1.71122452428ᴇ98 )
When in CAS it is a good idea to use integers, i.e. instead of 12.34 rather enter 1234/100.
For example in CAS, SIN(0.25*PI) gives 0.707106781187
but SIN(PI/4) gives the exact answer of 1/ROOT(2)
Hope this helps.
_________________________________________________________
calculator enthusiast
Didn't find what you were looking for? Ask the community