|
Scientific Calculator Math Center 2 Precision Question
|
|
07-17-2010, 06:07 PM
Post: #1
|
|||
|
|||
|
Scientific Calculator Math Center 2 Precision Question
Using Scientific Calculator Math Center Level 2, I sometimes get
results that differ from the exact results by more than 10^(-15). Isn't the precision of Scientific calculator 15 digits? |
|||
|
07-19-2010, 02:17 AM
(This post was last modified: 07-19-2010 02:18 AM by admin.)
Post: #2
|
|||
|
|||
|
RE: Scientific Calculator Math Center 2 Precision Question
When math software (including hand-held calculators) developers declare that precision of calculations is n digits, that means precision of 'standalone' arithmetic operations and standard functions. Most of hand-held calculators do just standalone calculations. Even a brief sequence of operations can lead to rounding error accumulation. Consider (1/3)*3. The exact result is 1. But in 5 digits calculator (1/3)*3=(3.3333E-1)*3=9.9999E-1. The most of calculators apply some guessing techniques which catch such situations and give exact numbers. In a long sequences of operations rounding errors can accumulate. For example, 1.0001 - (4.0000E-5)- (4.0000E-5) - (4.0000E-5) = 1.0001 - (4.0000E-5) - (4.0000E-5) = 1.0001 - (4.0000E-5) = 1.0001 and -(4.0000E-5) - (4.0000E-5) - (4.0000E-5) + 1.0001 = -(8.0000E-5) - (4.0000E-5) + 1.0001 = -(1.2000E-4) + 1.0001 = 1.0000 . It is supposed that rounding errors cancel each other in long sequences. It would be in purely stochastic (random) world. But in our reality patterns occur. Some very long sequences get very big rounding errors. Since functions are calculated with help of Taylor series, which are long sequences of arithmetic calculations, rounding errors are a serious problem for math software developers. In order to get desired precision for standard functions, developers use greater precision for arithmetic operations. Usually such precision is hidden for users and is named 'internal precision'.
|
|||
|
04-03-2011, 08:07 PM
(This post was last modified: 04-03-2011 08:56 PM by jeux flash gaspard.)
Post: #3
|
|||
|
|||
| de même qu jeu gratuit pourrez y effectuer | |||
|
04-04-2011, 02:11 AM
Post: #4
|
|||
|
|||
|
RE: Scientific Calculator Math Center 2 Precision Question
jeux flash, I appreciate your effort to promote your website here, but could you place your messege on French sub-forum?
|
|||
|
« Next Oldest | Next Newest »
|

Search
Member List
Calendar
Help


