[omniORB] bug in CORBA::Fixed constructor

[OKeeffe, Michael K]

Alexander Haarer writes:
 
 
1.10*1.10 => 1.2100

suggests, that the precision of the result would be higher than the
precision
of the source values - but this is just wrong, because the relative
errors
of the source values sum up in a multiplication operation. 

That means the relative error of the result is always bigger. Following
the argumentation of the mentioned web link, this should result in less
significant digits, but not in more significant digits.

Personally, i dont like the style of enconding precision into digits,
because
it restricts to discrete steps of precision values. 

IMO, most other reasons in the web link are only valid if user
presentation
requirements and internal data representation requirements are mixed up.
(which IMO is bad idea)

 
Alexander,
 
Yes, that's a good point.  I see I wasn't addressing Luke's specific
example, more the basic question of where trailing zeros are useful.
 
I think the link below (and example adding 1 and 0.001) explains their
reasoning: the idea is to treat the calculation as exact:
 
"Given that an addition operator does not know whether a number is a
measurement or an exact quantity then it has no option other than to
return the exact answer (the fourth case). How this result is
interpreted is up to the application, and that application is at liberty
to apply some rounding rule or to treat the result as exact, as
appropriate. 

So, default operations must treat calculations as exact. " 

There's the reasoning, but I'd agree with your point, that if it is
exact, I should have to specify that, i.e. "1.000", or "1".setExact().
It seems they have it defaulting the wrong way - another example is in
Java you have the option of:

decimalVal1.multiply(decimalVal2).stripTrailingZeros();

Where I should have to specify "keepTrailingZeros()" instead.

http://www2.hursley.ibm.com/decimal/decifaq4.html#signif
<http://www2.hursley.ibm.com/decimal/decifaq4.html#signif>  
 
Well, I will say it's not perfect, but better than it was, see Kahan's
page here for some funny tricks Excel, Java, Matlab, and even compilers
can play:
http://www.cs.berkeley.edu/~wkahan/
 
 
 
 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20070108/3ce9a1a3/attachment.htm