One-third

Here is an example in which a value rounds up in one type and down in another. Dividing 1 by 3 yields the binary result , where the bar indicates a repeating bit pattern. To round to the 24 significant bits of the float format, express the intermediate result in the normalized form , where the braces indicate bits beyond float precision. rounds up to when rounding to nearest. In double, the intermediate value is , which rounds down to . The situation is different because double has an odd number (53) of significant bits, versus an even 24 for float.


[Contents] [Previous] [Next]
Click the icon to mail questions or corrections about this material to Taligent personnel.
Copyright©1995 Taligent,Inc. All rights reserved.

Generated with WebMaker