Re: [S] Behavior of trunc(x)

Alan Zaslavsky (zaslavsk@hcp.med.harvard.edu)
Wed, 23 Sep 1998 05:59:31 -0400 (EDT)


On Tue, 22 Sep 1998, John wrote:
> This may be an old one that I've missed but can anyone explain the behavior
> illustrated below?
>
<snip>
> # Exhibit 2: trunc those integers ??
> > for(x in (10:19)/10)print(trunc(10*(x-trunc(x))))
> [1] 0
> [1] 1
> [1] 1
> [1] 3
> [1] 3
> [1] 5
> [1] 6
> [1] 7
> [1] 8
> [1] 9

This is a variant of an FAQ. THe problem is inherent in the way that
numbers are stored in floating point arithmetic. Because they are
represented as an integer times a power of 2, only binary fractions (those
with a terminating representation in base 2) have an exact representation.
So for example 1/2 or 7/16 can be represented exactly, but 1/10 cannot.
Consequently calculations with 1/10, 2/10 etc. always will involve a
little roundoff error. Your calculation is designed to be very sensitive
to that roundoff error since trunc() rounds down no matter how close you
are to the next integer.

As a practical matter, if you have a calculation that you know should give
an integer (if performed exactly), you should always round rather than
truncating to allow for this error.

This is the same reason that numerical comparisons by exact equality don't
always work (except for integers), e.g.

n*(1/n)-1 == 0

will often not be T, but

abs(n*(1/n)-1) < 1e-8

will be true for modest values of n (not too close to 0).

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news