RE: [S] reasonable p-values for Fisher exact's test - WAS strange ...

Traub, Richard J (Richard.Traub@pnl.gov)
Wed, 25 Mar 1998 08:10:15 -0800


Patrick Connoly asked:

>Is there a fundamental difference in the way different machines
> do such calculations?

when he compared the results from a sun machine and an intel based
machine.

The reason for the differences might be due to the numeric processor.

If we remember back when Intel came out with the 8087 math coprocessor
they made a (somewhat) big deal out of the fact that the 8087 used
80 bit registers for the arithmetic. If that has been carried forward
to
the pentiums then the pentium will also do 80 bit arithmetic as long
as the numbers stay in the registers. (Depending on how the numbers
are declared in our program they will be saved
in memory as single or double precision, however).

Other machines might not have a numeric processor that does
arithmetic in 80 bit precision. They might use 64 bit, for example.

Hope this helps explain some differences.

Rick Traub
Pacific Northwest Natl Lab.

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news