Date:         Mon, 25 Oct 2004 10:01:50 -0700
Reply-To:     "Nordlund, Dan" <NordlDJ@DSHS.WA.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
Comments:     To: Richard Ristow <wrristow@MINDSPRING.COM>
From:         "Nordlund, Dan" <NordlDJ@DSHS.WA.GOV>
Subject:      Re: Quiz: Numerical Precision
Comments: To: SAS-L@LISTSERV.VT.EDU
Content-Type: text/plain

In my previous post I didn't clarify one inaccuracy in this discussion of
numerical precision in SAS.  With a length of 3 bytes, there are only 12
bits available, not 13, for representing the mantissa: 1 sign-bit, 11 for
the exponent, 12 for the mantissa.  The reason you can get to 8191 in 12
bits is because of the normalization process.  As Richard pointed out, any
power of two can be represented exactly (up to the maximum).

Hope this helps,

Dan

Daniel J. Nordlund
Research and Data Analysis
Washington State Department of Social and Health Services
Olympia, WA  98504-5204


-----Original Message-----
From: Richard Ristow [mailto:wrristow@MINDSPRING.COM]
Sent: Monday, October 25, 2004 10:17 AM
To: SAS-L@VM.MARIST.EDU
Subject: Re: Quiz: Numerical Precision

At 03:17 AM 10/25/2004, ben.powell@CLA.CO.UK wrote:

>With three byte precision there are 13 bits available to describe the
>mantissa. But the largest number you can describe with 13 bits is 8191
>and not 8192, hence how can 3 byte integers display 8192?

I think it's a lot easier than you're making it. With 13 bits, you can
indeed represent from 0 through 2**13-1, i.e. 8,191. Add one to this,
and the 13 bits overflow; in (binary) floating point a new high bit is
added, and the old low bit is indeed lost. So, you can no longer
represent ODD integers. But the new integer, 8192, is an EVEN integer,
and is representable. It's the next one, 8193, that isn't.

The SAS precision table, like many such tables, is labeled wrong:

>>> From SAS online doc:
>>>
>>>Length...........Largest................Exponential.
>>>in...............Integer................Notation....
>>>Bytes............Represented........................
>>>.................Exactly............................
>>>....................................................
>>>3................................8,192  2**13 ......
>>>4............................2,097,152  2**21 ......
>>>5..........................536,870,912  2**29 ......
>>>6......................137,438,953,472  2**37 ......
>>>7...................35,184,372,088,832  2**45 ......
>>>8................9,007,199,254,740,992  2**53 ......

The "Largest integer represented exactly" is always the largest number
representable in the system -- it is an integer, right? The correct,
but clumsy, label for that column is, "Largest integer for which all
integers from 0 through that value can be represented exactly."

For all precisions, you have the "overflow to even" phenomenon that
gives you one more integer than you'd expect; that's why the upper
bounds have, correctly, the form 2**n rather than the more obvious
2**n-1.