l_fp, time, calendar
Gary E. Miller
gem at rellim.com
Mon Mar 27 02:46:31 UTC 2017
Yo Hal!
On Sun, 26 Mar 2017 19:04:22 -0700
Hal Murray <hmurray at megapathdsl.net> wrote:
> gem at rellim.com said:
> >> Isn't that as simple as convert a 64 bit integer to float and
> >> divide by 1<<32 to get the right scale?
> > Nope. Loss of precision. A double only has 52 bits of significant
> > digits. A float only 23. To fully encode a 64 bit int you need a
> > long double with (usually) 80 significant bits.
>
> Is there something I can say in python to get more precision? How
> widely is it supported?
You can use gmpy or numpy. Core Python is a bit sloppy on type precision.
Pulling in either once would annoy a lot of minimalists. OTOH there
are a lot of NUMPY fans.
RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
gem at rellim.com Tel:+1 541 382 8588
Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ntpsec.org/pipermail/devel/attachments/20170326/78c0f4f3/attachment.bin>
More information about the devel
mailing list