Aaaaarrrggghh!

Hal Murray hmurray at megapathdsl.net
Tue Dec 29 02:08:23 UTC 2015


Converting fractions between decimal and binary is messy.  You may have to 
dump them in hex.


> After discovering this, I checked and found that the unit tests don't check
> ulfptoa() with a nonzero fractional part.

I poked around a bit and couldn't get anything like what you reported.

I got the hex fractional constants from dc.
16o 12k
3660323067.1125955647 p
DA2C18FB.1CD3101C0

I had troubles with 1444359386.1798776096
I could get either ...095 or ...097

4212665562.739245376 also has problems.  It's one digit shorter.

This passes:

        l_fp test5 = {{4212665562UL}, 0x3C6BE7E6};
        l_fp test6 = {{4212665562UL}, 0x36222683};
        l_fp test7 = {{4212665562UL}, 0xBD3F2F5A};
        l_fp test8a = {{1444359386UL}, 0x2E0C7582};
        l_fp test8b = {{1444359386UL}, 0x2E0C7583};
        l_fp test9 = {{3660323067UL}, 0x1CD3101C};

        TEST_ASSERT_EQUAL_STRING("4212665562.2360215127", ulfptoa(&test5, 10));
        TEST_ASSERT_EQUAL_STRING("4212665562.2114585943", ulfptoa(&test6, 10));
        TEST_ASSERT_EQUAL_STRING("4212665562.739245376", ulfptoa(&test7, 9));
        TEST_ASSERT_EQUAL_STRING("1444359386.1798776095", ulfptoa(&test8a, 10));
        TEST_ASSERT_EQUAL_STRING("1444359386.1798776097", ulfptoa(&test8b, 10));
        TEST_ASSERT_EQUAL_STRING("3660323067.1125955647", ulfptoa(&test9, 10));


-- 
These are my opinions.  I hate spam.





More information about the devel mailing list