Results 1 to 3 of 3
  1. #1
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    15 precision Points (2k and 2k3)

    I have been playing around with this for a little while now and was just seeing if there was a work around. Let's look at these sums:

    11x11=121
    111x111=12321
    1111x1111=1234321

    ...

    111,111,111 x 111,111,111= 12345678987654321

    If we do this in Excel the sum totals 12345678987654300 due to the 15 precision points being that this number is 17 digits long. I found some very clumsy add-ins for this calculation on the web which used the method of converting the number to a text string. Any of you got a good idea for code that would do this? For some reason my little head states that I could do a binary conversion but I think I am going nutty <img src=/S/grin.gif border=0 alt=grin width=15 height=15>
    Jerry

  2. #2
    Plutonium Lounger
    Join Date
    Mar 2002
    Posts
    84,353
    Thanks
    0
    Thanked 29 Times in 29 Posts

    Re: 15 precision Points (2k and 2k3)

    Since Excel internally uses 15 significant digits, you must use some trick such as storing numbers as text if you want to work with extended precision numbers. You could write your own code for it, and it could be an educative excercise, but it would also be rather tedious. Add-ins such as xlPrecision (free version available) have already done the dirty work for you.

  3. #3
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: 15 precision Points (2k and 2k3)

    Thanks Hans

    Nice add-in, the other one I had, although clever just felt clumsy
    Jerry

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •