Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new Decimal(2).pow(3) is 7.999999999999999 #173

Open
ghoqyew opened this issue Jul 25, 2024 · 6 comments
Open

new Decimal(2).pow(3) is 7.999999999999999 #173

ghoqyew opened this issue Jul 25, 2024 · 6 comments

Comments

@ghoqyew
Copy link

ghoqyew commented Jul 25, 2024

Math.pow(2,3) is 8.

@MathCookie17
Copy link
Collaborator

I'm not sure if this one is fixable. Floating point numbers are naturally going to be somewhat imprecise - it's like how 0.1 + 0.2 = 0.3000000000000001.
break_eternity implements pow via taking log10, multiplying, and pow10'ing back, which works for any size number - so that means that pow's imprecision comes from log's imprecision, and I think we all know at this point that log's imprecision isn't going anywhere. I don't know how the JavaScript compiler works, but I'd assume it implements Math.pow, at least for integer exponents, via something like exponentiation by squaring, which is more accurate since it preserves integer values, but that wouldn't work when the exponent is something huge like 1e1,000,000.
I guess we could, like mod, use Math.pow instead when it returns a finite value? But that doesn't really get rid of the issue, just kicks it down the road and delays its activation. I'll talk to Patashu about this.

@HexaVault
Copy link

wouldnt a valid fix be to just round the values if both the exponent and base are integers (and final val < 9e15). Not the cleanest solution, sure, but works for most necessary cases

@MathCookie17
Copy link
Collaborator

wouldnt a valid fix be to just round the values if both the exponent and base are integers (and final val < 9e15). Not the cleanest solution, sure, but works for most necessary cases

The main problem there is that then pow might not quite be an increasing function anymore (random example I found: Decimal.pow(403.0000000000001, 3) is 65450826.99999999, which is less than the rounded 403^3 = 65450827), which could cause problems with things like logarithms. I'd rather have slight imprecision than give up the property that the function is increasing.

@HexaVault
Copy link

that only ever occurs when log(n) = log(floor(n)) and n is a integer very slightly larger than floor(n), so that n != floor(n)

@MathCookie17
Copy link
Collaborator

n is a integer very slightly larger than floor(n), so that n != floor(n)

That doesn't make sense. floor(n) is always equal to n when n is an integer.

@HexaVault
Copy link

n is a integer very slightly larger than floor(n), so that n != floor(n)

That doesn't make sense. floor(n) is always equal to n when n is an integer.

correct. Your point was that my solution would break non-integers to which i pointed out only affected a very small, unsubstantial amount of numbers.

Additionally, my fix wouldnt even do that since you could test for being integers and less than 9e15 after the log not before

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants