You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I also suspect there's a fast generalizeable way to detect this from bits, using toRawBits() and bit twiddling.
I looked at Float Toy and the manipulations of this gist, but ran out of energy and enthusiasm for reasoning through the corner cases.
I suspect that detecting "out of bounds" values in the double precision bit field is likely easier than straight conversion.
Taking b as the number of bits in the floating point representation, the current solution requires O(2^b) of auxiliary storage for storing the precision intervals and O(2^b) operations to compare the floating point number against the precision intervals.
A "more optimized" solution would ideally require no table of precision intervals, and would run in some constant number of operations.
The text was updated successfully, but these errors were encountered:
popematt
changed the title
See if it's possible to improve performance of ieee754_float constraint
Come up with a more optimized solution for ieee754_float constraint
Oct 11, 2022
Originally posted by @jobarr-amzn in #213 (comment)
Taking
b
as the number of bits in the floating point representation, the current solution requiresO(2^b)
of auxiliary storage for storing the precision intervals andO(2^b)
operations to compare the floating point number against the precision intervals.A "more optimized" solution would ideally require no table of precision intervals, and would run in some constant number of operations.
The text was updated successfully, but these errors were encountered: