Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sincos table under/overflow #498

Merged
merged 3 commits into from
Sep 10, 2024
Merged

Fix sincos table under/overflow #498

merged 3 commits into from
Sep 10, 2024

Conversation

solidpixel
Copy link
Contributor

@solidpixel solidpixel commented Aug 26, 2024

The current code computes a sincos table index based on the ideal weight for each texel, which is computed based on the current ideal endpoints. Ideal endpoints can be closer to the mean value than the principal axis projection for some texels and, for these texels, the ideal weights will be outside of the [0,1] range. The current code doesn't allow for this, which results in computed sincos table indices that end up wrapping inside the table bounds.

As part of fixing this I've replaced the float-to-int conversion that was implemented via float bit-twiddling with a straight round-to-nearest, which is faster on modern hardware.

Fixing this does change weight selection, and image PSNR changes are noisy in both directions, so regenerated test reference scores are required.

@solidpixel solidpixel added this to the 4.9.0 milestone Aug 26, 2024
@solidpixel solidpixel self-assigned this Sep 5, 2024
@solidpixel solidpixel merged commit f0e8a01 into main Sep 10, 2024
7 checks passed
@solidpixel solidpixel deleted the sincos_wrap branch September 10, 2024 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants