more careful float -> int range checks
After upgrading my emscripten toolchain I got a warning that
INT_MAX can't be represented with a float, so the comparison
if (fbits > INT_MAX)
may not be doing the comparison we want. In fact, it was not...
we want to exclude any fbits value that's equal to (float)INT_MAX,
since that's actually greater than INT_MAX.
So add explicit casts here to let that new toolchain know we know
what we're doing, and count INT_MAX among the infinite. INT_MIN
is a power of two so it's exactly representable as a float and that
comparison works as we were intending.
We do seem to hit these cases outside fuzzers, but I think only
when trying to fit curves that we'll fail to fit anyway.
Commit-Queue: Brian Osman <firstname.lastname@example.org>
Auto-Submit: Mike Klein <email@example.com>
Reviewed-by: Brian Osman <firstname.lastname@example.org>
1 file changed