check full round-tripped index for inf/overflow

We have been checking that our fractional table index is finite,
but not that the final (N-1)frac + 0.5 index is sensible.  Fix that.

Now we'll test that final roundtripped index is finite and in the range
of ints.  The tolerance check will handle the rest... we don't actually
look anything up with that index, so it's safe to be outside [0,N).

This didn't reproduce the exact error reported on oss-fuzz, but it
definitely reproduced _a_ problem.  Maybe they're not running full UBSAN
there?

Bug: oss-fuzz:8101
Change-Id: I7f3e7a4573c01b8065551f6ef8e064a51635da4a
Reviewed-on: https://skia-review.googlesource.com/124861
Auto-Submit: Mike Klein <mtklein@chromium.org>
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
4 files changed