Use FastDoubleParser where appropriate #9150

headius · 2025-12-28T05:59:14Z

This hooks up the FastDoubleParser project to our internal float parsing logic, excluding cases that are not 7-bit ASCII or which contain underscore characters (not currently allowed by FDP, see wrandelshofer/FastDoubleParser#85 for an attempt to add that feature).

FastDoubleParser is the Java implementation by @wrandelshofer of Daniel Lemire's fast float parsing algorithm. See the project page here: https://github.com/wrandelshofer/FastDoubleParser

This does not yet pass all float-parsing specs, primarily because it does not reject some numeric forms that Ruby's current parser rejects.

See ruby/ruby#15655 for a similar effort to add the Eisel-Lemire algorithm variant to CRuby, described in a blog post by @mensfeld here: https://mensfeld.pl/2025/12/ruby-string-to-float-optimization/.

Benchmarks are significantly faster than the previous implementation in JRuby and much faster than CRuby 4.0 (without @mensfeld's improvements).

BEFORE:

Simple decimals (1.5, 3.14)         1.087s
Prices (9.99, 19.95)                0.718s
Small integers (5, 42)              0.558s
Math constants (Pi, E)              1.888s
High precision decimals             1.474s
Scientific (1e5, 2e10)              0.850s
Simple decimals (1.5, 3.14)         0.656s
Prices (9.99, 19.95)                0.763s
Small integers (5, 42)              0.649s
Math constants (Pi, E)              1.497s
High precision decimals             1.348s
Scientific (1e5, 2e10)              0.774s
Simple decimals (1.5, 3.14)         0.744s
Prices (9.99, 19.95)                0.721s
Small integers (5, 42)              0.514s
Math constants (Pi, E)              1.403s
High precision decimals             1.223s
Scientific (1e5, 2e10)              0.725s
Simple decimals (1.5, 3.14)         0.731s
Prices (9.99, 19.95)                0.779s
Small integers (5, 42)              0.535s
Math constants (Pi, E)              1.699s
High precision decimals             1.583s
Scientific (1e5, 2e10)              0.815s
Simple decimals (1.5, 3.14)         0.832s
Prices (9.99, 19.95)                0.832s
Small integers (5, 42)              0.485s
Math constants (Pi, E)              1.486s
High precision decimals             1.396s
Scientific (1e5, 2e10)              0.780s

AFTER:

Simple decimals (1.5, 3.14)         0.107s
Prices (9.99, 19.95)                0.077s
Small integers (5, 42)              0.062s
Math constants (Pi, E)              0.112s
High precision decimals             0.089s
Scientific (1e5, 2e10)              0.075s
Simple decimals (1.5, 3.14)         0.070s
Prices (9.99, 19.95)                0.071s
Small integers (5, 42)              0.055s
Math constants (Pi, E)              0.093s
High precision decimals             0.093s
Scientific (1e5, 2e10)              0.069s
Simple decimals (1.5, 3.14)         0.069s
Prices (9.99, 19.95)                0.074s
Small integers (5, 42)              0.058s
Math constants (Pi, E)              0.090s
High precision decimals             0.085s
Scientific (1e5, 2e10)              0.071s
Simple decimals (1.5, 3.14)         0.071s
Prices (9.99, 19.95)                0.072s
Small integers (5, 42)              0.058s
Math constants (Pi, E)              0.084s
High precision decimals             0.086s
Scientific (1e5, 2e10)              0.070s
Simple decimals (1.5, 3.14)         0.070s
Prices (9.99, 19.95)                0.071s
Small integers (5, 42)              0.056s
Math constants (Pi, E)              0.087s
High precision decimals             0.089s
Scientific (1e5, 2e10)              0.069s

headius · 2025-12-28T06:00:31Z

CRuby 4.0 numbers for comparison:

Simple decimals (1.5, 3.14)         0.111s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.104s
Math constants (Pi, E)              0.615s
High precision decimals             0.472s
Scientific (1e5, 2e10)              0.109s
Simple decimals (1.5, 3.14)         0.109s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.104s
Math constants (Pi, E)              0.607s
High precision decimals             0.473s
Scientific (1e5, 2e10)              0.111s
Simple decimals (1.5, 3.14)         0.109s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.105s
Math constants (Pi, E)              0.624s
High precision decimals             0.464s
Scientific (1e5, 2e10)              0.110s
Simple decimals (1.5, 3.14)         0.111s
Prices (9.99, 19.95)                0.113s
Small integers (5, 42)              0.112s
Math constants (Pi, E)              0.681s
High precision decimals             0.523s
Scientific (1e5, 2e10)              0.122s
Simple decimals (1.5, 3.14)         0.123s
Prices (9.99, 19.95)                0.124s
Small integers (5, 42)              0.118s
Math constants (Pi, E)              0.676s
High precision decimals             0.534s
Scientific (1e5, 2e10)              0.123s

lemire · 2026-01-01T19:16:09Z

@headius So faster than CRuby ?

headius · 2026-01-02T05:52:59Z

@lemire Yes, faster than CRuby but without your algorithm from @mensfeld's PR. I will try to compare that soon. JVM having no native 128-bit primitive type limits what it can do somewhat.

wrandelshofer · 2026-01-02T12:10:26Z

Have you tried, if the performance improves even more, when JavaDoubleParser accesses the byte array contained in the BytesList?

Like this:

    public static double fastByteListToDouble(ByteList bytes) {
        return JavaDoubleParser.parseDouble(bytes.getUnsafeBytes(), bytes.getBegin(), bytes.getRealSize());
    }

wrandelshofer · 2026-01-02T12:25:23Z

I think, you can get some speed improvement and pass all tests in JRuby, if you revert the changes in class ConvertDouble, and only change the last line of method completeCalculation() from this:

        private double completeCalculation() {
            ...

            return Double.valueOf(new String(chars, 0, charsIndex));
        }

to this:

        private double completeCalculation() {
            ...

            return JavaDoubleParser.parseDouble(chars, 0, charsIndex);
        }

This hooks up the Java implementation of Daniel Lemire's fast float parsing algorithm to our internal float parsing logic, excluding cases that are not 7-bit ASCII or which contain underscore characters (not currently allowed by FDP, see wrandelshofer/FastDoubleParser#85 for an attempt to add that feature).

headius · 2026-01-07T22:13:00Z

JavaDoubleParser accesses the byte array

It doesn't make much difference based on my measurements (ByteList implements CharSequence by just accessing the byte array).

revert the changes in class ConvertDouble, and only change the last line of method completeCalculation()

That does indeed avoid the failures, but isn't as fast as my patch.

My patch:

Simple decimals (1.5, 3.14)         0.062s
Prices (9.99, 19.95)                0.063s
Small integers (5, 42)              0.059s
Math constants (Pi, E)              0.076s
High precision decimals             0.081s
Scientific (1e5, 2e10)              0.063s

Your patch:

Simple decimals (1.5, 3.14)         0.072s
Prices (9.99, 19.95)                0.080s
Small integers (5, 42)              0.064s
Math constants (Pi, E)              0.124s
High precision decimals             0.127s
Scientific (1e5, 2e10)              0.082s

It's quite a bit better than the original, though!

No patch:

Simple decimals (1.5, 3.14)         0.148s
Prices (9.99, 19.95)                0.163s
Small integers (5, 42)              0.136s
Math constants (Pi, E)              0.414s
High precision decimals             0.337s
Scientific (1e5, 2e10)              0.153s

It's possible my patch is faster because it's not handling all those other forms that we need for Ruby support.

@wrandelshofer I really want to figure out how to use your library in JRuby but Ruby has so many oddities in float parsing. For just String#to_f, with the patch you provided in wrandelshofer/FastDoubleParser#85, the following failures remain:

parsing things like "45.6 degrees" by terminating parsing at the first unexpected character:

- treats leading characters of self as a floating point number (FAILED - 1)
- treats any non-numeric character other than '.', 'e' and '_' as terminals (FAILED - 3)
- takes an optional sign (FAILED - 4)
- treats a second 'e' as terminal (FAILED - 5)
- treats a second '.' as terminal (FAILED - 6)
- treats a '.' after an 'e' as terminal (FAILED - 7)
- treats non-printable ASCII characters as terminals (FAILED - 8)

parsing "NaN", "Infinity", and "-Infinity"

- treats special float value strings as characters (FAILED - 2)

It's pretty close at this point.

headius · 2026-01-07T22:23:43Z

treats special float value strings as characters (FAILED - 2)

The Ruby behavior is to treat strings like "NaN" as non parseable and return 0.0:

[] jruby $ cx 4.0.0 ruby -e 'p "NaN".to_f'
0.0
[] jruby $ cx 4.0.0 ruby -e 'p "Infinity".to_f'
0.0

I can configure this behavior:

.withInfinity(Collections.EMPTY_SET)
.withNaN(Collections.EMPTY_SET)

One down!

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

headius · 2026-01-07T22:42:45Z

The remaining failures are all due to FDP rejecting any trailing non-float characters and returning 0.0 for such cases. @wrandelshofer I don't see any way to configure this and I was not clear where in the code it decide to bail out for bad input.

[] jruby $ cx 4.0.0 ruby -e 'p "45.6 degrees".to_f'               
45.6
[] jruby $ jruby -e 'p "45.6 degrees".to_f'                               
0.0

wrandelshofer · 2026-01-08T17:02:15Z

It doesn't make much difference based on my measurements (ByteList implements CharSequence by just accessing the byte array).

The purpose of this change, is to get reliable performance of the parser regardless of the JIT.
The JIT is not always able to inline methods. For example, when the JVM is low on memory for byte code or when the code - for whatever reason - is not considered 'hot' enough for the JIT.

wrandelshofer · 2026-01-08T17:06:16Z

The remaining failures are all due to FDP rejecting any trailing non-float characters and returning 0.0 for such cases. @wrandelshofer I don't see any way to configure this and I was not clear where in the code it decide to bail out for bad input.

Yes. There is currently no API for this in the fast double parser library.
However, you should be able to achieve quite a good result without such an API, by using a "fast path/slow path" approach.
Please take a look at this pull request: headius#6

headius · 2026-01-08T18:43:11Z

@wrandelshofer Thank you!

headius force-pushed the fast_float_parsing branch from 3628c5d to a402496 Compare January 7, 2026 21:41

headius changed the base branch from master to 10.1-dev January 7, 2026 21:41

headius mentioned this pull request Jan 7, 2026

Skip underscores wrandelshofer/FastDoubleParser#85

Closed

Use configurable parser

14a3511

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

headius force-pushed the fast_float_parsing branch from 1470290 to 14a3511 Compare January 7, 2026 22:40

headius mentioned this pull request Jan 7, 2026

Implement fast float parsing and to_s #9162

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use FastDoubleParser where appropriate #9150

Use FastDoubleParser where appropriate #9150

Uh oh!

headius commented Dec 28, 2025

Uh oh!

headius commented Dec 28, 2025

Uh oh!

lemire commented Jan 1, 2026

Uh oh!

headius commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

headius commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Use FastDoubleParser where appropriate #9150

Are you sure you want to change the base?

Use FastDoubleParser where appropriate #9150

Uh oh!

Conversation

headius commented Dec 28, 2025

Uh oh!

headius commented Dec 28, 2025

Uh oh!

lemire commented Jan 1, 2026

Uh oh!

headius commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

headius commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants