Skip to content

perf: skip redundant lookahead simulations in expression dispatch - #2827

Open
fudianchn wants to merge 2 commits into
JSQLParser:masterfrom
fudianchn:perf/condition-lookahead
Open

fudianchn wants to merge 2 commits into
JSQLParser:masterfrom
fudianchn:perf/condition-lookahead

Conversation

@fudianchn

@fudianchn fudianchn commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.

What

Skip redundant lookahead simulations on hot expression parse paths: the deep jj_2 phase simulation guarding the main branch of Condition() / SQLCondition(), and the Oracle/PostgresNamedFunctionParameter probes in ComplexExpressionList().

Also replace five complete single-token probes with exact token predicates: PRIOR, SQLCondition's OVERLAPS, time-key expressions, ALL, and boolean values.

Why

The main branch of Condition() / SQLCondition() is the last alternative of its choice, so its lookahead can only accept or reject what the productions themselves accept or reject immediately after: entering it directly and letting them fail is equivalent, without re-walking the whole expression tree first. Under JFR profiling of parseSQLStatements this simulation collected about 17% of the execution samples, paid by every plain condition on top of the real parse.

The named-parameter probes can never match a dotted name (f(a.b => 1) fails to parse on master with or without this change), so a check of the second token decides them exactly like the 2-token scan did.

How

  • Condition() / SQLCondition(): gate the main alternative with LOOKAHEAD({ !interrupted }) instead of the implicit deep simulation; the EXISTS alternatives ahead of it are unchanged.
  • ComplexExpressionList(): gate the four named-parameter alternatives with isNamedParameterAssignmentAhead(), a two-kind check on getToken(2). Keep the syntactic probes that validate the name.
  • Replace the five complete single-token syntactic probes with token predicates, retaining their existing interruption checks. Leave the multi-token disambiguation for Oracle (+), named-parameter names and ANY comparisons in place.

Root cause

Not applicable (performance).

Testing

  • Remote Linux verification (Core i9-13900KS, JDK 17.0.20): check jmhJar --no-daemon --max-workers=4, with a per-run repository mirror init script, passed on both the PR commit and master bdf8ac31.
  • PR: 10,404 tests discovered, 10,379 passed, 25 skipped, zero failures/errors. Master: 10,352 discovered, 10,327 passed, the same 25 skipped, zero failures/errors. Existing test sources and per-suite counts are unchanged; 52 cases were added. The earlier Mockito initialization and file-access failures did not reproduce.
  • The previously unverified WithItemTest.testInterruptedWithItemLookaheadStopsReading and CCJSqlParserUtilTest.testParserInterruptedByTimeout both passed remotely; ParserExecutionTest passed all 7 cases. The focused run completed 129 cases with 2 existing disabled cases and zero failures/errors.
  • LookaheadDispatchTest: 52 cases passed, checking AST kinds and operands, EOF consumption, round trips, quoted/non-reserved names, first/later named arguments, invalid names and malformed conditions. The two already-interrupted numeric-fragment cases document the !interrupted gate: master accepts the fragment, the PR rejects it before consumption, and removing the gate makes both tests fail.

Remote Linux four-version JMH comparison, run through ssh dev on a Core i9-13900KS with JDK 17.0.20. Each row uses its actual source tree; the two PRs are measured independently. The unchanged 54-statement performance.sql has Git blob a8c043ad99bab78597a118eacbd679e814fc1b06, and one operation parses the complete corpus with SIMPLE / withAllowComplexParsing(false).

All four states use the same frozen, unmodified project JSQLParserBenchmark.parseSQLStatements harness from master bdf8ac31, with the same isolated DynamicParserRunner class-loader path and locally built library JARs. Each window has 3 forks, 3 × 3 s warmup and 10 × 1 s measurement per fork, one thread, and -Xms1g -Xmx1g -Xss4m -Dfile.encoding=UTF-8. There are 30 samples per window and 60 per state. Execution order is 5.4 → master → #2827 → #2828, then the reverse; no Gradle build ran concurrently. Errors below are JMH 99.9% confidence half-widths for each score.

Source state Commit Window 1 (ms/op) Window 2 (ms/op)
Release 5.4 e847e94bb04671b5483fbe7814d7e9802e58bccc 21.678 ± 0.319 21.709 ± 0.237
master bdf8ac31bb17b2ae76400a8ddd22a39a990dcc89 22.493 ± 0.318 22.332 ± 0.269
This PR (#2827) 68c035bc058d5236ec8d30ca7909816a0a56e091 19.227 ± 0.353 19.317 ± 0.331
#2828, independent comparison 0f911fa7bf5dc1a80159eb46a9648e5abd748768 22.736 ± 0.316 22.649 ± 0.277
Latency change Window 1 Window 2 Score intervals overlap
master vs 5.4 +3.76% +2.87% No / No
This PR vs master -14.52% -13.50% No / No
This PR vs 5.4 -11.31% -11.02% No / No
#2828 vs master +1.08% +1.42% Yes / Yes
#2828 vs 5.4 +4.88% +4.33% No / No

Percentages are changes in score point estimates; the reported intervals belong to the individual scores, not the ratios. This PR consistently reduces latency in this corpus. #2828's intervals overlap master's in both windows, so these measurements do not establish a speedup for #2828. The master-to-5.4 difference measured here is 2.87–3.76%.

Comparability check: all four states parse 54 non-null statements and produce identical deparsed SQL (SHA-256 913caa84553bf65056593694fe90fa73765093d599782dc8e80106dbccd5dc12). The three snapshot states also produce identical semantic-field projections and serialized ASTs. Release 5.4 has an older model schema: shared field values agree, while snapshots add StringValue.escaping=STANDARD and filterWhereKeyword=true metadata; other extra fields are default/null. Release-to-snapshot timing therefore includes that model evolution. Every SQL sample was retained. No JavaCC choice conflicts were reported.

Malformed conditions can fail inside the productions instead of at choice dispatch; their failure class remains unchanged in the tested cases.

Verification of the original issue

Not applicable (no issue).

…expression lists

The main branch of Condition/SQLCondition is the last alternative of its
choice: entering it and letting the productions reject is equivalent to the
deep jj_2 phase simulation that ran for every plain condition first, so gate
it with a cheap !interrupted check instead. The named-parameter alternatives
of ComplexExpressionList only ever match a single-token name directly
followed by their assignment token, so a first check on the second token is
equivalent to their 2-token lookahead scan.

parseSQLStatements: 21.3-21.4 vs 24.6-24.8 ms/op against master in 3
interleaved JMH pairs (-13% to -14%, error intervals not overlapping);
behavior verified identical by 26 EXISTS/named-parameter probes, the full
10,344-test suite (same 35 pre-existing environment failures as master) and
a byte-identical AST dump of performance.sql.

Signed-off-by: 付典 <fudianchn@gmail.com>
@manticore-projects

Copy link
Copy Markdown
Contributor

This is a good one and to be honest: most of the Lookaheads where set arbitrarily when the Parser Generator requested for it. Nothing of that really went through a clean design ever. I would not be surprised when there are many obsolete Lookaheads left.

Does anyone have a good idea how to clean those up?

Use exact token predicates for PRIOR, OVERLAPS, time keywords, ALL and boolean literals while retaining interruption checks. Add 52 boundary cases for expression dispatch, named arguments, malformed input and cancellation.

Signed-off-by: 付典 <fudianchn@gmail.com>
@fudianchn fudianchn changed the title perf: skip redundant lookahead simulations in Condition dispatch and expression lists perf: skip redundant lookahead simulations in expression dispatch Oct 16, 2026
@fudianchn
fudianchn marked this pull request as ready for review October 16, 2026 02:14
@fudianchn

Copy link
Copy Markdown
Contributor Author

@manticore-projects The first phase looks good based on the tests and benchmark results so far. I'm now looking into the broader lookahead cleanup you raised and plan to handle that in a second phase.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants