Skip to content

Parser + lexer performance: consolidated 2–3× end-to-end speedup #378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
JanJakes wants to merge 30 commits into
base: trunk
Choose a base branch
from
Draft

Parser + lexer performance: consolidated 2–3× end-to-end speedup #378

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5a3f211
Inline terminal matching and defer parse node allocation
JanJakes Apr 24, 2026
cdc9713
Use per-branch FIRST sets to skip unreachable branches
JanJakes Apr 24, 2026
332c10a
Short-circuit nullable-fallback and inline single-branch fragments
JanJakes Apr 24, 2026
402554c
Strip epsilon markers and cache grammar refs on the parser
JanJakes Apr 24, 2026
c43425f
Return fragment results as children arrays, skip the intermediate node
JanJakes Apr 24, 2026
e35e14f
Append end-of-input sentinel token to drop range checks
JanJakes Apr 24, 2026
a3b8a08
Embed branch symbol sequences directly in the per-token selector
JanJakes Apr 24, 2026
25dad62
Compare selectStatement by rule id instead of by name
JanJakes Apr 24, 2026
e4a0951
Re-align grammar and parser whitespace after recent changes
JanJakes Apr 24, 2026
0e23320
Deduplicate selector entries while embedding branch sequences
JanJakes Apr 24, 2026
3164da9
Add direct-return fast path for single-candidate rules
JanJakes Apr 24, 2026
45c344a
Speed up the lexer with cheaper byte checks
JanJakes Apr 28, 2026
a763087
Skip parent constructor in WP_MySQL_Token
JanJakes Apr 28, 2026
60981f6
Use ! empty() in WP_Parser_Node::has_child()
JanJakes Apr 28, 2026
1aa326a
Inline leading-whitespace skip in lexer's token loops
JanJakes Apr 29, 2026
7256e62
Catch identifier and keyword tokens at the top of the chain
JanJakes Apr 29, 2026
ee75d4b
Add a single-byte operator dispatch table
JanJakes Apr 29, 2026
e36e2d2
Document non-obvious lexer dispatch conditions
JanJakes May 4, 2026
3867de9
Unroll whitespace check in '--' line-comment dispatch
JanJakes May 4, 2026
0e91429
Extract WP_Parser::set_tokens() helper for shared sentinel handling
JanJakes May 4, 2026
58d84c6
Update native parser to consume branches_for_token + nullable_branches
JanJakes May 4, 2026
b330676
Materialize native AST eagerly into plain WP_Parser_Node instances
JanJakes May 4, 2026
ec5a74e
Mark WP_Parser_Node as final
JanJakes Apr 24, 2026
2e20dc5
Document parent-ctor bypass and remaining_tokens duplication
JanJakes Jun 3, 2026
849c596
Add unit tests for grammar build-time transforms
JanJakes Jun 3, 2026
00a502a
Refresh native-extension benchmark numbers
JanJakes Jun 4, 2026
096a44e
Fall back to pure PHP when the native extension grammar ABI mismatches
JanJakes Jun 4, 2026
0679908
Speed up grammar construction with a worklist fixpoint and lazy selec…
JanJakes Jun 5, 2026
7368e34
Unify unit-test CI into one matrix and disable Xdebug
JanJakes Jun 5, 2026
d3716d3
Cache the Rust extension build in CI
JanJakes Jun 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 0 additions & 189 deletions .github/workflows/mysql-parser-extension-tests.yml
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions .github/workflows/mysql-proxy-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
uses: shivammathur/setup-php@v2
with:
php-version: '7.4'
coverage: none

- name: Install Composer dependencies
uses: ramsey/composer-install@v3
Expand Down
113 changes: 0 additions & 113 deletions .github/workflows/phpunit-tests-run.yml
View file
Open in desktop

This file was deleted.

Loading
Loading