Skip to content

Add stable ranges and structured diagnostics#92

Merged
marcbachmann merged 19 commits intomarcbachmann:mainfrom
AntonPieper:feat/stable-ranges-structured-diagnostics
Mar 26, 2026
Merged

Add stable ranges and structured diagnostics#92
marcbachmann merged 19 commits intomarcbachmann:mainfrom
AntonPieper:feat/stable-ranges-structured-diagnostics

Conversation

@AntonPieper
Copy link
Copy Markdown
Contributor

@AntonPieper AntonPieper commented Mar 18, 2026

Summary

  • add stable start/end ranges to AST nodes, parse errors, and diagnostics
  • add structured diagnostics for parse, type, and evaluation errors
  • keep the public error constructors compatible with main by preserving cause as the third argument and moving structured metadata to the optional fourth argument
  • preserve call-site ranges for both sync and async custom function errors and cover that parity in tests
  • recover the major eval regressions introduced by stable ranges without applying unrelated optimizations
  • recover the remaining parser regression by separating stable AST fields from mutable metadata in the parser hot path
  • clean up the branch history and drop the experimental async shortcut commit

Closes #90

Testing

  • npm test (1031/1031)

Benchmark Rerun

Same-machine rerun on main (5763f7c) vs this branch head (165246e):

  • 4 full-suite runs on each side
  • 112 benchmark cases total
  • 95% bootstrapped confidence intervals over the repeated samples
  • alternating branch/main runs to reduce drift

Overall

  • parse geomean: +2.25% (1.0225x, CI +0.37% to +3.84%)
  • eval geomean: +2.47% (1.0247x, CI +0.96% to +4.20%)

Parse suite highlights

  • accessors: +2.53% (CI +1.18% to +3.68%)
  • arithmetic: +1.94% (CI +0.29% to +3.40%)
  • logic: +2.70% (CI +1.80% to +3.70%)
  • collections: +1.51% (CI -0.20% to +3.20%)
  • functions: +2.00% (CI -0.22% to +4.41%)
  • literals: +3.40% (CI -0.58% to +8.61%)
  • macros: +1.56% (CI -2.04% to +5.20%)

Notable cases

  • accessors / parse / Int access: +6.75% (CI +3.29% to +10.11%)
  • logic / parse / Logical Expression: +6.35% (CI +5.73% to +6.97%)
  • collections / parse / Empty Map Creation: +6.05% (CI +3.13% to +8.37%)
  • literals / parse / String concat with variables: -5.47% (CI -9.83% to -1.64%)
  • collections / parse / List of Maps Access: -2.23% (CI -3.91% to -0.77%)

Readiness Gate

This PR is now ready for review: the benchmark results are confidently within, and in aggregate better than, the ±1% threshold against main.

@AntonPieper AntonPieper marked this pull request as draft March 18, 2026 11:54
@AntonPieper
Copy link
Copy Markdown
Contributor Author

History has been cleaned up and force-pushed as signed commits while preserving the original commit timestamps.

Latest validation on the rewritten branch head ab3d9e8:

  • npm test passes (1031/1031)
  • same-machine benchmark rerun vs main (5763f7c), 4 full-suite runs per side, 95% bootstrapped CI
  • overall eval: +1.43% (1.014x, CI +0.92% to +1.95%)
  • overall parse: -1.94% (0.981x, CI -2.42% to -1.42%)

Keeping the PR in draft because the branch is not yet confidently within the ±1% benchmark gate overall: eval is better than main, but parse remains confidently slower.

@AntonPieper
Copy link
Copy Markdown
Contributor Author

Added commit 165246e (perf(parser): split stable ast fields from mutable metadata).

Validation on the pushed commit:

  • npm test: 1031/1031 passing
  • Benchmarks: 4 full-suite runs on this branch (165246e) vs 4 full-suite runs on same-machine main (5763f7c)
  • Method: alternating branch/main runs with 95% bootstrapped confidence intervals over repeated run samples

Results:

  • Parse geomean: +2.25% (ratio 1.0225x, 95% CI +0.37% to +3.84%)
  • Eval geomean: +2.47% (ratio 1.0247x, 95% CI +0.96% to +4.20%)

Selected parse suite results:

  • accessors: +2.53% (CI +1.18% to +3.68%)
  • arithmetic: +1.94% (CI +0.29% to +3.40%)
  • logic: +2.70% (CI +1.80% to +3.70%)
  • collections: +1.51% (CI -0.20% to +3.20%)
  • functions: +2.00% (CI -0.22% to +4.41%)
  • literals: +3.40% (CI -0.58% to +8.61%)
  • macros: +1.56% (CI -2.04% to +5.20%)

This clears the previous parse gate: the branch is now confidently within, and in aggregate better than, the ±1% threshold against main.

@AntonPieper AntonPieper marked this pull request as ready for review March 18, 2026 18:31
@marcbachmann
Copy link
Copy Markdown
Owner

Thanks for the PR and sorry to let you wait. Isolating changes would help in merging quicker 😉
I'll commit against this PR again, but I don't have too much time and after Easter I'm 2 weeks on vacation.
But will try to land it before. I'll comment on a few things as I want to know more some parts, I hope you'll have time to answer them soonish, but I think no input besides this is needed from you.

@AntonPieper
Copy link
Copy Markdown
Contributor Author

No problem, thanks for the feedback :) I really didn't want performance regressions in comparison against main, which is why I added the accuPush optimization - in hindsight, I might should've just accepted a slight regression (as it stores more data) and place that optimization in another PR 😅 I'm here if you have any comments 👍

Copy link
Copy Markdown
Owner

@marcbachmann marcbachmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some questions while I'm already iterating on a few things.
No need for you to go into the code at the moment.

Comment thread lib/errors.js Outdated
Comment thread lib/errors.js Outdated
Comment thread lib/errors.js Outdated
Comment thread lib/evaluator.js Outdated
Comment thread lib/operators.js Outdated
Comment thread lib/parser.js Outdated
Comment thread lib/parser.js
Comment thread lib/parser.js
Comment thread lib/parser.js Outdated
@marcbachmann marcbachmann force-pushed the feat/stable-ranges-structured-diagnostics branch from 8114722 to 613353e Compare March 25, 2026 23:50
…meter to define code, message, cause, node and range
@marcbachmann
Copy link
Copy Markdown
Owner

I hope you're happy with that simplified solution. Thanks for the PR.

@marcbachmann marcbachmann merged commit d600a93 into marcbachmann:main Mar 26, 2026
@marcbachmann
Copy link
Copy Markdown
Owner

🎉 This PR is included in version 7.6.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add stable source ranges and structured diagnostics

2 participants