Skip to content

perf: lazy IR loading#14145

Draft
Kha wants to merge 5 commits into
masterfrom
lazy-ir-loading
Draft

perf: lazy IR loading#14145
Kha wants to merge 5 commits into
masterfrom
lazy-ir-loading

Conversation

@Kha

@Kha Kha commented Jun 22, 2026

Copy link
Copy Markdown
Member

No description provided.

Kha and others added 3 commits June 22, 2026 09:24
This PR threads `BaseIO` through `getModuleIREntries` and the interpreter/codegen lookups that consume imported IR, in preparation for loading imported IR on demand. There is no behavior change yet: the lookups still read the eagerly-imported state.

`getModuleIREntries`, `findInterpDecl`/`findInterpDeclBoxed`, `findEnvDecl`/`findEnvDecl'`, `getSorryDep`, and `getRegularInitAttrModIdxs` become `BaseIO`; `findExtEntry?` is split so the non-IR LCNF lookups stay pure and a new `findIRExtEntry?` carries the IR branch. The C-export ABI is unchanged: `BaseIO` compiles to a bare value-return (no world argument, no error wrapper), so `ir_interpreter.cpp` needs no changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This PR makes `[inherit_doc]` copy the resolved docstring when a public declaration inherits one from a private declaration, instead of storing an alias to the private name. An importer cannot resolve such an alias because the private target is not exported and has no module index, so the inherited docstring would be lost across modules. Since the source is available where the attribute is applied, its docstring is copied eagerly; all other cases keep the existing alias behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This PR loads a module's interpreter IR (`.ir`) lazily, on first use, instead of eagerly populating the IR extension state for every imported module at import time. Modules whose interpreter IR is already part of the imported `.olean` data are read as before; module-system modules with a separate `.ir` are memory-mapped and cached on first access.

`Kernel.Environment`'s `irBaseExts : Array EnvExtensionState` field becomes `irData : Option LazyIR`, holding the per-module `.ir` paths and a mutable `IO.Ref` cache populated via `modifyGet`. `getModuleIREntries` becomes `BaseIO`; the C-export ABI is unchanged because `BaseIO` compiles to a bare value return. A lazy index maps code-generator auxiliary constants discovered while loading `.ir` files, consulted by the interpreter and codegen lookups when `getModuleIdxFor?` misses. `const2ModIdx` auxiliaries are sourced from the already-loaded `.olean` data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Kha

Kha commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

!bench

@leanprover-radar

leanprover-radar commented Jun 22, 2026

Copy link
Copy Markdown

Benchmark results for ecebaec against bc5f89f are in. There are significant results. @Kha

  • 🟥 build//instructions: +128.3G (+1.13%)

Large changes (25✅, 30🟥)

Too many entries to display here. View the full report on radar instead.

Medium changes (2✅, 25🟥)

Too many entries to display here. View the full report on radar instead.

Small changes (21✅, 1880🟥)

Too many entries to display here. View the full report on radar instead.

@github-actions github-actions Bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Jun 22, 2026
@leanprover-bot

Copy link
Copy Markdown
Collaborator

Reference manual CI status:

  • ❗ Reference manual CI can not be attempted yet, as the nightly-testing-2026-06-19 tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto nightly-with-manual, reference manual CI should run now. You can force reference manual CI using the force-manual-ci label. (2026-06-22 14:57:21)

@github-actions github-actions Bot added the mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN label Jun 22, 2026
@mathlib-lean-pr-testing mathlib-lean-pr-testing Bot added the breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan label Jun 22, 2026
@mathlib-lean-pr-testing

mathlib-lean-pr-testing Bot commented Jun 22, 2026

Copy link
Copy Markdown

Mathlib CI status (docs):

@hargoniX

Copy link
Copy Markdown
Contributor

!bench

@leanprover-radar

leanprover-radar commented Jun 22, 2026

Copy link
Copy Markdown

Benchmark results for 46ccc24 against bc5f89f are in. There are significant results. @hargoniX

  • 🟥 build//instructions: +53.0G (+0.47%)

Large changes (26✅, 5🟥)

Too many entries to display here. View the full report on radar instead.

Medium changes (2✅, 17🟥)

  • compiled/incr_header_load//instructions: -107.0M (-17.20%)
  • 🟥 elab/big_beq//instructions: +116.2M (+1.33%)
  • 🟥 elab/big_deceq//instructions: +113.7M (+2.78%)
  • 🟥 elab/big_deceq_rec//instructions: +113.9M (+2.06%)
  • 🟥 elab/big_match//instructions: +110.0M (+1.07%)
  • 🟥 elab/big_match_nat//instructions: +100.9M (+2.19%)
  • 🟥 elab/big_match_partial//instructions: +126.9M (+0.92%)
  • 🟥 elab/cbv_dedup//instructions: +50.4M (+1.59%)
  • elab/cbv_leroy//maxrss: -125MiB (-6.78%)
  • 🟥 elab/delayed_assign//instructions: +110.7M (+3.41%)
  • 🟥 elab/delayed_sharing//instructions: +117.0M (+4.57%)
  • 🟥 elab/iterators//instructions: +44.5M (+1.81%)
  • 🟥 elab/simp_arith1//instructions: +88.9M (+4.12%)
  • 🟥 elab/string_simp_ne//instructions: +101.1M (+1.84%)
  • 🟥 interpreted/iterators//instructions: +47.8M (+0.13%)
  • 🟥 lake/inundation/config elab//instructions: +37.5M (+1.71%)
  • 🟥 lake/inundation/config import//instructions: +32.6M (+2.92%)
  • 🟥 lake/inundation/config tree//instructions: +33.9M (+2.95%)
  • 🟥 lake/inundation/env//instructions: +31.5M (+2.78%)

Small changes (44✅, 1440🟥)

Too many entries to display here. View the full report on radar instead.

mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Jun 22, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Jun 22, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Jun 22, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants