Skip to content

Conversation

@joehendrix
Copy link
Contributor

@joehendrix joehendrix commented Jan 29, 2026

This PR introduces a PySpec generator prototype along with several
supporting improvements and bug fixes.

Main Feature: PySpec Generator

Adds a new strata pySpecs subcommand that translates Python
specification files into binary PySpec files. Python specification
files capture method signatures and their pre/post-conditions as
idiomatic Python source code, facilitating LLM-assisted generation.
The binary PySpec format enables efficient loading when validating
source code that uses these imports.

New files:

  • Strata/Languages/Python/Specs.lean - Core specification
    translation logic
  • Strata/Languages/Python/Specs/DDM.lean - DDM types for PySpec
  • Strata/Languages/Python/Specs/Decls.lean - Declaration handling
  • Strata/Languages/Python/ReadPython.lean - Python file reading
    utilities
  • Test files: StrataTest/Languages/Python/Specs/{basetypes.py, main.py} and SpecsTest.lean
  • StrataTest/Util/Python.lean - Python testing utilities

Refactoring and Code Organization

Source range improvements:

  • Migrated source mapping types from Strata/DDM/AST.lean to
    dedicated modules:
    • Strata/DDM/Util/SourceRange.lean - SourceRange definition
    • Strata/Util/FileRange.lean - FileRange, Uri, DiagnosticModel
      types and formatting
  • Added Strata/Util/DecideProp.lean for decidability utilities
  • Reduces unnecessary code in DDM and improves modularity

Import organization:

  • Pruned unnecessary imports in StrataMain.lean
  • Filter builtin module imports from Python signatures

Bug Fixes

  1. DDM Ion parsing: Fixed bug in parsing DDM Ident types in Ion
    files (changed .asString to .asSymbolString in
    Strata/DDM/Ion.lean)
  2. DDM formatting:
  • Fixed precedence calculation in Strata/DDM/Format.lean
    (changed maxPrec to maxPrec + 1 in atom definition)
  • This fixes C_Simp formatting issues where output had missing
    spaces (e.g., (bool)proceduretrivial() → bool procedure trivial())
  • Made FormatState.empty public to enable external
    instantiation
  1. Python tooling:
  • Added recursion limit (2500) in Tools/Python/strata/gen.py
  • Improved parse error handling with dedicated exit code (100) for
    parse failures
  • Cleaned up handling of deprecated AST operations

Testing

  • Updated C_Simp example test expectations to reflect formatting
    fixes
  • Added comprehensive PySpec test cases

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.

@joehendrix joehendrix force-pushed the jhx/python_fixes branch 2 times, most recently from 5b4ee2b to 76f48c8 Compare January 29, 2026 22:17
@joehendrix joehendrix changed the title Lean code to run Python strata.gen Add pySpec generator Jan 29, 2026
@joehendrix joehendrix force-pushed the jhx/python_fixes branch 2 times, most recently from c535c4c to ac8d50d Compare January 29, 2026 23:20
@joehendrix joehendrix marked this pull request as ready for review January 30, 2026 06:55
@joehendrix joehendrix requested a review from a team as a code owner January 30, 2026 06:55
@joehendrix joehendrix force-pushed the jhx/python_fixes branch 4 times, most recently from 21ff87f to a0ca0b7 Compare February 3, 2026 16:28
shigoel
shigoel previously approved these changes Feb 5, 2026
@shigoel shigoel enabled auto-merge February 5, 2026 18:03
joehendrix and others added 11 commits February 5, 2026 11:03
- Fix DDM Format bug where identifiers got unnecessary parentheses by
  setting atom precedence to maxPrec + 1
- Improve Python specs output with bracket formatting for args and
  kwonly parameters
- Add NoArgs variants for types without arguments (cleaner output)
- Change arg names from Str to Ident for proper formatting
- Remove debug "Running" output from pythonToStrata
- Add guard_msgs test to validate clean output format

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ormatting

Python changes:
- Modified translateImportFrom to check if module is in preludeSig.rank
- Builtin modules (typing, builtins, etc.) no longer generate extern declarations
- Hoisted check outside loop for efficiency

C_Simp changes:
- Updated procedure pretty printer to add spaces around "procedure" keyword
- Ensures keywords are properly separated from identifiers in output

Test updates:
- Updated all C_Simp example tests to match new formatting
- Fixed DDMAxiomsExtraction expected output
- Updated Python SpecsTest expected output

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
joehendrix and others added 9 commits February 5, 2026 11:03
Wrap long comment lines in FileRange.lean, Python.lean, and Specs.lean
to adhere to the project's style guideline of 80 characters per line.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Use suffix pattern for function variants to match Lean conventions
(e.g., foldl/foldlM, map/mapM, translateModuleAux).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add concise comments to 19 functions and datatypes where the purpose is
not immediately obvious from the name:

- DDM/AST.lean: Document flatten, catbvarLevel, NewBindingM, and
  foldOverArgBindingSpecs
- DDM/Ion.lean: Explain StringOrSexp, StructArgMap, and mapFields
- DDM/Util/Fin.lean: Document Fin.Range for iteration
- Python/Specs.lean: Add context for Pred, Iterable, ModuleName,
  TypeTranslator, and resolveModule
- Python/Specs/DDM.lean: Document conversion functions
- Python/Specs/Decls.lean: Explain PythonIdent, MetadataType, unionAux,
  and SpecPred

All comments are 1-3 sentences and focus on semantic information not
obvious from names alone.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update deprecated String API calls to work with Lean 4.27:
- Replace String.trim with String.trimAscii.toString
- Update String.revFind to String.revFind?
- Fix String.Pos dependent type usage in Python.Specs
- Replace String.dropRight with String.dropEnd

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@shigoel shigoel added this pull request to the merge queue Feb 5, 2026
Merged via the queue into main with commit cc30781 Feb 5, 2026
15 checks passed
@shigoel shigoel deleted the jhx/python_fixes branch February 5, 2026 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants