Skip to content

parse python panics in LSPClient.Locate (slice bounds out of range) when following imports into .venv #186

@Valen-C12

Description

@Valen-C12

parse python panics in LSPClient.Locate (slice bounds out of range [13:6]) when following imports into .venv

Repo: cloudwego/abcoder
Labels: bug, language/python, crash

Summary

When parsing a Python project whose source imports modules from a .venv/ containing Python 3.12 syntax that the bundled jedi 0.19.1 / parso 0.8.4 cannot fully parse, abcoder hard-crashes:

panic: runtime error: slice bounds out of range [13:6]

goroutine 1 [running]:
github.com/cloudwego/abcoder/lang/lsp.(*LSPClient).Locate(...)
    /tmp/abcoder-build/lang/lsp/lsp_methods.go:316 +0x430
github.com/cloudwego/abcoder/lang/collect.(*Collector).getSymbolByLocation(...)
    /tmp/abcoder-build/lang/collect/collect.go:2114 +0xe58
github.com/cloudwego/abcoder/lang/collect.(*Collector).getSymbolByTokenWithLimit(...)
github.com/cloudwego/abcoder/lang/collect.(*Collector).getDepsWithLimit(...)
github.com/cloudwego/abcoder/lang/collect.(*Collector).processSymbol(...)
    /tmp/abcoder-build/lang/collect/collect.go:2245 +0x328
github.com/cloudwego/abcoder/lang/collect.(*Collector).Collect(...)
    /tmp/abcoder-build/lang/collect/collect.go:206 +0x3c0

The panic is preceded by hundreds of warnings like:

[ERROR] collect.go:2306: get type symbol for token {file:///.../.venv/lib/python3.12/site-packages/redis/_parsers/socket.py:115:24-115:32 function [] readline} failed: definition of token ... not found

…and pylsp stderr noise from the pylsp_semantic_tokens hook:

WARNING - pylsp.config.config - Failed to load hook pylsp_semantic_tokens:
  'Newline' object has no attribute 'children'
  AttributeError in jedi/inference/syntax_tree.py:460 (infer_or_test)

jedi/parso returns a malformed range (start 13 > end 6), and lang/lsp/lsp_methods.go:316 slices a string with that range without bounds-checking → panic.

Reproduction

# Project: any Python 3.12 project with a populated .venv that contains
# files using PEP 701 f-strings or other 3.12-only syntax (e.g. attrs, redis).
abcoder parse python . -o /tmp/ast.json
# …jedi follows imports into .venv/...redis/_parsers/socket.py
# panic: runtime error: slice bounds out of range [13:6]

Expected

  • LSPClient.Locate should validate the Range it receives (Start <= End, both within document bounds) and return an error instead of panicking.
  • Higher up, Collector.getSymbolByLocation should swallow such errors as a "skip this token" rather than propagating a panic.
  • Optionally: parse python should default to excluding common virtualenv directories (.venv, venv, env) since they're never the parse target — they're only ever followed transitively, which is what triggers the bad ranges.

Workaround

Pass --exclude .venv (and any other vendored Python trees):

abcoder parse python . -o /tmp/ast.json --exclude .venv

Root cause notes (for triage)

The malformed range originates from jedi's syntax-tree inference on certain 3.12 constructs that parso 0.8.4 represents incorrectly (e.g. Newline nodes appearing where parso normally yields simple_stmt children). Bumping parso to ≥0.8.5 in the bundled pylsp fork's deps may eliminate the source — but the Go side should still bounds-check defensively.

Environment

  • abcoder v0.0.0-20260427034146-9d8ecc5873be+dirty
  • macOS 25.3.0 (arm64)
  • jedi 0.19.1, parso 0.8.4 (in miniconda Python 3.10)
  • Project: Python 3.12 with .venv containing attrs, redis, pydantic, etc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions