Skip to content

THRIFT-6043: Limit struct read/write recursion depth in Python library#3592

Open
Jens-G wants to merge 1 commit into
apache:masterfrom
Jens-G:py-recursion-depth
Open

THRIFT-6043: Limit struct read/write recursion depth in Python library#3592
Jens-G wants to merge 1 commit into
apache:masterfrom
Jens-G:py-recursion-depth

Conversation

@Jens-G

@Jens-G Jens-G commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

Adds a configurable recursion depth limit (default 64) to struct and exception read/write in the Python library.

What is changed:

  • compiler/cpp/src/thrift/generate/t_py_generator.cc: generated read()/write() methods now wrap the body in increment_recursion_depth() / try / finally: decrement_recursion_depth().
  • lib/py/src/protocol/TProtocol.py: TProtocolBase gains _recursion_depth, DEFAULT_RECURSION_DEPTH = 64, increment_recursion_depth() (raises TProtocolException.DEPTH_LIMIT when exceeded), and decrement_recursion_depth().
  • lib/py/src/ext/protocol.h / protocol.tcc: The fastbinary C extension (TBinaryProtocolAccelerated, TCompactProtocolAccelerated) is also guarded. ProtocolBase gains a recursionDepth_ counter; readStruct() and encodeValue(T_STRUCT) use a RAII RecursionGuard that raises TProtocolException(DEPTH_LIMIT, …) via the Python C API when the limit is exceeded.
  • lib/py/test/test_recursion_depth.py + lib/py/Makefile.am: Generated-code round-trip tests over test/Recursive.thrift (RecTree struct, CoError exception) for TBinaryProtocol, TBinaryProtocolAccelerated, TCompactProtocol, TCompactProtocolAccelerated, and TJSONProtocol.

Note: The guard is placed after the fast-path early return in generated read(), so TBinaryProtocol (non-accelerated) and TJSONProtocol are protected via the Python path; TBinaryProtocolAccelerated and TCompactProtocolAccelerated are protected via the C extension guard. The limit (64) is hardcoded in the C extension; making it read from TConfiguration can be a follow-up.

skip() remains independently bounded via TProtocolBase.skip(max_depth=64) (unchanged).

Test plan

  • make -C lib/py check passes (Python 3.10–3.14, with and without skip-build-ext)
  • 12/12 round-trip tests in test_recursion_depth.py pass locally (Docker thrift:jammy)
  • Baseline proof: stripping the generator guard from the generated RecTree.read()/write() causes 8 over-limit assertions to fail

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com

Client: py

Adds a configurable recursion depth limit (default 64) to struct and
exception read/write in the Python library. Both the pure-Python path
and the fastbinary C extension are guarded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Jens-G Jens-G requested a review from mhlakhani as a code owner June 9, 2026 21:47
@mergeable mergeable Bot added python compiler build and general CI cmake, automake and build system changes labels Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build and general CI cmake, automake and build system changes compiler python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant