Python: Track instance attributes through type tracking

[Copilot]

Type-tracking-based modeling (Concepts, API graphs) misses values stored in a self attribute and read from another method. A typical symptom is a PEP 249 connection assigned to self._conn in one method not being recognized as a connection when used later, so downstream modeling such as SQL-execution sinks can be missed.

Changes

• TypeTrackingImpl.qll: Add localFieldStep, a no-call-graph type-tracking level step that models flow through self.attr across instance methods on the same class.
  • This complements the existing attribute store/read steps by relating writes to self.attr with later reads of the same attribute on the same class.
  • The step intentionally applies across instance methods in general, not just __init__, matching the existing instance-insensitive and order-insensitive treatment used for similar field tracking in Ruby and JavaScript.
• hdbcli/pep249.py: Add regression coverage for a connection stored in self._conn and used both directly and via a getter.
• attribute_tests.py: Update expectations to reflect tracking from values stored on self into later self.foo reads inside instance methods, while leaving external instance.foo reads unchanged.
• Experimental call-graph tests: Update inline expectations and the corresponding .expected file to reflect the improved type-tracker results.
• Change note: Add a Python library change note describing the additional analysis coverage.

Notes for reviewers

• The new step is deliberately an over-approximation for instance attributes on the same class; it does not distinguish different instances and does not require a write to be ordered before a read.
• External attribute reads such as instance.foo are still not modeled by this change.

Original prompt

This test has missing alerts when the object returned by connect() is stored in a class field (attribute). I believe the problem lies in execute(DataFlow::TypeTracker t). Investigate. This could be a missing step in type tracking, in which case we should fix it in general. There should be a store step into the attribute _conn in init (see attributeStoreStep in python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll) and then a read step from that attribute in get_connection (see attributeReadStep in python/ql/lib/semmle/python/dataflow/new/internal/DataFlowPrivate.qll). Another option is that we are misusing type tracking in this case. Please investigate and explain the cause to me, with suggestions for how to fix.

The user has attached the following file paths as relevant context:

• python/ql/test/library-tests/frameworks/hdbcli/pep249.py
• python/ql/lib/semmle/python/frameworks/PEP249.qll
• .github/instructions/ql-files.instructions.md

[Copilot]

Pull request overview

This PR improves Python type-tracking so that values stored into instance attributes (for example in __init__) can be recovered when those attributes are read in other methods, enabling downstream modeling (Concepts / API graphs) to recognize types such as PEP 249 connections when used outside the constructor.

Changes:

• Extend type-tracking with a no-call-graph “local field” level step to connect attribute writes (self.attr = ...) to later reads (self.attr) across methods on the same class.
• Add/extend Python library tests for PEP249 (hdbcli) and type-tracking attribute flows to validate the new behavior.
• Update experimental inline call-graph expected output to reflect improved type-tracker results through instance attributes.

Show a summary per file

File	Description
python/ql/lib/semmle/python/dataflow/new/internal/TypeTrackingImpl.qll	Adds a new type-tracking level step intended to model flow through instance attributes across methods.
python/ql/test/library-tests/frameworks/hdbcli/pep249.py	Adds a regression test for a DB-API connection stored on self._conn and used in other methods.
python/ql/test/library-tests/dataflow/typetracking/attribute_tests.py	Updates expectations for tracking reads of attributes assigned in __init__.
python/ql/test/experimental/library-tests/CallGraph/code/class_attr_assign.py	Updates inline expectations to reflect type tracking through instance attributes.
python/ql/test/experimental/library-tests/CallGraph/InlineCallGraphTest.expected	Updates expected output to reflect the improved type-tracker behavior.

Copilot's findings

• Files reviewed: 6/6 changed files
• Comments generated: 2

[yoff]

This looks fine and performance does not seem problematic. The first few new alerts look good. The file-not-closed ones seem a bit more involved and unclear. If the rest are fine, I am happy for this to be merged.

[BazookaMusic]

How about super classes? I see we take them into account for the ts/js implementation. Is there a reason why we don't model them here?

[owen-mc]

Good point. I'll implement that. I was about to re-run DCA anyway.

[owen-mc]

Note that I only implemented it for direct subclasses or superclasses, not mix-ins. We can always generalise it later if we find we need it.

[BazookaMusic]

Does this work nicely with @propertydecorators?

[owen-mc]

Not properly, but I think that's consistent with what the python library already does, from what I can see.

[owen-mc]

I went through the new results from the first DCA run in more detail. The new alerts for py/file-not-closed and py/should-use-with are FPs caused by this PR, and I have fixed them. In both cases we don't want to track through attribute reads. I've introduced a shared mechanism to exclude these results, which can be reused in future. Some other new results I looked at were FPs but ones we'd expect to find, which it doesn't make sense to fix as part of this PR. (In one case we now track taint through a local sanitizer, and the fix is to recognise that kind of sanitizer. In another we incorrectly identify some latitudes and longitudes as possibly sensitive information.)

[yoff]

Code changes look reasonable. Approved bearing DCA not raising any alarms.