Skip to content

chore: add targeted utils mutation tests#576

Merged
marandaneto merged 5 commits into
mainfrom
chore/add-utils-mutation-tests
May 26, 2026
Merged

chore: add targeted utils mutation tests#576
marandaneto merged 5 commits into
mainfrom
chore/add-utils-mutation-tests

Conversation

@marandaneto
Copy link
Copy Markdown
Member

@marandaneto marandaneto commented May 15, 2026

💡 Motivation and Context

Add targeted mutation testing coverage for posthog/utils.py so regressions in utility helpers and flag cache behavior are caught by tests.

This adds a focused mutmut configuration and CI job that only runs when the targeted mutation inputs change, with mutants/ cached across CI runs. The targeted utils tests now cover all statements in posthog/utils.py.

Dependency: Depends on PostHog/posthog-sdk-test-harness#19 landing first so the SDK compliance harness accepts api_key as an alias for the /flags token assertion.

💚 How did you test it?

  • uv run --extra test pytest posthog/test/test_utils.py posthog/test/test_size_limited_dict.py --timeout=30 -q
  • uv run --extra test coverage run -m pytest posthog/test/test_utils.py posthog/test/test_size_limited_dict.py --timeout=30 -q && uv run --extra test coverage report -m posthog/utils.py
  • rm -rf mutants && uv run --extra test --with mutmut mutmut run --max-children 1 && uv run --extra test --with mutmut mutmut results
  • uv run --extra test pytest --timeout=30 -q
  • Parsed .github/workflows/ci.yml as YAML

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

@marandaneto marandaneto requested a review from a team as a code owner May 15, 2026 22:42
@marandaneto marandaneto marked this pull request as draft May 15, 2026 22:42
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

posthog-python Compliance Report

Date: 2026-05-26 10:10:43 UTC
Duration: 176146ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 517ms
Format Validation.Event Has Uuid 1507ms
Format Validation.Event Has Lib Properties 1508ms
Format Validation.Distinct Id Is String 1506ms
Format Validation.Token Is Present 1508ms
Format Validation.Custom Properties Preserved 1507ms
Format Validation.Event Has Timestamp 1507ms
Retry Behavior.Retries On 503 9518ms
Retry Behavior.Does Not Retry On 400 3508ms
Retry Behavior.Does Not Retry On 401 3508ms
Retry Behavior.Respects Retry After Header 9513ms
Retry Behavior.Implements Backoff 23529ms
Retry Behavior.Retries On 500 7506ms
Retry Behavior.Retries On 502 7510ms
Retry Behavior.Retries On 504 7509ms
Retry Behavior.Max Retries Respected 23522ms
Deduplication.Generates Unique Uuids 1508ms
Deduplication.Preserves Uuid On Retry 7513ms
Deduplication.Preserves Uuid And Timestamp On Retry 14521ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7506ms
Deduplication.No Duplicate Events In Batch 1508ms
Deduplication.Different Events Have Different Uuids 1506ms
Compression.Sends Gzip When Enabled 1507ms
Batch Format.Uses Proper Batch Structure 1507ms
Batch Format.Flush With No Events Sends Nothing 1005ms
Batch Format.Multiple Events Batched Together 1506ms
Error Handling.Does Not Retry On 403 3509ms
Error Handling.Does Not Retry On 413 3508ms
Error Handling.Retries On 408 7513ms

Feature_Flags Tests

16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 1004ms
Request Payload.Flags Request Uses V2 Query Param 1006ms
Request Payload.Flags Request Hits Flags Path Not Decide 1006ms
Request Payload.Flags Request Omits Authorization Header 1007ms
Request Payload.Token In Flags Body Matches Init 1006ms
Request Payload.Groups Round Trip 1007ms
Request Payload.Groups Default To Empty Object 1006ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 1007ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 1007ms
Request Payload.Disable Geoip Omitted Defaults To False 1005ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 1007ms
Request Lifecycle.No Flags Request On Init Alone 503ms
Request Lifecycle.No Flags Request On Normal Capture 1506ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 1011ms
Request Lifecycle.Mock Response Value Is Returned To Caller 1002ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 1509ms

@socket-security
Copy link
Copy Markdown

socket-security Bot commented May 26, 2026

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
High CVE: AIOHTTP's HTTP Parser auto_decompress feature is vulnerable to zip bomb

CVE: GHSA-6mq8-rvhq-8wgg AIOHTTP's HTTP Parser auto_decompress feature is vulnerable to zip bomb (HIGH)

Affected versions: < 3.13.3

Patched version: 3.13.3

From: uv.lockpypi/aiohttp@3.12.13

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/aiohttp@3.12.13. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
posthog/test/test_utils.py:297-343
**Non-parameterised test for OS platform branches**

`test_get_os_info_branches` contains six independent platform scenarios (win32 with version, win32 without version, darwin, linux, freebsd, and a generic fallback) each written as a separate `with` block inside a single method. Per the team convention, these should each be a row in a `@parameterized.expand` table — as used elsewhere in this file — so that failures point to the exact platform case and each scenario is independently named and runnable.

### Issue 2 of 2
posthog/utils.py:202-204
**`# fmt: off` directives leak mutation-test infrastructure into production code**

Both occurrences of `# fmt: off` / `# fmt: on` (here and in `set_cached_flag` around the `FlagCacheEntry(…)` call) exist solely because the constant reference makes the line exceed the formatter's line-length limit. Suppressing the formatter in production source to satisfy mutmut's single-line requirement violates the "no superfluous parts" rule. A cleaner alternative would be to introduce a short local alias (e.g. `_STALE = CACHE_STALE_TTL`) at the call site, or to rename the constant to something shorter so the line fits naturally without bypassing `black`.

Reviews (1): Last reviewed commit: "Merge branch 'main' into chore/add-utils..." | Re-trigger Greptile

Comment thread posthog/test/test_utils.py Outdated
Comment thread posthog/utils.py Outdated
@marandaneto marandaneto merged commit 033d32b into main May 26, 2026
27 checks passed
@marandaneto marandaneto deleted the chore/add-utils-mutation-tests branch May 26, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants