Skip to content

refactor(pipeline): share psycopg scaffolding between checkpoint and audit#241

Merged
miguelgfierro merged 1 commit into
issue-147-pipeline-evolutionfrom
simplify/shared-psycopg-helper
May 28, 2026
Merged

refactor(pipeline): share psycopg scaffolding between checkpoint and audit#241
miguelgfierro merged 1 commit into
issue-147-pipeline-evolutionfrom
simplify/shared-psycopg-helper

Conversation

@miguelgfierro
Copy link
Copy Markdown
Contributor

Addresses finding #6 from the simplifier review of #232.

PostgresCheckpointer and PostgresAuditLog had four pieces of identical boilerplate copy-pasted between checkpoint.py and audit.py:

  1. try: import psycopg as _psycopg optional-dep guard.
  2. if _psycopg is None: raise ImportError(...) with the same install hint.
  3. (dsn is None) == (connection is None) xor-check.
  4. table_name.replace(\"_\", \"\").isalnum() validator (interpolated into DDL).
  5. _ddl_applied flag + _ensure_table lazy-DDL method.

This PR extracts them into a single PsycopgBackend base class in a new pipeline/_psycopg_backend.py module. Each concrete backend now only declares its _DDL and default table name:

class PostgresCheckpointer(PsycopgBackend):
    _DDL = \"\"\"CREATE TABLE IF NOT EXISTS {table} (...)\"\"\"

    def __init__(self, *, dsn=None, connection=None, table_name=\"firefly_checkpoints\"):
        super().__init__(kind=\"PostgresCheckpointer\", dsn=dsn, connection=connection, table_name=table_name)

Behavior

Unchanged. Same public API, same constructor signatures, same ImportError / ValueError messages (parametrized by kind).

Tests

The two test files used to monkeypatch audit_module._psycopg and checkpoint_module._psycopg to stub the optional dep. They now monkeypatch _psycopg_backend._psycopg (single source of truth). Same number of tests, same coverage.

Verification

  • pytest tests/unit/pipeline/ → 142 passed.
  • ruff check + ruff format --check clean.

Diff stats

audit.py        | -22 LOC
checkpoint.py   | -25 LOC
+ _psycopg_backend.py (new, ~75 LOC including Apache header + docstrings)
test_audit_log.py / test_checkpoint_backends.py: monkeypatch target swap

Net is mildly positive raw LOC, but the duplication is gone — single source of truth for the Postgres scaffolding, which is what the finding called out. Future Postgres-backed backends (if any) inherit the same surface for free.

…audit

Extract the duplicated Postgres setup (optional-dep guard, dsn-xor-connection
check, table-name validation, lazy idempotent DDL) into a single
PsycopgBackend base class in pipeline/_psycopg_backend.py.

PostgresCheckpointer and PostgresAuditLog now inherit from it; each only
declares its DDL and default table name.

Tests updated to monkeypatch _psycopg on the shared module.
@miguelgfierro miguelgfierro merged commit 236b814 into issue-147-pipeline-evolution May 28, 2026
@miguelgfierro miguelgfierro deleted the simplify/shared-psycopg-helper branch May 28, 2026 07:15
ancongui pushed a commit that referenced this pull request May 31, 2026
…-helper

refactor(pipeline): share psycopg scaffolding between checkpoint and audit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant