fix: Handle FinalStateCursor gracefully and detect final-state for retention check

[devin-ai-integration]

This PR targets the following PR:

• feat(cdk): Add cursor age validation to StateDelegatingStream #890

---

fix: Handle FinalStateCursor in retention check and detect final-state

Summary

Fixes two blockers in the StateDelegatingStream cursor age validation feature (PR #890):

1. FinalStateCursor crash in _is_cursor_older_than_retention_period

When iterating cursors [full_refresh.cursor, incremental.cursor], if the full_refresh cursor is a FinalStateCursor (no incremental_sync defined), it raises NotImplementedError from the base Cursor.get_cursor_datetime_from_state. Previously this was caught and re-raised as SystemError, preventing the incremental cursor from ever being tried. Now the loop continues to the next cursor instead.

2. Final-state detection in create_state_delegating_stream

When a full_refresh_stream without incremental_sync completes, it produces a "final state" ({NO_CURSOR_STATE_KEY: True}). The code now detects this sentinel state and skips the retention check entirely, returning the incremental stream directly. A completed full refresh means incremental is always correct.

Review & Testing Checklist for Human

• The continue path in _is_cursor_older_than_retention_period is not directly tested. The new test only exercises the early-return final-state detection (fix 2). There is no test that creates a stream where the full_refresh cursor is an actual FinalStateCursor that raises NotImplementedError and verifies the loop falls through to the incremental cursor. Consider whether this gap needs coverage.
• Silent fallback when ALL cursors lack the method: With the continue change, if every cursor in the list lacks get_cursor_datetime_from_state or raises NotImplementedError, cursor_datetime stays None and the method returns True (full refresh). Previously this was a hard SystemError. Verify this silent degradation is acceptable vs. raising an error.
• Final-state check uses truthy evaluation (stream_state.get(NO_CURSOR_STATE_KEY)) rather than is True. The actual FinalStateCursor sets the value to True, so this works in practice. Confirm this is sufficient.
• Manual test plan: Configure a StateDelegatingStream where full_refresh_stream has no incremental_sync, run a full refresh to completion (producing final state), then trigger a second sync with api_retention_period set — verify it transitions to incremental without crashing.

Notes

• Link to Devin run: https://app.devin.ai/sessions/c4a007a380b6473198492d9224bd39ef
• Requested by: Alfredo Garcia (@agarctfi)

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1771458776-fix-retention-period-final-state-cursor#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1771458776-fix-retention-period-final-state-cursor

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.