Skip to content

feat(check-stream): support configurable stream names#1030

Draft
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1779315328-configurable-check-streams
Draft

feat(check-stream): support configurable stream names#1030
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1779315328-configurable-check-streams

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 20, 2026

Summary

Adds support for overriding declarative CheckStream stream names from connector config using the top-level airbyte_check_stream_names key. When the key is absent, existing manifest stream_names behavior is preserved; when present, the value must be a list of strings and the selected names still go through the existing catalog membership and availability checks.

Review & Testing Checklist for Human

  • Verify airbyte_check_stream_names is the intended platform-facing config key for ADP/Sonar integration.
  • Confirm empty override list should mean “skip static stream availability checks” while still allowing dynamic stream checks to run when configured.
  • Validate with a low-code connector whose manifest stream_names includes an unselected stream and whose source config provides selected stream names through airbyte_check_stream_names.

Notes

Local checks run:

  • poetry run ruff format --check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
  • poetry run ruff check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
  • poetry run mypy --config-file mypy.ini airbyte_cdk/sources/declarative/checks/check_stream.py
  • poetry run pytest unit_tests/sources/declarative/checks/test_check_stream.py -q

Link to Devin session: https://app.devin.ai/sessions/2fe0f6c5174b40a1842828a49e32f69b

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1779315328-configurable-check-streams#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1779315328-configurable-check-streams

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 20, 2026

PyTest Results (Fast)

4 076 tests  +7   4 065 ✅ +7   7m 20s ⏱️ -24s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 344d0db. ± Comparison against base commit f67a9d9.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 20, 2026

PyTest Results (Full)

4 079 tests  +7   4 067 ✅ +7   10m 47s ⏱️ -10s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 344d0db. ± Comparison against base commit f67a9d9.

♻️ This comment has been updated with latest results.

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

CI note: the only current failed check is optional Pytest (All, Python 3.13, Ubuntu).

Failure details

The failed job is https://github.com/airbytehq/airbyte-python-cdk/actions/runs/26193981254/job/77069032809.

It failed in an existing JWT test, not in the changed check-stream tests:

FAILED unit_tests/sources/declarative/auth/test_jwt.py::TestJwtAuthenticator::test_get_signed_token - AssertionError: assert '***' == '***'

The same full pytest matrix passed on main yesterday: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/26100771442.

Targeted local verification still passes for this PR's changed area:

poetry run pytest unit_tests/sources/declarative/checks/test_check_stream.py -q
poetry run ruff check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
poetry run ruff format --check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
poetry run mypy --config-file mypy.ini airbyte_cdk/sources/declarative/checks/check_stream.py

The failing JWT assertion appears unrelated to this PR's two touched files (airbyte_cdk/sources/declarative/checks/check_stream.py and unit_tests/sources/declarative/checks/test_check_stream.py).


Devin session

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Updated the local variable in _get_stream_names from stream_names to configured_stream_names for clarity and pushed commit 344d0db6.

Re-ran targeted checks successfully:

poetry run ruff format --check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
poetry run ruff check airbyte_cdk/sources/declarative/checks/check_stream.py unit_tests/sources/declarative/checks/test_check_stream.py
poetry run mypy --config-file mypy.ini airbyte_cdk/sources/declarative/checks/check_stream.py
poetry run pytest unit_tests/sources/declarative/checks/test_check_stream.py -q

Devin session

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Description updated to include the mypy check that was run after the rename.


Devin session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants