recommended airflow provider version#37437
Open
mobuchowski wants to merge 2 commits into
Open
Conversation
Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com>
Contributor
Preview links (active after the
|
warrierr
requested changes
Jun 11, 2026
| @@ -19,6 +19,17 @@ For details about MWAA dependency and constraints behavior, see AWS documentatio | |||
|
|
|||
| For base setup steps, see [Enable Data Observability: Jobs Monitoring for Apache Airflow][1]. | |||
Contributor
There was a problem hiding this comment.
Cab we move this guide to be under Airflow in the nav? It seems to be floating on its own.
|
|
||
| ## OpenLineage provider compatibility | ||
|
|
||
| Data Observability: Jobs Monitoring uses the Apache Airflow OpenLineage provider to emit Airflow DAG and task events. Datadog recommends using the newest OpenLineage provider version that is compatible with your Airflow version. |
Contributor
There was a problem hiding this comment.
Recommend or require? If they don't do this are we ok with it, can we specifically call out what they are missing?
| apache-airflow-providers-openlineage==<PROVIDER_VERSION> | ||
| ``` | ||
|
|
||
| 4. If your deployment uses constraints, update the Airflow constraints file for every OpenLineage package that the provider requires. Include `apache-airflow-providers-openlineage`, `apache-airflow-providers-common-sql`, `apache-airflow-providers-common-compat`, `openlineage-integration-common`, `openlineage-python`, and `openlineage-sql` when they are present in your constraints. |
|
|
||
| 5. Redeploy Airflow, then trigger a DAG run and verify that it appears in Data Observability: Jobs Monitoring. | ||
|
|
||
| For Amazon MWAA, dependency resolution is controlled by MWAA constraints. If the newest compatible provider conflicts with MWAA defaults, follow the custom constraints pattern in the [MWAA upgrade guide](/data_observability/jobs_monitoring/airflow_mwaa_upgrade/). |
Contributor
There was a problem hiding this comment.
should just go under the MWAA instructions section
Contributor
|
Thanks for updating the docs! I've filed DOCS-14707 for an editorial review from the Docs team |
Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Clearly document what DD recomments for Airflow Data Observability users which OpenLineage provider version to use with given Airflow version.
Merge readiness:
AI assistance
Used codex for initial version, reviewed and edited the result.