Unpin KFP SDK for Kubeflow 1.11.0 and clean up residual dex-based examples#44
Open
Unpin KFP SDK for Kubeflow 1.11.0 and clean up residual dex-based examples#44
Conversation
Contributor
|
what about kubeflow-examples/pipelines/lightweight-components/mobile-price-classifications.ipynb ? |
Member
Author
good point, updated |
- pipelines/minimal-container-components: kfp 2.0.1 -> 2.15.0 - pipelines/lightweight-python-package: kfp[all] 2.12.1 -> 2.15.0 - mlflow/mlflow-kfp-example.ipynb: kfp[all] 2.9.0 -> 2.15.0
2c32cb6 to
00b8a15
Compare
The cluster uses Keycloak/oauth2-proxy/Dex JWT auth which cannot be bypassed with the basic Dex username/password flow in auth_session.py. Remote submission is not functional with this setup. Also remove the now-irrelevant 'Preparing for Remote KFP Connection' README section.
These files are not referenced by any Dockerfile, script, or CI config. The notebooks install kfp inline via pip magic commands, and the submit scripts import kfp directly from whatever environment they run in.
8017ada to
54ce95d
Compare
All prokube deployments now use Keycloak/oauth2-proxy — the Dex username/password session cookie approach no longer works. For remote pipeline submission, refer to the pipe-fiction example which documents the Keycloak OIDC client flow.
54ce95d to
5c94bdd
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aligns the Kubeflow Pipelines example content with the Kubeflow 1.11.0 platform upgrade by removing Dex-based remote-submission utilities and adjusting example dependency guidance around the KFP SDK.
Changes:
- Removed the Dex
auth_session.pyhelper and its docs (no longer compatible with current auth setup). - Removed the minimal-container-components remote submission script and the associated README instructions.
- Removed per-example
requirements.txtfiles and updated notebooks to reference (commented) KFP 2.15.0 availability in prokube notebook images.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pipelines/utils/README.md | Deleted Dex auth-session documentation. |
| pipelines/utils/auth_session.py | Deleted Dex cookie/session helper implementation. |
| pipelines/minimal-container-components/submit-remote.py | Deleted remote submission script that depended on Dex auth flow. |
| pipelines/minimal-container-components/requirements.txt | Removed pinned KFP requirement for this example. |
| pipelines/minimal-container-components/README.md | Removed remote-submission guidance; now only documents in-cluster submission. |
| pipelines/lightweight-python-package/requirements.txt | Removed pinned kfp[all] requirement for this example. |
| pipelines/lightweight-components/mobile-price-classifications.ipynb | Added commented dependency note referencing KFP 2.15.0. |
| mlflow/mlflow-kfp-example.ipynb | Replaced active pip install with commented dependency note referencing KFP 2.15.0. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unpin all KFP SDK pins to match KFP backend 2.15.0 (provided by our custom notebooks!) in Kubeflow v1.11.0. The hint at version is still there in the notebook, but it's no longer installed, to increase compatibility between our images and platform components.
Additionally, remove Dex-based remote submission utilities that are no longer functional.
KFP SDK bumps:
pipelines/minimal-container-components:kfp2.0.1 → Notebook's version (2.15.0 hint)pipelines/lightweight-python-package:kfp[all]2.12.1 → Notebook's version (2.15.0 hint)pipelines/mobile-price-classification: samemlflow/mlflow-kfp-example.ipynb: sameDex/auth cleanup:
utils/auth_session.pyand the entireutils/directory — Dex username/password session cookie flow no longer works with Keycloak/oauth2-proxy authsubmit-remote.pyfromminimal-container-componentsand the associated "Preparing for Remote KFP Connection" README section — remote submission is not functional with the current JWT auth setuprequirements.txtfiles from pipeline examples (not referenced by any Dockerfile, script, or CI config; notebooks install kfp inline via pip magic commands)For remote pipeline submission, refer to the pipe-fiction example which documents the Keycloak OIDC client flow.
Context
Part of the Kubeflow 1.11.0 upgrade. Companion PRs:
prokube/prokube— main platform upgrade (branchfeature/kubeflow-1.11.0)prokube/prokube-images— notebook image and MLflow server updates (PR #34)What was NOT changed (and why)
serving/): Already pinkserve>=0.15.2, compatible with KServe v0.15.2 in Kubeflow 1.11.0hparam-tuning/): Useskubeflow.org/v1beta1API (unchanged) withbatch/v1 Job(no Training Operator dependency)v1/v1beta1APIs still served in Istio 1.28mlflow==2.17.2): Not bumped — separate concern from Kubeflow upgrade