Skip to content

Unpin KFP SDK for Kubeflow 1.11.0 and clean up residual dex-based examples#44

Open
tmvfb wants to merge 6 commits intomainfrom
feature/kubeflow-1.11.0
Open

Unpin KFP SDK for Kubeflow 1.11.0 and clean up residual dex-based examples#44
tmvfb wants to merge 6 commits intomainfrom
feature/kubeflow-1.11.0

Conversation

@tmvfb
Copy link
Copy Markdown
Member

@tmvfb tmvfb commented Feb 27, 2026

Summary

Unpin all KFP SDK pins to match KFP backend 2.15.0 (provided by our custom notebooks!) in Kubeflow v1.11.0. The hint at version is still there in the notebook, but it's no longer installed, to increase compatibility between our images and platform components.

Additionally, remove Dex-based remote submission utilities that are no longer functional.

KFP SDK bumps:

  • pipelines/minimal-container-components: kfp 2.0.1 → Notebook's version (2.15.0 hint)
  • pipelines/lightweight-python-package: kfp[all] 2.12.1 → Notebook's version (2.15.0 hint)
  • pipelines/mobile-price-classification: same
  • mlflow/mlflow-kfp-example.ipynb: same

Dex/auth cleanup:

  • Removed utils/auth_session.py and the entire utils/ directory — Dex username/password session cookie flow no longer works with Keycloak/oauth2-proxy auth
  • Removed submit-remote.py from minimal-container-components and the associated "Preparing for Remote KFP Connection" README section — remote submission is not functional with the current JWT auth setup
  • Removed unused requirements.txt files from pipeline examples (not referenced by any Dockerfile, script, or CI config; notebooks install kfp inline via pip magic commands)

For remote pipeline submission, refer to the pipe-fiction example which documents the Keycloak OIDC client flow.

Context

Part of the Kubeflow 1.11.0 upgrade. Companion PRs:

  • prokube/prokube — main platform upgrade (branch feature/kubeflow-1.11.0)
  • prokube/prokube-images — notebook image and MLflow server updates (PR #34)

What was NOT changed (and why)

  • KServe examples (serving/): Already pin kserve>=0.15.2, compatible with KServe v0.15.2 in Kubeflow 1.11.0
  • Katib example (hparam-tuning/): Uses kubeflow.org/v1beta1 API (unchanged) with batch/v1 Job (no Training Operator dependency)
  • Istio VirtualService examples: Use v1/v1beta1 APIs still served in Istio 1.28
  • MLflow version in notebook (mlflow==2.17.2): Not bumped — separate concern from Kubeflow upgrade

@geier
Copy link
Copy Markdown
Contributor

geier commented Mar 3, 2026

what about kubeflow-examples/pipelines/lightweight-components/mobile-price-classifications.ipynb ?

@tmvfb
Copy link
Copy Markdown
Member Author

tmvfb commented Mar 3, 2026

what about kubeflow-examples/pipelines/lightweight-components/mobile-price-classifications.ipynb ?

good point, updated

tmvfb added 2 commits April 7, 2026 15:03
- pipelines/minimal-container-components: kfp 2.0.1 -> 2.15.0
- pipelines/lightweight-python-package: kfp[all] 2.12.1 -> 2.15.0
- mlflow/mlflow-kfp-example.ipynb: kfp[all] 2.9.0 -> 2.15.0
@tmvfb tmvfb force-pushed the feature/kubeflow-1.11.0 branch from 2c32cb6 to 00b8a15 Compare April 7, 2026 13:11
tmvfb added 3 commits April 7, 2026 15:21
The cluster uses Keycloak/oauth2-proxy/Dex JWT auth which cannot be
bypassed with the basic Dex username/password flow in auth_session.py.
Remote submission is not functional with this setup.

Also remove the now-irrelevant 'Preparing for Remote KFP Connection'
README section.
These files are not referenced by any Dockerfile, script, or CI config.
The notebooks install kfp inline via pip magic commands, and the submit
scripts import kfp directly from whatever environment they run in.
@tmvfb tmvfb force-pushed the feature/kubeflow-1.11.0 branch from 8017ada to 54ce95d Compare April 7, 2026 13:31
@tmvfb tmvfb changed the title Bump KFP SDK to 2.15.0 for Kubeflow 1.11.0 Bump KFP SDK to 2.15.0 for Kubeflow 1.11.0 and cleanup residual dex-based examples Apr 7, 2026
@tmvfb tmvfb changed the title Bump KFP SDK to 2.15.0 for Kubeflow 1.11.0 and cleanup residual dex-based examples Bump KFP SDK to 2.15.0 for Kubeflow 1.11.0 and clean up residual dex-based examples Apr 7, 2026
All prokube deployments now use Keycloak/oauth2-proxy — the Dex
username/password session cookie approach no longer works.

For remote pipeline submission, refer to the pipe-fiction example
which documents the Keycloak OIDC client flow.
@tmvfb tmvfb force-pushed the feature/kubeflow-1.11.0 branch from 54ce95d to 5c94bdd Compare April 7, 2026 13:34
@tmvfb tmvfb requested a review from Copilot April 7, 2026 13:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aligns the Kubeflow Pipelines example content with the Kubeflow 1.11.0 platform upgrade by removing Dex-based remote-submission utilities and adjusting example dependency guidance around the KFP SDK.

Changes:

  • Removed the Dex auth_session.py helper and its docs (no longer compatible with current auth setup).
  • Removed the minimal-container-components remote submission script and the associated README instructions.
  • Removed per-example requirements.txt files and updated notebooks to reference (commented) KFP 2.15.0 availability in prokube notebook images.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pipelines/utils/README.md Deleted Dex auth-session documentation.
pipelines/utils/auth_session.py Deleted Dex cookie/session helper implementation.
pipelines/minimal-container-components/submit-remote.py Deleted remote submission script that depended on Dex auth flow.
pipelines/minimal-container-components/requirements.txt Removed pinned KFP requirement for this example.
pipelines/minimal-container-components/README.md Removed remote-submission guidance; now only documents in-cluster submission.
pipelines/lightweight-python-package/requirements.txt Removed pinned kfp[all] requirement for this example.
pipelines/lightweight-components/mobile-price-classifications.ipynb Added commented dependency note referencing KFP 2.15.0.
mlflow/mlflow-kfp-example.ipynb Replaced active pip install with commented dependency note referencing KFP 2.15.0.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pipelines/minimal-container-components/README.md
Comment thread mlflow/mlflow-kfp-example.ipynb
Comment thread pipelines/lightweight-components/mobile-price-classifications.ipynb
@tmvfb tmvfb changed the title Bump KFP SDK to 2.15.0 for Kubeflow 1.11.0 and clean up residual dex-based examples Unpin KFP SDK for Kubeflow 1.11.0 and clean up residual dex-based examples Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants