Skip to content

experimental/ssh: add --environment flag to ssh connect#5706

Open
anton-107 wants to merge 1 commit into
mainfrom
environment-flag-implement-https-databricks-20
Open

experimental/ssh: add --environment flag to ssh connect#5706
anton-107 wants to merge 1 commit into
mainfrom
environment-flag-implement-https-databricks-20

Conversation

@anton-107

Copy link
Copy Markdown
Contributor

Changes

Adds a user-facing --environment flag to databricks ssh connect that maps to compute.Environment.BaseEnvironment in the serverless submit path. It accepts three input forms:

  • an env.yaml path (leading /),
  • a workspace-base-environments/... resource ID,
  • a bare display name, resolved via Environments.ListWorkspaceBaseEnvironmentsAll.

Supporting behavior:

  • Rejects --environment together with --environment-version (mutually exclusive in the SDK) or with --cluster (serverless only).
  • Serializes the flag through ToProxyCommand so it survives serverless reconnect (quoted, since paths/display names may contain spaces).
  • Folds the environment into the default connection name so distinct environments map to distinct servers, mirroring how --accelerator is handled.
  • Adds an environment slot to SshTunnelEvent telemetry.

Why

Implements DECO-27423. Serverless SSH users could pin an environment version but not point the session at a custom base environment (their own env.yaml or a workspace base environment). No SDK bump is needed — databricks-sdk-go already exposes BaseEnvironment and the workspace-base-environments listing API.

Because a serverless server bakes in its environment at startup, the environment is part of the server's identity; folding it into the default connection name avoids silently reusing a server started with a different environment. The remaining explicit---name reuse case and the interactive-shell Python path are tracked as follow-ups (DECO-27498, DECO-27499).

Tests

  • Unit tests for Validate (both incompatibility errors, valid serverless/GPU cases), ToProxyCommand round-trip, default-connection-name differentiation, and resolveBaseEnvironment (path/ID passthrough, display-name resolution, not-found, ambiguous) using the SDK mock client.
  • ./task test-exp-ssh (unit + TestAccept/ssh acceptance) passes; ./task lint-q clean.
  • Manually verified all three input forms end-to-end against a dogfood workspace with a base environment that installs a distinctive package.

This pull request and its description were written by Isaac.

Adds a user-facing --environment flag mapping to
compute.Environment.BaseEnvironment in the serverless submit path. It
accepts an env.yaml path, a workspace-base-environments resource ID, or a
bare display name (resolved via ListWorkspaceBaseEnvironments).

- Rejects --environment with --environment-version or --cluster.
- Serializes through ToProxyCommand so it survives serverless reconnect.
- Folds the environment into the default connection name so distinct
  environments map to distinct servers (mirrors --accelerator).
- Adds an environment slot to SshTunnelEvent telemetry.

DECO-27423

Co-authored-by: Isaac
@anton-107 anton-107 temporarily deployed to test-trigger-is June 24, 2026 14:55 — with GitHub Actions Inactive
@anton-107 anton-107 temporarily deployed to test-trigger-is June 24, 2026 14:55 — with GitHub Actions Inactive
@anton-107 anton-107 marked this pull request as ready for review June 24, 2026 15:08
@anton-107 anton-107 requested review from ilia-db and rugpanov June 24, 2026 15:08
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/libs/telemetry/ - needs approval

Files: libs/telemetry/protos/ssh_tunnel.go
Suggested: @simonfaltum
Also eligible: @Divyansh-db, @renaudhartert-db, @hectorcast-db, @parthban-db, @tanmay-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

General files (require maintainer)

4 files changed
Based on git history:

  • @pietern -- recent work in experimental/ssh/internal/client/

Any maintainer (@andrewnester, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d4d1e64

Run: 28107667018

Env ❌​FAIL 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 7 13 244 1024 5:55
💚​ aws windows 7 13 246 1022 6:36
💚​ aws-ucws linux 7 13 334 940 7:18
💚​ aws-ucws windows 7 13 336 938 7:39
❌​ azure linux 3 3 15 242 1022 16:26
❌​ azure windows 2 6 1 15 241 1020 18:05
❌​ azure-ucws linux 3 3 15 334 936 10:49
❌​ azure-ucws windows 3 1 15 338 934 10:49
💚​ gcp linux 1 15 246 1024 5:14
💚​ gcp windows 1 15 248 1022 6:03
29 interesting tests: 13 SKIP, 7 flaky, 6 RECOVERED, 3 FAIL
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🔄​ TestAccept 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 🔄​f 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/deployment/bind/alert 🙈​s 🙈​s 🙈​s 🙈​s ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/generate/alert ✅​p ✅​p ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestSyncIncrementalFileOverwritesFolder ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
❌​ TestFetchRepositoryInfoAPI_FromRepo ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ✅​p ✅​p
❌​ TestFetchRepositoryInfoAPI_FromRepo/root ✅​p ✅​p ✅​p ✅​p ❌​F 🔄​f ❌​F ❌​F ✅​p ✅​p
❌​ TestFetchRepositoryInfoAPI_FromRepo/subdir ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ✅​p ✅​p
Top 17 slowest tests (at least 2 minutes):
duration env testname
4:54 azure windows TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
4:34 azure windows TestExportDirWithOverwriteFlag
4:12 azure windows TestFilerWorkspaceFilesExtensionsDelete
3:47 azure linux TestAccept/bundle/run_as/job_default/DATABRICKS_BUNDLE_ENGINE=direct
3:24 azure windows TestAccept/bundle/run_as/job_default/DATABRICKS_BUNDLE_ENGINE=direct
3:21 azure windows TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
3:21 gcp windows TestAccept
3:19 aws-ucws windows TestAccept
3:08 aws windows TestAccept
3:00 azure-ucws windows TestAccept
2:54 azure windows TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform
2:48 azure windows TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct
2:44 azure windows TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:20 azure linux TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 azure windows TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 azure linux TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
2:00 azure linux TestLock

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants