Dynamic provider discovery from llmspy pod

[bussyjd]

Summary

Remove the hardcoded providerEnvKeys map (anthropic/openai only) and instead query the running llmspy pod at runtime for provider metadata. This means any provider llmspy supports (zai, deepseek, google, mistral, groq, etc.) works with obol model setup without code changes.

Before

var providerEnvKeys = map[string]string{
    "anthropic": "ANTHROPIC_API_KEY",
    "openai":    "OPENAI_API_KEY",
}

Adding a new provider required editing Go code, rebuilding, and releasing.

After

Provider info is discovered dynamically by executing a Python script inside the llmspy pod that reads providers.json — the same file llmspy itself uses. The CLI becomes a thin layer over llmspy's own provider registry.

What changed

internal/model/model.go

• Removed providerEnvKeys hardcoded map
• Added getProviderEnvKey() — queries the llmspy pod via kubectl exec for a single provider's env var name
• Added GetAvailableProviders() — returns all providers that accept API keys, discovered from the running pod
• Refactored ConfigureLLMSpy() to use dynamic env var lookup instead of the hardcoded map
• Refactored GetProviderStatus() to cross-reference dynamic providers with ConfigMap and Secret state
• Extracted pure functions (parseProviderEnvKey, parseAvailableProviders, buildProviderStatus, patchLLMsJSON) from kubectl-calling wrappers for testability

cmd/obol/model.go

• promptModelConfig() now accepts cfg and queries llmspy for available providers dynamically
• Interactive menu shows all discovered providers instead of hardcoded Anthropic/OpenAI
• obol model status table widened to accommodate longer provider names
• Added hint: Run 'obol model setup' to configure a provider.

internal/model/model_test.go (new)

• 23 unit tests covering all extracted pure functions:
  • TestParseProviderEnvKey — 5 cases (valid output, whitespace, empty, unknown)
  • TestParseAvailableProviders — 6 cases (empty, single, multiple, malformed lines)
  • TestBuildProviderStatus — 6 cases (full status, ollama injection, missing env vars, invalid JSON)
  • TestPatchLLMsJSON — 6 cases (enable, create new, idempotent, preserve fields, invalid JSON)

How it works

obol model setup --provider=zai --api-key=xxx
  │
  ├── kubectl exec deploy/llmspy -n llm -- python3 -c "read providers.json, print env var"
  │   → "ZHIPU_API_KEY"
  │
  ├── kubectl patch secret llms-secrets -n llm (set ZHIPU_API_KEY=xxx)
  ├── kubectl patch configmap llmspy-config -n llm (enable zai in llms.json)
  └── kubectl rollout restart deploy/llmspy -n llm

Interactive mode (obol model setup with no flags) now shows all providers from llmspy:

Available providers:
  [1] Anthropic (anthropic)
  [2] DeepSeek (deepseek)
  [3] Google (google)
  [4] OpenAI (openai)
  [5] Z.AI (zai)
  ...

Test plan

• go build ./cmd/obol compiles cleanly
• go test ./... passes (23 new tests + existing tests)
• obol model setup --provider=zai --api-key=<key> with running cluster
• obol model status shows dynamically discovered providers
• obol model setup (interactive) lists all llmspy providers

[OisinKyne]

will merge this in after a 0.3.1 tag, don't want to add more problems to critical path again today

[OisinKyne]

merge conflict to fix