feat(ui): workspace-level skill-builder LLM config (closes #92)#96
Merged
Conversation
Pre-#92 the forge ui skill builder borrowed credentials from whichever agent the operator clicked into: it read the picked agent's forge.yaml + .env, called os.Setenv to push those values into the UI process env, applied a hardcoded model upgrade (gpt-4.1 for openai, claude-opus-4-6 for anthropic), and used them to call the LLM in-process. That model produced four problems documented in the issue: 1. Empty workspaces had no answer — the skill builder needed an agent context to borrow credentials from. 2. Cross-agent skill development was arbitrary — picking one of several agents to borrow from determined billing/model/endpoint. 3. The hardcoded codegen upgrade ignored OPENAI_BASE_URL — any agent pointed at a custom OpenAI-compatible endpoint (OpenRouter, vLLM, litellm, self-hosted Kimi/Llama) got a request for gpt-4.1 against an endpoint that didn't host it. 4. The UI process env was stomped when switching agents — agent A's OPENAI_API_KEY would leak into the process, then agent B's would overwrite it. This fix decouples the skill-builder LLM from any specific agent. What landed: * NEW package forge-ui/uiconfig — workspace-level config loader with a three-tier resolution precedence: 1. <workspace>/.forge/ui.yaml (primary, per-workspace) 2. ~/.forge/ui.yaml (fallback, operator's machine-wide) 3. The picked agent's forge.yaml + .env (deprecated; emits a deprecation warning the UI surfaces) Plus a forward-compat-ready Save function and a Validate function used by both the file loader and the HTTP write handler. * NEW EnvLookupForWorkspace + SetEnvFileValue in uiconfig — file-first env lookup layer (with OS env as fallback) and atomic writer for <workspace>/.forge/.env. SetEnvFileValue writes with mode 0600 via temp-file-and-rename, edits in place (no duplicate lines on rewrite), removes the key on empty value, and auto-generates a <workspace>/.forge/.gitignore containing ".env" so the secret file is auto-protected. Preserves operator-added gitignore lines. * Settings HTTP API at /api/settings/skill-builder (GET + PUT). PUT accepts an optional api_key field on the request body kept off the persisted YAML struct via a wrapping skillBuilderSettingsRequest type — the key value can never leak into ui.yaml. When present, the key is persisted to <workspace>/.forge/.env under the operator's api_key_env name (or the provider default). Path-less GET /api/skill-builder/provider added for first-run detection in empty workspaces (no agent picked yet). * forge-ui handler refactor — handleSkillBuilderProvider and handleSkillBuilderChat call the loader instead of reading per-agent .env / encrypted secrets. ZERO os.Setenv calls remain. Credentials are threaded as request-scoped values through LLMStreamOptions.LLM. * SkillBuilderCodegenModel reduced to a Deprecated: no-op shim that returns the configured model verbatim. The operator's chosen model is used — no hardcoded gpt-4.1 / claude-opus-4-6 override. * forge-cli/cmd/ui.go llmStreamFunc simplified from ~120 lines of env-reading + ResolveModelConfig + OAuth + codegen-upgrade logic to ~30 lines that consume opts.LLM directly. No more coreruntime.ResolveModelConfig coupling. OAuth path removed from the skill-builder code path — workspace config requires an explicit API key (the issue #83 OAuth-precedence guardrail still applies). * Frontend (forge-ui/static/app.js) — new SkillBuilderSettingsModal component (preact/htm). Provider picker, model field, optional base URL (openai-only), api_key_env override, password field for the API key value with show/hide toggle. Shows "(saved; leave blank to keep)" when a key is already persisted — submitting an empty key leaves the saved value untouched. Status banner surfaces the resolution source (workspace / user / agent_fallback / unset) and the deprecation warning when the fallback shim resolves. * docs/ui/skill-builder-llm.md — full configuration reference, three setup recipes (UI / file / API), status banner semantics table, trust-boundary explanation for why ui.yaml + .env are split. Regression tests: forge-ui/uiconfig (18 tests): Load + Save round-trip; workspace > user > agent_fallback precedence; ollama-no-key-needed; api_key_env override; envfile create + 0600 perm + auto-gitignore; in-place update; empty-value removal; comments/other-keys preservation; file-precedes-OS precedence; missing-file fallback to OS env; gitignore idempotence; operator-gitignore preservation. forge-ui (19 settings + skill-builder tests): PUT persists + 0600 perm + auto-gitignore + no leak into ui.yaml; PUT honors custom api_key_env name; PUT omitting api_key preserves existing key; PUT validation rejects bad provider / missing model; GET after PUT returns workspace source; agent-fallback returns deprecation warning + preserves configured model verbatim (no codegen upgrade); env-stomp regression — switching between two agents with different OPENAI_API_KEY values in their .env files does NOT leak either key into the UI process's env. go test -race ./forge-ui/... ./forge-cli/cmd/... ./forge-cli/runtime/... golangci-lint and gofmt clean across both modules.
Cleans up stale per-agent skill-builder doc references after #92 and applies the requested README title/lede update. docs/reference/web-dashboard.md Rewrote the Skill Builder "How It Works" section: no longer claims the builder uses the agent's own LLM; no longer describes the hardcoded gpt-4.1 / claude-opus-4-6 codegen upgrade. Now points operators at docs/ui/skill-builder-llm.md for the workspace-LLM model + 3-tier resolution. API Endpoints table adds GET /api/skill-builder/provider (path-less, for first-run detection) and GET / PUT /api/settings/skill- builder with explicit notes on the inline-api_key flow and the <workspace>/.forge/.env destination. Architecture file listing adds handlers_settings.go and the uiconfig/ package. docs/skills/skills-cli.md Replaced "uses the agent's own LLM provider" with a cross-link to the workspace-LLM page. README.md Title -> "Forge — An Open, Secure, Portable AI Agent Runtime for the Enterprise". Lede rewritten around Anthropic's Agent Skills standard and the SKILL.md -> portable agent -> deploy anywhere narrative. Broken-link check on the edited pages and the workspace LLM doc (README + 3 docs + ../ui/skill-builder-llm.md) passed.
…ation (#83) PR #93 (issue #83) made `forge init` normalize `provider: custom` -> `provider: openai` + OPENAI_BASE_URL / OPENAI_API_KEY at scaffold time for OpenAI-compatible endpoints (OpenRouter, vLLM, litellm, self-hosted Kimi/Llama). The generated forge.yaml has never carried provider: custom since that landed, but the reference docs still listed "custom" as a forge.yaml model.provider value, which is misleading. docs/reference/forge-yaml-schema.md Drop "custom" from the model.provider option list. Add a short note explaining the OpenAI-compatible endpoint recipe and that the Custom wizard option normalizes to this shape. docs/reference/cli-reference.md Keep "custom" as a valid --model-provider flag value (the CLI input still accepts it; the normalizer rewrites at scaffold time) but clarify inline that it's an alias for the OpenAI-compatible endpoint scaffold. Both edits surfaced from the sync-docs pass on PR #96 as adjacent doc-debt from #83.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #92.
Pre-#92 the
forge uiskill builder borrowed credentials from whichever agent the operator clicked into. It read the picked agent'sforge.yaml+.env, calledos.Setenvto push those values into the UI process env, applied a hardcoded model upgrade (gpt-4.1for openai,claude-opus-4-6for anthropic), and used them to call the LLM in-process. That model produced four problems:OPENAI_BASE_URL. Any agent pointed at a custom OpenAI-compatible endpoint (OpenRouter, vLLM, litellm, self-hosted Kimi/Llama) got a request forgpt-4.1against an endpoint that didn't host it.OPENAI_API_KEYwould leak into the UI process, then agent B's would overwrite it.This PR decouples the skill-builder LLM from any specific agent.
What landed
New
forge-ui/uiconfigpackage (the foundation)SkillBuilderConfig(persisted yaml/json shape) +SkillBuilderLLM(resolved runtime view).LoadSkillBuilderLLM(workspaceDir, agentDir, envLookup)with three-tier precedence:<workspace>/.forge/ui.yaml— primary, per-workspace.~/.forge/ui.yaml— fallback, operator's machine-wide.forge.yaml+.env— deprecated; emits a deprecation warning the UI surfaces.SaveSkillBuilderLLM(workspaceDir, cfg)— persists to<workspace>/.forge/ui.yaml.ValidateSkillBuilderConfig(cfg)— same rules apply to file load + HTTP write.EnvLookupForWorkspace(workspaceDir)— file-first env-lookup layer that consults<workspace>/.forge/.env, falling back to OS env. Zero process-env mutation.SetEnvFileValue(workspaceDir, name, value)— atomic writer (temp + rename) at mode0600. Edits in place (no duplicate lines on rewrite), removes the key on empty value, auto-generates<workspace>/.forge/.gitignorecontaining.env, preserves operator-added gitignore lines.Inline API key storage (Option 2 from the design discussion)
The Settings modal accepts the API key value via a password field. On save, the key is written to
<workspace>/.forge/.envunder the operator'sapi_key_envname (or the provider default). An auto-generated<workspace>/.forge/.gitignoreprotects the file from accidental commits. The key value never appears inui.yamland never appears in the GET response — only the persisted env-var name comes back. Submitting an emptyapi_keyleaves the saved value untouched (for non-key field rotation).Handler refactor
handleSkillBuilderProviderandhandleSkillBuilderChatcall the loader instead of reading per-agent.envor encrypted secrets. Zeroos.Setenvcalls remain. Credentials are threaded as request-scoped values viaLLMStreamOptions.LLM.SkillBuilderCodegenModelreduced to aDeprecated:no-op shim that returns the configured model verbatim. The operator's chosen model is used — no hardcodedgpt-4.1/claude-opus-4-6override.forge-cli/cmd/ui.gollmStreamFuncsimplified from ~120 lines of env-reading +ResolveModelConfig+ OAuth + codegen-upgrade logic to ~30 lines that consumeopts.LLMdirectly.Settings HTTP API
GET /api/settings/skill-builder— current config + metadata for the form (source,has_key, providers list).PUT /api/settings/skill-builder— validates + persistsui.yaml; persistsapi_keyto.forge/.envif provided. The PUT body uses a wrappingskillBuilderSettingsRequesttype that keepsapi_keyoff the persisted YAML struct, so the value can never leak intoui.yaml.GET /api/skill-builder/provider(path-less) for first-run detection in empty workspaces.Frontend
SkillBuilderSettingsModalcomponent (preact/htm). Provider picker, model field, optional base URL (openai-only),api_key_envoverride, password field for the API key value with show/hide toggle.workspace/user/agent_fallback/unset) and the deprecation warning when the fallback shim resolves.Regression tests
forge-ui/uiconfig— 18 tests:api_key_envoverrideforge-ui— 19 settings + skill-builder tests:ui.yamlapi_key_envnameapi_keypreserves existing keysource: workspaceOPENAI_API_KEYvalues in their.envfiles does NOT leak either key into the UI process's env. Pre-feat(ui): workspace-level LLM config for skill builder (decouple build-time codegen from per-agent runtime credentials) #92 this would have failed.Gates
go test -race -count=1 ./forge-ui/... ./forge-cli/cmd/... ./forge-cli/runtime/...— greengolangci-lint run— 0 issues across both modulesgofmt -l forge-ui/ forge-cli/— cleannode -c forge-ui/static/app.js— cleanDecoupling rules the implementation pins
These are guaranteed by the tests above:
os.Setenvon theforge uiprocess.SkillBuilderCodegenModelmapping is gone — the operator's chosen model is used verbatim.<workspace>/.forge/.env(mode 0600, gitignored) and never inui.yaml.Acceptance criteria
mkdir empty && forge ui --dir emptyand use the skill builder before scaffolding any agent.forge ui.provider: openai+OPENAI_BASE_URL) doesn't cause the skill builder to issue requests forgpt-4.1against that endpoint.forge uiprocess's env (TestSkillBuilderProvider_DoesNotStompProcessEnvBetweenAgentspins this).Out of scope (deliberately deferred)
forge ui set-llm— listed as optional in the issue body. Backend has the full API + persistence; CLI helper is a fast follow-up.~/.forge/secrets.encAES-256-GCM keyring (gated byFORGE_PASSPHRASE) is the obvious next step for operators who don't trust plain.env. The current implementation matchesforge init's.envpattern; encrypted-store integration is a fast follow-up.Test plan
go test -race -count=1 ./forge-ui/... ./forge-cli/cmd/... ./forge-cli/runtime/...greengolangci-lint runcleangofmt -lcleanTestPutSkillBuilderSettings_PersistsAndEchoes+TestPutSkillBuilderSettings_PersistsAPIKeyToEnvFile)OPENAI_API_KEY(pinned by env-stomp regression test)gpt-4.1(pinned byTestSkillBuilderProvider_AgentFallback_PreservesConfiguredModel)