Skip to content

AIServicePlugin should auto-detect and initialize LLM provider based on environment variables #1070

@hotlong

Description

@hotlong

Problem

Currently, only the CLI server (packages/cli/src/commands/serve.ts) contains logic to auto-detect LLM providers (OpenAI, Anthropic, Google, Vercel AI Gateway) via environment variables and construct a VercelLLMAdapter accordingly. In non-CLI deployments (e.g., Vercel via apps/studio/server/index.ts), AIServicePlugin is always constructed with no adapter supplied, so it falls back to the test-only MemoryLLMAdapter.

This leads to inconsistent behavior: production deployments risk silently using only the echo AI, unless every entrypoint duplicates the CLI detection logic.

Proposal

Refactor and move the entire LLM provider auto-detection and adapter selection logic into the core AIServicePlugin. When no adapter is supplied in options, AIServicePlugin should:

  1. Detect LLM provider preference and settings from standard environment variables, in priority order:
    • AI_GATEWAY_MODEL → use @ai-sdk/gateway via VercelLLMAdapter
    • OPENAI_API_KEY → use @ai-sdk/openai via VercelLLMAdapter
    • ANTHROPIC_API_KEY → use @ai-sdk/anthropic via VercelLLMAdapter
    • GOOGLE_GENERATIVE_AI_API_KEY → use @ai-sdk/google via VercelLLMAdapter
  2. Fallback to the MemoryLLMAdapter only if no config is available
  3. Log which provider is selected or a warning if falling back to memory adapter

Accept dynamic import failures as a soft error: optional peer dependencies should not break boot, just emit a warning and try the next provider.

Benefits

  • No need to repeat auto-detect logic for every serverless or containerized entrypoint
  • All use cases (CLI, Vercel, Docker, custom node) consistently get best-available real AI
  • One place to extend with new providers in the future
  • Reduced operational risk from silent fallback

Tasks

  • Refactor LLM adapter auto-detection into AIServicePlugin (private method during init)
  • Remove redundant LLM detection logic from CLI serve.ts
  • Add logging to report which adapter/provider is active
  • Add unit test (mock env) for provider detection logic
  • Update roadmap and changelog

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions