Chore[Docs-AI]: use Datadog Feature Flags when streaming quality degrade#35336
Conversation
onceLearner
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5f36d9340
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const USE_LEGACY_MODEL_FLAG_KEY = 'docs-ai-use-legacy-model'; | ||
| const DISABLE_STREAMING_FLAG_KEY = 'docs-ai-disable-streaming'; |
There was a problem hiding this comment.
This change renames both Datadog flag keys to docs-ai-* and drops the previously used DOCS_AI_* names without a fallback check. If the existing flags are still configured under the old keys (as in prior code), resolveFlags() will never enable the non-streaming path or legacy model, so the operational kill switch for streaming degradation can silently fail until flags are recreated/renamed in Datadog.
Useful? React with 👍 / 👎.
StefonSimmons
left a comment
StefonSimmons
left a comment
There was a problem hiding this comment.
same prompt for preview and live site produces the same results however the preview has an uncorrupted response 👍
2 participants
What does this PR do? What is the motivation?
Non-streaming fallback with feature flag: Added a
DOCS_AI_DISABLE_STREAMINGfeature flag to switch Docs AI responses from streaming (SSE) to a single synchronous request. This mitigates a known upstream issue where streamed chunks arrive with dropped/corrupted tokens — worsened by faster model output (GPT-5.2) and higher traffic. When streaming is disabled, users see a rotating status indicator ("Searching documentation…", "Reviewing relevant pages…", etc.) while the full response is generated.Merge instructions
Merge readiness:
Note: The
DOCS_AI_DISABLE_STREAMINGflag should be enabled in Datadog Feature Flags when streaming quality degrades. It can be toggled off once the upstream fix is released.AI assistance
Additional notes
https://docs-staging.datadoghq.com/reda.elissati/temp-disable-streaming-docs-ai/
The non-streaming path returns the complete Docs AI response in a single request. Response quality and content are identical — only the delivery method changes. The feature flag allows toggling between modes without a deploy.