Skip to content

Fix tool-call fallback for structured completion content#2278

Open
suger-m wants to merge 2 commits intomicrosoft:mainfrom
suger-m:fix/issue-2263-tool-call-json-fallback
Open

Fix tool-call fallback for structured completion content#2278
suger-m wants to merge 2 commits intomicrosoft:mainfrom
suger-m:fix/issue-2263-tool-call-json-fallback

Conversation

@suger-m
Copy link

@suger-m suger-m commented Mar 12, 2026

Summary

  • Fix structured response parsing when provider responses return empty assistant content and place JSON in function tool-call arguments.
  • Add fallback in LLMCompletionResponse.content to read tool_calls[].function.arguments when message content is empty.
  • Update gather_completion_response and gather_completion_response_async to use response.content.
  • Add unit regression tests for content fallback, gather helpers, and structured parsing.

Issue

Fixes #2263

Validation

  • uvx ruff check packages/graphrag-llm/graphrag_llm/types/types.py packages/graphrag-llm/graphrag_llm/utils/gather_completion_response.py tests/unit/utils/test_completion_response.py
  • python -m pytest tests/unit/utils/test_completion_response.py -q

suger-m added 2 commits March 12, 2026 16:29
Handles LiteLLM structured-output fallbacks that return JSON in function tool-call arguments with empty assistant content.\n\n- Add fallback in LLMCompletionResponse.content to return function tool-call arguments\n- Update gather_completion_response helpers to use response.content\n- Add regression tests for content gathering and structured parsing via tool calls\n\nFixes microsoft#2263
@suger-m suger-m requested a review from a team as a code owner March 12, 2026 08:58
@suger-m
Copy link
Author

suger-m commented Mar 12, 2026

@microsoft-github-policy-service agree

@microsoft-github-policy-service

@suger-m the command you issued was incorrect. Please try again.

Examples are:

@microsoft-github-policy-service agree

and

@microsoft-github-policy-service agree company="your company"

@suger-m
Copy link
Author

suger-m commented Mar 12, 2026

@microsoft-github-policy-service agree company="your company"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: model_supports_json is no longer supported on v3, breaking custom model setups

1 participant