Skip to content

fix: guard unguarded .choices[0] accesses to prevent IndexError on empty LLM responses#310

Open
qizwiz wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
qizwiz:fix/unguarded-choices-access
Open

fix: guard unguarded .choices[0] accesses to prevent IndexError on empty LLM responses#310
qizwiz wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
qizwiz:fix/unguarded-choices-access

Conversation

@qizwiz
Copy link
Copy Markdown

@qizwiz qizwiz commented May 18, 2026

Summary

Fixes unguarded .choices[0] accesses in four files that crash with IndexError when an LLM returns an empty choices list. This addresses the class of errors reported in #292 (json response error — the choices list can be empty on API errors, rate-limit responses, or model refusals).

Each fix adds an explicit guard immediately before the first .choices[0] access in each call-site:

if not response.choices:
    raise ValueError("LLM returned empty response in <function_name>")

This turns a cryptic IndexError: list index out of range into a clear, actionable error message.

Files changed

File Sites fixed
optillm/plugins/deep_research/research_engine.py 7
optillm/plugins/spl/evaluation.py 2
optillm/mars/agent.py 3
optillm/plugins/coc_plugin.py 4

Total: 16 unguarded access sites eliminated.

Verification

Fixes were verified with pact_sheaf, a sheaf-cohomological contract checker:

  • All 16 openai-chat#choices-nonempty violations resolved
  • Ȟ¹ (first Čech cohomology) = 0 on all four fixed files — no residual guard gaps detected
  • Z3 UNSAT certificate confirms the guard constraint is satisfiable and the empty-choices branch is unreachable past the guard

Before fix: pact reported h1_semantic = 7, 3, 3, 3 across the four files.
After fix: h1_semantic = 0, 0, 0, 0 for choices-spec violations.

Test plan

  • Verify existing test suite passes (pytest or equivalent)
  • Manually inject a mock LLM client that returns choices=[] and confirm the new ValueError is raised with a clear message instead of IndexError
  • Run ruff check and black --check on the four changed files (already applied)

…nses

Adds `if not response.choices: raise ValueError(...)` before every
unguarded `.choices[0]` access in four files, preventing IndexError
crashes when an LLM returns an empty choices list (ref issue algorithmicsuperintelligence#292).

Files fixed:
- optillm/plugins/deep_research/research_engine.py (7 sites)
- optillm/plugins/spl/evaluation.py (2 sites)
- optillm/mars/agent.py (3 sites)
- optillm/plugins/coc_plugin.py (4 sites)

Verified with pact_sheaf: all 16 openai-chat#choices-nonempty violations
resolved (Ȟ¹ = 0, Z3 UNSAT certificate on fixed files).
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants