Skip to content

fix(types): add top_logprobs to LogprobsPart to match API response shape#450

Open
xodn348 wants to merge 1 commit into
togethercomputer:mainfrom
xodn348:fix/logprobs-top-logprobs-type
Open

fix(types): add top_logprobs to LogprobsPart to match API response shape#450
xodn348 wants to merge 1 commit into
togethercomputer:mainfrom
xodn348:fix/logprobs-top-logprobs-type

Conversation

@xodn348
Copy link
Copy Markdown

@xodn348 xodn348 commented May 21, 2026

Have you read the Contributing Guidelines?

Issue #443

Describe your changes

LogprobsPart (in src/together/types/common.py) was missing a typed field for top_logprobs, even though the Together API returns it alongside tokens and token_logprobs when logprobs=N is requested.

Because BaseModel uses extra="allow", the data was silently stored as an untyped extra attribute. Two problems result:

  1. No static type information — IDEs and mypy treat response.choices[0].logprobs.top_logprobs as Any, losing all type safety.
  2. PydanticSerializationUnexpectedValue warnings on model_dump() — when serializing, Pydantic sees a list value for a field that has no declared schema and emits a warning.

The fix adds:

top_logprobs: List[Dict[str, float]] | None = None

to LogprobsPart, matching the actual API shape: one dict per token mapping each candidate token string to its log-probability.

Three unit tests are added to tests/unit/test_logprobs_type.py:

  • Round-trip correctness: verify top_logprobs is parsed as list[dict]
  • Warning-free model_dump() with warnings-as-errors enabled
  • Default None when field is absent

Summary

Added top_logprobs: List[Dict[str, float]] | None = None to LogprobsPart so that the field is fully typed, IDE-visible, and serializes without Pydantic warnings. Three regression tests cover the happy path, warning-free serialization, and the optional-field default.

Issue

Fixes #443

Local verification

$ PYTHONPATH=/tmp/together-python/src python -m pytest tests/unit/test_logprobs_type.py -v

============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/together-python
configfile: pyproject.toml
collected 3 items

tests/unit/test_logprobs_type.py::test_logprobs_part_top_logprobs_is_list PASSED [ 33%]
tests/unit/test_logprobs_type.py::test_logprobs_part_model_dump_no_warning PASSED [ 66%]
tests/unit/test_logprobs_type.py::test_logprobs_part_top_logprobs_optional PASSED [100%]

$ PYTHONPATH=/tmp/together-python/src python -m pytest tests/unit/ -q

============ 6 failed, 177 passed, 14 warnings, 26 errors in 0.70s =============
(6 failures and 26 errors are pre-existing on upstream/main, confirmed by running
the same suite on the unpatched branch)

$ pre-commit run --files src/together/types/common.py tests/unit/test_logprobs_type.py

fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
check docstring is first.................................................Passed
check for merge conflicts................................................Passed
black....................................................................Passed
mypy.....................................................................Passed
=== LOCAL_TEST_PASSED ===

Risk

Purely additive type annotation — no runtime behaviour change. The field defaults to None, so all existing callers that don't use top_logprobs are unaffected. Users who do access response.choices[0].logprobs.top_logprobs already receive the correct list value (stored via extra="allow"); this PR simply makes the field explicit and statically typed.

The Together API returns `top_logprobs` as a List[Dict[str, float]] (one
dict per token mapping token string to log-probability), but LogprobsPart
had no field for it.  Since BaseModel uses extra="allow", the data was
stored as an extra field with no declared type, causing:
  - Missing type information for static analysis / IDE autocompletion
  - PydanticSerializationUnexpectedValue warnings on model_dump()

Add `top_logprobs: List[Dict[str, float]] | None = None` to LogprobsPart
and add three unit tests covering the list round-trip, warning-free
serialization, and the default-None path.

Fixes togethercomputer#443

Signed-off-by: xodn348 <xodn348@tamu.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LogProbs.top_logprobs typed as Dict but API returns List[Dict]

1 participant