Skip to content

feat: context window info #76

Description

@naqerl

Problem

Version: v0.6.1

Cptr uses hardcoded openai models info (except anthropic) and compaction threshold as a fixed amount of tokens (configurable).

Reasoning

Resource like https://models.dev/api.json provides a complete info about popular models (utilized by opencode). Using it will eliminate wrong context size in the UI (e.g. 216%) and make it much easier to use a fraction of the context for compaction over configurable tokens amount e.g. 0.6 instead of 80K, as almost any model would have a context window info.

Proposed solution

For portability, it could be fetched on startup and stored in the app.db with TTL, which will be used by the background task for refreshing.

Requested deliverables

  1. Valid context window for the popular modules, not onlyopenai & antropic
  2. Using context fraction as a threshold for compaction with fixed amount as a fallback or ability to choose from these 2 option (yet the fraction is a more generic version of the fixed amount, so for me this replacement is straightforward)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions