Skip to content

local-tokenizer does not support gemini-3.1-flash-lite #2602

@jayjk13

Description

@jayjk13

local-tokenizer does not support gemini-3.1-flash-lite

Same class of issue as #1784 (gemini-3-pro-preview) and #1972 (gemini-3-flash-preview).

Environment details

  • Programming language: Python
  • OS: macOS
  • Language runtime version: 3.13.2
  • Package version: google-genai==2.8.0 (latest on PyPI at time of report)

Steps to reproduce

from google.genai.local_tokenizer import LocalTokenizer
from google.genai.types import CountTokensResult

tokenizer = LocalTokenizer(model_name="gemini-3.1-flash-lite")
result: CountTokensResult = tokenizer.count_tokens("test")

Actual behavior

ValueError: Model gemini-3.1-flash-lite is not supported. Supported models: gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.0-flash, gemini-2.0-flash-lite, gemini-2.5-pro-preview-06-05, gemini-2.5-pro-preview-05-06, gemini-2.5-pro-exp-03-25, gemini-live-2.5-flash, gemini-2.5-flash-preview-05-20, gemini-2.5-flash-preview-04-17, gemini-2.5-flash-lite-preview-06-17, gemini-2.0-flash-001, gemini-2.0-flash-lite-001, gemini-3-pro-preview.

Expected behavior

LocalTokenizer(model_name="gemini-3.1-flash-lite") should work, since:

  1. gemini-3.1-flash-lite is a supported Vertex model (added to SDK model enums in v2.1.0).
  2. client.models.count_tokens(model="gemini-3.1-flash-lite", ...) works via the API.
  3. Sibling Gemini 3 models like gemini-3-pro-preview already work with LocalTokenizer on 2.8.0.

Metadata

Metadata

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions