OVMS 2026.1-gpu fails to load OpenVINO/gemma-4-E4B-it-int4-ov with Unsupported 'gemma4' VLM model type

[jthakare17]

Describe the bug
OVMS fails to load the VLM model OpenVINO/gemma-4-E4B-it-int4-ov and exits during initialization with:

Unsupported 'gemma4' VLM model type

This happens when running openvino/model_server:2026.1-gpu.

The behavior appears inconsistent with the current openvinotoolkit/openvino.genai source, where gemma4 is explicitly recognized in VLM config parsing.

To Reproduce

1. Prepare a model repository containing:

   • models/OpenVINO/gemma-4-E4B-it-int4-ov/
   • The model used is OpenVINO/gemma-4-E4B-it-int4-ov

2. Launch OVMS:

   docker run -it --rm \
     -p 8000:8000 \
     --device /dev/dri \
     --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
     -v $(pwd)/models:/models:ro \
     openvino/model_server:2026.1-gpu \
     --rest_port 8000 \
     --model_name OpenVINO/gemma-4-E4B-it-int4-ov \
     --model_path ./models/OpenVINO/gemma-4-E4B-it-int4-ov/

3. No client request is needed. The failure happens during model initialization at server startup.

4. Observe the error:

   Error during llm node initialization ...
   Unsupported 'gemma4' VLM model type
   

Expected behavior
OVMS should recognize model_type: "gemma4" and initialize the model successfully, or the image/docs should clearly state that Gemma4 is not supported by this OVMS image.

Logs
Startup version info:

[2026-05-05 08:15:34.612][1][serving][info][server.cpp:88] OpenVINO Model Server 2026.1.0.72cc06244
[2026-05-05 08:15:34.612][1][serving][info][server.cpp:89] OpenVINO backend 2026.1.0-21367-63e31528c62-releases/2026/1
[2026-05-05 08:15:34.612][1][serving][info][server.cpp:92] OpenVINO GenAI backend 2026.1.0.0-2957-1dabb8c2255

Relevant failure logs:

[2026-05-05 08:06:22.232][1][modelmanager][info][servable_initializer.cpp:448] Initializing Visual Language Model Continuous Batching servable
[2026-05-05 08:06:23.062][1][serving][error][servable_initializer.cpp:214] Error during llm node initialization for models_path: //./models/OpenVINO/gemma-4-E4B-it-int4-ov/./ exception: Exception from ../../../../../repos/openvino.genai/src/cpp/src/visual_language/vlm_config.cpp:34:
Unsupported 'gemma4' VLM model type

[2026-05-05 08:06:23.062][1][modelmanager][error][servable_initializer.cpp:453] Error during LLM node resources initialization: The LLM Node resource initialization failed
[2026-05-05 08:06:23.062][1][serving][error][mediapipegraphdefinition.cpp:474] Failed to process LLM node graph OpenVINO/gemma-4-E4B-it-int4-ov
[2026-05-05 08:06:23.062][1][modelmanager][info][pipelinedefinitionstatus.hpp:59] Mediapipe: OpenVINO/gemma-4-E4B-it-int4-ov state changed to: LOADING_PRECONDITION_FAILED after handling: ValidationFailedEvent:
[2026-05-05 08:06:23.062][1][modelmanager][error][modelmanager.cpp:184] Couldn't start model manager
[2026-05-05 08:06:23.062][1][serving][error][servablemanagermodule.cpp:58] ovms::ModelManager::Start() Error: The LLM Node resource initialization failed

There is no client log because the server fails before serving requests.

Configuration

1. OVMS version:

   • openvino/model_server:2026.1-gpu
   • Logged as: OpenVINO Model Server 2026.1.0.72cc06244

2. OVMS config.json file:

   • Not used
   • Model was provided via CLI arguments:
     • --model_name OpenVINO/gemma-4-E4B-it-int4-ov
     • --model_path ./models/OpenVINO/gemma-4-E4B-it-int4-ov/

3. CPU / accelerator versions if applicable:

   • OpenVINO backend: 2026.1.0-21367-63e31528c62-releases/2026/1
   • OpenVINO GenAI backend: 2026.1.0.0-2957-1dabb8c2255
   • Device exposed in container: GPU via /dev/dri

4. Model repository directory structure:

   models/
   └── OpenVINO/
       └── gemma-4-E4B-it-int4-ov/
           ├── config.json
           ├── preprocessor_config.json
           ├── tokenizer.json
           ├── tokenizer_config.json
           ├── openvino_language_model.xml
           ├── openvino_language_model.bin
           ├── ...
   

5. Model or publicly available similar model that reproduces the issue:

   • OpenVINO/gemma-4-E4B-it-int4-ov

Additional context
The current openvinotoolkit/openvino.genai source appears to support gemma4 in VLM config parsing, for example in src/cpp/src/visual_language/vlm_config.cpp, where gemma4 is mapped to VLMModelType::GEMMA4.

Because of that, this looks like one of:

• OVMS 2026.1-gpu does not actually include effective Gemma4 support yet
• the bundled GenAI backend is inconsistent with the source support
• Gemma4 support exists in openvino.genai repo but is not yet supported in the shipped OVMS image

If helpful, I can test a nightly image or provide additional model files / exact directory contents.

[mzegla]

Gemma4 is not supported in OVMS 2026.1. The work on this model is still very much in progress, but you can check it with new weekly release that we will publish with at least partial support to that model.