Skip to content

Gemma4:latest fail to run #966

@iofirag

Description

@iofirag

while trying run Gemma4:latest failed due to
background model preload failed: preload failed: status=500 body=unable to load runner: error waiting for runner to be ready: llama.cpp terminated unexpectedly: llama.cpp failed: D:\a\inference-engine-llama.cpp\inference-engine-llama.cpp\llamacpp\native\vendor\llama.cpp\tools\mtmd\clip.cpp:4391: Unknown projector type

Microsoft Windows [Version 10.0.26200.8655]
(c) Microsoft Corporation. All rights reserved.

C:\Users\iofir>docker model run gemma4
> background model preload failed: preload failed: status=500 body=unable to load runner: error waiting for runner to be ready: llama.cpp terminated unexpectedly: llama.cpp failed: D:\a\inference-engine-llama.cpp\inference-engine-llama.cpp\llamacpp\native\vendor\llama.cpp\tools\mtmd\clip.cpp:4391: Unknown projector type
 Intel(R) Core(TM) Ultra 7 265H (64980 MiB, 36723 MiB free)
0.00.183.149 I system_info: n_threads = 16 (n_threads_batch = 16) / 16 | CUDA : ARCHS = 500,610,700,750,800,860,890,900,1200,1210 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | REPACK = 1 |
0.00.183.153 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.183.204 I srv          init: using 15 threads for HTTP server
0.00.183.295 I srv         start: setting address family to AF_UNIX
0.00.190.684 I srv  llama_server: loading model
0.00.190.695 I srv    load_model: loading model 'C:\Users\iofir\.docker\models\bundles\sha256\44aa4db60f30abffcbd669214e3949d685d5e7f384e0637fc4b7c56702e24a43\model\gemma-4-E4B-it-UD-Q4_K_XL.gguf'

Docker desktop > Models > [Model]:

docker.io/gemma4
44aa4db60f3

Docker desktop > Models > Inspect tab:

Model Characteristics:
ARCHITECTURE gemma4
PARAMETERS 7.52B
QUANTIZATION MOSTLY_Q4_K_M
FORMAT gguf
SIZE 7.52B

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions