Conversation
There was a problem hiding this comment.
Pull request overview
Updates the OpenWebUI integration demo documentation to use newer default models (including a VLM) and adds guidance for newer “agentic” OpenWebUI features like Web Search, Memory, and Code Interpreter.
Changes:
- Switched the primary chat model example to
OpenVINO/gpt-oss-20b-int4-ovand standardized the OpenWebUI Model ID toovms-model. - Replaced the VLM example model with
Junrui2021/Qwen3-VL-8B-Instruct-int4and added a new screenshot for image upload. - Added new documentation sections for Web Search, Memory/context, and Code Interpreter configuration in OpenWebUI.
Reviewed changes
Copilot reviewed 1 out of 13 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| demos/integration_with_OpenWebUI/README.md | Updates model pull/config instructions and adds new OpenWebUI feature sections (Web Search, Memory, Code Interpreter). |
| demos/integration_with_OpenWebUI/upload_images.png | Adds/updates a screenshot used by the VLM “upload images” step. |
| ```bash | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model OpenVINO/InternVL2-2B-int4-ov --model_repository_path models --model_name OpenVINO/InternVL2-2B-int4-ov --task text_generation | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --add_to_config --config_path /models/config.json --model_path OpenVINO/InternVL2-2B-int4-ov --model_name OpenVINO/InternVL2-2B-int4-ov | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation |
There was a problem hiding this comment.
In the Docker VLM pull command, --model_repository_path is set to models (relative path inside the container), while the volume is mounted to /models. This likely writes to a non-persisted path and breaks the subsequent --add_to_config --config_path /models/config.json step. Use /models here for consistency with other Docker examples above.
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation | |
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path /models --model_name ovms-model-vl --task text_generation |
| ```bash | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model OpenVINO/InternVL2-2B-int4-ov --model_repository_path models --model_name OpenVINO/InternVL2-2B-int4-ov --task text_generation | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --add_to_config --config_path /models/config.json --model_path OpenVINO/InternVL2-2B-int4-ov --model_name OpenVINO/InternVL2-2B-int4-ov | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation |
There was a problem hiding this comment.
Windows VLM section uses --pipeline_type VLM_CB, but the Linux/Docker command omits it. If AUTO doesn’t reliably detect VLM, the Docker path may start the wrong pipeline. Align the Linux command with Windows by specifying the same --pipeline_type (and keep both examples consistent).
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation | |
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation --pipeline_type VLM_CB |
| ### Step 2: Chat with VLM | ||
|
|
||
| 1. Start a **New Chat** with model set to `OpenVINO/InternVL2-2B-int4-ov` | ||
| 1. Start a **New Chat** | ||
| 2. Click **+More** to upload images, by capturing the screen or uploading files. The image used in this demo is [http://raw.githubusercontent.com/openvinotoolkit/model_server/refs/heads/releases/2025/3/demos/common/static/images/zebra.jpeg](http://raw.githubusercontent.com/openvinotoolkit/model_server/refs/heads/releases/2025/3/demos/common/static/images/zebra.jpeg). |
There was a problem hiding this comment.
Step 2 for VLM no longer tells the reader to select the ovms-model-vl model in OpenWebUI (it just says “Start a New Chat”). Without selecting the correct model, the subsequent image-upload chat steps may run against the default text model instead of the VLM endpoint.
|
|
||
| ### Step 3: Use Web Search in the chat | ||
|
|
||
| 1. Open new Chat (it should have blue Web Search icon blow) |
There was a problem hiding this comment.
Typo: “blow” should be “below” in “blue Web Search icon blow”.
| 1. Open new Chat (it should have blue Web Search icon blow) | |
| 1. Open new Chat (it should have blue Web Search icon below) |
|
|
||
| ### Reference | ||
|
|
||
|  |
There was a problem hiding this comment.
The “Reference” is currently written as an image link (), which renders as a broken image instead of a hyperlink. Use a normal markdown link here.
|  | |
| [Open WebUI Web Search documentation](https://docs.openwebui.com/features/chat-conversations/web-search/agentic-search/) |
|
|
||
|  | ||
|
|
||
| It's possible to have multiple manageble memory records. |
There was a problem hiding this comment.
Typos: “manageble” should be “manageable”.
| It's possible to have multiple manageble memory records. | |
| It's possible to have multiple manageable memory records. |
|
|
||
| 1. Go to **Workspace** → **Models** | ||
| 2. Choose model or create it. | ||
| 3. In **Buildin Tools** section enable **Memory** |
There was a problem hiding this comment.
Typo: “Buildin Tools” should be “Built-in Tools” (or whatever wording matches the OpenWebUI UI label).
| 3. In **Buildin Tools** section enable **Memory** | |
| 3. In **Built-in Tools** section enable **Memory** |
|
|
||
|  | ||
|
|
||
| > **Note**: There is no way to make searching memory default on the beggingng of the conversation in Open Web UI. User should tell model to use it to make it work. |
There was a problem hiding this comment.
Typo: “beggingng” should be “beginning” in the note about searching memory.
| > **Note**: There is no way to make searching memory default on the beggingng of the conversation in Open Web UI. User should tell model to use it to make it work. | |
| > **Note**: There is no way to make searching memory default on the beginning of the conversation in Open Web UI. User should tell model to use it to make it work. |
| ovms.exe --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation --pipeline_type VLM_CB | ||
| ovms.exe --add_to_config --config_path models\config.json --model_path Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_name ovms-model-vl | ||
| ``` |
There was a problem hiding this comment.
In the VLM Windows example, --pull uses --model_name ovms-model-vl, but --add_to_config points --model_path to Junrui2021/Qwen3-VL-8B-Instruct-int4. If pull mode follows the documented behavior (downloads into <model_repository_path>/<model_name>), the model path should match ovms-model-vl (or omit --model_name in the pull step). As written, --add_to_config will likely fail to find the pulled model directory.
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --pull --source_model Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_repository_path models --model_name ovms-model-vl --task text_generation | ||
| docker run --rm -u $(id -u):$(id -g) -v $PWD/models:/models openvino/model_server:weekly --add_to_config --config_path /models/config.json --model_path Junrui2021/Qwen3-VL-8B-Instruct-int4 --model_name ovms-model-vl |
There was a problem hiding this comment.
Similarly in the Docker VLM example, --pull sets --model_name ovms-model-vl, but --add_to_config uses --model_path Junrui2021/Qwen3-VL-8B-Instruct-int4. If pull stores the model under /models/ovms-model-vl, --model_path should point to ovms-model-vl (or the pull step should not override --model_name).
🛠 Summary
CVS-183785
Changing models used in OpenWebUI, adding sections about new agentic features.
TODO: Update screenshots to use ovms-model model name instead of
Godreign/llama-3.2-3b-instruct-openvino-int4-model🧪 Checklist
``