Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the Continuous Batching RAG demo documentation to remove Docker-based instructions that relied on the Python (latest-py) OVMS image, aligning the demo setup guidance with the intended distribution approach.
Changes:
- Removed the “With Docker” model download instructions for OpenVINO-org models.
- Removed the “With Docker” model download instructions for non-OpenVINO-org (advanced) models that referenced
openvino/model_server:latest-py.
| @@ -5,21 +5,6 @@ | |||
| ### 1. Download the preconfigured models using ovms --pull option from [HugginFaces Hub OpenVINO organization](https://huggingface.co/OpenVINO) (Simple usage) | |||
There was a problem hiding this comment.
The link text has a typo: "HugginFaces" should be "Hugging Face" (or "Hugging Face Hub") to match the correct project name.
There was a problem hiding this comment.
@copilot apply changes based on this feedback
There was a problem hiding this comment.
Fixed in commit 726003d. Corrected "HugginFaces" → "Hugging Face" in both section headings, and also removed the unnecessary single-item tab-set wrapper in section 2 (Advanced usage).
| @@ -57,20 +42,6 @@ ovms --add_to_config --config_path c:\models\config.json --model_name OpenVINO/b | |||
| ### 2. Download the preconfigured models using ovms --pull option for models outside [HugginFaces Hub OpenVINO organization](https://huggingface.co/OpenVINO) in HuggingFace Hub. (Advanced usage) | |||
There was a problem hiding this comment.
This heading/link text has a typo: "HugginFaces" should be "Hugging Face" (or "Hugging Face Hub") for the correct name.
| ::::{tab-set} | ||
|
|
||
| :::{tab-item} With Docker | ||
| **Required:** Docker Engine installed | ||
| ```bash | ||
| mkdir models | ||
| docker run --user $(id -u):$(id -g) -e HF_HOME=/hf_home/cache --rm -v $(pwd)/models:/models:rw -v /opt/home/user/.cache/huggingface/:/hf_home/cache openvino/model_server:latest-py --pull --model_repository_path /models --source_model meta-llama/Meta-Llama-3-8B-Instruct --task text_generation --weight-format int8 | ||
| docker run --user $(id -u):$(id -g) -e HF_HOME=/hf_home/cache --rm -v $(pwd)/models:/models:rw -v /opt/home/user/.cache/huggingface/:/hf_home/cache openvino/model_server:latest-py --pull --model_repository_path /models --source_model Alibaba-NLP/gte-large-en-v1.5 --task embeddings --weight-format int8 | ||
| docker run --user $(id -u):$(id -g) -e HF_HOME=/hf_home/cache --rm -v $(pwd)/models:/models:rw -v /opt/home/user/.cache/huggingface/:/hf_home/cache openvino/model_server:latest-py --pull --model_repository_path /models --source_model BAAI/bge-reranker-large --task rerank --weight-format int8 | ||
|
|
||
| docker run --user $(id -u):$(id -g) --rm -v $(pwd)/models:/models:rw openvino/model_server:latest-py --add_to_config --config_path /models/config.json --model_name meta-llama/Meta-Llama-3-8B-Instruct --model_path meta-llama/Meta-Llama-3-8B-Instruct --weight-format int8 | ||
| docker run --user $(id -u):$(id -g) --rm -v $(pwd)/models:/models:rw openvino/model_server:latest-py --add_to_config --config_path /models/config.json --model_name Alibaba-NLP/gte-large-en-v1.5 --model_path Alibaba-NLP/gte-large-en-v1.5 --weight-format int8 | ||
| docker run --user $(id -u):$(id -g) --rm -v $(pwd)/models:/models:rw openvino/model_server:latest-py --add_to_config --config_path /models/config.json --model_name BAAI/bge-reranker-large --model_path BAAI/bge-reranker-large --weight-format int8 | ||
| ``` | ||
| ::: | ||
|
|
||
| :::{tab-item} On Baremetal Host | ||
| **Required:** OpenVINO Model Server package - see [deployment instructions](../../../docs/deploying_server_baremetal.md) for details. | ||
|
|
There was a problem hiding this comment.
This tab-set currently contains only a single tab item ("On Baremetal Host"). Using a tab-set for a single option tends to render as unnecessary UI and adds maintenance overhead; consider removing the ::::{tab-set} / ::{tab-item} wrappers here (or add the missing alternatives if there are meant to be multiple choices).
There was a problem hiding this comment.
@copilot apply changes based on this feedback
…in section 2 Agent-Logs-Url: https://github.com/openvinotoolkit/model_server/sessions/76bdc9f4-5f03-4afd-a8ac-6c88629dc6d0 Co-authored-by: rasapala <58549742+rasapala@users.noreply.github.com>
🛠 Summary
JIRA CVS-179954
Updates the Continuous Batching RAG demo documentation to remove Docker-based instructions that relied on the Python (
latest-py) OVMS image. Also fixes typos in section headings ("HugginFaces" → "Hugging Face") and removes an unnecessary single-item tab-set wrapper in the Advanced usage section.🧪 Checklist
``