[RL] [KVCache] let cache transfer managers update key prefix after weight update and add unit tests#7083
Conversation
|
Thanks for your contribution! |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #7083 +/- ##
==========================================
Coverage ? 74.15%
==========================================
Files ? 376
Lines ? 52873
Branches ? 8251
==========================================
Hits ? 39207
Misses ? 10908
Partials ? 2758
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
31ee731 to
26489aa
Compare
26489aa to
4a38b8a
Compare
fastdeploy-bot
left a comment
There was a problem hiding this comment.
🤖 AI Code Review |
{{TIMESTAMP}}
📋 Review 摘要
PR 概述:在 update_weights 操作中添加缓存 key_prefix 的更新,使缓存元数据与新模型版本更早保持一致
变更范围:cache_manager/、engine/、测试文件
影响面 Tag:KVCache Engine
问题
未发现阻塞性问题。
代码亮点
-
request_id 改进:将硬编码的
pause_transfer/resume_transfer改为基于父请求 ID 的命名方式(如{control_request.request_id}_pause_transfer),提升了请求追踪的可调试性。 -
一致的条件检查:
_handle_update_weights和_handle_resume都正确检查了storage_backend_type is not None才调用_update_key_prefix()。 -
测试覆盖充分:新增测试覆盖了正常路径(有 storage backend)和跳过路径(无 storage backend),以及相关的控制方法行为。
总体评价
代码实现清晰,逻辑正确,测试覆盖充分。PR 标题和描述均符合规范,可以合入。
Merge with upstream test_engine.py (PR PaddlePaddle#7083) and add comprehensive coverage for LLMEngine: lifecycle, worker signals, requests, utils, stop_profile, and start error handling.
Motivation
This PR updates the mooncake
key_prefixduringupdate_weights.Previously,
key_prefixwas only refreshed inresume. To make cache metadata align with the new model version earlier, this PR adds one more refresh inupdate_weights, while keeping the existing refresh inresume.Modifications
EngineService._control_update_weights()update_weightshandler inCacheTransferManagerresumerefreshingkey_prefixupdate_weightsandresumepathsUsage or Command
No new API.
Behavior after this PR:
update_weightsrefresheskey_prefixresumealso refresheskey_prefixAccuracy Tests
N/A
Checklist