perf(index): optimize search index slice allocation#2074
Closed
MEKXH wants to merge 3 commits intoOpenListTeam:mainfrom
Closed
perf(index): optimize search index slice allocation#2074MEKXH wants to merge 3 commits intoOpenListTeam:mainfrom
MEKXH wants to merge 3 commits intoOpenListTeam:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes search index batch-building by reducing slice reallocations and avoiding intermediate slice conversions, with an added benchmark to quantify allocation improvements.
Changes:
- Preallocate
[]model.SearchNodeinBatchIndexto avoidappendgrowth. - Replace
utils.MustSliceConvert+BatchIndexin index building with a direct preallocated conversion path for MQ messages. - Add benchmarks to compare allocation behavior before/after the optimization.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| internal/search/search.go | Preallocates searchNodes in BatchIndex and assigns by index to reduce allocations. |
| internal/search/build.go | Introduces batchIndexMessages to build searchNodes directly from MQ messages and preallocates toAddObjs capacity. |
| internal/search/benchmark_test.go | Adds benchmarks simulating old vs new slice allocation strategies. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
perf(search): optimize search index slice allocation
Description / 描述
优化搜索索引构建过程中的切片内存分配策略:
build.go: 在batchIndexMessages函数中使用预分配切片替代append动态增长search.go: 在BatchIndex函数中同样采用预分配策略,移除utils.MustSliceConvert中间转换核心变更
Motivation and Context / 背景
搜索索引批量构建是一个高频操作,尤其在初始化索引或批量更新时。原实现存在以下性能问题:
append动态扩容: 导致多次内存分配和数据拷贝utils.MustSliceConvert中间切片: 产生额外内存分配开销通过预分配确切大小的切片,可显著减少 GC 压力并提升批量索引性能。
Relates to performance optimization
How Has This Been Tested? / 测试
添加了 Benchmark 测试对比优化前后性能:
go test -bench=BenchmarkSearchAllocation -benchmem ./internal/search/测试结果
Checklist / 检查清单
我已阅读 CONTRIBUTING 文档。
go fmtor prettier.我已使用
go fmt或 prettier 格式化提交的代码。我已为此 PR 添加了适当的标签(如无权限或需要的标签不存在,请在描述中说明,管理员将后续处理)。
建议标签:
performance,search我已在适当情况下使用"Request review"功能请求相关代码作者进行审查。
我已相应更新了相关仓库(若适用)。