Skip to content

Make chunk_size to uint64_t to avoid overflow and support chunk size>=4GB#861

Open
shosseinimotlagh wants to merge 1 commit intoeBay:masterfrom
shosseinimotlagh:adjust_chunk_size
Open

Make chunk_size to uint64_t to avoid overflow and support chunk size>=4GB#861
shosseinimotlagh wants to merge 1 commit intoeBay:masterfrom
shosseinimotlagh:adjust_chunk_size

Conversation

@shosseinimotlagh
Copy link
Contributor

Currently in HomeBlks, overflow can happen if chunk size is set for more than 4GB. To avoid it, change chunk size type from uint32_t to uint64_t

@codecov-commenter
Copy link

codecov-commenter commented Feb 9, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 73.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.38%. Comparing base (1a0cef8) to head (3680d76).
⚠️ Report is 315 commits behind head on master.

Files with missing lines Patch % Lines
src/lib/device/device_manager.cpp 0.00% 0 Missing and 1 partial ⚠️
src/lib/index/wb_cache.cpp 66.66% 0 Missing and 1 partial ⚠️
src/lib/replication/repl_dev/solo_repl_dev.h 0.00% 1 Missing ⚠️
src/lib/replication/service/raft_repl_service.cpp 85.71% 0 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #861      +/-   ##
==========================================
- Coverage   56.51%   48.38%   -8.13%     
==========================================
  Files         108      110       +2     
  Lines       10300    12869    +2569     
  Branches     1402     6176    +4774     
==========================================
+ Hits         5821     6227     +406     
+ Misses       3894     2554    -1340     
- Partials      585     4088    +3503     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xiaoxichen
Copy link
Collaborator

Do we have strong reason for that?

With 4GB and 64K chunks w can support to 256TB disk, is that having a bigger disk support is a must have?

Asking as it is an incompatibility changes, though not that bad as only impacting meta part.

Regarding the upgradability, can we do that in code? maybe we need to introduce version number

@xiaoxichen
Copy link
Collaborator

Oh are you hitting the 4GB limit when you tuning the logstore chunk size?

@JacksonYao287
Copy link
Contributor

can you put more context why you have to change chunksize to more than 4G? let`s see if there is any workaround and not make this change.

we`d better keep backward compatibility and not diverge the code

@xiaoxichen
Copy link
Collaborator

@JacksonYao287 we discussed this changes. It is not too bad as it only touches the vdev header part, we have 1M/8M reservation for FAST/DATA respectively, by changing it to u64 we used 512k reservation.

We will figure out how this benefit performance , and if the change is needed, inline upgrade is needed

@JacksonYao287
Copy link
Contributor

if compatibility can be well handled and can do inplace upgrade, I am good with this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments