Make chunk_size to uint64_t to avoid overflow and support chunk size>=4GB#861
Make chunk_size to uint64_t to avoid overflow and support chunk size>=4GB#861shosseinimotlagh wants to merge 1 commit intoeBay:masterfrom
Conversation
61b3a73 to
56b7d3f
Compare
…greater equal than 4GB
56b7d3f to
3680d76
Compare
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #861 +/- ##
==========================================
- Coverage 56.51% 48.38% -8.13%
==========================================
Files 108 110 +2
Lines 10300 12869 +2569
Branches 1402 6176 +4774
==========================================
+ Hits 5821 6227 +406
+ Misses 3894 2554 -1340
- Partials 585 4088 +3503 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Do we have strong reason for that? With 4GB and 64K chunks w can support to 256TB disk, is that having a bigger disk support is a must have? Asking as it is an incompatibility changes, though not that bad as only impacting meta part. Regarding the upgradability, can we do that in code? maybe we need to introduce version number |
|
Oh are you hitting the 4GB limit when you tuning the logstore chunk size? |
|
can you put more context why you have to change chunksize to more than 4G? let`s see if there is any workaround and not make this change. we`d better keep backward compatibility and not diverge the code |
|
@JacksonYao287 we discussed this changes. It is not too bad as it only touches the vdev header part, we have 1M/8M reservation for FAST/DATA respectively, by changing it to u64 we used 512k reservation. We will figure out how this benefit performance , and if the change is needed, inline upgrade is needed |
|
if compatibility can be well handled and can do inplace upgrade, I am good with this change. |
Currently in HomeBlks, overflow can happen if chunk size is set for more than 4GB. To avoid it, change chunk size type from uint32_t to uint64_t