Skip to content

fix: drop old reservation before creating new one in claim() (#10139)#10167

Open
sandy-sachin7 wants to merge 1 commit into
apache:mainfrom
sandy-sachin7:fix/claim-race-condition
Open

fix: drop old reservation before creating new one in claim() (#10139)#10167
sandy-sachin7 wants to merge 1 commit into
apache:mainfrom
sandy-sachin7:fix/claim-race-condition

Conversation

@sandy-sachin7

Copy link
Copy Markdown

Which issue does this PR close?

Closes #10139.

Rationale for this change

The claim() method on Bytes and MutableBuffer creates a new pool reservation before dropping the existing one. This causes a transient period where both the old and new reservations exist simultaneously, resulting in the memory pool reporting up to 2x the actual memory in use.

The problematic sequence in the old code:

*self.reservation.lock().unwrap() = Some(pool.reserve(self.capacity()));
// Equivalent to:
//   let guard = self.reservation.lock().unwrap();  // old reservation alive
//   let new_claim = pool.reserve(self.capacity());  // BOTH old + new alive (2x count)
//   *guard = Some(new_claim);                       // old dropped, back to 1x

For pools with capacity limits, or when another thread reads the pool counter during this window, the pool would report double the actual memory in use.

What changes are included in this PR?

  1. arrow-buffer/src/bytes.rs: Bytes::claim now drops the existing reservation (guard.take()) before calling pool.reserve().
  2. arrow-buffer/src/buffer/mutable.rs: Same fix for MutableBuffer::claim.
  3. arrow-buffer/src/bytes.rs: Added test_claim_does_not_double_count using a MaxTrackerPool wrapper that detects transient double-counting deterministically.

Are these changes tested?

Yes — the new test_claim_does_not_double_count test uses a custom pool that records the maximum used() value observed during any reserve() call. Before the fix, claiming the same buffer twice would produce max_used = 2048 (double the 1024 buffer size). After the fix, max_used = 1024.

All existing arrow-buffer tests (53) and downstream tests continue to pass.

Are there any user-facing changes?

No API changes. The behavior is identical except that there is no longer a window where the pool counter is double-counted during re-claiming.

…10139)

The claim() method on Bytes and MutableBuffer created a new pool
reservation before dropping the existing one, causing transient
double-counting of memory in the pool during the window between
reserve() and the old reservation being dropped.

For pools with capacity limits, or when another thread reads the
pool counter during this window, the pool would report up to 2x
the actual memory in use.

Fix: call guard.take() to drop the old reservation before calling
pool.reserve() to create the new one.
@github-actions github-actions Bot added the arrow Changes to the arrow crate label Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claim API Race Condition when reclaiming existing memory

2 participants