Skip to content

chore(ilp): review changes, reducing allocations#22

Merged
bluestreak01 merged 2 commits into
mainfrom
vi_sf
May 13, 2026
Merged

chore(ilp): review changes, reducing allocations#22
bluestreak01 merged 2 commits into
mainfrom
vi_sf

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

No description provided.

bluestreak01 and others added 2 commits May 13, 2026 17:02
SegmentManager's hot-spare provisioning previously allocated a fresh
byte[] and a NATIVE_PATH-tagged native malloc on every rotation:
StringSink.toString() produced a String, Files.openCleanRW(String)
encoded it back into native memory via pathPtr(), and the same
encoding repeated for any cleanup remove(). With small segments and
high throughput the per-rotation churn becomes proportional to the
rotation rate.

Replace the StringSink with a reusable DirectUtf8Sink (pathScratch)
that the worker thread writes directly. nextSparePath() builds the
path bytes in place, captures the display String once for
MmapSegment.path() and exception messages, then appends a NUL byte
so the same buffer is a valid C string for the long-ptr Files
overloads.

Files / FilesFacade / DefaultFilesFacade gain three new overloads
that accept a pre-encoded native pointer: openRW(long),
openCleanRW(long, long), length(long). Each is a direct passthrough
to the existing JNI native, no allocation.

MmapSegment.create grows a parallel long-ptr factory:
create(FilesFacade, long pathPtr, String displayPath, ...). The
existing String-taking factories delegate through it via
allocNativePath, preserving the public API while concentrating the
open + allocate + mmap dance in one place. SegmentManager calls the
new factory directly with pathScratch.ptr().

Per rotation the byte[] and the NATIVE_PATH malloc/free pair are
gone. The one remaining String allocation (pathScratch.toString())
stays because MmapSegment.path() returns String for trim and log
lines -- killing that would require a different segment-identity
representation and is out of scope here.

Test seam: FaultyFilesFacade in MmapSegmentTest mirrors the new
long-ptr overloads (openCleanRW(long, long), openRW(long),
length(long), remove(long) counter) so the create-time fault
injection contracts still fire after the String factory routes
internally through the pointer factory.

Drive-by: replace frameCount++ in MmapSegment.tryAppend with an
explicit volatile read + write to silence the "non-atomic increment
of volatile" inspection. Semantics unchanged -- the field has a
single writer by design (producer thread in tryAppend), so the
read-modify-write race the inspection warns about cannot occur.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bluestreak01 bluestreak01 changed the title chore: review changes chore(qwp): review changes, reducing allocations May 13, 2026
@mtopolnik
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 23 / 29 (79.31%)

file detail

path covered line new line coverage
🔵 io/questdb/client/std/Files.java 1 3 33.33%
🔵 io/questdb/client/std/DefaultFilesFacade.java 1 3 33.33%
🔵 io/questdb/client/cutlass/qwp/client/sf/cursor/MmapSegment.java 9 11 81.82%
🔵 io/questdb/client/cutlass/qwp/client/sf/cursor/SegmentManager.java 12 12 100.00%

@bluestreak01 bluestreak01 changed the title chore(qwp): review changes, reducing allocations chore(ilp): review changes, reducing allocations May 13, 2026
@bluestreak01 bluestreak01 merged commit 67bb5e4 into main May 13, 2026
14 checks passed
@bluestreak01 bluestreak01 deleted the vi_sf branch May 13, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants