Skip to content

[wip] PatchedArray#6815

Draft
a10y wants to merge 6 commits intodevelopfrom
aduffy/patched
Draft

[wip] PatchedArray#6815
a10y wants to merge 6 commits intodevelopfrom
aduffy/patched

Conversation

@a10y
Copy link
Contributor

@a10y a10y commented Mar 6, 2026

Summary

  • new PatchedArray
  • On read, BitPackedArray w/Patches -> PatchedArray(BitPackedArray)
  • Slice
  • Filter
  • Take
  • ALP
  • ALP-RD (?)

Closes: #000

Testing

a10y added 5 commits March 5, 2026 10:16
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 6, 2026

Merging this PR will improve performance by 22.59%

⚡ 3 improved benchmarks
✅ 391 untouched benchmarks
🆕 6 new benchmarks
⏩ 2052 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation bench_patch_transpose[100] N/A 201 µs N/A
🆕 Simulation bench_patch_transpose[10] N/A 197.2 µs N/A
🆕 Simulation bench_patch_transpose[1] N/A 197.3 µs N/A
🆕 Simulation bench_patch_transpose[2048] N/A 277.5 µs N/A
🆕 Simulation bench_patch_transpose[65536] N/A 2.7 ms N/A
🆕 Simulation bench_patch_transpose[1024] N/A 238.4 µs N/A
Simulation take_map[(0.1, 1.0)] 4.2 ms 3.6 ms +17.09%
Simulation take_map[(0.1, 0.5)] 2.6 ms 2.1 ms +22.59%
Simulation take_map[(0.1, 0.1)] 1,007.5 µs 911.1 µs +10.57%

Comparing aduffy/patched (5580564) with develop (5d6a3c8)2

Open in CodSpeed

Footnotes

  1. 2052 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (761c404) during the generation of this report, so 5d6a3c8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@gatesn
Copy link
Contributor

gatesn commented Mar 6, 2026

Moving my comment from another PR:

Unless we have some way of passing down an "is_defined" mask into execution, there's no way for the child array to know whether some checked operation should fail or not.

e.g. the child of a Patches array may overflow on some arithmetic, but the patch would have succeeded. We need a way to tell the child to ignore the overflow for the invalid positions.

@a10y
Copy link
Contributor Author

a10y commented Mar 6, 2026

do you think that's something we could ferry in the executionctx?

@a10y
Copy link
Contributor Author

a10y commented Mar 6, 2026

Alternatively you can just not pushdown scalarfns to the child and force it to be done after patching? It's clear to me why you'd wanna push down cardinality reducing ops (e.g. Slice) but idk why it matters if fallible scalar fns like Sum or Binary need the patched result

Signed-off-by: Andrew Duffy <andrew@a10y.dev>
@gatesn
Copy link
Contributor

gatesn commented Mar 6, 2026

The push-down makes sense, but it seems a shame that we cannot have a specialized kernel for ALP::compare because there's a patch array in the middle. I guess compare can be pushed down? Because it isn't a checked operation?

In fact, I think we already have this defined as ScalarFnVTable::is_fallible

@gatesn
Copy link
Contributor

gatesn commented Mar 6, 2026

Other option ofc is ArrayVTable::execute(selection: Mask, defined: Mask) - but 🤷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants