refactor: move from 3-variant index search result to single generic variant

[westonpace]

What

Replaces the three-variant enum:

pub enum IndexExprResult {
    Exact(RowAddrMask),
    AtMost(RowAddrMask),   // upper bound on the answer
    AtLeast(RowAddrMask),  // lower bound on the answer
}

with a single struct carrying both bounds:

pub struct IndexExprResult {
    pub lower: RowAddrMask,  // rows definitely in the answer
    pub upper: RowAddrMask,  // rows that may be in the answer
                             // (rows outside `upper` are definitely out)
}

Same change applied to NullableIndexExprResult (the during-evaluation form
that carries SQL 3VL info via NullableRowAddrMask).

The three pre-existing shapes map onto degenerate intervals:

Old variant	New form
Exact(m)	{lower: m, upper: m}
AtMost(m)	{lower: allow_nothing(), upper: m}
AtLeast(m)	{lower: m, upper: all_rows()}

Constructed via IndexExprResult::{exact, at_most, at_least} associated
functions; inspected via is_exact() / is_at_most() / is_at_least()
predicates.

Why

Some indices can return three-way info per row range — "definitely
matches", "maybe matches", "definitely doesn't match" — and the old
representation couldn't express it. The new form makes that fourth shape
representable as a non-degenerate interval (nonempty lower strictly
inside non-universe upper), and the boolean algebra collapses to
elementwise lattice operations on the endpoints:

!{l, u}              = {!u, !l}
{l1, u1} & {l2, u2}  = {l1 & l2, u1 & u2}
{l1, u1} | {l2, u2}  = {l1 | l2, u1 | u2}

The motivating case: a zone-map IS NOT NULL search can now produce
{lower: zones_with_zero_nulls, upper: not_all_null_zones}. The read
planner can both skip the all-null zones entirely (I/O saving on the
50%-clustered-nulls case) and skip the recheck on the
guaranteed-non-null zones (CPU saving). The old algebra had to either
drop the lower bound (resulting in an AtMost that loses the
all-null-skip) or drop the upper bound (resulting in an AtLeast that
loses the recheck-skip).

3VL semantics are preserved: NULL info lives inside each
NullableRowAddrMask (in its nulls field), and NullableRowAddrMask::Not
flips Allow↔Block without touching nulls, so !{l, u} = {!u, !l}
preserves NULL on both endpoints.

Wire format

The INDEX_EXPR_RESULT_SCHEMA columns change from
(result: Binary, discriminant: UInt32, fragments_covered: Binary) to
(lower: Binary, upper: Binary, fragments_covered: Binary). The schema
is internal to the in-process hand-off from ScalarIndexExec to
FilteredReadExec, so this isn't a persisted-format break.

Other touchpoints

• apply_index_to_fragment in filtered_read.rs: switches from a
  3-arm match to a predicate chain. Existing shapes keep their
  existing read-plan semantics. A non-degenerate Refined case is
  handled conservatively for now (read upper, recheck via
  full_filter); a follow-up will split to_read into
  guaranteed-match ranges (skip recheck) and recheck ranges (apply
  full filter) so the new representation actually pays off.
• MaterializeIndexExec in scalar_index.rs uses result.upper
  for the candidate-row set. AtLeast still bails (upper is
  universe); everything else works.
• New NullableRowAddrMask::allow_nothing() / all_rows()
  constructors for the at_most / at_least builders.
• NullableRowAddrMask gains PartialEq so is_exact() can check
  lower == upper.
• IndexExprResult::{from_parts, serialize_to_arrow} (inherent methods
  on the old enum) → index_expr_result_from_parts(lower, upper) /
  serialize_index_expr_result(result, frags) free functions on
  lance-index::scalar::expression.
• Existing certainty-propagation tests in expression.rs rewritten to
  use the new constructors and is_at_most() / is_at_least()
  predicates. All assertions still hold under the new algebra.

Performance

Bench from lance-select/benches/index_expr_result.rs at N=10M rows.
Both runs at criterion --measurement-time 3 --warm-up-time 2 --sample-size 30, same workload (single contiguous run, AllowList with
no nulls), same machine, idle CPU. Median time per op:

Op	Variant pair	Baseline	2-mask	Delta
!	Exact	8 ns	13 ns	+60%
!	AtMost / AtLeast	8 ns	14 ns	+75%
&	Exact_Exact	7.24 µs	13.84 µs	+91%
&	AtMost_AtMost	7.49 µs	7.32 µs	−2%
&	AtLeast_AtLeast	7.22 µs	7.35 µs	+2%
&	Exact_AtMost	7.15 µs	10.16 µs	+42%
&	Exact_AtLeast	1.63 µs	7.19 µs	+341% (was a drop)
&	AtMost_AtLeast	1.66 µs	1.76 µs	+6%
|	Exact_Exact	3.82 µs	7.54 µs	+97%
|	AtMost_AtMost	3.90 µs	3.87 µs	−1%
|	AtLeast_AtLeast	3.82 µs	4.00 µs	+5%
|	Exact_AtMost	3.67 µs	3.75 µs	+2%
|	Exact_AtLeast	3.74 µs	12.49 µs	+234% (was a drop)
|	AtMost_AtLeast	3.04 µs	11.70 µs	+285% (was a drop)

Reading the deltas:

• NOT: a few extra ns. Was one mask flip; now two endpoint flips +
  swap. Irrelevant in absolute terms.
• Same-variant AtMost_AtMost / AtLeast_AtLeast: roughly
  unchanged. The second mask op composes empty-with-empty or
  universe-with-universe and short-circuits in RowAddrMask::BitAnd.
• Exact_Exact: ~1.9-2× slower. Both endpoints carry the full mask
  (the exact() constructor clones), so we genuinely do two mask ops
  instead of one. Expected.
• Cases that the old algebra resolved by dropping a bound
  (Exact_AtLeast for both ops, AtMost_AtLeast for OR): 3-4× slower
  in absolute terms. The old code returned without doing a mask op;
  the new code computes both endpoints and preserves both bounds.
  Follow-up optimization here is to add BlockList(empty) /
  AllowList(empty) fast paths to NullableRowAddrMask::{BitAnd, BitOr}, which would catch the universe/empty operands directly.

Trade-off: a few extra microseconds per binary op on 10M-row masks (no
impact on query latency — millisecond-scale and dominated by I/O) in
exchange for richer interval results. The Refined case unlocks
acceleration paths that weren't reachable before.

Verification

• cargo test -p lance-select --lib: 99 passed
• cargo test -p lance-index --lib: 311 passed
• cargo test -p lance --lib io::exec::filtered_read: 23 passed
• cargo test -p lance --lib io::exec::scalar_index: 2 passed
• cargo clippy --workspace --tests -- -D warnings: clean
• cargo fmt --all --check: clean

Follow-ups

1. Per-range filter pushdown in filtered_read::apply_index_to_fragment
   so the Refined case can split to_read into definitely-match (skip
   recheck) and recheck ranges (apply full filter) within one fragment.
   That's what makes the new representation actually pay off for IS NOT
   NULL.
2. Empty/universe short-circuit in NullableRowAddrMask::{BitAnd, BitOr}. Detect AllowList(empty_set) and BlockList(empty_set)
   operands and bypass the per-row work. Erases the regressions on
   cross-variant cases.
3. Zone map IsNotNull query that produces a Refined result
   directly, instead of going through Not(IsNull).

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[westonpace]

@claude review once

[claude]

⚠️ Code review skipped — your organization has reached its monthly code review spending cap.

An organization admin can view or raise the cap at claude.ai/admin-settings/claude-code. The cap resets at the start of the next billing period.

Once the cap resets or is raised, comment @claude review on this pull request to trigger a review.