Skip to content

Append constant runs and sparse patches without slicing or canonicalizing [builders-child-stack] - #9131

Open
robert3005 wants to merge 5 commits into
claude/builders-lazy-validity-9ze0t6from
claude/constant-fast-paths-9ze0t6
Open

Append constant runs and sparse patches without slicing or canonicalizing [builders-child-stack]#9131
robert3005 wants to merge 5 commits into
claude/builders-lazy-validity-9ze0t6from
claude/constant-fast-paths-9ze0t6

Conversation

@robert3005

@robert3005 robert3005 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Stacked on #8966, below #8967.

Sparse list and fixed-size-list canonicalization regressed badly when #9064 replaced its per-row appends with bulk ones. This is the other half of that change: the bulk appends stay, but they stop paying a fixed cost per appended run.

Three things were doing that:

Slicing is not constant-time. ArrayRef::slice ends in .optimize(), so every slice pays a full optimizer pass, and slicing a ListView is four of them - the array's offsets, sizes and elements are each sliced in turn. A profile of canonicalize_sparse_list[(512, 7, 4)] put 42% of the benchmark in optimizer::try_optimize under ArrayRef::slice. So take each patch's elements with list_elements_at instead of slicing runs of patches out of the patch array: one optimizer pass over a primitive array rather than four over a list view.

Constant runs were canonicalized before being appended. Constant::append_to_builder had fast paths only for the flat dtypes; everything else was built as an array and then copied into the builder. The nested dtypes with a builder can all skip that, and two of them can skip the values entirely - a constant struct is a constant array per field, and an extension array is its storage wearing a dtype. Union is the only dtype left on the fallback, and it has no builder.

The fill was rebuilt per gap. Both sparse paths now materialize the fill's elements once, up front, and every gap points its rows at that one array. For lists the rows share a single copy through their views; a fixed-size list holds its elements back to back so it tiles them, unless they are all the same scalar, in which case the tile stays constant-encoded and costs nothing.

append_listview_array also stops rebuilding: trimming through ListViewRebuildMode::TrimElements subtracted the window start from every offset with a compute kernel only for the rebase to add the builder's base straight back on, and the sizes cast was another kernel for what is a copy. It now computes the referenced window itself and rebases in the pass that was already walking the offsets.

Benchmarks

sparse_canonical medians, against develop:

benchmark develop before after
canonicalize_sparse_list[(512, 7, 4)] 24.1 µs 385.6 µs 20.2 µs
canonicalize_sparse_list[(1024, 17, 8)] 33.4 µs 323.9 µs 16.7 µs
canonicalize_sparse_fixed_size_list[(512, 7, 4)] 21.1 µs 79.0 µs 25.6 µs
canonicalize_sparse_fixed_size_list[(1024, 17, 8)] 29.1 µs 139.5 µs 32.9 µs
canonicalize_sparse_list[(8192, 1024, 4)]* 180.6 µs — 5.0 µs
canonicalize_sparse_fixed_size_list[(8192, 1024, 4)]* 150.1 µs — 140.1 µs

* added locally to cover long gaps; not part of the committed benchmark set.

Lists now beat develop. Fixed-size lists are within 1.13-1.21x of it, and what is left is the per-row copy of the tile: a fixed-size list cannot point its rows at one shared copy, so a run of n rows copies the tile n times. Removing that needs an encoding that expresses a repeated multi-element tile lazily, which does not exist yet - RunEnd cannot, since its runs must cover the whole logical length, so a single run is just a constant.

Behaviour change worth reviewing

Patches now go into the builder one at a time, and #8964 keeps every appended array as a chunk of its own, so a patch run of length n leaves n chunks in the elements child where it previously left one. test_sparse_list_chunks_elements_per_run_not_per_patch asserted the old shape and is now test_sparse_list_chunks_elements_per_patch_and_once_per_gap. Gaps still cost exactly one chunk however many rows they cover. The trade-off shows up on dense patches: (4096, 2, 4), which patches every other row, is 532 µs against develop's 384 µs.

Checks

  • cargo nextest run --workspace - 7234 passed
  • cargo clippy --all-targets --all-features - clean
  • cargo +nightly fmt --all --check - clean

@robert3005
perf(array): append a constant list run straight into the list builders
edd7758 
`Constant::append_to_builder` canonicalized every dtype it had no fast path
for. For a list that builds a whole `ListViewArray` only for
`append_listview_array` to rebuild it and cast its offsets and sizes back to
the builder's types - fixed cost per appended run, paid by every caller that
covers a run of rows with one repeated list.

A list builder can record the run from the scalar alone. `ListViewBuilder`
stores the elements once and points every view at them; `ListBuilder` repeats
them, because its offsets can only describe contiguous, in-order lists.
Dispatch to both through `match_each_list_builder!` and leave every other
builder on the canonical path.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005
perf(array): drop the rebuild and the sizes cast from append_listview…
7196ddd 
…_array

Trimming through `rebuild(ListViewRebuildMode::TrimElements)` subtracts the
window start from every offset with a compute kernel, only for the rebase in
`append_listview_array` to add this builder's elements base straight back on:
two passes, one of them through the compute stack, for one addition per offset.
Casting the sizes to the builder's type is another kernel for what is a copy.

Compute the referenced window here instead. An exact source covers it back to
back, so its first and last view bound it, and only some other layout has to be
searched for. Slice the elements to that window and rebase in the single pass
that was already walking the offsets. The sizes go straight into the builder's
`uninit_range` - as a `copy_from_slice` when they already have its type, and a
typed conversion loop when they do not.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005
perf(sparse): append patches and fills without slicing the patch array
c01dcc2 
Slicing is not the constant-time operation its doc comment claims:
`ArrayRef::slice` ends in `.optimize()`, so every slice pays a full optimizer
pass. Slicing a `ListView` is four of them, because slicing the array slices
its offsets, its sizes and its elements in turn. A profile of
`canonicalize_sparse_list[(512, 7, 4)]` put 42% of the benchmark in
`optimizer::try_optimize` under `ArrayRef::slice`.

So take each patch's elements instead of slicing runs of patches out of the
patch array: `list_elements_at` slices `elements` alone, which is one
optimizer pass over a primitive array rather than four over a list view. The
same reasoning applies to the fixed-size-list path, which sliced its elements
and its validity per run.

Gaps keep their bulk append, but reach it without a `ConstantArray`: the
fill's elements are materialized once, up front, and every gap points its rows
at that one array through `append_array_as_repeated_list`. The fill's elements
are now stored once for the whole result rather than once per gap, and a gap
costs nothing per row it covers.

`canonicalize_sparse_list` medians, against develop:

  (512, 7, 4)       24.1 us -> 22.5 us
  (1024, 17, 8)     33.4 us -> 18.7 us
  (4096, 8, 4)     173.3 us -> 119.6 us
  (4096, 64, 4)     99.9 us -> 17.3 us
  (8192, 1024, 4)  180.6 us -> 5.6 us

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005
perf(array): append constant structs, extensions and fixed-size lists…
dc479bf 
… directly

`Constant::append_to_builder` had fast paths for the flat dtypes and fell back
to canonicalizing everything else, which builds the whole run as an array only
to copy it into the builder. The nested dtypes with a builder can all skip
that, and two of them can skip the values entirely:

- a constant struct is a constant array per field, so each field's builder
  takes one as a chunk and the fields stay constant-encoded;
- an extension array is its storage wearing a dtype, so a constant one is a
  constant storage array;
- a fixed-size list cannot share one copy of its elements between rows the way
  a list view can, since its elements sit back to back. A null value still
  needs only placeholders, and a value whose elements are all the same scalar
  still tiles to a constant array. Otherwise the tile is copied in per row -
  one copy of each element, where canonicalizing first made two.

That last case is why `ChildBuilder` grows `append_array_values`: appending the
same tiny array over and over is the one case where a chunk per append costs
more than copying the values, and only the caller can see it.

`canonicalize_sparse_fixed_size_list` medians:

  (512, 7, 4)      79.0 us -> 55.0 us
  (1024, 17, 8)   139.5 us -> 63.9 us
  (8192, 1024, 4) 375.9 us -> 142.3 us

`Union` is the only dtype left on the fallback, and it has no builder yet.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005
perf(sparse): build the fixed-size-list fill tile once, not once per gap
681bd37 
`append_fill` handed the fill scalar to `Constant::append_to_builder` per gap,
which materialized the scalar's elements into an array every time. Hoist that
out of the loop the way the list path already does: the fill's elements become
an array once, up front, and every gap tiles that same array through
`FixedSizeListBuilder::append_array_as_repeated_list`.

Elements that are all the same scalar stay a constant array, so tiling them
over a gap costs nothing at all however many rows it covers. Anything else is
copied in per row, because a fixed-size list holds its elements back to back
and cannot point its rows at one shared copy.

`canonicalize_sparse_fixed_size_list` medians, against develop:

  (512, 7, 4)      21.1 us -> 25.6 us   (was 55.0 us)
  (1024, 17, 8)    29.1 us -> 32.9 us   (was 63.9 us)
  (8192, 1024, 4) 150.1 us -> 140.1 us  (was 142.3 us)

What is left of the gap against develop is the per-row copy: each row pays its
own append for `list_size` elements, where tiling a run could double the region
already written instead.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005 robert3005 added the changelog/performance A performance improvement label Jul 31, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 31, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚡ 6 improved benchmarks
❌ 1 regressed benchmark
✅ 1835 untouched benchmarks
⏩ 55 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation execute_scalar_struct_simple 427.2 µs 478.5 µs -10.72%
⚡ Simulation canonicalize_sparse_list[(512, 7, 4)] 2,960.6 µs 250.1 µs ×12
⚡ Simulation canonicalize_sparse_list[(1024, 17, 8)] 2,530.3 µs 221.9 µs ×11
⚡ Simulation canonicalize_sparse_fixed_size_list[(1024, 17, 8)] 1,282.7 µs 389.7 µs ×3.3
⚡ Simulation canonicalize_sparse_fixed_size_list[(512, 7, 4)] 715.4 µs 311.5 µs ×2.3
⚡ Simulation extend_from_array_zctl[(10000, 8)] 629.4 µs 490.3 µs +28.36%
⚡ Simulation extend_from_array_non_zctl_overlapping[(10000, 8)] 634.8 µs 495.2 µs +28.19%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

Comparing claude/constant-fast-paths-9ze0t6 (681bd37) with claude/builders-lazy-validity-9ze0t6 (dd4420b)

Open in CodSpeed

Footnotes

  1. 55 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog/performance A performance improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@robert3005