Skip to content

Tags: zarr-developers/zarr-python

Tags

zarr_metadata-v0.5.0

Toggle zarr_metadata-v0.5.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore(zarr-metadata): build 0.5.0 changelog (#4266)

* chore(zarr-metadata): note the sdist allowlist, and let misc entries speak

#4248 gave this package's sdist an explicit allowlist and merged without a
news fragment, so a user-visible packaging change was about to miss the
0.5.0 notes. Add one.

Filing it as `misc` exposed that towncrier's built-in `misc` type sets
`showcontent = false`: the entry would render as a bare PR link, which tells
a reader nothing. Restate all five types the `changes/README.md` menu offers
— declaring any type replaces the built-in set — as the defaults verbatim
except for `misc`, which now shows its content. A change worth a release
note is worth a sentence, whatever its category.

Assisted-by: ClaudeCode:claude-opus-5

* chore(zarr-metadata): build 0.5.0 changelog

Consume the pending news fragments — #4232's constant-naming-grammar removal
note, #4264's note widening `JSONValue`'s array arm to the covariant
`Sequence`, and #4248's sdist allowlist — into CHANGELOG.md via towncrier
for the zarr_metadata-v0.5.0 release.

Minor, not patch: the `JSONValue` widening changes a published type's
meaning for every consumer that annotates against it, and #4232 removes the
old version-last constant spellings outright.

Assisted-by: ClaudeCode:claude-opus-5

zarr_indexing-v0.2.1

Toggle zarr_indexing-v0.2.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fix(docs): build subpackage docs from the package directory (#4252)

Read the Docs pointed `-f` at each package's mkdocs.yml from the repo
root. mkdocs resolves some settings relative to the current working
directory rather than to the config file, so building from elsewhere
looks for them in the wrong place -- and silently, because the paths are
valid, just wrong.

zarr-indexing hit this at v0.2.0: `pymdownx.snippets` has a relative
`base_path` of `[docs, examples]`, so `--8<-- "snippets/canonical_slice.py"`
resolved against the repo root and searched zarr-python's own docs/
rather than the package's. The build failed with SnippetMissingError
while `just docs-check` passed, because that runs from the package
directory.

Building from the package directory makes the Read the Docs invocation
identical to the local and CI ones, so a green build there means a green
build here. $READTHEDOCS_OUTPUT is absolute, so the cd does not affect
where the site lands.

Applied to all three packages. Only zarr-indexing is failing today;
zarr-metadata and zarr-http-server do not use snippets, so for them this
is preventive -- the hazard is any config resolved against the working
directory, and it would show up only on Read the Docs.

Assisted-by: ClaudeCode:claude-opus-5

zarr_indexing-v0.2.0

Toggle zarr_indexing-v0.2.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat(zarr-indexing): LazyArray — generic lazy indexing over array-API…

… arrays (#4222)

* feat(zarr-indexing): LazyArray — generic lazy indexing over array-API arrays

A generic wrapper for any array-API-like source (numpy, zarr, cupy, ...)
adding TensorStore-style lazy indexing with a positional NumPy dialect:
eager __getitem__, .lazy/.oindex/.vindex composing transforms without
data access, result()/__array__ materializing.

Resolution is partition-based: parts() iterates the base array's
partitions projected through the view as resolvable sub-LazyArrays
(Partition carries global box coordinates, out placement, and
completeness); with_parts() re-partitions the same base explicitly; a
single lowering engine serves both partitioned and whole-array sources.
Partitioning is discovered from the source (read_chunk_sizes, .chunks)
or declared, and never surfaced as a chunks vocabulary.

Box selections (no index arrays; affine, interval-representable) are
first-class: is_box, bounding_box() (exact hull up to stride), and
strides() complete the slab-read story; the design-notes page records
the box-vs-query taxonomy and the relationship to TensorStore.

Dunders: __dask_tokenize__ (deterministic, canonical-ndsel-body-based),
__len__, __iter__, 0-d conversions, pickling. Degenerate all-singleton
index-array maps now collapse to constant maps in the transform algebra,
and NumPy advanced-index placement rules are implemented faithfully.

Assisted-by: ClaudeCode:claude-fable-5

* feat(zarr-indexing): negative-step slices, per merged ndsel 1.0-draft.2

`arr[::-1]` reverses. One desugaring rule covers both signs, as TensorStore
0.1.84 does and as ndsel PR #2 now specifies: omitted bounds resolve on the
side the traversal starts and stops (`hi-1` and `lo-1` going down), the source
interval is [start, stop) going up and [stop+1, start+1) going down, an empty
interval is legal at any coordinate, an interval running the wrong way is an
error rather than a silent empty, and the origin is trunc(start/step) for
either sign.

The corpus is re-vendored from ndsel 92d6a32 in this same commit, because it is
the definition of correct here: `slice.json` gains ten negative-step fixtures,
`errors.json` retires `negative_step_unsupported` for three
`bounds_out_of_order` fixtures, and the message layer is changed to satisfy
them. The retired reason code is documented as such rather than removed.

A latent bug that only negative steps could reach: `_reindex_array` built
`slice(pos, pos + size*step, step)`, and a downward walk reaching the front of
the array computes a negative stop, which NumPy reads as counting from the end
— `slice(6, -1, -1)` selects nothing where `slice(6, None, -1)` selects seven
elements reversed. Both reindex helpers now go through `_positional_slice`.

The stride<0 branches of `_intersect_dimension_map` and
`iter_chunk_transforms` were written defensively and had never been reachable.
They are now, and they were right: the parts-coverage test gains two reversing
views, and the seeded sweep generates downward slices (4,352 of 7,200 chains
carry one) across every partitioning.

At the wrapper boundary the dialect stays NumPy's, which differs in one place:
a reversed *positional* interval like `lazy[2:5:-1]` is empty, not an error,
because that is what `x[2:5:-1]` means. Only literal coordinates call it a
direction error.

Tests: the study's recorded TensorStore corpus lands in
`test_tensorstore_parity.py` — fifteen desugarings with their domains, offsets
and strides, the three rows that discriminate trunc from floor and ceil, both
error families, empty-outside-the-domain, five recorded compositions, and the
recorded index-array reversal (a negative step over a gathered axis reverses
the array rather than attaching a stride).

Assisted-by: ClaudeCode:claude-fable-5

* polish(zarr-indexing): re-review minors — step-zero ValueError, kw-only Partition, strides docs

- slice step zero now raises ValueError, matching NumPy in the wrapper's
  positional dialect (was IndexError)
- Partition is keyword-only: box was inserted mid-field-list, so
  positional construction would silently misbind
- strides() documents the empty-box case (bounding_box None, strides
  still defined)

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): plain technical language throughout

Rewrite the package's documentation surfaces — docs/index.md,
docs/design-notes.md, docs/ndsel.md, docs/api/index.md, the LazyArray,
boundary, and transform docstrings, and the 267 changelog fragment — in
plain declarative English. Metaphor, personification, rhetorical framing,
and emphasis used for effect are replaced with statements of the same
technical content. No technical claim, API name, example, or example
output changes.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): American spelling (flavour -> flavor)

Assisted-by: ClaudeCode:claude-fable-5

* ci(zarr-indexing): scoped lint ignores for the deliberate blind excepts

A new ruff release (the CI job floats via uvx) flags BLE001/S110 at the
chunk-discovery tolerance and tokenize-fallback sites. Both catches are
intentional contracts: discovery must degrade to no-information on any
foreign-object failure, and a token call must never raise. Configured as
per-file-ignores rather than noqa comments because the pinned pre-commit
ruff strips the comments as unused (RUF100) while the floating CI ruff
requires them.

Assisted-by: ClaudeCode:claude-fable-5

* ci(zarr-indexing): pin ruff in the lint job and justfile

Mirrors the main-branch pin (d-v-b#271) so this PR's workflow runs the
same ruff version; bump together with the pyproject pin.

Assisted-by: ClaudeCode:claude-fable-5

* docs: add lazy-indexing examples for NumPy and Dask

Two runnable examples in the house style: wrapping a NumPy array in
LazyArray (attribute forwarding, composing selections, box vs query
selections, partitions), and using a LazyArray with Dask (from_array,
one task per partition, deterministic tokens).

Assisted-by: ClaudeCode:claude-fable-5

* docs: compare dask task graphs with fused transforms in the dask example

Adds a timed comparison of chained selections through dask.array against
the same selections composed into one transform, and a section on when
each is the right tool: dask's graph earns its cost when there is
computation across chunks, and is overhead when it only defers indexing.

Assisted-by: ClaudeCode:claude-fable-5

* test: run examples against this repository's local packages

The example runner rewrote only the `zarr` dependency to the local
checkout, so an example depending on an in-repo package resolved it from
git main instead — and the lazy-indexing examples failed in CI, since
LazyArray is not on main yet. Rewrite every package this repository
ships, leaving dependencies an example does not declare alone.

Assisted-by: ClaudeCode:claude-fable-5

* feat(zarr-indexing): negotiate what indexing a source supports

`LazyArray` assumed every wrapped array could do basic slicing and did all
fancy work itself, reading a block and post-indexing it with NumPy. That
over-reads when the source could gather natively, and it walks a source
axis by axis with `take` where one request would do.

Each wrapper now carries an `IndexingSupport` level — BASIC, OUTER,
OUTER_1VECTOR, VECTORIZED, the taxonomy and member names of xarray's
`IndexingSupport` — and every read is split into the largest part of the
selection that level can express, asked of the source in one call through
`oindex`/`vindex` when it has them, and a residual transform applied to
the block that comes back. The split is applied per partition as well as
per whole-array read, so a part costs one request.

The level is resolved at construction: an explicit `with_indexing_support`
wins, then the source's own `__zarr_indexing_support__` (read defensively),
then conservative inference — a NumPy array or zarr's `oindex`/`vindex`
pair reads as VECTORIZED, everything else as BASIC, the only assumption
that is always correct. A multi-array outer request only ever goes to an
`oindex` accessor, because a bare `__getitem__` key with two arrays means
an outer product to HDF5 and a correlated gather to NumPy.

The level decides how much data crosses the boundary, never what
`result()` returns. Tests hold that invariant directly: every selection
case, at all four levels, against NumPy and zarr sources, partitioned and
not; plus test doubles that raise when handed a key their declared level
forbids, so exceeding a declaration fails loudly rather than working by
accident.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): token the data, not how it is read

The token included the partitioning while deliberately excluding the
indexing-support level, though both are read strategies that leave the
values unchanged. Excluding both means two wrappers that describe the
same data token alike, so a consumer caching on tokens reuses one result
across partitionings and support levels.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): an empty downward walk selects nothing

A slice with a negative step whose start lies before the front of the
axis selects nothing, but the positional slice was written as
`slice(start, stop, step)` with a negative stop, which NumPy reads as
counting from the end: an empty selection of a fancy axis returned the
whole axis reversed, and the correlated form raised a broadcast error.
Write an empty selection out explicitly.

The randomized basic-selection generator drew negative-step starts from
`[0, size)` only, so a start before the front of the axis was
unreachable and the suite could not see this. It now draws from below
`-size` as well, and three cases pin the behavior directly.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): a selection of slices is not a fancy selection

An `oindex`/`vindex` step whose entries are all slices carries no
coordinates: it narrows the view's own axes and must compose like basic
indexing. `_reindex_array_oindex` instead applied each entry positionally
to the corresponding axis of the existing index array, without asking
whether that axis is one the array varies over or a singleton it merely
broadcasts along — the distinction its basic-indexing sibling
`_reindex_array` has always made. A slice starting past 0 therefore
indexed a size-1 broadcast axis out of range and truncated the whole
index array to size 0. A view with no coordinates left resolves to no
parts, and `result()` handed back its unwritten `np.empty` buffer: live,
on the default path, for any source that advertises chunks.

`_reindex_array_oindex` now takes the `ArrayMap` and applies an entry
only along its dependency axes (plus the `input_dimension` that breaks
the tie for a degenerate length-1 orthogonal selection), preserving a
broadcast singleton whatever the slice says. Coordinates never reach a
broadcast axis — `_guard_fancy_after_fancy` still rejects genuine
fancy-after-fancy with `NotImplementedError`, now under test.

Four defects from the same review ride along:

- `_array_map_dependency_axes` counted a length-**0** axis as an axis the
  array varies over, so an empty orthogonal selection classified as
  correlated and `array_map_dependent_axis` rejected it — a raise on the
  unpartitioned path where every partitioned path returned the right
  empty answer. An axis of size 0 carries no dependency any more than a
  singleton does.
- `parts()` raised on a view emptied by a slice over an axis of extent 1.
  A correlated selection of one point normalizes to an all-singleton
  index array, so emptying the domain leaves the array at size 1 and the
  resolver went looking for a chunk. An empty input domain now yields no
  parts and meets no output domain, matching `result()`.
- `sub_transform_to_selections` built `slice(stop + 1, start + 1, stride)`
  for a negative stride — endpoints swapped, step still negative, so it
  selected nothing where the reversed axis was meant. Both branches now
  lower through `_positional_slice`, the same walk an `ArrayMap` axis is
  reindexed by, which knows a downward walk reaching the front must stop
  at `None`.
- `compose()` evaluated an inner index array over `range(size)` rather
  than over the outer domain's own range, and addressed it from 0 rather
  than from the inner domain's origin. Every coordinate resolved to the
  wrong cell whenever a domain did not start at 0 — which a step-1 slice
  and a negative-step slice both produce routinely here.
- `transform_from_canonical` now rejects a non-integer `index_array` with
  an `NdselError` carrying `invalid_json`, instead of silently truncating
  `[0.9, 1.9]` to cells 0 and 1, coercing booleans, or leaking NumPy's own
  conversion error for strings.

The fuzzer missed the first two because `_random_chain` drew at most one
fancy step and had no way to spell a step that goes through a fancy
accessor while carrying only slices. It now draws such a step separately,
and the seeded sweep gained a `parts()` counterpart: `result()` can
absorb a defect that the iteration contract cannot, since an empty view
assembles correctly from no parts at all. Both sweeps fail on the
pre-fix source, as does the new exhaustive stride/extent sweep over the
chunk-selection bridge.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): count a domain axis no output map depends on

A `vindex` coordinate array with a singleton broadcast axis contributes an
axis it does not vary over. A later basic index that consumes the axis it
*does* vary over collapses the map to a `ConstantMap` and leaves the
broadcast axis in the domain, referenced by nothing. Three places assumed
that could not happen:

- `sub_transform_to_selections` built `out_selection` with one entry per
  output map, so a view with such an axis got an index tuple of lower rank
  than the buffer. `out[out_selection] = value` then placed the part against
  the leading axes and broadcast the rest — silently wrong data on a
  partitioned read, and a `parts()` walk that left cells unwritten.
- `_restore_domain_axis_order` put an unreferenced axis back as a singleton
  whatever the domain said. At extent 0 that fabricated a row for a
  selection whose own `shape` reported it empty.
- `_lower_correlated` built its flat gather index from the domain's
  broadcast shape but added coordinates straight off the stored index array,
  which is singleton on the axes it does not vary over. The two disagree
  exactly when a correlated map is constant along a shared broadcast axis.

`out_selection` is now built per domain dimension throughout, an
unreferenced axis is restored at its own extent, and a correlated map's
coordinates are broadcast to the block before being combined.

The randomized chain sweep never generated the shape at fault: `_random_vindex`
only produced `(length,)` and `(length, 1)` coordinate arrays, neither of
which leaves a singleton axis for a later step to strand. It now draws a
broadcast rank and places each array's varying axis within it, which
reproduces all three failures on the unfixed code.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): a materialized view never hands back the source

Five fixes to the wrapper's edges, none of which changes what a selection
means.

`result()` and `__array__` no longer alias the wrapped array. An
unpartitioned read of a basic selection lowers to plain slicing, so it came
back as a *view* of the source; NumPy 2 hands whatever `__array__` returns
straight to the caller, so `numpy.array(view, copy=True)` aliased it and a
write reached through. Under any partitioning the same read allocates, so
this also made the answer depend on how the read was divided. The result is
now detached whenever it may share memory with the wrapped array, and the
`copy=False` refusal no longer justifies itself with a claim the other
branch violated.

`result()` verifies that the partition walk covered the output before
returning it. The buffer is deliberately uninitialized, so any defect in the
walk was reported as plausible-looking numbers rather than as an error. The
cells each part addresses are counted from the selectors' own shapes —
nothing is read — and a walk that does not add up to the view's size raises.
Measured on a 16 MiB read: 51 us of accounting against 6.5 ms of read for 64
parts, within noise end to end, and +1.7% at 512 parts.

The `BASIC` floor is a promise about the *source*, not about the blocks it
returns. The residual is finished with `take`, `reshape` and `transpose`,
which were applied to the block unconverted — so a source meeting exactly
the documented floor crashed on `oindex[[4, 0, 0], :, :]`. A block that is
neither a NumPy array nor an array-API namespace of its own is now coerced,
which leaves a device array where it is.

`numpy.matrix` is refused at construction: it never reduces rank, so a
view's shape and its result disagree on every rank-reducing selection. A
`numpy.ma` source keeps its mask through a partitioned read, which allocates
a masked buffer. A declaration holding a *foreign* enum member that names
one of these four levels — xarray's `IndexingSupport`, whose members these
are borrowed from — is honored rather than discarded, since discarding it
fell through to inference and answered with a *more* permissive level than
the source asked for.

`is_complete` is true for a reversing view, which reads every cell of its
box back to front; the stride-1 test it failed was about direction, not
coverage. `with_parts` accepts `(0,)` and `(0, 0)` for a zero-length axis,
which said the same thing as the `()` and uniform spellings it already took,
and the positivity error names the working form.

Above the token digest limit and without dask, `__dask_tokenize__` returns a
value that matches nothing rather than a shape-and-dtype description that
two different 4 MiB arrays shared. A cache keyed on it misses instead of
lying.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): correct claims a reviewer found false

Every statement below was executed before being rewritten, and the
replacement was executed too.

- "`view + 1`" / "arithmetic materializes through `__array__`" is false.
  `LazyArray` defines no arithmetic dunders, so `view + 1` raises
  `TypeError`. What does work is a NumPy *function* — `numpy.add(view, 1)`,
  `numpy.sum(view)`, `numpy.stack([view, view])` — and an ndarray on the
  left of the operator. Corrected in the module docstring, `docs/index.md`
  and the changelog fragment.
- "An empty selection returns `None` from both" is false: an empty *box*
  reports `strides()` and only `bounding_box()` is `None`. The `strides()`
  docstring already said so; the design notes now agree with it.
- "A box touches a contiguous run of parts" is false for a strided box —
  `[::4]` over 2-wide parts visits every other part. The true property, and
  the one a partition-walk optimizer would want, is a regularly-spaced run
  in increasing order, each part at most once.
- "TensorStore permits a lower-rank index array" is backwards. Checked
  against tensorstore 0.1.84: its JSON parser rejects a rank-1 array over a
  rank-2 domain and accepts full rank with singletons, which is what we
  emit. *Our* loader is the permissive one. The passage now says both models
  want full rank, keeps the real rationale (the singletons are what makes
  the orthogonal/vectorized distinction derivable), and describes our
  lower-rank acceptance as the compatibility affordance it is.
- "Two limits remain" omitted fancy-after-fancy, which is a live
  `NotImplementedError` reachable from the documented surface, while
  `index.md` invited chaining fancy steps "anywhere in the chain". Current
  scope now lists five limits, including the diagonal-view and mixed
  correlated/orthogonal ones, and both prose pages point at it.
- "A single whole-array part stays in the wrapped array's namespace" is only
  true with *no* partitioning: `result()` branches on whether a partitioning
  is in force, not on how many boxes it has, so `with_parts((4, 6))` on a
  4x6 array returns a plain ndarray.
- The changelog stated the support-detection precedence backwards
  (declaration wins, not inference); `index.md` had a sentence missing its
  noun; the module docstring's one-line `bounding_box()` summary dropped the
  stride caveat the three other locations keep; and the package README, the
  PyPI long description, never mentioned `LazyArray`.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): hold the full-rank invariant inside the engine

The index-array rank was checked only from above, so a lower-rank array
could exist inside the engine and be read for dependency axes it did not
have. Nothing produced one: the tolerance was there for a test asserting
compatibility with a body TensorStore itself rejects (verified against
0.1.84 — a rank-1 array over a rank-3 domain is an error in its JSON
parser). Require the full input rank in the type, widen a lower-rank
array at the JSON boundary where external input arrives, and give the
test the shape TensorStore accepts.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): an index array spans the domain it is read over

An index array axis must be the domain's extent or a singleton it
broadcasts over. Any other size leaves input coordinates with no entry,
which read as a smaller selection rather than as the error it is: the
truncated array behind one of this review's silent-corruption bugs was a
(3, 0) array over a (3, 2) domain, which this rejects at construction.

Two fixtures carried the inconsistency they were meant to exercise — an
empty array over a domain with room for two coordinates, and a widening
case whose array covered three of four positions — and now describe
domains their arrays span.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-indexing): a state machine for chained indexing, and the rank-0 part it found

Adds `zarr_indexing.testing`, behind a `testing` extra: a Hypothesis
state machine that composes indexing steps onto a LazyArray and checks
each step's shape, `result()`, and `parts()` assembly against NumPy, plus
the selection strategies on their own. A project can point it at its own
array by overriding one method.

The machine asserts the documented assembly literally — a part's values
must arrive at the shape its out_selection addresses — which is how it
found the defect it also fixes: intersecting a correlated transform with
a part's bounds collapsed the surviving broadcast block into one axis
even when the block was already rank 0, so a view narrowed to a single
point produced parts of rank 1. A rank-0 block now stays rank 0, and
`result()` drops the reshape that was absorbing the mismatch.

Merged from the branch that produced it, which predates the coverage
guard in `result()`; the guard stays and the reshape it compensated with
goes.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): the defects an adversarial review found at the boundaries

Six reviewers went at the package, each proving findings by execution. The
algebra held: ~85k chained selections against NumPy, ~24k part assemblies with
poisoned buffers, 3.8k transform round-trips evaluated as coordinate maps, all
clean. Everything below was at a boundary.

`result()` and `__array__(copy=True)` could hand back a live view of the
source. `_detach` asked whether the *source* was an ndarray, but a duck array
that merely stores its data in NumPy returns NumPy views, and those went
straight to the caller — while three docstrings promised the opposite
unconditionally. It now asks whether the *result* owns its buffer, so memory is
released only when sharing is disproved rather than when it cannot be
established.

The wire format could not reload its own output. `tolist()` renders every empty
array as `[]` once the leading axis is the zero-length one, so an ordinary empty
selection lost the axis it varied over, and the loader put it back on a
different one by prepending singletons. Nested lists cannot express the shape
either, so the body carries it. `index_domain_from_json` was a second
undefended way into the same objects: a bare `int()` that truncated 3.9, coerced
"3" and True, and let a non-string label into a tuple[str, ...]. It goes through
the message layer now, as a transform body always did. With it: a rank ceiling,
i64 checks on desugared bounds so normalization stays idempotent, ordered
index_array_bounds, and typed errors where raw ones leaked.

`sub_transform_to_selections` transposed its blocks two ways. A ConstantMap was
emitted as a bare integer, which NumPy counts among the *advanced* indices
whenever an index array is present, moving the broadcast axis to the front when
a slice separates them — while `out_selection` is built positionally. And the
correlated branch documented a points-major block but assembled the chunk
selection in output order, so a residual slice before the coordinates arrived
slice-major. Constants are length-one slices now, named in drop_axes, and the
correlated scatter is permuted to the block NumPy actually returns. Nothing
caught either one because `LazyArray` resolves a part through its own lowering
and never reads `chunk_selection`; one of the two was even asserted as correct
in a passing test's comment.

Three further failures were one stale field: `_apply_vindex` carried
`input_dimension` onto a map the vindex had just made correlated. The value
outlived the shape that justified it and was believed later by a scatter that
filed positions under the wrong axis — which is why one view's answer depended
on how it was partitioned. The dependency is read back off the array now, and
`__post_init__` checks the field it was wrong about: ArrayMap was the one map
whose `input_dimension` nothing validated.

Also: `oindex` over a correlated view applied its index tuple positionally,
NumPy's vectorized rule, collapsing two arrays into one axis; `compose()`
indexed an inner array's broadcast singletons by the raw coordinate and sized
its one-dimensional shortcut by the input rank while gating on the output rank;
and neither checked that the outer transform's output lands inside the inner
domain, where a negative would have wrapped.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-indexing): generate the selections that were never generated

A mutation audit ran 31 mutations and 10 survived. They were not scattered:
the invariants check thoroughly what a view *returns* and never what it *claims
about itself*, and the strategies could not draw whole classes of selection.

The generators now draw them. Orthogonal slices carry a step and may stop
early, so a strided or reversed slice reaches `oindex` at all. Coordinate lists
and masks can be empty, so a fancy selection that selects nothing exists — the
shape that lost its axis on the way through JSON. Vectorized coordinate arrays
can be multi-dimensional, so a rank-raising `vindex` is generated. Measured
over 4000 draws each, every one of those counts was previously 0.

The first run of the widened generators found a live defect: an empty Python
list carries no element type and NumPy defaults it to float64, so `oindex[[]]`
was refused as a non-integer index array. `json.py` already had that case; the
boundary did not. NumPy takes `a[np.ix_([])]`, and so does this now.

`Partition.is_complete` gets an invariant. It is what a consumer reads to
decide it may take a whole-box read, so a wrongly-`True` one is silent
corruption — and three separate mutations to `_covers_whole_part` survived the
entire suite, including ones reporting `is_complete` for a part carrying two of
its three cells. Asserted one way only: the flag is documented as conservative
and only the claim to cover everything has to be earned.

Two more state machines. The sorted one-dimensional fancy path needs both ranks
to be 1, and the output rank is the *source's*, so the rank-3 default source
walled it off entirely — 0 hits from the machine against 105 from the unit
tests, in the path where reordering and duplicate coordinates are partitioned.
A rank-1 source now reaches it 322 times per run. A source with an extent-1
axis covers the other side of a distinction the code draws from the domain
rather than the array.

Finally the two remaining mutants, both checked by mutating and confirming the
failure: the index-array bound checks are probed one past the boundary rather
than comfortably outside it, and `_out_selection_cell_count`'s guard is tested
directly, since a partition walk only ever produces the forward in-bounds
intervals that never reach it.

Assisted-by: ClaudeCode:claude-fable-5

* refactor(zarr-indexing)!: settle the API decisions that get dearer after 1.0

`with_parts` decided what it had been given by inspecting the type of it: a
sequence of integers meant uniform boxes, a sequence of sequences meant
per-axis sizes, and `None` meant no partitioning at all — which also sent
`result()` down an entirely different code path. Three semantics behind one
parameter, and no way to ask for one of them and be told when you had spelled
it wrong. They are now `with_parts`, `with_parts_per_axis` and
`unpartitioned`. A harness that draws from a list of mixed partitionings still
needs the dispatch, so it exists once, as `zarr_indexing.testing.repartition`.

The sizes those methods take are relative to the array being read, not to the
view reading it, and a narrowed view partitions the base extents — which could
only be discovered from an error message. `base_shape` says it.

`ArrayMap`, `IndexTransform` and `Partition` are `frozen=True`, which reads as
a promise that a value can be compared and hashed. Both raised: `==` on the
index arrays returned an array and then `ValueError: the truth value of an
array is ambiguous`, and `hash()` refused an ndarray outright. So no transform
could enter a set or key a cache, and the package had already grown an internal
`_is_identity_transform` because of it.

An `ArrayMap` was frozen but the array inside it was not, so reaching through
a view's transform to `index_array[0] = 9` silently changed what the view
returned. It is held through a read-only view now — a view rather than a flag
on the caller's array, since constructing a map should not take away the right
to write to an array you still own.

`IndexDomain.narrow` clamped a slice bound to the domain, in a package whose
stated invariant is no clamping and no negative wrapping. `narrow(slice(-3,
None))` on `[0, 10)` therefore returned the whole axis — reading as the NumPy
spelling of "the last three" and answering with something else — and a stop
past the end returned a domain its own parent did not contain. Both raise
`BoundsCheckError` now, and a stride raises `ValueError` like every other
unimplemented request rather than `IndexError`.

`Partition.array` was a view of the array while `LazyArray.array` was the raw
source: adjacent types, one name, inverted meanings. The part's is `view`.

Smaller: `BoundsCheckError` and `VindexInvalidSelectionError` are exported at
the top level, being what exported functions raise; `errors.py` no longer
claims `zarr.errors` re-exports them by identity, which is false and would have
been believed; `parts()` says it is single-use; and
`sub_transform_to_selections` says it is provisional rather than implying the
rest of the API's stability.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): correct the claims a reviewer could check, and two dialect gaps

`boundary.py` justified applying scalars before the advanced indices by
asserting NumPy does the same, citing `a[0, [1, 2], :]`. NumPy groups a scalar
*with* the advanced indices for placement, so the two disagree the moment a
slice separates them: `a[0, ..., [1, 2]]` is `(2, 3)` where `a[0][..., [1, 2]]`
is `(3, 2)`. Scalar-first is this package's documented dialect and stays; the
reasoning was wrong, and a wrong reason invites someone to "fix" the correct
end later.

Two places where the dialect really did diverge, both now matching NumPy. A
zero-dimensional integer array is a scalar — `a[np.array(2), :]` drops its axis
— but only Python and NumPy integers counted, so a 0-d array was widened into a
length-1 index array and kept an axis; that was a third answer, agreeing with
neither NumPy nor eager zarr. And a multi-dimensional array in an orthogonal
selection is refused where the rule lives, instead of surfacing two layers down
as a rank complaint about an `index_array` the caller never wrote.

`iter_chunk_transforms` documented two shapes for `out_indices` and returns
three: the `dict[int, ndarray]` an orthogonal selection with several index
arrays produces was missing, from the function downstream integrators use most.

Packaging: the README is the PyPI long description, so the monorepo-only
development section moved to CONTRIBUTING.md and the dead relative link to the
justfile went with it. The examples pinned `zarr-indexing` to a moving `main`
rather than to a release they document. Three of the six docs pins claimed to
match the repo root and did not, and the justfile's ruff comment contradicted
the package's own pin.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): collapse an empty index array instead of extending the format

The previous commit fixed the empty-selection round trip by carrying an
`index_array_shape` field, on the reasoning that JSON nested lists cannot
express the shape of an array with a leading zero axis — `[]` is the only
spelling of every empty shape, and `[[]]` is (1, 0) with nothing for (0, 1).
That much is true, but the conclusion was wrong: it invented a field ndsel does
not define, so the documents this package wrote stopped being ndsel documents.

The reference implementation does not have the problem, because it never emits
an empty index array. `t[ts.d[0][[]]]` in TensorStore is `out[0] = 0`, emitted
as `{}` — a constant map. An empty index array names no cell, and it can only
be empty because an input dimension is, since the full-rank invariant makes
every axis either 1 or the domain's extent. Nothing is ever read through it, the
emptiness is carried by the domain, and the map is degenerate in exactly the way
a size-1 array is — which this format already collapses.

So it collapses the same way, and the extension is gone. The loader still
recovers the axis from the domain for an empty array arriving from a producer
that does emit one, since ndsel does not forbid it; that path just no longer has
a first-party caller.

Checked against tensorstore 0.1.84: every canonical body from 4000 randomized
transforms loads into `ts.IndexTransform`, and every one re-emits itself
unchanged. No spec change needed.

Assisted-by: ClaudeCode:claude-fable-5

* fix(indexing): address lazy array review findings

Assisted-by: Codex:gpt-5

* fix(zarr-indexing): keep an empty masked result masked whichever parts are in force

An empty view is now answered without reading the source, and the shortcut
reached for the array namespace's own `empty`, which knows nothing about masks.
An unpartitioned empty view over a masked source therefore came back a plain
array while the same view partitioned came back masked — no cells either way,
so no value changed, but the caller's type depended on how the read had been
divided, which `result()` promises it never does. A masked source goes through
`_output_buffer`, which is the branch that knows.

Assisted-by: ClaudeCode:claude-fable-5

* docs(indexing): design chunk projection API

Assisted-by: Codex:gpt-5.6

* feat(indexing): add reusable chunk plans

Assisted-by: Codex:gpt-5.6

* feat(indexing): project chunks through paired transforms

Assisted-by: Codex:gpt-5.6

* refactor(indexing): build lazy parts from projections

Assisted-by: Codex:gpt-5.6

* refactor(indexing): expose projection-only chunk planning

Assisted-by: Codex:gpt-5.6

* docs(indexing): design visual indexing guide

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain coordinate origins

Assisted-by: Codex:gpt-5.6

* docs(indexing): motivate negative chunk coordinates

Assisted-by: Codex:gpt-5.6

* feat(indexing-docs): add SVG diagram renderer

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): harden diagram rendering

Assisted-by: Codex:gpt-5.6

* feat(indexing-docs): add accessible guide diagrams

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): correct guide figure semantics

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): prevent selection label overlap

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): validate arrow label offsets

Assisted-by: Codex:gpt-5.6

* test(indexing-docs): add executable guide examples

Assisted-by: Codex:gpt-5.6

* docs(indexing): add NumPy-first visual tour

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain chunk projections visually

Assisted-by: Codex:gpt-5.6

* docs(indexing): add indexing and integration references

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): satisfy strict example typing

Assisted-by: Codex:gpt-5.6

* docs(indexing): connect visual guide to reference docs

Assisted-by: Codex:gpt-5.6

* fix(indexing-docs): source landing quickstart from example

Assisted-by: Codex:gpt-5.6

* ci(indexing): verify executable visual docs

Assisted-by: Codex:gpt-5.6

* fix(indexing): address visual guide review

Assisted-by: Codex:gpt-5.6

* fix(indexing): improve chunk overlay on phones

Assisted-by: Codex:gpt-5.6

* docs(indexing): design system-memory chunk cache example

Assisted-by: Codex:gpt-5.6

* docs(indexing): introduce half-open intervals

Assisted-by: Codex:gpt-5.6

* test(indexing): narrow diagram label elements

Assisted-by: Codex:gpt-5.6

* docs(indexing): demonstrate a system-memory chunk cache

Assisted-by: Codex:gpt-5.6

* chore(indexing): stop tracking design specs

Assisted-by: Codex:gpt-5.6

* feat(indexing): apply and invert transforms

Assisted-by: Codex:gpt-5.6

* fix(indexing): handle scalar and wide transform coordinates

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain the chunk-cache lifecycle

Assisted-by: Codex:gpt-5.6

* fix(indexing): keep lifecycle diagram readable

Assisted-by: Codex:gpt-5.6

* fix(indexing): keep lifecycle caption stationary

Assisted-by: Codex:gpt-5.6

* fix(indexing): keep guide diagrams readable

Assisted-by: Codex:gpt-5.6

* test(indexing): enforce unique guide figure wrappers

Assisted-by: Codex:gpt-5.6

* test(indexing): scan all guide sources for figure duplicates

Assisted-by: Codex:gpt-5.6

* docs(indexing): distinguish cache indexing modes

Assisted-by: Codex:gpt-5.6

* docs(indexing): scope lazy examples to package

Assisted-by: Codex:gpt-5.6

* fix(indexing): expose docs modules to root tests

Assisted-by: Codex:gpt-5.6

* docs(indexing): omit text from diagram legends

Assisted-by: Codex:gpt-5.6

* docs(indexing): strengthen chunk outlines

Assisted-by: Codex:gpt-5.6

* docs(indexing): label unselected chunk cells

Assisted-by: Codex:gpt-5.6

* fix(indexing): show coordinates in basic selection

Assisted-by: Codex:gpt-5.6

* docs(indexing): clarify coordinate-value mapping

Assisted-by: Codex:gpt-5.6

* docs(indexing): simplify half-open intervals

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain ordered concatenation

Assisted-by: Codex:gpt-5.6

* docs(indexing): consolidate visual guide

Assisted-by: Codex:gpt-5.6

* docs(indexing): clarify basic selection figure

Assisted-by: Codex:gpt-5.6

* docs(indexing): simplify coordinate introduction

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain result axis construction

Assisted-by: Codex:gpt-5.6

* docs(indexing): enclose slice result axis

Assisted-by: Codex:gpt-5.6

* fix(indexing): clarify result array comparison

Assisted-by: Codex:gpt-5.6

* fix(indexing): preserve tutorial result ranks

Assisted-by: Codex:gpt-5.6

* docs(indexing): promote chunk cache example

Assisted-by: Codex:gpt-5.6

* docs(indexing): render chunk cache source

Assisted-by: Codex:gpt-5.6

* fix(indexing): close final correctness gaps

Assisted-by: Codex:gpt-5.6

* docs(indexing): replace diagrams with ascii

Assisted-by: Codex:gpt-5.6

* chore(indexing): remove svg diagram pipeline

Assisted-by: Codex:gpt-5.6

* docs(indexing): simplify guide navigation

Assisted-by: Codex:gpt-5.6

* fix(indexing): align selection diagram columns

Assisted-by: Codex:gpt-5.6

* feat(indexing): add explicit array readers

Assisted-by: Codex:gpt-5.6

* refactor(indexing): resolve lazy arrays through readers

Assisted-by: Codex:gpt-5.6

* docs(indexing): exercise readers in chunk cache example

Assisted-by: Codex:gpt-5.6

* fix(indexing): preserve cache request event ordering

Assisted-by: Codex:gpt-5.6

* refactor(indexing): remove indexing capability taxonomy

Assisted-by: Codex:gpt-5.6

* docs(indexing): explain explicit reader execution

Assisted-by: Codex:gpt-5.6

* test(indexing): avoid constructor spelling assertion

Assisted-by: Codex:gpt-5.6

* fix(indexing): complete reader migration

Assisted-by: Codex:gpt-5.6

* fix(indexing): keep reader helpers private

Assisted-by: Codex:gpt-5.6

* docs(indexing): finalize reader safety contract

Assisted-by: Codex:gpt-5.6

* feat(indexing): add compact chunk grids

Assisted-by: Codex:gpt-5.6

* fix(indexing): check affine coordinate arithmetic

Assisted-by: Codex:gpt-5.6

* fix(indexing): validate composed constants

Assisted-by: Codex:gpt-5.6

* fix(indexing): expose grid size representation

Assisted-by: Codex:gpt-5.6

* fix(indexing): validate direct advanced selections

Assisted-by: Codex:gpt-5.6

* fix(indexing): handle boolean list masks

Assisted-by: Codex:gpt-5.6

* fix(indexing): project sparse affine selections directly

Assisted-by: Codex:gpt-5.6

* fix(indexing): normalize chunk planner positions

Assisted-by: Codex:gpt-5.6

* fix(indexing): lower reader transforms through bounded slabs

Assisted-by: Codex:gpt-5.6

* refactor(indexing): expose global partition read context

Assisted-by: Codex:gpt-5.6

* fix(indexing): resolve read context annotations

Assisted-by: Codex:gpt-5.6

* feat(indexing): reuse prepared partition plans

Assisted-by: Codex:gpt-5.6

* fix(indexing): validate prepared partition coverage

Assisted-by: Codex:gpt-5.6

* test(indexing): enforce transform materialization laws

Assisted-by: Codex:gpt-5.6

* docs(indexing): clarify chunk projection locality

Assisted-by: Codex:gpt-5.6

* fix(indexing): harden transform and partition validation

Assisted-by: Codex:gpt-5.6

* docs(indexing): separate examples from snippets

Assisted-by: Codex:gpt-5.6

* feat(indexing): support index protocol selectors

Assisted-by: Codex:gpt-5.6

* feat(indexing): compose fancy selections without restriction

A second oindex/vindex/mask step may now land on any axis of an
already-fancy view, including axes an existing index array merely
broadcasts along. Array-carrying transforms route through compose()
instead of being rewritten in place: the selection is applied to an
identity transform over the current domain (same dialect by
construction) and chained on, which evaluates the existing index arrays
at the new coordinates. The in-place reindex machinery and its
fancy-after-fancy guard are deleted.

Resolution classifies transforms by structure (the new public
index_array_structure): pure per-axis outer products keep the orthogonal
resolvers; correlated maps, mixtures, and index arrays sharing an input
axis (diagonal gathers) all take the pointwise path, whose intersect and
lower stages now broadcast per-map blocks instead of assuming full-block
index arrays. Only hand-built affine diagonals (an index array and a
slice map bound to the same axis) remain unsupported.

This removes the crash where a slice-only vindex step
(view.lazy.vindex[...] or vindex[..., scalar]) after a correlated gather
misclassified the gather as orthogonal and failed at result(), and fixes
a stale input_dimension surviving integer indexing of an empty map's
pinned axis.

Also, from the same review: result(parts=...) raises ValueError instead
of AssertionError when supplied parts do not tile the view; with_parts
and with_parts_per_axis raise the documented ValueError for non-iterable
input; prepared-part validation uses plain assignment for box parts; and
__dask_tokenize__ digests the canonical transform body instead of
embedding it, keeping tokens small for large fancy selections. The
testing state machine now draws any number of fancy steps per chain.

Assisted-by: ClaudeCode:claude-fable-5

* refactor(indexing)!: retire ArrayMap.input_dimension

What an index-array map depends on is now read from one place: its
full-rank array's shape, whose non-singleton axes are the dependency
axes. The retired field pinned the orthogonal axis redundantly and could
contradict the array it rode on; every bug found in two adversarial
review rounds traced back to its bookkeeping (stale values surviving
reindexing, misclassification of correlated maps, dangling axes after
integer indexing).

The one shape the field disambiguated - a single-coordinate,
all-singleton array - is normalized away instead: the selection and
composition layers build it as the ConstantMap it equals
(output_map.array_map_or_constant), exactly as the JSON serializer has
always collapsed it on the wire. A consequence is that a length-1 fancy
selection now classifies as a box (is_box, bounding_box, strides),
which is the semantically sharper answer. Hand-built all-singleton,
empty, or shared-axis ArrayMaps remain valid values and resolve through
the pointwise path.

Fallout removed with the field: the post-init consistency validation,
the basic-indexing renumbering of pinned axes, the JSON loader's global
dependency-axis reconstruction, and composition's binding carry-through.
The sorted 1-D chunk-planning fast path now applies to either fancy
spelling, since the flavors coincide in one dimension.

The wire format is unaffected - it never carried the field. BREAKING:
ArrayMap.__init__ no longer accepts input_dimension, and the attribute
is gone; array_map_dependent_axis now answers from the shape alone.

Assisted-by: ClaudeCode:claude-fable-5

* feat(zarr-indexing): a reader for sources that only accept unit-step slices

BasicReader reads the minimum by pushing strided and descending selections
down as positive-step slices, which assumes the source accepts any step.
Integrating a zarrs-backed array showed how common the narrower contract is:
FFI bindings and HTTP range endpoints support nothing but
slice(start, stop, 1), leaving every such backend to rewrite the same
cover-and-restride lowering in its facade.

UnitStepReader moves that lowering behind the Reader boundary. The
decomposition covers each DimensionMap with the smallest ascending unit-step
slab and replays the original stride in the residual — the same move the
basic decomposition already makes for direction, extended to magnitude. The
residual lowering needed no change: positive strides slice the block,
descending ones were always gathered.

The cost is explicit in the docstring: a strided selection over-reads its
cover by the stride factor, bounded by partitioning the wrapping LazyArray.
The existing reader contract cases and the affine-overflow parity test now
run across all three built-in readers, through a source that rejects
anything but ascending unit-step slices inside its bounds.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): the dense-box re-partition idiom

is_box and strides() exist so a consumer can choose a read strategy, but the
three lines that act on them were only discoverable by deriving them. The
integrations guide now states the policy: a dense box resolves best as one
backend slab read — re-partition to the base shape and let the backend
dispatch, decode in parallel, and partial-decode shards on its own side —
while strided boxes and gathers keep the partitioning, which bounds every
cover by one part and makes hull-sized reads of sparse selections
structurally impossible.

The snippet is executable and pins both regimes by observed reads: the
corner gather touches four single cells, never the hull; the dense box is
exactly one call. A closing subsection points sources that only accept
unit-step slices at unit_step_reader.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-indexing): re-vendor ndsel conformance corpus at 49b9e1db

Re-vendor from zarr-developers/ndsel main (49b9e1db1ca93c55f320b025a666367de87a9014,
merge of ndsel PR #3). Only transform.json changed vs the previously vendored
92d6a32d: two new fixtures pin empty index_array serialization —
normalize carries an empty index_array verbatim rather than rewriting it
to a constant map, while a producer SHOULD collapse it to a constant
output map, which zarr_indexing.json already does. All other corpus files
are byte-identical.

Assisted-by: ClaudeCode:claude-fable-5

* ci(zarr-indexing): run the tensorstore parity tests

test_ndsel_tensorstore.py skipped everywhere because tensorstore was in
no dependency group or workflow. Add a dedicated step to the test job
that overlays tensorstore (>=0.1.84, wheels cover the whole 3.12-3.14
matrix) and runs the two parity modules; the main pytest run stays
byte-identical to the local canonical invocation. Verified locally
against tensorstore 0.1.85 on CPython 3.14: 88 passed, 0 skipped.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-indexing): derive the docs include graph instead of registering it

The doc-example tests accreted during the documentation build-out as one-off
guardrails: hand-maintained registries of snippet regions, exact heading and
navigation strings, substring pins on teaching prose, and tombstones for
migrations that already happened. Each new snippet had to be registered by
hand, and editing a sentence could fail CI. The registries also missed the
one failure they existed to prevent: a page including a region nobody
registered was invisible to them.

The suite now states two kinds of contract and nothing else. Structural:
every '--8<--' include in the rendered markdown — discovered by scanning, so
new snippets are covered the day they are written — resolves to exactly one
file with a balanced, non-empty region, and every snippet executes, its
inline assertions serving as the value check where expected values were
previously duplicated into test tables. Behavioral: the pattern matrix, the
wrapped-source contract, and the chunk-cache lifecycle keep their tests,
because an example cannot assert its own error paths.

pymdownx.snippets gains check_paths: true, so the strict docs build now
fails on an unresolvable include instead of silently rendering nothing.
Verified both directions: a deliberately broken region name fails the scan
test (naming page, file, and region) and the docs build. Editorial choices —
section order, wording, nav — return to review, where they belong.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): lead every page with the simple idea

The guide opened with two meta-paragraphs and an annotated table of contents;
the one-sentence mental model — lazy indexing builds a view, planning
partitions it, result() materializes it — sat below them. It opens with that
sentence now, and says plainly that the first four sections serve anyone
indexing arrays while the last two serve integrators, so most readers know
they can stop early.

Advanced material moves out of the beginner path. Negative-origin domains,
grid prepending, and the EdgeDimensionGrid/DimensionGridLike comparison sat
in section two, before the reader had met a transform; they now live in the
design notes as 'Negative-origin domains and prependable grids', linked from
the two places that want them. The paired-projection section introduces the
cell domain concretely — a table with one row per selected cell, chunk-local
address on one side, result position on the other — before naming it.

The landing page gains the missing why: many arrays support only plain
slicing, and this package grafts the full NumPy dialect onto them. The
pattern reference leads with the selection matrix readers come for and moves
the positions-vs-literal-coordinates table after it. The materialization
warning becomes a list, and captions that narrated their own code are cut.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): document every public method of the public API

Every public class, method, and property reachable from the package root now
carries a docstring stating its contract: the coordinate frame it speaks
(global source, chunk-grid, or zero-origin chunk-local), whether a chunk
length is the declared codec size or the boundary-clipped data extent, what
is bounds-checked and what extrapolates, and which inputs raise which
errors. Protocol members (DimensionGridLike, DimensionGrid) are written as
implementer obligations, since the docstring is the contract a third-party
grid must satisfy.

Dataclass-generated __init__ methods are left to their class docstrings;
adding a docstring there would mean hand-writing the constructor for no
behavioral reason.

Docstrings only — no code, signature, or existing-docstring changes.
Verified: an introspection audit over __all__ reports zero public members
without docstrings; the full suite and pyright are unchanged.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): attribute docstrings for every public field

The previous pass documented methods and properties; dataclass and TypedDict
fields — IndexDomain.exclusive_max and 41 siblings across fifteen classes —
carried no per-attribute documentation, only prose in their class docstrings.
Each public field now has an attribute docstring stating what the value
means, its coordinate frame or units, and the invariant it carries (literal
bounds may be negative, edges are declared codec sizes unclipped by extent,
derived fields say what they are derived from, wire bounds admit the
infinities the engine refuses to lower).

Found by an AST audit, since attribute docstrings are invisible to runtime
introspection; that audit now reports zero undocumented public fields.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): the input/output-to-request/source dictionary, and each map kind in NumPy terms

An IndexTransform is a function between coordinate spaces, and its fields
speak function vocabulary (input, output) while every array-minded reader
speaks request and source. The confusion this causes is concentrated in one
word: 'output' looks like data, but names the output side of the coordinate
function — which is where values are read FROM, since data flows against
the arrow. The transform section of the guide and the transform API page
now state the dictionary outright, in a two-row table, at the moment a
reader first meets the fields, along with why the neutral names exist:
composition, where an interior transform has neither a request nor a
source side.

The three output map kinds are now demonstrated executably against their
NumPy counterparts: DimensionMap against basic and negative-step slices,
ArrayMap against fancy indexing with order and duplicates preserved, and
ConstantMap against numpy.broadcast_to — stated as the value-faithful
counterpart precisely because no NumPy selection spells a retained
constant axis; an integer index drops it, and a repeated fancy index
matches the values while degrading the description to a coordinate list.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): flat landing page — motivation, example, links

The grid cards misrendered (misaligned card bodies) and earned their keep
poorly: two navigation targets dressed as a layout feature. The landing page
now follows the shape convention of projects like pydantic — motivation
paragraph, install, one quickstart with a sentence stating the lazy/eager
boundary, then a single annotated link list. The two cards' start-here
targets survive as the visual guide entry, which names both audiences and
their entry points in one line each.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): a single-point bounds error never mentions a batch

apply() delegates to the vectorized kernel shared with apply_many() — the
right direction, since the batch path is the hot one — but the kernel's
diagnostic leaked through it: apply((11,)) on a [-10, 10) domain reported
'point at batch position ()', naming a batch the caller never formed, from
a private frame the caller never called.

The kernel now raises an internal structured signal (dimension, value,
bounds, batch position) and each public entry formats it in its own
vocabulary, 'from None' so the traceback ends at the API layer: apply says
'coordinate 11 on input dimension 0 is outside the domain [-10, 10)';
apply_many keeps the batch-position form, where that context is exactly
right. Message-only change; BoundsCheckError remains the type on both
paths.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): annotation syntax in docstring type slots

Parameters and Returns entries now state types as annotations —
Sequence[int], tuple[int, ...], numpy.typing.NDArray[numpy.intp] — instead
of prose like 'sequence of int' or bare 'tuple'. The annotation is the
type's one precise spelling, matches the signature beside it, and names
the exact shape where prose left it to the description (both boundary
functions' bare 'tuple' entries now state their element structure).

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): the chunk cache example is named for what it is

The example was born napari_chunk_cache, and the docs spent two bold
disclaimers insisting it is a napari-like consumer, not a napari
integration — while the nav entry, section headings, and class names had
already settled on 'system-memory chunk cache'. A name that needs
disclaimers is the wrong name. The directory, script, and docs page are now
system_memory_chunk_cache, matching everything else; napari remains where
it belongs, in prose, as the motivating access pattern.

Path-only rename: no code, region names, or prose claims change. The
derived include-graph test and the strict docs build (check_paths) verify
every include and link followed the move.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-indexing): an empty domain reads as empty through every reader

Composing a fancy selection onto an empty-domain view emits an ArrayMap
that is legitimately empty along the vanished axis — a shape the package
promises resolves like any other. The resolvers broke that promise:
_correlated_map_coords tried to reshape the 0-size array to its non-zero
singleton block axes and raised ValueError from all three built-in readers
on a direct read_into, a sequence the pre-composition engine handled.
LazyArray.result() masked it only through its own size-0 short-circuit.

_lower now answers an empty domain first — nothing is selected, so no
resolver needs to evaluate maps that may be empty along vanished axes —
and the correlated path independently returns no coordinates for an empty
broadcast block. Found by an adversarial review fuzzing composed selection
chains (3 of 400 random chains hit it); the regression test pins the exact
public-API reproduction across all three readers.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-indexing): execute the CLI examples; strict builds guard anchors and nav

Two claims from the doc-test restructure were false, and an adversarial
review proved both empirically. First, the lazy_indexing_* examples were
said to run under the repository-root example runner; that runner globs
only the root examples directory, so the two CLI examples ran under no
test at all. They now run as subprocesses here, the dask one skipping
where dask is absent. Second, the module docstring claimed mkdocs --strict
covered the deleted anchor and nav guards; a strict build passed with a
deliberately broken cross-page anchor and with a page omitted from nav,
because both are INFO-level by default. mkdocs.yml now sets those
validations to warn, which strict promotes to errors — verified failing
on a broken anchor and passing clean.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): the vindex error's real contract, and neutral map vocabulary

VindexInvalidSelectionError's docstring claimed it covered every
non-coordinate vindex form; in fact only the wrapper's validation raises
it, only for slices — other invalid entries raise plain IndexError, and
the engine-level IndexTransform.vindex accepts residual slice dimensions
without raising. The docstring now states the actual raise site.

The transform-algebra docstrings (transform, output_map, composition,
json) also drop 'storage' for neutral input/output vocabulary: a
transform's output side is just output coordinates — in a composition
chain an interior transform has no storage side at all. The
request/source/storage translation stays where it belongs, in the guide's
vocabulary table and the endpoint layers (readers, grids) that really do
face arrays.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): an executable Examples section on every public object

Every public class and function in the package root now carries a numpydoc
Examples section in doctest form: 37 new examples, each a small
intuition-builder in the shape the IndexTransform walk-through set — the
domain is the result's coordinates, the output maps are the rule, and where
an object corresponds to a NumPy indexing concept the example demonstrates
the equivalence (DimensionMap against a slice, ArrayMap against fancy
indexing with duplicates surviving, ConstantMap against numpy.broadcast_to,
compose against chained slicing). The error classes demonstrate their
actual raise; the wire types round-trip real bodies.

The examples are enforced, not decorative — and closing that loop exposed
that the package's existing doctests were never collected anywhere: the
package pyproject shadows the repository root's pytest configuration, and
no invocation named src. The package config now enables --doctest-modules
with the root's option flags, testpaths includes src/zarr_indexing, and the
justfile recipe and CI workflow collect it explicitly. 1280 tests pass,
45 of them doctests.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): apply and apply_many say what they locate, not their signature

'Map one input coordinate to an output coordinate' restates the type
signature in prose — any function maps inputs to outputs. The summaries now
speak the class docstring's array-indexing frame: apply maps a coordinate
of the domain (a result cell) to the source coordinate its value is read
from, by evaluating each output map; apply_many is the batch form. Both
gain a doctest locating cells of the [::2] transform, and both state that
no data is touched — this is the coordinate arrow, running result to
source.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): method summaries never lean on the ambiguous naked input/output

'input' and 'output' are reserved algebra terms in this package, and in a
method summary they collide with ordinary function-speak: identity's
'input coordinate i maps to output coordinate i' reads equally as the
algebra statement and as a vacuous description of any function — the same
trap apply's summary fell into. identity, intersect, and translate now
speak the array frame instead: every result cell reads the source at its
own address; keep only the cells whose source coordinates fall inside the
box; shift the source coordinates every cell reads. Field docstrings and
class summaries keep the naked terms where no call is in sight and the
technical reading is the only one available.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): drop non-behavior statements that disambiguate nothing

'No data is touched' in apply/apply_many and 'without I/O' on the
oindex/vindex accessors stated what the functions do not do without
clarifying what they do: nothing about a coordinate lookup, or about an
accessor documented to return a new transform, suggests data movement.
Removed. The statements that earn their negation stay: the module thesis,
the class contract, and __getitem__ — where subscription syntax genuinely
suggests an eager read to anyone arriving from zarr.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): a flat sidebar — sections must earn their existence

'Use lazy indexing' held one page: a disclosure triangle and a competing
label with no organization gained. 'Practical reference' classified
nothing and grouped two pages serving different audiences — the exact
split the guide's opening and the landing page's annotated links already
route explicitly. Both dissolve into top-level entries. Examples and API
Reference keep their sections, being the only real collections at this
site's size; the ndsel wire format and design notes gain explicit labels
and sit in the for-builders tail before the API.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): the three guide pages are one Guide section

Visual guide, indexing patterns, and integration boundaries sat at top
level with the same rank as the fifteen-page API Reference — three pages
wearing category clothes. They are one collection, and the docs/guide/
directory said so all along: learn it, look it up, apply it at the…

zarr_http_server-v0.1.0

Toggle zarr_http_server-v0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
HTTP server that exposes stores, arrays, groups (#3732)

* implement store server and node server

* update tests and add v2 -> v3 example

* add __all__ and clean up tests

* add docs and changelog

* add proper server

* rework examples (simplify) and make server a context manager

* minor tweaks to server

* fix: byte-order handling for structured dtypes in the bytes codec (#220)

* fix: byte-order handling for structured dtypes in the bytes codec

The bytes codec neither byte-swapped structured-dtype fields to its
configured endian on encode (numpy reports byteorder '|' for void
dtypes, so the top-level byteorder comparison never detected a
mismatch) nor honored its endian when decoding, silently corrupting
any structured data whose field byte order differed from the stored
one (e.g. virtual references to external big-endian data).

Encode now detects byte-order mismatches by comparing full dtypes via
newbyteorder, and decode reinterprets raw bytes in the stored byte
order before converting to the data type's declared byte order, so the
stored layout (codec state) and the in-memory layout (array data type)
are independent.

Closes #4141

Assisted-by: ClaudeCode:claude-fable-5

* test: fold structured byte-order cases into existing bytes codec tests

Extend test_endian's parametrization with structured dtypes and
test_bytes_codec_sync_roundtrip with endian/dtype parametrization plus
stored-layout and decoded-dtype assertions, instead of adding parallel
test functions for the same properties.

Assisted-by: ClaudeCode:claude-fable-5

* refactor: rename stored_dtype to view_dtype in BytesCodec decode

The variable is the dtype used to view the raw chunk bytes (byte order
from the codec's endian configuration), not a property of the stored
data or of the returned buffer, which always carries the array's
declared dtype.

Assisted-by: ClaudeCode:claude-fable-5

* docs: note that the decode-side byte-order conversion copies the chunk

Assisted-by: ClaudeCode:claude-fable-5

* fix: adapt server branch to current starlette/uvicorn

starlette 1.3 deprecated using httpx with its TestClient, which the
warnings-as-errors filter turns into a collection error; add httpx2 to
the test dependency group. uvicorn 0.51 removed
Server.install_signal_handlers and now skips signal-handler setup off
the main thread natively, so drop the monkey-patch workaround.

Assisted-by: ClaudeCode:claude-fable-5

* feat(zarr-server): scaffold packages/zarr-server

Assisted-by: ClaudeCode:claude-fable-5

* feat(zarr-server): move server code, tests, and example into the package

The moved code now uses only public zarr API: zarr.buffer.cpu replaces
zarr.core.buffer.cpu, spec-defined key names are inlined, chunk-key
encodings are duck-typed on their spec names, and the shard grid shape is
computed locally from public Array attributes.

Also mirrors the root repo's [tool.numpydoc_validation] override in the
package's pyproject.toml, since numpydoc-validation resolves config from
the nearest pyproject.toml and would otherwise apply its stricter default
checks to the moved docstrings.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-server): package README; retarget root changelog fragment at the core fix

Assisted-by: ClaudeCode:claude-fable-5

* refactor!: remove the HTTP server from zarr core

The server now lives in packages/zarr-server (published as zarr-server).
The feature never shipped in a zarr release, so there is no deprecation
shim. The decode_chunk_key strictness fix stays in zarr core.

Assisted-by: ClaudeCode:claude-fable-5

* ci(zarr-server): add package test and release workflows

Assisted-by: ClaudeCode:claude-fable-5

* chore(zarr-server): refresh package lockfile after root server-extra removal

The package lock embeds the workspace-root zarr project's metadata; Task 4
removed the root server extra and httpx2 test dep after this lock was first
generated.

Assisted-by: ClaudeCode:claude-fable-5

* fix: reject path-traversal segments in zarr-server request handling

store_app (and thus serve_store) passed the request path straight to
the store without validation, since the is_valid_node_key gate only
ran for node_app. Starlette percent-decodes path params, so a request
like GET /..%2fsecret.txt arrived as literal ".." and LocalStore
resolved it outside the store root, allowing arbitrary file read via
GET and arbitrary file write via PUT. Closes this path-traversal
vulnerability by rejecting any "." or ".." path segment in
_handle_request before the store is touched, for both store_app and
node_app.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-server): close absolute-key path-traversal bypass in request guard

The path-traversal guard in _handle_request only rejected "." and ".."
segments. A percent-encoded leading slash (e.g. "/%2fetc%2fhostname")
decodes to an absolute path param ("/etc/hostname"), whose split()
produces an empty leading segment with no "." or ".." segment, so the
guard let it through. LocalStore resolves an absolute key by discarding
its configured root, allowing arbitrary filesystem read/write outside
the store. Reject empty segments too, closing the bypass.

Assisted-by: ClaudeCode:claude-fable-5

* ci(zarr-server): bump actions/attest to v4.2.0 to match sibling workflow

Keeps the zarr-server release workflow's pinned action SHAs in sync with
the dependabot bump that landed in zarr-metadata-release.yml via the merge.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-server): fold backslash separators and reject drive-qualified keys

The traversal guard in _handle_request split only on "/", so backslash-
separated segments like "..\\..\\win.ini" and drive-qualified or
UNC-rooted keys like "C:/Windows/win.ini" or "\\host\share\x" passed
through untouched. On Windows, LocalStore joins keys onto its root via
pathlib, which discards the root entirely for a drive-qualified or
rooted key -- turning percent-encoded backslash paths into arbitrary
file read/write.

Fold backslashes to "/" before the segment check (mirroring zarr's own
normalize_path) and add an ntpath.splitdrive check to catch drive
letters and UNC prefixes that don't produce empty/"."/".." segments.

Added TDD coverage that fails against the old guard (via a store double
that raises if get/set is ever called) and passes against the fix.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-server): fix vacuous two-level traversal test

The `/..%2f..%2fsecret.txt` case in test_encoded_traversal_variants_
return_404 climbed two levels from tmp_path/store_root to tmp_path/..,
but the secret was written at tmp_path/secret.txt (one level up) --
so the case passed for the wrong reason (no such file, not "guard
blocked it") even with the traversal guard deleted entirely.

Nest the store root exactly `climb_depth` directories below tmp_path
per case, so every case's ".." segments resolve to tmp_path/secret.txt.

Verified by mutation: copied _serve.py to a scratch dir (never the
repo), deleted the guard body, and ran TestPathTraversalProtection
against it via PYTHONPATH. Before this fix, 12/13 traversal tests
failed against the mutant (1 false pass -- this vacuous case). After
this fix, 13/13 fail against the mutant, and all still pass against
the real guard.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-server): bound BackgroundServer shutdown instead of blocking forever

shutdown() joined the server thread with no timeout, and uvicorn's
graceful wait is itself unbounded: force_exit is only set by a signal
handler uvicorn deliberately skips off the main thread. A client mid
request could wedge __exit__ unrecoverably.

Bound uvicorn's graceful wait with timeout_graceful_shutdown and fall
back to force_exit if the thread outlives it, both driven by a new
shutdown_timeout parameter on serve_store/serve_node.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-server)!: only accept HTTP methods the handler implements

_handle_request special-cased PUT and let every other verb fall through
to the read path, so a server built with DELETE/POST/PATCH answered them
with the key's contents and changed nothing. HTTPMethod advertised all
of them.

Narrow HTTPMethod to GET/PUT/HEAD and reject anything else when the app
is built. Doing this before the first release avoids narrowing a
published type later.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-server): document client deps; lint the package's changelog in CI

The README's round-trip example reads back through FsspecStore, which
needs an HTTP-capable fsspec that zarr-server does not depend on, so a
clean install failed on the PyPI landing page's headline snippet. Same
for examples/serve.py under a plain interpreter, which needs httpx.

check_changelogs.yml also never visited packages/zarr-server/changes, so
a malformed fragment would have passed PR CI and failed at release time.

Assisted-by: ClaudeCode:claude-fable-5

* test(zarr-server): add docs dependency group and cover the README round-trip

Reading a served array back with zarr.open_array(url) routes through
FsspecStore -- zarr's only URL-string backend -- so the README's headline
example needs an HTTP-capable fsspec that the package itself has no
reason to depend on. Carry that in a docs group, mirroring the root
project's group for running examples, and use it to test the round-trip
in CI so the example cannot rot.

Assisted-by: ClaudeCode:claude-fable-5

* refactor(zarr-http-server)!: rename zarr-server to zarr-http-server

The package is HTTP-specific end to end -- Starlette/ASGI, byte-range
headers, CORS, HTTP verbs -- so the unqualified name overclaimed its
scope and squatted the generic name that a future transport (an
S3-compatible or WebDAV frontend) would want.

Renames the distribution, the zarr_http_server module, the package
directory, the zarr_http_server-v* release tags, both workflows and
their artifact and PyPI environment names. Nothing is published yet, so
this costs nothing now and would be permanent after the first upload.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-http-server): reject an empty methods set instead of failing open

Starlette's Route treats a falsy `methods` as "match every method", and
an empty set passes the unsupported-verb check trivially, so
store_app(store, methods=set()) served GET, DELETE, POST, PATCH and
TRACE alike and accepted PUT writes -- the opposite of what the caller
asked for, and the same fail-open the method narrowing set out to close.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-http-server): report the bound port, unblock the loop, 404 unopenable children

Three defects the adversarial re-review reproduced:

BackgroundServer reported the requested port, so port=0 produced
http://host:0 while the socket was bound elsewhere -- clients following
the documented url reached an unrelated service.

Group key validation opened children through zarr's synchronous API
directly on the event loop, serializing every concurrent request behind
it (20 concurrent requests: 10.5s, now 1.1s) and outlasting the shutdown
timeout. Move it to a worker thread.

Child lookup caught only KeyError, so a corrupt metadata document or a
codec from an uninstalled plugin surfaced as 500 and made a whole
subtree unservable during ordinary operation.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-http-server): close the S5-S13 follow-ups from the adversarial review

Hostile keys the store cannot express (embedded NUL, over-long names) now
answer 404 instead of surfacing ValueError/OSError as a 500.

PUT bodies are capped at DEFAULT_MAX_BODY_SIZE and answer 413 past it;
Store.set takes a whole Buffer, so a body cannot be streamed and one
request would otherwise size the server's memory.

206 responses carry Content-Range, and a range beyond the end or an
inverted one answers 416 rather than an empty 206.

A 0-d v2 array's sole chunk, stored under "0", is now servable: the key
decodes to a 1-tuple no 0-d grid could match, so it needs the array's
dimensionality to disambiguate.

Array and group metadata key sets are split, so a v2 group no longer
claims .zarray as its own; bind failures report the likely cause instead
of a bare timeout; and serve_* gain a background: bool overload.

The two out-of-bounds chunk tests planted no data, so they passed for the
wrong reason -- mutating the bounds check to return True left the suite
green. They now plant data at the out-of-grid key, and that mutation
fails 3 tests.

Adds the missing coverage for this branch's one core change, and widens
the workflow's path filter to src/zarr, since the package resolves zarr
from the repo root and a core change can break it.

Assisted-by: ClaudeCode:claude-fable-5

* fix(zarr-http-server): stop reporting I/O failures as misses; enforce the body cap while reading

The round-3 review found the previous round's error handling was too broad
and its body cap too narrow.

except (ValueError, OSError) around store.get and store.set caught the whole
errno family, so EACCES, EROFS and ENOSPC all answered 404. Under the v3 spec
an absent chunk is an uninitialized one and a reader is right to substitute
the array's fill value, so 404 asserts something about the store's contents:
an unreadable chunk answered that way has a correct client silently
materialize fill values over data that exists, and a failed write looks like
a pointless one rather than a failure. Only ENAMETOOLONG and EINVAL now mean
"this key names nothing"; everything else surfaces. NUL bytes are rejected in
the guard instead, so the store's own errors always mean real I/O trouble.

max_body_size only consulted Content-Length, which a chunked request does not
send, so request.body() buffered the whole thing before the check: 256 MiB
passed a 1 KiB cap at 572 MiB peak RSS. The body is now read incrementally
and abandoned at the cap -- the same attack peaks at 59 MiB.

Also: a range too wide to allocate answers 416 rather than raising
MemoryError; the force_exit fallback join is bounded, so shutdown cannot
outlast its timeout; DEFAULT_MAX_BODY_SIZE is importable, since it is the
documented default of three public functions; and _make_starlette_app loses a
parameter it never read.

Adds the coverage the review found missing: shard-grid bounds (the mutant
served an out-of-grid shard key against a green suite), wrong-arity chunk
keys, the chunked body path, I/O failures as 5xx, and a parser-level
assertion for inverted ranges, which the status code alone could not
distinguish on a MemoryStore.

The drive-letter check stays unconditional. It over-rejects a first-segment
node name like a:b, which is legal on POSIX -- but ntpath treats any single
character before a colon as a drive, so there is no safe subset, and the
guard is a string gate in front of an arbitrary Store whose path semantics
this package cannot know.

Assisted-by: ClaudeCode:claude-fable-5

* fix(ci): silence BLE001 on two intentional broad excepts

ruff 0.16 selects BLE001 under the root config's `B` prefix, so the
package's `uvx ruff check .` job failed on two deliberate blind excepts.
Both are intentional and already documented, so they get targeted noqa
comments, matching how #4213 handled the same rule in the core tree.

Assisted-by: ClaudeCode:claude-opus-5

* chore: drop stale changelog fragment for the chunk key fix

The DefaultChunkKeyEncoding.decode_chunk_key fix this fragment described
was split out into #4219 and has already shipped -- its text is in
docs/release-notes.md verbatim. Leaving the fragment here would emit the
same paragraph a second time under a #3732 link, for a change this branch
no longer contains.

Assisted-by: ClaudeCode:claude-opus-5

* ci: check zarr-http-server changelogs and align action pins

check_changelogs.yml validated the root, zarr-metadata and zarr-indexing
changes/ directories but not zarr-http-server's, so the new package's
fragments were the only ones whose filenames went unchecked.

The two new workflows were also written before #4241 bumped the actions
group, so they pinned older checkout/setup-uv/attest/pypi-publish SHAs
than their siblings. Bump them to the versions main already uses; the
release workflow is now identical to zarr-metadata's modulo the package
name.

Assisted-by: ClaudeCode:claude-opus-5

* docs(http-server): add justfile and Read the Docs scaffold

Brings the package in line with zarr-metadata and zarr-indexing, which
each own a justfile and a separate Read the Docs site. The docs content is
a scaffold -- an overview page and an API reference over the public
namespace -- meant to be filled in later; the point is to get the site
wired up and building under --strict now.

The `docs` dependency group previously held the runtime deps for the
README examples, but Read the Docs and `just docs-check` both expect it to
carry the mkdocs toolchain, as it does in the sibling packages. Those
example deps move to a new `examples` group and the test job follows.

CI gains a `docs` job matching the siblings', so a scaffold that stops
building fails the gate. The repo-root .readthedocs.yaml skips PR builds
confined to this package now that it has its own site, and the root docs
nav links out to it.

Creating the Read the Docs project itself is a manual step: point its
configuration-file path at packages/zarr-http-server/.readthedocs.yaml.

Assisted-by: ClaudeCode:claude-opus-5

* fix(http-server): correct silent-corruption and HTTP-conformance defects

Two of these answered a request successfully while leaving the client
with data that does not exist.

Chunk keys were validated by decoding coordinates and bounds-checking
them, never by re-encoding. `int` is lenient in ways a store key is not
-- leading zeros, a leading `+`/`-`, surrounding whitespace, underscore
separators, non-ASCII decimal digits -- so `c/00/00` decoded to (0, 0)
and validated, then went to the store verbatim. A PUT answered 204 and
stored the body under a key no reader looks up: success reported, data
invisible. Validation now requires the key to equal
`metadata.encode_chunk_key(coords)`, which makes the accepted set exactly
the set zarr can read. Decoding delegates to the encoding's own decoder
rather than reimplementing the default/v2 grammars, so a new or
third-party chunk key encoding works without changes here.

Resolving a group child caught bare `Exception` and returned False, which
turned an unreadable child -- EACCES, EIO, a corrupt metadata document, a
missing codec plugin -- into 404. Under the v3 spec an absent chunk is an
uninitialized one, so a correct reader answers that 404 by substituting
the fill value over data that exists. Only KeyError is caught now; a key
that could not be judged surfaces as 5xx rather than being reported
absent.

The rest are conformance fixes on the same request path:

- A Range header the server cannot use is now ignored with a 200 rather
  than refused with a 416, per RFC 9110 §14.2. This covers an
  unrecognized unit and a multi-range request, both legal to send.
- A suffix range resolves against the object's size, so a 206 always
  carries the Content-Range that RFC 9110 §15.3.7 requires. Sharding
  reads a shard index this way, so the header was missing on a hot path.
- A last-byte-pos wider than the store can materialize is clamped to the
  end of the object per §14.1.2 instead of raising out of the store as a
  500.
- A byte position is parsed as 1*DIGIT rather than by `int`, which
  accepted `+0`, ` 0` and `0_0`.
- PUT to a read-only store answers 403 instead of letting the store's
  ValueError surface as a 500.

Assisted-by: ClaudeCode:claude-opus-5

* test(http-server): property tests over a real endpoint

Adds hypothesis properties that drive a real uvicorn server over a
socket, so the assertions cover what only exists on the wire: header
parsing, method dispatch, status codes.

Each property checks the response *and* the backing store. That pairing
is what the previous suite could not do: a PUT to a non-canonical chunk
key answered 204 and wrote a key no reader consults, which a
response-only assertion cannot see. Refused requests assert the store is
byte-for-byte unchanged; accepted writes assert the bytes landed under
the key the client named and that a zarr client reads back the values.

Keys are generated in two families -- in-band (the node's metadata and
the canonical spelling of each chunk key in its grid) and out-of-band
(non-canonical spellings, out-of-grid coordinates, traversal probes, a
sibling node's keys).

Two details worth keeping:

- Traversal probes are percent-encoded. An HTTP client resolves
  dot-segments before sending, so httpx turns "../secret" into "/secret"
  and a literal probe asserts nothing; encoded, it reaches the server and
  Starlette decodes it back into a real ".." segment.
- The matrix includes LocalStore, not just MemoryStore. MemoryStore
  slices a `bytes` and accepts any range bound, so it cannot distinguish
  a clamped over-wide range from a refused one -- mutation-testing the
  suite showed that property passing against deliberately broken code
  until a filesystem-backed server was added.

Verified by reverting each fix in turn and confirming the corresponding
property fails.

Assisted-by: ClaudeCode:claude-opus-5

* fix(http-server): stop reading EINVAL from a store as a missing key

`_names_nothing` reclassified two errnos as absence so a client could not
turn a freely chosen key into a 5xx. `ENAMETOOLONG` earns that: it is the
store answering about the name -- nothing can be stored under a name it
cannot express -- and `encode_chunk_key` never produces a segment near a
filesystem's length limit, so it is unreachable for real data.

`EINVAL` does not. It is POSIX's catch-all, reachable on a perfectly
ordinary short key through a bad seek or an unsupported filesystem
feature, and under the v3 spec an absent chunk is an uninitialized one --
so answering 404 has a correct reader write fill values over a chunk that
exists but could not be read. That is the same defect already fixed in
the group-child lookup and in chunk key validation: reporting "something
went wrong" as "it is not there".

Nothing exercised EINVAL in practice. A NUL in a key raises ValueError
and is rejected before the store anyway, and an over-long key raises
ENAMETOOLONG, so this narrows the guard to the case that was doing the
work.

The traversal, absolute-key and drive-letter guards deliberately stay
where they are rather than deferring to the store. LocalStore.get/set
are `self.root / key` with no validation -- zarr's normalize_path applies
at the StorePath layer, not to raw store keys -- so `../sibling.txt` and
an absolute key both write outside the store root. This server is the
component that feeds a Store unvalidated strings from the network, so it
is the component that has to reject them.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): stop sealing over CORSMiddleware and uvicorn.Config

Wrapping an API means taking responsibility for its parameters, not
hiding the ones we did not think to name. Two wrappers were doing the
latter.

`CorsOptions` carried 2 of `CORSMiddleware`'s 8 parameters, so
`allow_headers`, `allow_credentials`, `allow_origin_regex`,
`allow_private_network`, `expose_headers` and `max_age` were unreachable
without bypassing this package. It now mirrors the full signature, with
every key optional.

Two of the defaults are ours rather than Starlette's, because the server
knows what its caller should not have to. It emits `Content-Range` on
every ranged response, which is not a CORS-safelisted response header, so
`expose_headers` defaults to `["Content-Range"]` -- without it a browser
client could read the bytes but not learn which bytes it got, which made
the suffix-range Content-Range fix invisible to exactly the clients CORS
exists for. It accepts a `Range` request header, so `allow_headers`
defaults to `["Range"]`; Starlette's empty default answered a preflight
naming Range with 400. Defaults apply only to absent keys, so an explicit
`expose_headers: []` means "expose nothing".

`_start_server` passed 4 of `uvicorn.Config`'s 52 parameters, which put
TLS, `proxy_headers`/`forwarded_allow_ips`, `root_path`, `log_level`,
`limit_concurrency` and unix-socket binds out of reach entirely. A
`uvicorn_options: Mapping[str, object] | None` is merged over the three
options set here, so a caller key wins. uvicorn ships no TypedDict for
Config -- its only TypedDicts are ASGI protocol events -- so the mapping
is hand-typed and the cast is confined to the call site.

Un-sealing uvicorn makes two BackgroundServer attributes reachable that
could previously only be one thing. `url` now reports the scheme actually
in use, so configuring TLS yields https, and `host`/`port`/`url` are None
for a uds or fd bind rather than naming an address nothing is listening
on.

Assisted-by: ClaudeCode:claude-opus-5

* fix(http-server): lifecycle, HEAD cost, and CORS/route method agreement

Clears the findings left open from the review.

Lifecycle. `_start_server` raised on its startup timeout without ever
signalling the server, so the thread went on to bind the port and serve
forever as a daemon with no handle to stop it -- and a retry on the same
port then failed with the other error. It now sets should_exit and
force_exit and joins before raising. `shutdown()` returned normally when
the thread survived both joins, reporting success for a server still
bound and still serving; it now raises. The first join also matched
uvicorn's own `timeout_graceful_shutdown` exactly, and uvicorn spends
~0.2s tearing down before that wait even begins, so the join always
expired first and escalated to force_exit on the orderly path -- which
makes uvicorn skip ASGI lifespan shutdown. The join now outlasts the
graceful bound by a margin, and reads that bound from the config so it
stays correct when a caller sets it through uvicorn_options.

HEAD. A HEAD body is discarded at the wire, but HEAD fell through to the
GET handler, so answering one transferred the whole value: measured at 10
MB read to report a length. It is now answered from `Store.getsize` --
a stat on a filesystem store, an info call on a remote one -- and reads
zero bytes. HEAD is served whenever GET is, which is what Starlette does
and what RFC 9110 asks of an origin server; the README and docstrings
said otherwise and now say so.

CORS. `cors_options["allow_methods"]` was passed through unchecked, so an
app could advertise methods its route rejects: a browser caches that
preflight and every later cross-origin call fails with 405 after a
successful handshake. Advertising an unserved method is now a ValueError
at construction, consistent with how unsupported `methods` are already
rejected, and `"*"` expands to what is actually served rather than to
every verb Starlette knows. An absent `allow_methods` is left alone --
widening it to everything served would newly advertise PUT cross-origin
on a write-enabled app that never asked for it.

Media type. The JSON content type was keyed off a third hardcoded
"zarr.json", so a v2 array's `.zarray` was served as octet-stream. It is
now derived from the same tables that decide which keys a node owns.

Also documents that `store_app` does not validate keys: it proxies the
raw key space and has no array semantics to check against, so a client
that misspells a chunk key gets a successful write to a key no reader
consults. `node_app` rejects that with 404.

Assisted-by: ClaudeCode:claude-opus-5

* ci(http-server): run the justfile's recipes instead of copies of them

The workflow repeated the commands the justfile already defines -- `uvx ruff
check .` and the mypy invocation were byte-identical copies, and the pytest
step differed only by the sync that precedes it. Two definitions of the same
verb drift silently: renaming the `docs` dependency group to `examples`
required the same edit in both places, and updating only one would have left
`just check` and CI testing different things with nothing failing.

CI now calls `just test`, `just lint` and `just typecheck` (it already called
`just docs-check`), keeping the python matrix and caching, which are
genuinely CI's concern. This matches zarr-metadata, whose workflow already
states the arrangement; zarr-indexing remains half-converted.

Delegating also meant fixing what the shared recipe would otherwise spread:
`just lint` ran an unpinned `uvx ruff`, which is precisely how this job broke
before -- ruff 0.16 began selecting BLE001 under the root config's `B` prefix
and failed on rules the pre-commit-pinned ruff never enforced, with no code
change to blame. The recipe now pins the same version
.pre-commit-config.yaml does, so the local gate, the pre-commit gate and CI
enforce one standard.

Assisted-by: ClaudeCode:claude-opus-5

* ci: build the zarr-http-server docs on pre-push

A dead cross-reference or a nav entry pointing at a removed file only
fails at `mkdocs build --strict`, which until now happened first in CI.
This catches it before the code leaves the machine.

Scoped deliberately. `stages: [pre-push]` overrides the repo default of
running on every commit: this builds the whole site, which is too slow to
pay per commit and is only actionable before pushing. `files:` limits it
to changes that touch the package, and `pass_filenames: false` because
mkdocs builds a site rather than a list of files.

It delegates to `just docs-check` so the build has one definition shared
with CI, and is added to `ci.skip` alongside mypy for the same reason
that one is skipped: pre-commit.ci's runners have neither `uv` nor the
repo checkout needed to resolve the environment. The zarr-http-server
workflow covers it there.

Note this hook and CI still declare their toolchains separately -- the
hook shells out to the local `just`/`uv`, CI installs them itself. That
is inherent to pre-commit.ci not being able to run them, and is the same
trade already accepted for mypy.

Also refreshes packages/zarr-http-server/uv.lock, which references the
root project's dependency groups, for the hypothesis and uv bumps that
arrived with the main merge.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): make the read-only guarantee explicit and enforced

Read-only was already the default -- `store_app(store)` answers 405 to
PUT, POST, DELETE and PATCH, and POST is unconfigurable because there is
no handler behavior for it -- but nothing said so and little pinned it.
Only PUT was covered against the default app; POST, DELETE and PATCH were
covered only against a fixture built with writes enabled, so "the default
app is read-only" was not actually a tested claim.

Adds a test class covering both layers the guarantee rests on: `methods`,
which decides what the route answers, and the store, which decides
whether a write could succeed at all. Each refusal also asserts the value
is unchanged, matching the property tests -- a 405 that still wrote would
otherwise pass.

Serving PUT from a read-only store is now a ValueError at construction. A
store's `read_only` is fixed when it is built, so that combination can
never succeed; it previously surfaced as a 403 to whichever client tried
to write first, long after whoever misconfigured it had moved on. The
handler's 403 stays as a backstop for a store whose read_only is not
fixed, and the test for it builds the app through the private builder
since the public entry points now reject the combination.

Documents `store.with_read_only(True)` as the categorical recipe: it is
the stronger of the two layers because it holds even if the HTTP layer is
misconfigured.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): name the read-only and read-write method sets

`READ_ONLY_METHODS` and `READ_WRITE_METHODS` let a call site say which it
is, rather than leaving that to the presence or absence of an argument.

The read-only one is exactly the default, so passing it changes nothing
except that the intent is written down. The value is the other direction:
a writable app must name a method set, so `grep -r 'methods='` finds
every place that opts into writes -- which is what makes a deployment
auditable without a separate read-only entry point.

Both are frozensets, so one caller cannot widen the default for every
other, and both name HEAD explicitly: Starlette serves it wherever GET
goes, and a constant that omitted it would misdescribe the route.

`methods` now accepts any `AbstractSet`, which is what lets a frozenset
constant be passed where a `set` was previously required.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): model read-only methods in the type domain

Renames the constants to READ_ONLY_HTTP_METHODS / READ_WRITE_HTTP_METHODS
so they say what kind of method they hold, matching the HTTPMethod type
they are drawn from.

Adds `ReadOnlyHTTPMethod = Literal["GET", "HEAD"]`, which moves the
distinction from a runtime convention to something a checker enforces: a
`frozenset[ReadOnlyHTTPMethod]` cannot contain "PUT", so a read-only
interface can be declared rather than merely configured. Verified against
mypy --strict -- assigning either a set containing "PUT" or
READ_WRITE_HTTP_METHODS to that annotation is an error, while
READ_ONLY_HTTP_METHODS is accepted.

HTTPMethod is now the union of that and a private `_WriteHTTPMethod`
rather than a third hand-written list of the same strings, and both
constants plus _SUPPORTED_METHODS are derived from the Literals via
get_args. The runtime sets and the static types therefore cannot
disagree about what this server serves: widening a Literal is the only
edit needed, and a test pins the contents so that widening is deliberate.

Assisted-by: ClaudeCode:claude-opus-5

* docs(http-server): add a notebook example, and run both examples in tests

The only example used `with serve_node(...)`, which is the one form that
cannot work in a notebook: it shuts the server down when the cell ends, so
anyone copying it gets a dead server by the next cell. Nothing in the
package mentioned notebooks at all.

Adds examples/serve_notebook.ipynb covering the lifecycle a kernel needs --
start with background=True and keep the handle, use it across cells, then
shutdown() -- plus metadata and chunk reads, a byte range, and a refused
PUT. Two arguments carry it: background=True runs uvicorn in a daemon
thread with its own loop so the kernel's loop is untouched, and port=0
means re-running a start cell picks a new port instead of failing with
"address already in use". A README section says the same in prose.

The notebook is executed by the suite through nbclient, in a real kernel,
and asserts its own expectations, so a behavior change fails there rather
than in someone's notebook. Verified by mutation: making writes the
default breaks the notebook's `assert refused.status_code == 405` and
surfaces as a CellExecutionError naming the cell.

examples/serve.py is now executed too, which required fixing the same
fixed-port footgun the notebook section warns about -- it bound 8000, so
it failed if anything else held that port. It runs in-process rather than
under `uv run`, because its inline script metadata resolves
zarr-http-server from git and would test main instead of the working tree.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): expose serve() for arbitrary ASGI apps

Serving two nodes did not need two servers, but the only way to run
several was to reach past this package: `serve_store`/`serve_node` each
take exactly one store or node, so a composed app had no route to the
background-server ergonomics -- `port=0` into `server.url`, and
`shutdown()` -- only a blocking `uvicorn.run`. `_start_server` already
did this for any Starlette app; it was just private.

`serve(app, ...)` makes it public, with the same background/blocking
overloads the shorthands have. `serve_store` and `serve_node` now
delegate to it and stay, because they are the common case and are what
the docs and examples use; retiring them is still available later.

The split the pair muddles is now visible: what an app *serves*
(`methods`, `cors_options`, `max_body_size`) is settled when the app is
built, and `serve` only decides how it runs.

Documents the three ways to serve several nodes -- serve their common
parent group, serve the whole store, or mount separate apps and run the
result -- with tests covering mounted nodes in *separate* stores, that
each mount serves only its own data, and that `serve` runs the composed
app in the background.

Also derives the bounded-shutdown test's threshold from the timeouts that
produce it. It hard-coded 3.0s, which was generous when shutdown could
take at most 2x shutdown_timeout and marginal once the join margin was
added -- it began failing under load rather than at the moment the
constant changed.

Assisted-by: ClaudeCode:claude-opus-5

* refactor(http-server)!: split blocking and background into two functions

`background: bool` decided whether a call returns immediately with a
handle or never returns at all -- the largest difference a call site can
have, hidden in a keyword. The return type depended on it too, which is
why every runner carried three @overload stanzas: nine in total, all of
them working around that one flag, and `background=False` returned None,
a value meaningless half the time.

`serve(app)` now blocks and `serve_background(app)` returns a
BackgroundServer. Neither needs an overload.

Splitting forced the shorthand question, since the axes multiply: keeping
serve_store/serve_node alongside two modes means six runner functions.
The public surface is instead two builders and two runners --
`serve_background(store_app(store))` replaces
`serve_store(store, background=True)`. That is one more call, and it puts
the two halves where they belong: what an app serves is settled when it
is built, and the runner only decides how it runs.

_serve.py drops from 1301 to ~1100 lines with the duplicated signatures
and docstrings gone.

`serve_background` defaults to `port=0` where `serve` defaults to 8000.
Deliberate: a background server is reached through `server.url`, and a
fixed default makes starting a second one -- or re-running a notebook
cell -- fail on a collision, while a blocking server usually wants a port
others already know.

BREAKING CHANGE: serve_store and serve_node are removed. The package is
unreleased, so nothing depends on them yet.

Assisted-by: ClaudeCode:claude-opus-5

* feat(http-server): default both runners to port="auto"

`serve` defaulted to 8000 and `serve_background` to 0, which read as an
arbitrary disagreement between two sibling functions about a shared
parameter. Both now default to `"auto"`: prefer 8000, fall back to any
free port if it is taken, and report the result through `server.url` and
uvicorn's own startup line.

What makes the fallback safe is that it applies only to the default. An
explicit port still binds exactly that or fails, because a caller who
names one usually has a proxy or a container port mapping expecting the
server there -- silently moving would break it while looking healthy.
`port=0` keeps its OS meaning of "any free port, no preference".

The port is bound here and the socket handed to `Server.run(sockets=...)`
rather than probing for a free port and passing uvicorn the number:
probing releases the port before uvicorn claims it, which is the
bind-then-close race that makes "find a free port" helpers flaky. Holding
the socket means nothing can take it in between. `Config.port` is set to
what was actually bound, so uvicorn's "running on ..." line does not name
a port it is not serving.

Two cases the mechanism has to respect: a `uds` or `fd` bind in
uvicorn_options skips the TCP bind entirely, and the address family comes
from `getaddrinfo` rather than a hard-coded AF_INET, which would bind the
wrong family for an IPv6 host. Both are covered by tests, as is that an
explicit taken port still raises.

Assisted-by: ClaudeCode:claude-opus-5

* revert: drop changes to files this package does not own

The PR should not reach outside packages/zarr-http-server and .github,
and four files did.

Two were unrelated churn: docs/api/zarr/experimental.md renamed a heading
in the *core* zarr docs about zarr.experimental.cache_store, left over
from when this server lived at zarr.experimental.serve, and uv.lock
carried an idna bump nothing here asked for. Reverted; `uv lock --check`
is clean.

Two were premature rather than wrong. mkdocs.yml added a nav link to
zarr-http-server.readthedocs.io and .readthedocs.yaml skipped the
repo-root docs build for changes confined to this package. Both belong
with a Read the Docs project that does not exist yet -- until it does,
the nav link 404s and the build skip means neither site builds the
package's docs. mkdocs.yml is also what GitHub reported a conflict on,
which is what surfaced this. They should land in the follow-up that
creates the RTD project.

What remains outside the package is two root files that have nowhere
else to live: the pre-push docs hook, since a pre-commit hook is
necessarily repo-level, and a comment-only change to pyproject.toml
noting that the release workflow's `zarr_http_server-v*` tags are among
those the `git describe --match v*` filter exists to exclude.

Assisted-by: ClaudeCode:claude-opus-5

zarr_indexing-v0.1.0

Toggle zarr_indexing-v0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat: add the zarr-indexing package (TensorStore-style index transfor…

…ms, ndsel wire format) (#4196)

* fix: byte-order handling for structured dtypes in the bytes codec (#220)

* fix: byte-order handling for structured dtypes in the bytes codec

The bytes codec neither byte-swapped structured-dtype fields to its
configured endian on encode (numpy reports byteorder '|' for void
dtypes, so the top-level byteorder comparison never detected a
mismatch) nor honored its endian when decoding, silently corrupting
any structured data whose field byte order differed from the stored
one (e.g. virtual references to external big-endian data).

Encode now detects byte-order mismatches by comparing full dtypes via
newbyteorder, and decode reinterprets raw bytes in the stored byte
order before converting to the data type's declared byte order, so the
stored layout (codec state) and the in-memory layout (array data type)
are independent.

Closes #4141

Assisted-by: ClaudeCode:claude-fable-5

* test: fold structured byte-order cases into existing bytes codec tests

Extend test_endian's parametrization with structured dtypes and
test_bytes_codec_sync_roundtrip with endian/dtype parametrization plus
stored-layout and decoded-dtype assertions, instead of adding parallel
test functions for the same properties.

Assisted-by: ClaudeCode:claude-fable-5

* refactor: rename stored_dtype to view_dtype in BytesCodec decode

The variable is the dtype used to view the raw chunk bytes (byte order
from the codec's endian configuration), not a property of the stored
data or of the returned buffer, which always carries the array's
declared dtype.

Assisted-by: ClaudeCode:claude-fable-5

* docs: note that the decode-side byte-order conversion copies the chunk

Assisted-by: ClaudeCode:claude-fable-5

* feat: add the zarr-indexing package (TensorStore-style index transforms)

Standalone workspace package extracted from the lazy-indexing branch
(#3906): composable, lazy coordinate transforms
(IndexTransform / IndexDomain / output maps), dependency-aware chunk
resolution against a DimensionGridLike protocol, and an ndsel-conformant
JSON wire format validated against the vendored conformance corpus.

zarr itself does not depend on zarr-indexing yet — the runtime wiring
lands separately once 0.1.0 is published. The package is numpy-only;
its tests exercise chunk resolution against zarr's concrete ChunkGrid,
so they run from the workspace root (uv sync --all-packages).

Assisted-by: ClaudeCode:claude-fable-5

* style: conventional submodule import in the chunk-resolution tests

Assisted-by: ClaudeCode:claude-fable-5

* perf(zarr-indexing): joint chunk enumeration for correlated vindex maps

Candidate-chunk enumeration took the cartesian product of each correlated
ArrayMap's per-dimension distinct chunk ids and relied on intersect() to
filter untouched combinations. For a diagonal selection of P scattered
points that is P**2 intersect calls — quadratic in the number of selected
points, the same workload shape as #4174 (400 points:
~2.6s; 10k points: ~30min).

Group correlated maps jointly instead: broadcast their per-point chunk
ids, take the distinct rows (np.unique(axis=0), O(P log P)), and
enumerate exactly the touched combinations. Candidate slots now carry
chunk-coordinate tuples covering one or more output dimensions;
orthogonal/constant/slice dimensions keep their existing per-dimension
candidates. 400-point diagonal resolution drops from 2628ms to 14ms and
scales linearly.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): standalone documentation site; add package justfile

Mirror the treatment zarr-metadata received in #4208/#4210 onto
zarr-indexing: a self-contained mkdocs site under the package (own
mkdocs.yml, landing page, ndsel wire-format guide, mkdocstrings page per
module, and .readthedocs.yaml for a dedicated RTD project), so the
package presents as a separate project with docs versioned by its own
zarr_indexing-v* release tags rather than zarr-python's. The zarr-python
site's API Reference nav links out to it, and each RTD project now skips
PR builds that do not touch its half of the repo.

The package gains a pinned docs dependency group, a docs build job in its
CI workflow, and a justfile with package-scoped dev recipes. Two recipes
deviate from the zarr-metadata original by design:

- `test` runs against the workspace-root environment (`uv run --project
  ../.. --all-packages --group test`), because the chunk-resolution tests
  exercise this package against zarr's chunk grids and `zarr` is
  deliberately not a dependency of this package.
- `typecheck` uses plain `pyright`, unpinned and on the default
  interpreter, mirroring this package's own CI invocation. The
  zarr-metadata pin exists for a PEP 661 sentinel regression that
  zarr-indexing's sources do not hit.

composition.py gains the module docstring the other modules already have,
since mkdocstrings renders it as the page introduction.

Assisted-by: ClaudeCode:claude-fable-5

* chore: drop the already-released 4141 changelog fragment

The bytes-codec byte-order fix this fragment describes shipped upstream
and its entry is already in docs/release-notes.md; the fragment survived
on this branch only as a rebase remnant, and would emit a duplicate entry
in the next release.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): canonicalize ndsel references to zarr-developers/ndsel

Also aligns the zarr-indexing workflow's setup-uv pin (v8.3.2) with the
rest of the repo. The vendored-corpus sha is present upstream; the
historical d-v-b/ndsel#1 PR reference stays as provenance.

Assisted-by: ClaudeCode:claude-fable-5

* chore: drop the root uv-workspace wiring for zarr-indexing

Per review: the root pyproject.toml should not change in this PR. The
package now operates fully standalone (like zarr-metadata); the test
invocations layer the package into the repo-root environment as an
editable overlay instead (python -m pytest, since a base-env console
script would not see the overlay).

Assisted-by: ClaudeCode:claude-fable-5

v3.3.0

Toggle v3.3.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
docs: dev blog, performance examples, and compiled 3.3.0 release notes (

#4191)

Adds a development blog to the documentation site with a 3.3.0 release
post covering the FusedCodecPipeline and sharded partial-read
coalescing, plus two runnable examples referenced by the post
(examples/codec_pipeline_performance, examples/sharding_coalescing).

Compiles every pending changelog fragment into a single 3.3.0 release
notes section dated 2026-07-30, merging it with the section compiled
earlier in #4148, and empties changes/ — making this commit suitable to
tag as v3.3.0.

Assisted-by: ClaudeCode:claude-fable-5

zarr_metadata-v0.4.0

Toggle zarr_metadata-v0.4.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore(zarr-metadata): build 0.4.0 changelog (#4211)

Consume the pending news fragments (the #4119 model layer's feature and
removal notes, plus a new doc fragment for the standalone documentation
site and justfile from #4208/#4210) into CHANGELOG.md via towncrier for
the zarr_metadata-v0.4.0 release.

Assisted-by: ClaudeCode:claude-fable-5

zarr_metadata-v0.3.0

Toggle zarr_metadata-v0.3.0's commit message
Release zarr-metadata 0.3.0

zarr_metadata-v0.2.0

Toggle zarr_metadata-v0.2.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Widen ChunksLike type alias (#3990)

* Widen ChunksLike type alias

Fixes #3869

* Add changelog file.

* Prefer if/else statement to if/else expression

v3.2.1

Toggle v3.2.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
release : 3.2.1 release notes (#3942)

* release:3.2.1 release notes

* docs: add changelog entry

* docs: remove changelog

* Fix date and add link

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>