diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
index 29281f5be9..9503f3df8a 100644
--- a/.github/CONTRIBUTING.md
+++ b/.github/CONTRIBUTING.md
@@ -1,4 +1,4 @@
Contributing
============
-Please see the [project documentation](https://zarr.readthedocs.io/en/stable/contributing.html) for information about contributing to Zarr.
+Please see the [project documentation](https://zarr.readthedocs.io/en/stable/developers/contributing.html) for information about contributing to Zarr.
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
index 705cd31cb5..e765e3136e 100644
--- a/.github/ISSUE_TEMPLATE/bug_report.yml
+++ b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -57,7 +57,22 @@ body:
id: reproduce
attributes:
label: Steps to reproduce
- description: Minimal, reproducible code sample, a copy-pastable example if possible.
+ description: Minimal, reproducible code sample. Must list dependencies in [inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/#example). When put in a file named `issue.py` calling `uv run issue.py` should show the issue.
+ value: |
+ ```python
+ # /// script
+ # requires-python = ">=3.12"
+ # dependencies = [
+ # "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
+ # ]
+ # ///
+ #
+ # This script automatically imports the development branch of zarr to check for issues
+
+ import zarr
+ # your reproducer code
+ # zarr.print_debug_info()
+ ```
validations:
required: true
- type: textarea
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
index edbd88eaf2..d219a73737 100644
--- a/.github/ISSUE_TEMPLATE/config.yml
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -1,11 +1,11 @@
blank_issues_enabled: true
contact_links:
- - name: Propose a new major feature
+ - name: Propose a new Zarr specification feature
url: https://github.com/zarr-developers/zarr-specs
- about: A new major feature should be discussed in the Zarr specifications repository.
+ about: A new feature for the Zarr storage specification should be opened on the zarr-specs repository.
- name: Discuss something on ZulipChat
url: https://ossci.zulipchat.com/
- about: For questions like "How do I do X with Zarr?", you can move to our ZulipChat.
+ about: For questions like "How do I do X with Zarr?", consider posting your question to our developer chat.
- name: Discuss something on GitHub Discussions
url: https://github.com/zarr-developers/zarr-python/discussions
about: For questions like "How do I do X with Zarr?", you can move to GitHub Discussions.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
new file mode 100644
index 0000000000..f067655f22
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,11 @@
+name: Feature Request
+description: Request a new feature for zarr-python
+# labels: []
+body:
+- type: textarea
+ attributes:
+ label: Describe the new feature you'd like
+ description: >
+ Please provide a description of what new feature or functionality you'd like to see in zarr-python.
+ validations:
+ required: true
diff --git a/.github/ISSUE_TEMPLATE/release-checklist.md b/.github/ISSUE_TEMPLATE/release-checklist.md
new file mode 100644
index 0000000000..309c76b4dc
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/release-checklist.md
@@ -0,0 +1,53 @@
+---
+name: Zarr-Python release checklist
+about: Checklist for a new Zarr-Python release. [For project maintainers only!]
+title: Release Zarr-Python vX.Y.Z
+labels: release-checklist
+assignees: ''
+
+---
+
+**Release**: [v3.x.x](https://github.com/zarr-developers/zarr-python/milestones/?)
+**Scheduled Date**: 20YY/MM/DD
+
+**Priority PRs/issues to complete prior to release**
+
+- [ ] Priority pull request #X
+
+**Before release**:
+
+- [ ] Check [SPEC 0](https://scientific-python.org/specs/spec-0000/#support-window) to see if the minimum supported version of Python or NumPy needs bumping.
+- [ ] Verify that the latest CI workflows on `main` are passing: [Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml), [GPU Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml), [Hypothesis](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml), [Docs](https://github.com/zarr-developers/zarr-python/actions/workflows/docs.yml), [Lint](https://github.com/zarr-developers/zarr-python/actions/workflows/lint.yml), [Wheels](https://github.com/zarr-developers/zarr-python/actions/workflows/releases.yml).
+- [ ] Run the [downstream tests](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml) against `main`: go to the workflow page, click "Run workflow", and select the `main` branch. Verify that the Xarray and numcodecs integration tests pass.
+- [ ] Open a release PR with the changelog entries for the upcoming release, generated with `uv run --only-group release towncrier build --version x.y.z`.
+- [ ] Review the release PR and verify the changelog in `docs/release-notes.md` looks correct.
+- [ ] Merge the release PR.
+
+**Release**:
+
+- [ ] [Draft a new GitHub Release](https://github.com/zarr-developers/zarr-python/releases/new) with tag `vX.Y.Z` targeting `main`. Use "Generate release notes" for the description.
+- [ ] Verify the release is published on [PyPI](https://pypi.org/project/zarr/) and [ReadTheDocs](https://zarr.readthedocs.io/en/stable/).
+
+**After release**:
+
+- [ ] Review and merge the pull request on the conda-forge [zarr-feedstock](https://github.com/conda-forge/zarr-feedstock) that will be automatically generated.
+
+---
+
+- [ ] Party :tada:
+
+---
+
+
+Releasing from a branch other than main
+
+In rare cases (e.g. patch releases for an older minor version), you may need to release from a dedicated release branch (e.g. `3.1.x`):
+
+- Create the release branch from the appropriate tag if it doesn't already exist.
+- Cherry-pick or backport the necessary commits onto the branch.
+- Run `towncrier build --version x.y.z` and open the release PR against the release branch instead of `main`.
+- Run the downstream tests against the release branch instead of `main`.
+- When drafting the GitHub Release, set the target to the release branch instead of `main`.
+- After the release, ensure any relevant changelog updates are also reflected on `main`.
+
+
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
index a0d41f9841..47adb4b19e 100644
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,9 +1,27 @@
-[Description of PR]
+
+
+## Summary
+
+[Describe what this PR changes and why, in your own words.]
+
+## For reviewers
+
+[What would you most value a second look at? What are you already confident in? For a refactor, say whether behavior is meant to be unchanged.]
+
+## Author attestation
+
+- [ ] I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.
+
+
+
+## TODO
-TODO:
* [ ] Add unit tests and/or doctests in docstrings
* [ ] Add docstrings and API docs for any new/modified user-facing classes and functions
-* [ ] New/modified features documented in docs/tutorial.rst
-* [ ] Changes documented in docs/release.rst
+* [ ] New/modified features documented in `docs/user-guide/*.md`
+* [ ] Changes documented as a new file in `changes/`
* [ ] GitHub Actions have all passed
* [ ] Test coverage is 100% (Codecov passes)
diff --git a/.github/codecov.yml b/.github/codecov.yml
deleted file mode 100644
index e9b99c8214..0000000000
--- a/.github/codecov.yml
+++ /dev/null
@@ -1,15 +0,0 @@
-coverage:
- status:
- project:
- default:
- target: 100
- threshold: 0.1
- patch:
- default:
- target: 100
-comment:
- layout: "diff, files"
- behavior: default
- require_changes: true # if true: only post the comment if coverage changes
- branches: # branch names that can post comment
- - "main"
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index a437a5c269..0c794d9b08 100644
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -1,15 +1,7 @@
---
version: 2
updates:
- # Updates for v3 branch (the default branch)
- - package-ecosystem: "pip"
- directory: "/"
- schedule:
- interval: "daily"
- groups:
- actions:
- patterns:
- - "*"
+ # Updates for main
- package-ecosystem: "github-actions"
directory: "/"
schedule:
@@ -18,23 +10,24 @@ updates:
actions:
patterns:
- "*"
-
- # Same updates, but for main branch
- - package-ecosystem: "pip"
+ cooldown:
+ default-days: 7
+ # Keep the pinned dev tooling in pyproject.toml's [dependency-groups] and the
+ # uv.lock current. Without this the exact pins (e.g. pytest) would never be
+ # bumped automatically and would silently rot.
+ #
+ # `allow: dependency-type: direct` restricts updates to dependencies declared
+ # in pyproject.toml. Transitive deps in uv.lock are then only updated as a
+ # side effect of a direct bump, never via a standalone PR.
+ - package-ecosystem: "uv"
directory: "/"
- target-branch: "main"
- schedule:
- interval: "daily"
- groups:
- requirements:
- patterns:
- - "*"
- - package-ecosystem: "github-actions"
- directory: "/"
- target-branch: "main"
+ allow:
+ - dependency-type: "direct"
schedule:
interval: "weekly"
groups:
- actions:
+ python-dependencies:
patterns:
- "*"
+ cooldown:
+ default-days: 7
diff --git a/.github/labeler.yml b/.github/labeler.yml
index dbc3b95333..7eb74211ea 100644
--- a/.github/labeler.yml
+++ b/.github/labeler.yml
@@ -1,2 +1,4 @@
needs release notes:
-- all: ['!docs/release.rst']
+ - all:
+ - changed-files:
+ - all-globs-to-all-files: '!changes/*.md'
diff --git a/.github/workflows/check_changelogs.yml b/.github/workflows/check_changelogs.yml
new file mode 100644
index 0000000000..c391f63738
--- /dev/null
+++ b/.github/workflows/check_changelogs.yml
@@ -0,0 +1,37 @@
+name: Check changelog entries
+
+on:
+ pull_request:
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ check-changelogs:
+ name: Check changelog entries
+ runs-on: ubuntu-latest
+
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+
+ - name: Check zarr-python changelog entries
+ run: uv run --no-sync python ci/check_changelog_entries.py
+
+ - name: Check zarr-metadata changelog entries
+ run: uv run --no-sync python ci/check_changelog_entries.py packages/zarr-metadata/changes
+
+ - name: Check zarr-indexing changelog entries
+ run: uv run --no-sync python ci/check_changelog_entries.py packages/zarr-indexing/changes
+
+ - name: Check zarr-http-server changelog entries
+ run: uv run --no-sync python ci/check_changelog_entries.py packages/zarr-http-server/changes
diff --git a/.github/workflows/codspeed.yml b/.github/workflows/codspeed.yml
new file mode 100644
index 0000000000..f36fbff233
--- /dev/null
+++ b/.github/workflows/codspeed.yml
@@ -0,0 +1,40 @@
+name: CodSpeed Benchmarks
+
+on:
+ schedule:
+ - cron: '0 9 * * 1' # Every Monday at 9am UTC
+ pull_request:
+ types: [labeled]
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ benchmarks:
+ name: Run benchmarks
+ runs-on: codspeed-macro
+ if: |
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'benchmark'))
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
+ - name: Run the benchmarks
+ uses: CodSpeedHQ/action@0ca9cbbf4623b599a6c3ed4fc8a922942705d9f1 # v5.0.2
+ env:
+ ZARR_BENCHMARK_CLEAR_CACHE: '1'
+ with:
+ mode: walltime
+ run: hatch run test.py3.12-minimal:pytest tests/benchmarks --codspeed
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
new file mode 100644
index 0000000000..792ee431ab
--- /dev/null
+++ b/.github/workflows/docs.yml
@@ -0,0 +1,40 @@
+name: Docs
+
+on:
+ push:
+ branches: [main]
+ pull_request:
+ branches: [main]
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ docs:
+ name: Check docs
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - run: uv sync --group docs
+ # Fast source-level guards that need no built site, so they run before the (slower)
+ # build for a quick failure: every public export is in the API reference, and no
+ # docstring/Markdown carries reStructuredText markup that MkDocs won't render.
+ - run: uv run python ci/check_documented_exports.py docs/api
+ - run: uv run python ci/lint_docs.py
+ # --strict turns warnings into errors, so a docs code block that fails to execute
+ # at build time (e.g. a non-exec python fence disrupting a later exec="true" block)
+ # fails CI instead of merging as a silent warning.
+ - run: uv run mkdocs build --strict
+ env:
+ DISABLE_MKDOCS_2_WARNING: "true"
+ NO_MKDOCS_2_WARNING: "true"
+ - run: uv run python ci/check_unlinked_types.py
+ continue-on-error: true
diff --git a/.github/workflows/downstream.yml b/.github/workflows/downstream.yml
new file mode 100644
index 0000000000..98cd0fee3f
--- /dev/null
+++ b/.github/workflows/downstream.yml
@@ -0,0 +1,129 @@
+name: Downstream
+
+on:
+ workflow_dispatch:
+ pull_request:
+ types: [labeled, synchronize, opened, reopened]
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ xarray:
+ name: Xarray zarr backend tests
+ if: |
+ github.event_name == 'workflow_dispatch'
+ || contains(github.event.pull_request.labels.*.name, 'run-downstream')
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out zarr-python
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Check out xarray
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ repository: pydata/xarray
+ path: xarray
+ persist-credentials: false
+
+ # We install xarray with plain pip/uv rather than pixi. pixi solves
+ # xarray's entire manifest (it has no committed lockfile), which drags in
+ # the `mypy-upstream` environment; that environment sources numcodecs from
+ # git and fails to build under newer pixi with
+ # `meson-python: error: Unknown option "pixi-conda-environment"`, breaking
+ # the job before any test runs. Tests that need a backend we don't install
+ # are skipped via xarray's `requires_*` markers, not failed.
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: '3.13'
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+
+ - name: Install xarray and test dependencies
+ working-directory: xarray
+ run: |
+ uv venv
+ # xarray's pytest tooling lives in the PEP 735 `dev` dependency group;
+ # the zarr-relevant backends come from the `io` and `parallel` extras.
+ uv pip install --group dev ".[io,parallel,accel]"
+
+ - name: Override zarr-python with branch version
+ working-directory: xarray
+ run: uv pip install --no-deps ..
+
+ - name: Show versions
+ working-directory: xarray
+ run: |
+ uv run python -c "
+ import zarr; print(f'zarr {zarr.__version__}')
+ import xarray; print(f'xarray {xarray.__version__}')
+ "
+
+ - name: Run xarray zarr backend tests
+ working-directory: xarray
+ run: |
+ uv run python -m pytest --no-header -q \
+ xarray/tests/test_backends.py \
+ xarray/tests/test_backends_api.py \
+ xarray/tests/test_backends_datatree.py
+
+ numcodecs:
+ name: numcodecs zarr3 codec tests
+ if: |
+ github.event_name == 'workflow_dispatch'
+ || contains(github.event.pull_request.labels.*.name, 'run-downstream')
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out zarr-python
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Check out numcodecs
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ repository: zarr-developers/numcodecs
+ fetch-depth: 0
+ path: numcodecs
+ submodules: recursive
+ persist-credentials: false
+
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: '3.13'
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+
+ - name: Install numcodecs with test-zarr-main group
+ working-directory: numcodecs
+ run: |
+ uv venv
+ uv pip install --group dev
+ uv sync --group dev --group test-zarr-main
+ uv pip install --no-build-isolation -e .
+
+ - name: Override zarr-python with branch version
+ working-directory: numcodecs
+ run: uv pip install --no-deps ..
+
+ - name: Show versions
+ working-directory: numcodecs
+ run: |
+ uv run python -c "
+ import zarr; print(f'zarr {zarr.__version__}')
+ import numcodecs; print(f'numcodecs {numcodecs.__version__}')
+ "
+
+ - name: Run numcodecs zarr3 tests
+ working-directory: numcodecs
+ run: uv run python -m pytest -x --no-header -q tests/test_zarr3.py
diff --git a/.github/workflows/gpu_test.yml b/.github/workflows/gpu_test.yml
index b13da7d36f..bf8700400e 100644
--- a/.github/workflows/gpu_test.yml
+++ b/.github/workflows/gpu_test.yml
@@ -5,13 +5,18 @@ name: GPU Test
on:
push:
- branches: [ main ]
+ branches: [ main, 3.1.x ]
pull_request:
- branches: [ main ]
+ branches: [ main, 3.1.x ]
workflow_dispatch:
env:
LD_LIBRARY_PATH: /usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64
+ # Use the uv from astral-sh/setup-uv instead of hatch's bundled (pyapp) uv.
+ HATCH_ENV_TYPE_VIRTUAL_UV_PATH: uv
+
+permissions:
+ contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -19,17 +24,20 @@ concurrency:
jobs:
test:
- name: py=${{ matrix.python-version }}, np=${{ matrix.numpy-version }}, deps=${{ matrix.dependency-set }}
-
+ name: py=${{ matrix.python-version }}
+ environment:
+ name: codecov-upload
+ deployment: false
runs-on: gpu-runner
strategy:
matrix:
- python-version: ['3.11']
- numpy-version: ['2.1']
- dependency-set: ["minimal"]
+ python-version: ['3.12']
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0 # grab all branches and tags
+ persist-credentials: false
# - name: cuda-toolkit
# uses: Jimver/cuda-toolkit@v0.2.16
# id: cuda-toolkit
@@ -49,18 +57,31 @@ jobs:
echo $LD_LIBRARY_PATH
nvcc -V
- name: Set up Python
- uses: actions/setup-python@v5
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- - name: Install Hatch and CuPy
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
- name: Set Up Hatch Env
+ env:
+ HATCH_ENV: gputest.py${{ matrix.python-version }}
run: |
- hatch env create gputest.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }}
- hatch env run -e gputest.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} list-env
+ hatch env create "$HATCH_ENV"
+ hatch env run -e "$HATCH_ENV" list-env
- name: Run Tests
+ env:
+ HATCH_ENV: gputest.py${{ matrix.python-version }}
run: |
- hatch env run --env gputest.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run-coverage
+ hatch env run --env "$HATCH_ENV" run-coverage
+
+ - name: Upload coverage
+ uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+ with:
+ token: ${{ secrets.CODECOV_TOKEN }}
+ flags: gpu
+ verbose: true # optional (default = false)
diff --git a/.github/workflows/hypothesis.yaml b/.github/workflows/hypothesis.yaml
index 1029063ef4..f463397c85 100644
--- a/.github/workflows/hypothesis.yaml
+++ b/.github/workflows/hypothesis.yaml
@@ -1,23 +1,36 @@
name: Slow Hypothesis CI
on:
push:
- branches:
- - "main"
+ branches: [main, 3.1.x]
pull_request:
- branches:
- - "main"
+ branches: [main, 3.1.x]
types: [opened, reopened, synchronize, labeled]
schedule:
- cron: "0 0 * * *" # Daily “At 00:00” UTC
workflow_dispatch: # allows you to trigger manually
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
env:
FORCE_COLOR: 3
+ # Use the uv from astral-sh/setup-uv instead of hatch's bundled (pyapp) uv.
+ HATCH_ENV_TYPE_VIRTUAL_UV_PATH: uv
jobs:
hypothesis:
name: Slow Hypothesis Tests
+ permissions:
+ contents: read
+ issues: write
+ environment:
+ name: codecov-upload
+ deployment: false
runs-on: "ubuntu-latest"
defaults:
run:
@@ -25,29 +38,43 @@ jobs:
strategy:
matrix:
- python-version: ['3.11']
- numpy-version: ['2.1']
+ python-version: ['3.12']
dependency-set: ["optional"]
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Set HYPOTHESIS_PROFILE based on trigger
+ env:
+ EVENT_NAME: ${{ github.event_name }}
+ run: |
+ if [[ "$EVENT_NAME" == "schedule" || "$EVENT_NAME" == "workflow_dispatch" ]]; then
+ echo "HYPOTHESIS_PROFILE=nightly" >> $GITHUB_ENV
+ else
+ echo "HYPOTHESIS_PROFILE=ci" >> $GITHUB_ENV
+ fi
- name: Set up Python
- uses: actions/setup-python@v5
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Install Hatch
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
- name: Set Up Hatch Env
+ env:
+ HATCH_ENV: test.py${{ matrix.python-version }}-${{ matrix.dependency-set }}
run: |
- hatch env create test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }}
- hatch env run -e test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} list-env
+ hatch env create "$HATCH_ENV"
+ hatch env run -e "$HATCH_ENV" list-env
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
- name: Restore cached hypothesis directory
id: restore-hypothesis-cache
- uses: actions/cache/restore@v4
+ uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}
@@ -57,26 +84,37 @@ jobs:
- name: Run slow Hypothesis tests
if: success()
id: status
+ env:
+ HATCH_ENV: test.py${{ matrix.python-version }}-${{ matrix.dependency-set }}
+ PYTEST_ADDOPTS: "--report-log=output-${{ matrix.python-version }}-log.jsonl"
run: |
- hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run-hypothesis
+ echo "Using Hypothesis profile: $HYPOTHESIS_PROFILE"
+ hatch env run --env "$HATCH_ENV" run-hypothesis
# explicitly save the cache so it gets updated, also do this even if it fails.
- name: Save cached hypothesis directory
id: save-hypothesis-cache
if: always() && steps.status.outcome != 'skipped'
- uses: actions/cache/save@v4
+ uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}
+ - name: Upload coverage
+ uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+ with:
+ token: ${{ secrets.CODECOV_TOKEN }}
+ flags: tests
+ verbose: true # optional (default = false)
+
- name: Generate and publish the report
if: |
failure()
&& steps.status.outcome == 'failure'
&& github.event_name == 'schedule'
&& github.repository_owner == 'zarr-developers'
- uses: xarray-contrib/issue-from-pytest-log@v1
+ uses: scientific-python/issue-from-pytest-log-action@35b4e0a9e06f8e7e261778289cf4b968b722662d # v1.6.2
with:
log-path: output-${{ matrix.python-version }}-log.jsonl
issue-title: "Nightly Hypothesis tests failed"
- issue-label: "topic-hypothesis"
+ issue-label: "automated issue"
diff --git a/.github/workflows/issue-metrics.yml b/.github/workflows/issue-metrics.yml
index 34bda59ff6..adbd2748a0 100644
--- a/.github/workflows/issue-metrics.yml
+++ b/.github/workflows/issue-metrics.yml
@@ -7,13 +7,17 @@ on:
permissions:
contents: read
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
jobs:
build:
name: issue metrics
runs-on: ubuntu-latest
permissions:
- issues: write
- pull-requests: read
+ issues: write # Required to create the metrics report issue
+ pull-requests: read # Required to read PR metrics
steps:
- name: Get dates for last month
shell: bash
@@ -29,13 +33,13 @@ jobs:
echo "last_month=$first_day..$last_day" >> "$GITHUB_ENV"
- name: Run issue-metrics tool
- uses: github/issue-metrics@v3
+ uses: github-community-projects/issue-metrics@df8c49d20958f9345281fa2124858bd0ad227e1f # v5.0.0
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SEARCH_QUERY: 'repo:zarr-developers/zarr-python is:issue created:${{ env.last_month }} -reason:"not planned"'
- name: Create issue
- uses: peter-evans/create-issue-from-file@v5
+ uses: peter-evans/create-issue-from-file@fca9117c27cdc29c6c4db3b86c48e4115a786710 # v6.0.0
with:
title: Monthly issue metrics report
token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/links.yml b/.github/workflows/links.yml
new file mode 100644
index 0000000000..d52639a708
--- /dev/null
+++ b/.github/workflows/links.yml
@@ -0,0 +1,32 @@
+name: Check links
+
+on:
+ repository_dispatch:
+ workflow_dispatch:
+ # pull_request:
+ schedule:
+ - cron: "00 18 * * 1" # weekly, Mondays at 18:00 UTC
+
+jobs:
+ linkChecker:
+ runs-on: ubuntu-latest
+ permissions:
+ issues: write # required for peter-evans/create-issue-from-file
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+
+ - name: Link Checker
+ id: lychee
+ uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0
+ with:
+ fail: false
+
+ - name: Create Issue From File
+ if: steps.lychee.outputs.exit_code != 0
+ uses: peter-evans/create-issue-from-file@fca9117c27cdc29c6c4db3b86c48e4115a786710 # v6.0.0
+ with:
+ title: Link Checker Report
+ content-filepath: ./lychee/out.md
+ labels: report, automated issue
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
new file mode 100644
index 0000000000..83cc0a1b3e
--- /dev/null
+++ b/.github/workflows/lint.yml
@@ -0,0 +1,33 @@
+name: Lint
+
+on:
+ push:
+ branches: [main, 3.1.x]
+ pull_request:
+ branches: [main, 3.1.x]
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ lint:
+ name: Lint
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Set up Python
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: "3.12"
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
diff --git a/.github/workflows/needs_release_notes.yml b/.github/workflows/needs_release_notes.yml
index d81ee0bdc4..e001e8cd43 100644
--- a/.github/workflows/needs_release_notes.yml
+++ b/.github/workflows/needs_release_notes.yml
@@ -1,14 +1,27 @@
name: "Pull Request Labeler"
on:
- - pull_request_target
+ # pull_request_target is needed to label PRs from forks.
+ # This workflow only runs actions/labeler (no code checkout), so it's safe.
+ pull_request_target: # zizmor: ignore[dangerous-triggers]
+ types: [opened, reopened, synchronize]
+
+permissions: {}
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
+ cancel-in-progress: true
jobs:
- triage:
- if: ${{ github.event.pull_request.user.login != 'dependabot[bot]' }} && ${{ github.event.pull_request.user.login != 'pre-commit-ci[bot]' }}
+ labeler:
+ name: Label pull request
+ if: ${{ github.event.pull_request.user.login != 'dependabot[bot]' && github.event.pull_request.user.login != 'pre-commit-ci[bot]' }}
+ permissions:
+ contents: read # Required to read label configuration
+ pull-requests: write # Required to add labels to PRs
runs-on: ubuntu-latest
steps:
- - uses: actions/labeler@main
+ - uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13 # v7.0.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
sync-labels: true
diff --git a/.github/workflows/nightly_wheels.yml b/.github/workflows/nightly_wheels.yml
new file mode 100644
index 0000000000..5b99c523a1
--- /dev/null
+++ b/.github/workflows/nightly_wheels.yml
@@ -0,0 +1,48 @@
+name: Nightly Wheels
+
+on:
+ schedule:
+ # Run nightly at 2 AM UTC
+ - cron: '0 2 * * *'
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ build_and_upload_nightly:
+ name: Build and upload nightly wheels
+ environment:
+ name: nightly-wheel-upload
+ deployment: false
+ runs-on: ubuntu-latest
+
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ submodules: true
+ fetch-depth: 0
+ persist-credentials: false
+
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ name: Install Python
+ with:
+ python-version: '3.14'
+
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
+
+ - name: Build wheel and sdist
+ run: hatch build
+
+ - name: Upload nightly wheels
+ uses: scientific-python/upload-nightly-action@e76cfec8a4611fd02808a801b0ff5a7d7c1b2d99
+ with:
+ artifacts_path: dist
+ anaconda_nightly_upload_token: ${{ secrets.ANACONDA_ORG_UPLOAD_TOKEN }}
diff --git a/.github/workflows/releases.yml b/.github/workflows/releases.yml
index 1b23260c2e..a08d5a6d3f 100644
--- a/.github/workflows/releases.yml
+++ b/.github/workflows/releases.yml
@@ -1,6 +1,21 @@
name: Wheels
-on: [push, pull_request]
+on:
+ release:
+ types:
+ - published
+ push:
+ branches: [main]
+ pull_request:
+ branches: [main]
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
jobs:
@@ -11,32 +26,34 @@ jobs:
fail-fast: false
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
submodules: true
fetch-depth: 0
+ persist-credentials: false
- - uses: actions/setup-python@v5.2.0
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
name: Install Python
with:
- python-version: '3.11'
+ python-version: '3.12'
- - name: Install PyBuild
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
- name: Build wheel and sdist
run: hatch build
- - uses: actions/upload-artifact@v4
+ - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: releases
path: dist
test_dist_pypi:
+ name: Test distribution artifacts
needs: [build_artifacts]
runs-on: ubuntu-latest
steps:
- - uses: actions/download-artifact@v4
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: releases
path: dist
@@ -47,16 +64,25 @@ jobs:
ls dist
upload_pypi:
- needs: [build_artifacts]
+ name: Upload to PyPI
+ needs: [build_artifacts, test_dist_pypi]
runs-on: ubuntu-latest
- if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
+ if: github.event_name == 'release'
+ environment:
+ name: releases
+ url: https://pypi.org/p/zarr
+ permissions:
+ id-token: write # Required for OIDC trusted publishing to PyPI
+ attestations: write # Required for artifact attestation
+ artifact-metadata: write # Required for artifact attestation metadata
steps:
- - uses: actions/download-artifact@v4
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: releases
path: dist
- - uses: pypa/gh-action-pypi-publish@v1.12.3
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
- user: __token__
- password: ${{ secrets.pypi_password }}
- # To test: repository_url: https://test.pypi.org/legacy/
+ subject-path: dist/*
+ - name: Publish package to PyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 5309ea4565..50bb85ff5c 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -5,118 +5,183 @@ name: Test
on:
push:
- branches: [ main ]
+ branches: [ main, 3.1.x ]
pull_request:
- branches: [ main ]
+ branches: [ main, 3.1.x ]
workflow_dispatch:
+permissions:
+ contents: read
+
+env:
+ # Use the uv from astral-sh/setup-uv; without an explicit path hatch
+ # bootstraps its own (pyapp) uv, which fails on non-3.12 runners.
+ HATCH_ENV_TYPE_VIRTUAL_UV_PATH: uv
+
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
- name: os=${{ matrix.os }}, py=${{ matrix.python-version }}, np=${{ matrix.numpy-version }}, deps=${{ matrix.dependency-set }}
+ name: os=${{ matrix.os }}, py=${{ matrix.python-version }}, deps=${{ matrix.dependency-set }}
+ environment:
+ name: codecov-upload
+ deployment: false
+ defaults:
+ run:
+ shell: bash
strategy:
matrix:
- python-version: ['3.11', '3.12', '3.13']
- numpy-version: ['1.25', '2.1']
+ python-version: ['3.12', '3.13', '3.14']
dependency-set: ["minimal", "optional"]
os: ["ubuntu-latest"]
include:
- - python-version: '3.11'
- numpy-version: '1.25'
+ - python-version: '3.12'
dependency-set: 'optional'
os: 'macos-latest'
- - python-version: '3.13'
- numpy-version: '2.1'
+
+ - python-version: '3.14'
dependency-set: 'optional'
os: 'macos-latest'
- - python-version: '3.11'
- numpy-version: '1.25'
+
+ - python-version: '3.12'
dependency-set: 'optional'
os: 'windows-latest'
- - python-version: '3.13'
- numpy-version: '2.1'
+
+ - python-version: '3.14'
dependency-set: 'optional'
os: 'windows-latest'
runs-on: ${{ matrix.os }}
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0 # grab all branches and tags
+ persist-credentials: false
- name: Set up Python
- uses: actions/setup-python@v5
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Install Hatch
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ run: python -m pip install hatch==1.16.5
- name: Set Up Hatch Env
+ env:
+ HATCH_ENV: test.py${{ matrix.python-version }}-${{ matrix.dependency-set }}
run: |
- hatch env create test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }}
- hatch env run -e test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} list-env
+ hatch env create "$HATCH_ENV"
+ hatch env run -e "$HATCH_ENV" list-env
- name: Run Tests
+ env:
+ HYPOTHESIS_PROFILE: ci
+ HATCH_ENV: test.py${{ matrix.python-version }}-${{ matrix.dependency-set }}
run: |
- hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run
+ hatch env run --env "$HATCH_ENV" run-coverage
+ - name: Upload coverage
+ if: ${{ matrix.dependency-set == 'optional' && matrix.os == 'ubuntu-latest' }}
+ uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+ with:
+ token: ${{ secrets.CODECOV_TOKEN }}
+ flags: tests
+ verbose: true # optional (default = false)
test-upstream-and-min-deps:
name: py=${{ matrix.python-version }}-${{ matrix.dependency-set }}
-
+ environment:
+ name: codecov-upload
+ deployment: false
runs-on: ubuntu-latest
strategy:
matrix:
- python-version: ['3.11', "3.13"]
+ python-version: ['3.12', "3.14"]
dependency-set: ["upstream", "min_deps"]
exclude:
- - python-version: "3.13"
+ - python-version: "3.14"
dependency-set: min_deps
- - python-version: "3.11"
+ - python-version: "3.12"
dependency-set: upstream
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0
+ persist-credentials: false
- name: Set up Python
- uses: actions/setup-python@v5
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Install Hatch
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ run: python -m pip install hatch==1.16.5
- name: Set Up Hatch Env
+ env:
+ HATCH_ENV: ${{ matrix.dependency-set }}
run: |
- hatch env create ${{ matrix.dependency-set }}
- hatch env run -e ${{ matrix.dependency-set }} list-env
+ hatch env create "$HATCH_ENV"
+ hatch env run -e "$HATCH_ENV" list-env
- name: Run Tests
+ env:
+ HATCH_ENV: ${{ matrix.dependency-set }}
run: |
- hatch env run --env ${{ matrix.dependency-set }} run
+ hatch env run --env "$HATCH_ENV" run-coverage
+ - name: Upload coverage
+ uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+ with:
+ token: ${{ secrets.CODECOV_TOKEN }}
+ flags: tests
+ verbose: true # optional (default = false)
doctests:
name: doctests
runs-on: ubuntu-latest
steps:
- - uses: actions/checkout@v4
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0 # required for hatch version discovery, which is needed for numcodecs.zarr3
+ persist-credentials: false
- name: Set up Python
- uses: actions/setup-python@v5
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.13'
cache: 'pip'
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Install Hatch
- run: |
- python -m pip install --upgrade pip
- pip install hatch
+ run: python -m pip install hatch==1.16.5
- name: Set Up Hatch Env
run: |
- hatch env create doctest
- hatch env run -e doctest list-env
+ hatch run doctest:pip list
- name: Run Tests
run: |
- hatch env run --env doctest run
+ hatch run doctest:test
+
+ benchmarks:
+ name: Benchmark smoke test
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+ - name: Set up Python
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: '3.13'
+ cache: 'pip'
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - name: Install Hatch
+ run: python -m pip install hatch==1.16.5
+ - name: Run Benchmarks
+ env:
+ ZARR_BENCHMARK_CLEAR_CACHE: '1'
+ run: |
+ hatch env run --env "test.py3.13-minimal" run-benchmark
test-complete:
name: Test complete
@@ -125,7 +190,8 @@ jobs:
[
test,
test-upstream-and-min-deps,
- doctests
+ doctests,
+ benchmarks
]
if: always()
runs-on: ubuntu-latest
diff --git a/.github/workflows/zarr-http-server-release.yml b/.github/workflows/zarr-http-server-release.yml
new file mode 100644
index 0000000000..b78fa29ab7
--- /dev/null
+++ b/.github/workflows/zarr-http-server-release.yml
@@ -0,0 +1,117 @@
+name: zarr-http-server release
+
+on:
+ workflow_dispatch:
+ push:
+ tags:
+ - 'zarr_http_server-v*'
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: false
+
+jobs:
+ build:
+ name: Build wheel and sdist
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-http-server
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ fetch-depth: 0 # hatch-vcs needs full history + tags
+
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
+
+ - name: Build
+ run: hatch build
+
+ - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: zarr-http-server-dist
+ path: packages/zarr-http-server/dist
+
+ test_artifacts:
+ name: Test built artifacts
+ needs: [build]
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-http-server-dist
+ path: dist
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: false
+
+ - name: Set up Python
+ run: uv python install 3.12
+
+ - name: Install built wheel and run import smoke test
+ run: |
+ wheel=$(ls dist/*.whl)
+ uv run --with "${wheel}" --python 3.12 --no-project \
+ python -c "import zarr_http_server; print('zarr_http_server', zarr_http_server.__version__)"
+
+ upload_pypi:
+ name: Upload to PyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/zarr_http_server-v')
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-http-server-releases
+ url: https://pypi.org/p/zarr-http-server
+ permissions:
+ id-token: write # required for OIDC trusted publishing
+ attestations: write # required for artifact attestations
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-http-server-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to PyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+
+ upload_testpypi:
+ name: Upload to TestPyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'workflow_dispatch'
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-http-server-releases-test
+ url: https://test.pypi.org/p/zarr-http-server
+ permissions:
+ id-token: write
+ attestations: write
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-http-server-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to TestPyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+ with:
+ repository-url: https://test.pypi.org/legacy/
diff --git a/.github/workflows/zarr-http-server.yml b/.github/workflows/zarr-http-server.yml
new file mode 100644
index 0000000000..16589f0d7d
--- /dev/null
+++ b/.github/workflows/zarr-http-server.yml
@@ -0,0 +1,139 @@
+name: zarr-http-server
+
+# Job steps delegate to packages/zarr-http-server/justfile, the single source
+# of truth for this package's verbs; CI owns only the python matrix and
+# caching. Keeping the commands in one place is what makes `just check`
+# locally mean the same thing as a green run here.
+
+on:
+ push:
+ branches: [main]
+ paths:
+ - 'packages/zarr-http-server/**'
+ - '.github/workflows/zarr-http-server.yml'
+ # The package resolves zarr from the repo root for its own tests, so a
+ # core change can break it. Run this suite when core changes too.
+ - 'src/zarr/**'
+ pull_request:
+ paths:
+ - 'packages/zarr-http-server/**'
+ - '.github/workflows/zarr-http-server.yml'
+ # The package resolves zarr from the repo root for its own tests, so a
+ # core change can break it. Run this suite when core changes too.
+ - 'src/zarr/**'
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ test:
+ name: pytest py=${{ matrix.python-version }}
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-http-server
+ strategy:
+ fail-fast: false
+ matrix:
+ python-version: ['3.12', '3.13', '3.14']
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Set up Python ${{ matrix.python-version }}
+ run: uv python install ${{ matrix.python-version }}
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Sync test dependency groups
+ # The examples group carries the deps the README examples need, so the
+ # test that reads a served array back with a zarr client runs here
+ # instead of silently skipping.
+ run: uv sync --group test --group examples --python ${{ matrix.python-version }}
+ - name: Run pytest
+ run: just test
+
+ ruff:
+ name: ruff
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-http-server
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Run ruff
+ run: just lint
+
+ mypy:
+ name: mypy
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-http-server
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Set up Python
+ run: uv python install 3.12
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Sync test dependency group
+ run: uv sync --group test --python 3.12
+ - name: Run mypy
+ run: just typecheck
+
+ docs:
+ name: docs
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-http-server
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Build docs
+ run: just docs-check
+
+ zarr-http-server-complete:
+ name: zarr-http-server complete
+ needs: [test, ruff, mypy, docs]
+ if: always()
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check failure
+ if: |
+ contains(needs.*.result, 'failure') ||
+ contains(needs.*.result, 'cancelled')
+ run: exit 1
+ - name: Success
+ run: echo Success!
diff --git a/.github/workflows/zarr-indexing-release.yml b/.github/workflows/zarr-indexing-release.yml
new file mode 100644
index 0000000000..2c35abad9f
--- /dev/null
+++ b/.github/workflows/zarr-indexing-release.yml
@@ -0,0 +1,117 @@
+name: zarr-indexing release
+
+on:
+ workflow_dispatch:
+ push:
+ tags:
+ - 'zarr_indexing-v*'
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: false
+
+jobs:
+ build:
+ name: Build wheel and sdist
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-indexing
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ fetch-depth: 0 # hatch-vcs needs full history + tags
+
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
+
+ - name: Build
+ run: hatch build
+
+ - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: zarr-indexing-dist
+ path: packages/zarr-indexing/dist
+
+ test_artifacts:
+ name: Test built artifacts
+ needs: [build]
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-indexing-dist
+ path: dist
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: false
+
+ - name: Set up Python
+ run: uv python install 3.12
+
+ - name: Install built wheel and run import smoke test
+ run: |
+ wheel=$(ls dist/*.whl)
+ uv run --with "${wheel}" --python 3.12 --no-project \
+ python -c "import zarr_indexing; print('zarr_indexing', zarr_indexing.__version__)"
+
+ upload_pypi:
+ name: Upload to PyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/zarr_indexing-v')
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-indexing-releases
+ url: https://pypi.org/p/zarr-indexing
+ permissions:
+ id-token: write # required for OIDC trusted publishing
+ attestations: write # required for artifact attestations
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-indexing-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to PyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+
+ upload_testpypi:
+ name: Upload to TestPyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'workflow_dispatch'
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-indexing-releases-test
+ url: https://test.pypi.org/p/zarr-indexing
+ permissions:
+ id-token: write
+ attestations: write
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-indexing-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to TestPyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+ with:
+ repository-url: https://test.pypi.org/legacy/
diff --git a/.github/workflows/zarr-indexing.yml b/.github/workflows/zarr-indexing.yml
new file mode 100644
index 0000000000..61776df913
--- /dev/null
+++ b/.github/workflows/zarr-indexing.yml
@@ -0,0 +1,136 @@
+name: zarr-indexing
+
+on:
+ push:
+ branches: [main]
+ paths:
+ - 'packages/zarr-indexing/**'
+ - '.github/workflows/zarr-indexing.yml'
+ pull_request:
+ paths:
+ - 'packages/zarr-indexing/**'
+ - '.github/workflows/zarr-indexing.yml'
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ test:
+ name: pytest py=${{ matrix.python-version }}
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-indexing
+ strategy:
+ fail-fast: false
+ matrix:
+ python-version: ['3.12', '3.13', '3.14']
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Set up Python ${{ matrix.python-version }}
+ run: uv python install ${{ matrix.python-version }}
+ # The transform tests exercise chunk resolution against zarr's ChunkGrid,
+ # so they run against the repo-root environment (which provides `zarr`)
+ # with this package as an editable overlay rather than in package
+ # isolation. The recipes carry that invocation; this step only fixes the
+ # interpreter the matrix asked for.
+ - name: Sync test dependency group
+ run: uv sync --project ../.. --group test --python ${{ matrix.python-version }}
+ - name: Run pytest
+ # Suites and invocation live in packages/zarr-indexing/justfile.
+ run: just test
+ - name: Run pytest (tensorstore parity)
+ run: just test-tensorstore
+
+ ruff:
+ name: ruff
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-indexing
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Run ruff
+ # The ruff version pin lives in packages/zarr-indexing/justfile.
+ run: just lint
+
+ pyright:
+ name: pyright
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-indexing
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Set up Python
+ run: uv python install 3.12
+ - name: Sync test dependency group
+ run: uv sync --group test --python 3.12
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Run pyright
+ # The pyright invocation lives in packages/zarr-indexing/justfile.
+ run: just typecheck
+
+ docs:
+ name: docs
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-indexing
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Build docs
+ # The strict mkdocs build lives in packages/zarr-indexing/justfile.
+ run: just docs-check
+
+ zarr-indexing-complete:
+ name: zarr-indexing complete
+ needs: [test, ruff, pyright, docs]
+ if: always()
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check failure
+ if: |
+ contains(needs.*.result, 'failure') ||
+ contains(needs.*.result, 'cancelled')
+ run: exit 1
+ - name: Success
+ run: echo Success!
diff --git a/.github/workflows/zarr-metadata-release.yml b/.github/workflows/zarr-metadata-release.yml
new file mode 100644
index 0000000000..17c285ded6
--- /dev/null
+++ b/.github/workflows/zarr-metadata-release.yml
@@ -0,0 +1,117 @@
+name: zarr-metadata release
+
+on:
+ workflow_dispatch:
+ push:
+ tags:
+ - 'zarr_metadata-v*'
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: false
+
+jobs:
+ build:
+ name: Build wheel and sdist
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-metadata
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ fetch-depth: 0 # hatch-vcs needs full history + tags
+
+ - name: Install Hatch
+ uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
+ with:
+ version: '1.16.5'
+
+ - name: Build
+ run: hatch build
+
+ - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: zarr-metadata-dist
+ path: packages/zarr-metadata/dist
+
+ test_artifacts:
+ name: Test built artifacts
+ needs: [build]
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-metadata-dist
+ path: dist
+
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: false
+
+ - name: Set up Python
+ run: uv python install 3.12
+
+ - name: Install built wheel and run import smoke test
+ run: |
+ wheel=$(ls dist/*.whl)
+ uv run --with "${wheel}" --python 3.12 --no-project \
+ python -c "import zarr_metadata; print('zarr_metadata', zarr_metadata.__version__)"
+
+ upload_pypi:
+ name: Upload to PyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/zarr_metadata-v')
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-metadata-releases
+ url: https://pypi.org/p/zarr-metadata
+ permissions:
+ id-token: write # required for OIDC trusted publishing
+ attestations: write # required for artifact attestations
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-metadata-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to PyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+
+ upload_testpypi:
+ name: Upload to TestPyPI
+ needs: [build, test_artifacts]
+ if: github.event_name == 'workflow_dispatch'
+ runs-on: ubuntu-latest
+ environment:
+ name: zarr-metadata-releases-test
+ url: https://test.pypi.org/p/zarr-metadata
+ permissions:
+ id-token: write
+ attestations: write
+ steps:
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: zarr-metadata-dist
+ path: dist
+
+ - name: Generate artifact attestation
+ uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
+ with:
+ subject-path: dist/*
+
+ - name: Publish package to TestPyPI
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+ with:
+ repository-url: https://test.pypi.org/legacy/
diff --git a/.github/workflows/zarr-metadata.yml b/.github/workflows/zarr-metadata.yml
new file mode 100644
index 0000000000..5b3b83b0e0
--- /dev/null
+++ b/.github/workflows/zarr-metadata.yml
@@ -0,0 +1,125 @@
+name: zarr-metadata
+
+# Job steps delegate to packages/zarr-metadata/justfile, the single source of
+# truth for this package's verbs; CI owns only the python matrix and caching.
+
+on:
+ push:
+ branches: [main]
+ paths:
+ - 'packages/zarr-metadata/**'
+ - '.github/workflows/zarr-metadata.yml'
+ pull_request:
+ paths:
+ - 'packages/zarr-metadata/**'
+ - '.github/workflows/zarr-metadata.yml'
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ test:
+ name: pytest py=${{ matrix.python-version }}
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-metadata
+ strategy:
+ fail-fast: false
+ matrix:
+ python-version: ['3.11', '3.12', '3.13', '3.14']
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Set up Python ${{ matrix.python-version }}
+ run: uv python install ${{ matrix.python-version }}
+ - name: Sync test dependency group
+ run: uv sync --group test --python ${{ matrix.python-version }}
+ - name: Run pytest
+ run: just test
+
+ ruff:
+ name: ruff
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-metadata
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Run ruff
+ run: just lint
+
+ pyright:
+ name: pyright
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-metadata
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Run pyright
+ # The pyright version and interpreter pins live in the justfile.
+ run: just typecheck
+
+ docs:
+ name: docs
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ shell: bash
+ working-directory: packages/zarr-metadata
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+ - name: Install uv
+ uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+ with:
+ enable-cache: true
+ - name: Install just
+ uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
+ - name: Build docs
+ run: just docs-check
+
+ zarr-metadata-complete:
+ name: zarr-metadata complete
+ needs: [test, ruff, pyright, docs]
+ if: always()
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check failure
+ if: |
+ contains(needs.*.result, 'failure') ||
+ contains(needs.*.result, 'cancelled')
+ run: exit 1
+ - name: Success
+ run: echo Success!
diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml
new file mode 100644
index 0000000000..c90ba718f6
--- /dev/null
+++ b/.github/workflows/zizmor.yml
@@ -0,0 +1,35 @@
+name: GitHub Actions Security Analysis
+
+on:
+ push:
+ branches: [main]
+ paths:
+ - '.github/workflows/**'
+ - '.github/actions/**'
+ pull_request:
+ branches: ["**"]
+ paths:
+ - '.github/workflows/**'
+ - '.github/actions/**'
+ workflow_dispatch:
+
+permissions: {}
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ zizmor:
+ name: Run zizmor
+ runs-on: ubuntu-latest
+ permissions:
+ security-events: write # Required by zizmor-action to upload SARIF files
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ persist-credentials: false
+
+ - name: Run zizmor
+ uses: zizmorcore/zizmor-action@3dc1ecc9bcb9e94e9b2c709687979e1298497054 # v0.6.2
diff --git a/.gitignore b/.gitignore
index 153ca39df0..59b6632a3c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -49,9 +49,9 @@ coverage.xml
# Django stuff:
*.log
-# Sphinx documentation
+# Documentation
+site/
docs/_build/
-docs/_autoapi
docs/data
data
data.zip
@@ -83,9 +83,18 @@ src/zarr/_version.py
data/*
src/fixture/
fixture/
+junit.xml
.DS_Store
tests/.hypothesis
.hypothesis/
zarr/version.py
+zarr.egg-info/
+
+# Local agent / planning notes (not versioned)
+.claude/
+CLAUDE.md
+docs/superpowers/
+# zarr-metadata package lockfile (a library, not an app)
+packages/zarr-metadata/uv.lock
diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc
new file mode 100644
index 0000000000..3dfdf96856
--- /dev/null
+++ b/.markdownlint-cli2.jsonc
@@ -0,0 +1,54 @@
+// markdownlint-cli2 configuration for zarr-python docs.
+//
+// We keep the rules that catch real rendering/structure problems and disable those that
+// are pure style, conflict with house conventions, or fire false positives against our
+// MkDocs/mkdocstrings + pymdownx toolchain. Complementary, not overlapping, with
+// ci/lint_docs.py (RST residue + list-breaking fences) and `mkdocs build --strict`.
+{
+ "config": {
+ "default": true,
+
+ // House style: Markdown paragraphs are single unwrapped lines, so line length is not
+ // a meaningful constraint.
+ "MD013": false,
+
+ // Purely stylistic marker/emphasis choices -- not worth the churn across existing docs.
+ "MD004": false, // ul bullet style (-, *, +)
+ "MD007": false, // ul indentation width
+ "MD050": false, // strong (bold) style
+ "MD035": false, // hr style
+
+ // False positives from our toolchain:
+ // mkdocstrings cross-refs `[`X`][zarr.X]` read as undefined reference links (MD052);
+ // pymdownx.magiclink auto-links bare URLs (MD034);
+ // md_in_html lets us embed intentional raw HTML (MD033);
+ // generated/included files (api stubs, snippets) need not open with an H1 (MD041).
+ "MD052": false,
+ "MD034": false,
+ "MD033": false,
+ "MD041": false,
+
+ // Duplicate headings are legitimate under different sections (e.g. repeated
+ // "Documentation"); only flag true sibling duplicates.
+ "MD024": { "siblings_only": true },
+
+ // Opinionated table/link/command rules with low value for these docs.
+ "MD055": false, // table pipe style
+ "MD060": false, // table column style
+ "MD059": false, // "descriptive" link text (no "click here")
+ "MD014": false, // $ before commands without shown output
+
+ // markdownlint does not understand MkDocs `!!!` admonitions, so it reads their
+ // 4-space-indented bodies as indented code blocks and flags them (and, via inferred
+ // file style, flags real fenced blocks too). Cannot coexist with our admonitions.
+ "MD046": false // code block style (fenced vs indented)
+ // Kept on (structural / real rendering bugs): MD012 (multiple blanks), MD022/MD031/MD032
+ // (blanks around headings/fences/lists), MD025 (single H1), MD029 (ordered-list prefix),
+ // MD040 (fenced code language), MD042 (empty links),
+ // MD047 (trailing newline), MD056 (table column count), among others.
+ },
+ "globs": ["docs/**/*.md"],
+ "ignores": [
+ "docs/api/**" // mkdocstrings stubs (`::: zarr.X`)
+ ]
+}
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index ea1cd4dbab..54345c819e 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -1,51 +1,98 @@
ci:
autoupdate_commit_msg: "chore: update pre-commit hooks"
autoupdate_schedule: "monthly"
- autofix_commit_msg: "style: pre-commit fixes"
autofix_prs: false
+ # Both of these are `language: system` hooks that shell out to the local
+ # toolchain — `uv` for mypy, `uv` and `just` for the docs build — and need
+ # the repo checkout to resolve their environments from `uv.lock`. Neither is
+ # available on pre-commit.ci's runners. Each is covered instead by a GitHub
+ # Actions job (Lint for mypy, zarr-http-server for the docs build) and by
+ # local prek runs.
+ skip: [mypy, zarr-http-server-docs]
+
default_stages: [pre-commit, pre-push]
+
+default_language_version:
+ python: python3.12
+
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
- rev: v0.8.2
+ rev: v0.16.0
hooks:
- - id: ruff
- args: ["--fix", "--show-fixes"]
- - id: ruff-format
+ - id: ruff-check
+ args: ["--fix", "--show-fixes"]
+ - id: ruff-format
- repo: https://github.com/codespell-project/codespell
- rev: v2.3.0
+ rev: v2.4.2
hooks:
- id: codespell
args: ["-L", "fo,ihs,kake,te", "-S", "fixture"]
- repo: https://github.com/pre-commit/pre-commit-hooks
- rev: v5.0.0
+ rev: v6.0.0
hooks:
- - id: check-yaml
- - id: trailing-whitespace
- - repo: https://github.com/pre-commit/mirrors-mypy
- rev: v1.13.0
+ - id: check-yaml
+ exclude: mkdocs.yml
+ - id: trailing-whitespace
+ - repo: https://github.com/DavidAnson/markdownlint-cli2
+ rev: v0.22.1
+ hooks:
+ # Markdown structure/hygiene. Rule selection and ignores are in
+ # .markdownlint-cli2.jsonc; complements ci/lint_docs.py (RST residue,
+ # list-breaking fences) and `mkdocs build --strict`. Scoped to docs/ to
+ # match the config's globs (pre-commit passes filenames, which would
+ # otherwise override that scoping and lint all repo Markdown).
+ - id: markdownlint-cli2
+ files: ^docs/
+ - repo: local
hooks:
- id: mypy
- files: src|tests
- additional_dependencies:
- # Package dependencies
- - packaging
- - donfig
- - numcodecs[crc32c]
- - numpy==2.1 # until https://github.com/numpy/numpy/issues/28034 is resolved
- - typing_extensions
- - universal-pathlib
- # Tests
- - pytest
+ name: mypy
+ language: system
+ entry: uv run --frozen mypy
+ pass_filenames: false
+ always_run: true
+ types_or: [python, pyi]
+ # Builds the zarr-http-server docs site with warnings as errors, which
+ # catches a dead cross-reference or a nav entry pointing at a file that
+ # no longer exists before it reaches CI.
+ #
+ # `stages: [pre-push]` overrides the default of running on every commit:
+ # this is a whole-site build, too slow to pay per commit and only
+ # actionable before the code leaves the machine. `files:` limits it to
+ # changes that touch the package, and `pass_filenames: false` because
+ # mkdocs builds the site, not a list of files. Delegating to the
+ # justfile keeps one definition of the build shared with CI.
+ - id: zarr-http-server-docs
+ name: zarr-http-server docs build
+ language: system
+ entry: >-
+ just --justfile packages/zarr-http-server/justfile
+ --working-directory packages/zarr-http-server docs-check
+ pass_filenames: false
+ files: ^packages/zarr-http-server/
+ stages: [pre-push]
- repo: https://github.com/scientific-python/cookie
- rev: 2024.08.19
+ rev: 2026.06.18
hooks:
- id: sp-repo-review
- - repo: https://github.com/pre-commit/pygrep-hooks
- rev: v1.10.0
- hooks:
- - id: rst-directive-colons
- - id: rst-inline-touching-normal
- repo: https://github.com/numpy/numpydoc
- rev: v1.8.0
+ rev: v1.11.0rc0
hooks:
- id: numpydoc-validation
+ - repo: local
+ hooks:
+ - id: ban-lstrip-rstrip
+ name: ban lstrip/rstrip
+ language: pygrep
+ # Matches .lstrip() or .rstrip() where the string argument is 2+ characters.
+ entry: "\\.(lstrip|rstrip)\\([\"'][^\"']{2,}[\"']\\)"
+ types: [python]
+ files: ^(src|tests)/
+ - repo: https://github.com/zizmorcore/zizmor-pre-commit
+ rev: v1.26.1
+ hooks:
+ - id: zizmor
+ - repo: https://github.com/twisted/towncrier
+ rev: 25.8.0
+ hooks:
+ - id: towncrier-check
diff --git a/.python-version b/.python-version
new file mode 100644
index 0000000000..e4fba21835
--- /dev/null
+++ b/.python-version
@@ -0,0 +1 @@
+3.12
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
index 32a3f0e4e1..dddf8449a4 100644
--- a/.readthedocs.yaml
+++ b/.readthedocs.yaml
@@ -4,16 +4,30 @@ build:
os: ubuntu-22.04
tools:
python: "3.12"
-
-sphinx:
- configuration: docs/conf.py
- fail_on_warning: true
-
-formats: all
-
-python:
- install:
- - method: pip
- path: .
- extra_requirements:
- - docs
+ jobs:
+ post_checkout:
+ # Cancel pull request builds whose changes are confined to the packages
+ # that have their own Read the Docs projects. Exit code 183 cancels the
+ # build and reports success to the Git provider. Scoped to PR builds
+ # ("external" versions) because origin/main is only a meaningful diff
+ # base there. Read the Docs strips shell quoting from commands, so the
+ # exclude pathspecs must use the quote-free :! form, not ':(exclude)'.
+ - |
+ if [ "$READTHEDOCS_VERSION_TYPE" = "external" ] && git diff --quiet origin/main -- :!packages/zarr-metadata :!packages/zarr-indexing;
+ then
+ exit 183;
+ fi
+ install:
+ - pip install --upgrade pip
+ - pip install .[remote] --group docs
+ pre_build:
+ - |
+ if [ "$READTHEDOCS_VERSION_TYPE" != "tag" ];
+ then
+ towncrier build --version Unreleased --yes;
+ fi
+ build:
+ html:
+ - mkdocs build --strict --site-dir $READTHEDOCS_OUTPUT/html
+mkdocs:
+ configuration: mkdocs.yml
diff --git a/LICENSE.txt b/LICENSE.txt
index a4de1c39d3..1e8da4d242 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -1,6 +1,6 @@
The MIT License (MIT)
-Copyright (c) 2015-2024 Zarr Developers
+Copyright (c) 2015-2025 Zarr Developers
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
diff --git a/README.md b/README.md
index 5ee6748ada..330c1da5ea 100644
--- a/README.md
+++ b/README.md
@@ -4,110 +4,28 @@
# Zarr
-
-
- | Latest Release |
-
-
- 
-
- |
-
- |
-
-
- 
-
- |
-
-
- | Package Status |
-
-
- 
-
- |
-
-
- | License |
-
-
- 
-
- |
-
-
- | Build Status |
-
-
- 
-
- |
-
-
- | Pre-commit Status |
-
-
- 
-
- |
-
-
-
- | Coverage |
-
-
- 
-
- |
-
-
- | Downloads |
-
-
- 
-
- |
-
-
- | Zulip |
-
-
- 
-
- |
-
-
- | Funding |
-
-
- 
-
- |
-
- Citation |
-
-
- 
-
- |
-
-
-
+[](https://pypi.org/project/zarr/)
+[](https://anaconda.org/anaconda/zarr/)
+[](https://pypi.org/project/zarr/)
+[](https://github.com/zarr-developers/zarr-python/blob/main/LICENSE.txt)
+[](https://app.codecov.io/gh/zarr-developers/zarr-python)
+[](https://zarr.readthedocs.io/en/stable/)
+[](https://ossci.zulipchat.com/#narrow/channel/423692-Zarr-Python)
+[](https://doi.org/10.5281/zenodo.3773450)
## What is it?
-Zarr is a Python package providing an implementation of compressed, chunked, N-dimensional arrays, designed for use in parallel computing. See the [documentation](https://zarr.readthedocs.io) for more information.
+The `zarr` library is a Python implementation of the [Zarr storage format](https://zarr.dev/). `zarr` delivers compressed, chunked, N-dimensional arrays that work well for parallel computing and object storage. See the [documentation](https://zarr.readthedocs.io/en/stable/) for more information.
## Main Features
-- [**Create**](https://zarr.readthedocs.io/en/stable/tutorial.html#creating-an-array) N-dimensional arrays with any NumPy `dtype`.
-- [**Chunk arrays**](https://zarr.readthedocs.io/en/stable/tutorial.html#chunk-optimizations) along any dimension.
-- [**Compress**](https://zarr.readthedocs.io/en/stable/tutorial.html#compressors) and/or filter chunks using any NumCodecs codec.
-- [**Store arrays**](https://zarr.readthedocs.io/en/stable/tutorial.html#tutorial-storage) in memory, on disk, inside a zip file, on S3, etc...
-- [**Read**](https://zarr.readthedocs.io/en/stable/tutorial.html#reading-and-writing-data) an array [**concurrently**](https://zarr.readthedocs.io/en/stable/tutorial.html#parallel-computing-and-synchronization) from multiple threads or processes.
-- Write to an array concurrently from multiple threads or processes.
-- Organize arrays into hierarchies via [**groups**](https://zarr.readthedocs.io/en/stable/tutorial.html#groups).
+- [**Create**](https://zarr.readthedocs.io/en/stable/user-guide/arrays/#creating-an-array) N-dimensional arrays with NumPy-compatible `dtype`s.
+- [**Chunk arrays**](https://zarr.readthedocs.io/en/stable/user-guide/performance/#chunk-optimizations) along any dimension.
+- [**Encode**](https://zarr.readthedocs.io/en/stable/user-guide/arrays/#compressors) chunks using a variety of useful encodings (e.g., compression).
+- [**Store arrays**](https://zarr.readthedocs.io/en/stable/user-guide/storage/) in memory, on disk, inside a zip file, on S3, etc...
+- [**Read**](https://zarr.readthedocs.io/en/stable/user-guide/arrays/#reading-and-writing-data) an array [**concurrently**](https://zarr.readthedocs.io/en/stable/user-guide/performance/#parallel-computing-and-synchronization) from multiple threads or processes.
+- [**Write**](https://zarr.readthedocs.io/en/stable/user-guide/arrays/#reading-and-writing-data) to an array concurrently from multiple threads or processes.
+- Organize arrays into hierarchies via [**groups**](https://zarr.readthedocs.io/en/stable/quick-start/#hierarchical-groups).
## Where to get it
@@ -123,4 +41,12 @@ or via `conda`:
conda install -c conda-forge zarr
```
-For more details, including how to install from source, see the [installation documentation](https://zarr.readthedocs.io/en/stable/index.html#installation).
+For more details, including how to install from source, see the [installation documentation](https://zarr.readthedocs.io/en/stable/#installation).
+
+## Repository sub-packages
+
+In addition to the primary `zarr` implementation, this repository contains other packages that provide specialized functionality with minimal dependencies:
+
+- [`zarr-metadata`](https://github.com/zarr-developers/zarr-python/tree/main/packages/zarr-metadata): Tools for Zarr metadata. Install with `pip install zarr-metadata`.
+- [`zarr-indexing`](https://github.com/zarr-developers/zarr-python/tree/main/packages/zarr-indexing): Tools for lazily indexing chunked arrays. Install with `pip install zarr-indexing`.
+- [`zarr-http-server`](https://github.com/zarr-developers/zarr-python/tree/main/packages/zarr-http-server): An HTTP server implementation targeting Zarr data. Install with `pip install zarr-http-server`.
diff --git a/TEAM.md b/TEAM.md
index e6975d7c04..ce9de1d486 100644
--- a/TEAM.md
+++ b/TEAM.md
@@ -10,6 +10,8 @@
- @dstansby (David Stansby)
- @dcherian (Deepak Cherian)
- @TomAugspurger (Tom Augspurger)
+- @maxrjones (Max Jones)
+- @ilan-gold (Ilan Gold)
## Emeritus core-developers
- @alimanfoo (Alistair Miles)
diff --git a/bench/compress_normal.py b/bench/compress_normal.py
index 179520a0e4..64204cd1e5 100644
--- a/bench/compress_normal.py
+++ b/bench/compress_normal.py
@@ -1,11 +1,11 @@
import sys
import timeit
+import blosc
import line_profiler
import numpy as np
import zarr
-from zarr import blosc
if __name__ == "__main__":
sys.path.insert(0, "..")
diff --git a/changes/.gitignore b/changes/.gitignore
new file mode 100644
index 0000000000..f935021a8f
--- /dev/null
+++ b/changes/.gitignore
@@ -0,0 +1 @@
+!.gitignore
diff --git a/changes/3285.feature.md b/changes/3285.feature.md
new file mode 100644
index 0000000000..3809c0ab02
--- /dev/null
+++ b/changes/3285.feature.md
@@ -0,0 +1,13 @@
+JSON metadata validation now delegates to ``msgspec.convert`` for the type
+coercions it supports (``Literal`` membership, ``int`` / ``bool`` strictness,
+list-to-tuple), replacing the per-field hand-written ``parse_*`` logic. A small
+fallback validates the recursive JSON values msgspec cannot, now with an
+explicit nesting-depth limit, and a latent generator-exhaustion bug in
+``parse_storage_transformers`` is fixed. See #3285.
+
+As a result some metadata inputs are now parsed more strictly. The previous
+per-field checks compared values with ``==``, which accepts any numerically
+equal object, so a float such as ``2.0`` was accepted as ``zarr_format``; it is
+now rejected because it is not an ``int``. Booleans are likewise no longer
+accepted where an ``int`` is expected, since ``bool`` is an ``int`` subclass.
+Metadata that conforms to the Zarr specification is unaffected.
diff --git a/changes/4149.doc.md b/changes/4149.doc.md
new file mode 100644
index 0000000000..8a473acac9
--- /dev/null
+++ b/changes/4149.doc.md
@@ -0,0 +1 @@
+Added a Roadmap page to the documentation outlining future plans and intended changes to the library.
diff --git a/changes/4189.bugfix.md b/changes/4189.bugfix.md
new file mode 100644
index 0000000000..76ef7e7a5e
--- /dev/null
+++ b/changes/4189.bugfix.md
@@ -0,0 +1 @@
+Allow `Group.require_array` to accept a `ZDType` for `dtype`, matching the other array creation methods. Previously an existing array could only be required with a string or NumPy dtype.
diff --git a/changes/4193.doc.md b/changes/4193.doc.md
new file mode 100644
index 0000000000..0972e8be2c
--- /dev/null
+++ b/changes/4193.doc.md
@@ -0,0 +1,4 @@
+Converted remaining reStructuredText-style double-backtick markup to Markdown
+single backticks in the docstrings of `zarr.api.asynchronous`,
+`zarr.api.synchronous`, `zarr.core.array`, `zarr.registry`, and
+`zarr.storage._common`. No functional changes.
diff --git a/changes/4213.misc.md b/changes/4213.misc.md
new file mode 100644
index 0000000000..150e60b57b
--- /dev/null
+++ b/changes/4213.misc.md
@@ -0,0 +1 @@
+Updated ruff to 0.16.0 and fixed the violations surfaced by its expanded default rule set: narrowed a blind `except Exception` in `StorePath.__eq__` to `AttributeError`, removed unnecessary `global` declarations in `zarr.core.sync`, made `subprocess.run` calls in tests pass `check=False` explicitly, and applied automatic fixes (`None` moved to the end of type unions, unused `noqa` directives removed).
diff --git a/changes/4227.bugfix.md b/changes/4227.bugfix.md
new file mode 100644
index 0000000000..18293178bd
--- /dev/null
+++ b/changes/4227.bugfix.md
@@ -0,0 +1 @@
+Consolidated metadata is now reconstructed independently of the order the keys appear in on disk. Previously, sibling subtrees whose keys were not adjacent in the persisted mapping lost their children, which made nodes unreachable through consolidated metadata -- most visibly for sibling groups whose names differ only by case.
diff --git a/changes/4239.bugfix.md b/changes/4239.bugfix.md
new file mode 100644
index 0000000000..b5bc92f18b
--- /dev/null
+++ b/changes/4239.bugfix.md
@@ -0,0 +1 @@
+`FsspecStore.from_mapper` and `FsspecStore.from_url` no longer fail when converting a synchronous instance of an async-capable filesystem whose storage options contain objects that cannot be serialized to JSON (e.g. an `azure.identity.DefaultAzureCredential`). The async instance is now constructed from the original filesystem arguments instead of a JSON round-trip.
diff --git a/changes/4247.doc.md b/changes/4247.doc.md
new file mode 100644
index 0000000000..6dbca1d3d6
--- /dev/null
+++ b/changes/4247.doc.md
@@ -0,0 +1,5 @@
+Added a "Related Projects" page to the documentation listing the companion
+packages developed in this repository — `zarr-metadata` and `zarr-indexing` —
+and linked it from the landing page. Links to those packages now use the
+canonical `https://zarr.readthedocs.io/projects/...` URLs, and each companion
+package's documentation links back to the `zarr-python` docs.
diff --git a/changes/4257.bugfix.md b/changes/4257.bugfix.md
new file mode 100644
index 0000000000..6f2740ccb4
--- /dev/null
+++ b/changes/4257.bugfix.md
@@ -0,0 +1 @@
+Numpy integers are accepted as chunk sizes again. Since 3.3.0 a per-dimension chunk size that was a numpy integer (e.g. `chunks=(np.int64(2), np.int64(2))`, as produced by any computed chunk shape) raised `TypeError: 'numpy.int64' object is not iterable`, because the scalar chunk path narrowed on `int` while its caller dispatched on `numbers.Integral`. Numpy arrays are now also accepted as chunk specifications, and a chunk specification that is neither an integer nor iterable now reports the offending value instead of failing with an opaque iteration error.
diff --git a/changes/4260.bugfix.md b/changes/4260.bugfix.md
new file mode 100644
index 0000000000..b703c47a35
--- /dev/null
+++ b/changes/4260.bugfix.md
@@ -0,0 +1 @@
+The `cast_value` codec now requires `cast-value-rs>=0.4.2`. Earlier versions of that backend silently corrupted data when handed an array that was not row-major — the layout the `transpose` codec produces — so a `cast_value` codec next to a `transpose` codec would either write transposed values with no error or fail with `ValueError: Input array must be contiguous`. The minimum version is enforced at runtime as well as in the package metadata, so an environment that already has an older `cast-value-rs` installed now raises `ImportError` when the codec is used, instead of corrupting data.
diff --git a/changes/4261.misc.md b/changes/4261.misc.md
new file mode 100644
index 0000000000..ca2a3fbb1e
--- /dev/null
+++ b/changes/4261.misc.md
@@ -0,0 +1 @@
+The contents of the `zarr` source distribution are now defined by an explicit allowlist rather than a blocklist. Previously the sdist bundled the whole `packages/` tree — `zarr-indexing`, `zarr-metadata` and `zarr-http-server`, which are released as their own distributions — along with CI configuration and other repository files. The sdist also now ships `docs/`, so the test suite it carries can be collected and run from an unpacked sdist.
diff --git a/changes/4265.bugfix.md b/changes/4265.bugfix.md
new file mode 100644
index 0000000000..6daf0fc7d0
--- /dev/null
+++ b/changes/4265.bugfix.md
@@ -0,0 +1,13 @@
+Accept [universal-pathlib](https://github.com/fsspec/universal_pathlib) `UPath` objects wherever
+zarr accepts a `StoreLike` value. A remote `UPath` now creates an `FsspecStore` using the
+filesystem and storage options the `UPath` already carries, and a local `UPath` creates a
+`LocalStore`, so that `UPath('/data')` and `Path('/data')` behave the same.
+
+Previously this worked only by accident: in universal-pathlib < 0.3 every `UPath` subclassed
+`pathlib.Path` and implemented `__fspath__`, so remote paths were either converted to a URI string
+by the caller or wrapped in a `LocalStore` that happened to dispatch through fsspec. Since
+universal-pathlib 0.3 remote paths do neither, and passing one raised
+`TypeError: Unsupported type for store_like`.
+
+`FsspecStore.from_upath` also now converts the `UPath`'s filesystem to async mode, instead of
+raising `TypeError` for synchronous filesystems and warning for sync-mode instances of async ones.
diff --git a/changes/4279.bugfix.md b/changes/4279.bugfix.md
new file mode 100644
index 0000000000..7d99a49ccf
--- /dev/null
+++ b/changes/4279.bugfix.md
@@ -0,0 +1 @@
+A `scale_offset` codec configured with a string-valued zero scale is now rejected. `scale` accepts strings, and no string is ever equal to `0`, so `"0"`, `"0.0"` and the hex form `"0x0000000000000000"` skipped the "scale must be non-zero" check that the numeric `0` triggers. On float data types the array was created, every chunk was written as zero and read back as `nan` with no error, and the zero scale was persisted to the metadata so reopening the store reproduced it; on integer data types the codec raised `ZeroDivisionError` instead of `ValueError`. The check now runs on the parsed scalar rather than the value as supplied.
diff --git a/changes/README.md b/changes/README.md
new file mode 100644
index 0000000000..889a52baa4
--- /dev/null
+++ b/changes/README.md
@@ -0,0 +1,14 @@
+Writing a changelog entry
+-------------------------
+
+Please put a new file in this directory named `xxxx..md`, where
+
+- `xxxx` is the pull request number associated with this entry
+- `` is one of:
+ - feature
+ - bugfix
+ - doc
+ - removal
+ - misc
+
+Inside the file, please write a short description of what you have changed, and how it impacts users of `zarr-python`.
diff --git a/ci/check_changelog_entries.py b/ci/check_changelog_entries.py
new file mode 100644
index 0000000000..42d7cc1708
--- /dev/null
+++ b/ci/check_changelog_entries.py
@@ -0,0 +1,70 @@
+"""
+Check changelog entries have the correct filename structure.
+
+Usage:
+ python check_changelog_entries.py [DIRECTORY]
+
+DIRECTORY defaults to the repo-root `changes/`.
+"""
+
+import sys
+from pathlib import Path
+
+VALID_CHANGELOG_TYPES = ["feature", "bugfix", "doc", "removal", "misc"]
+REPO_ROOT = Path(__file__).parent.parent.resolve()
+DEFAULT_DIRECTORY = REPO_ROOT / "changes"
+
+
+def is_int(s: str) -> bool:
+ try:
+ int(s)
+ except ValueError:
+ return False
+ else:
+ return True
+
+
+def check(directory: Path) -> int:
+ print(f"Looking for changelog entries in {directory}")
+ entries = list(directory.glob("*"))
+ entries = [e for e in entries if e.name not in [".gitignore", "README.md"]]
+ print(f"Found {len(entries)} entries")
+ print()
+
+ bad_suffix = [e for e in entries if e.suffix != ".md"]
+ bad_issue_no = [e for e in entries if not is_int(e.name.split(".")[0])]
+ # Only flag bad_type for files that have already passed the prior two
+ # checks; otherwise `e.name.split(".")[1]` may raise IndexError on a
+ # malformed name like `notes.md`.
+ bad_type = [
+ e
+ for e in entries
+ if e.suffix == ".md"
+ and is_int(e.name.split(".")[0])
+ and e.name.split(".")[1] not in VALID_CHANGELOG_TYPES
+ ]
+
+ if bad_suffix or bad_issue_no or bad_type:
+ if bad_suffix:
+ print("Changelog entries without .md suffix")
+ print("-------------------------------------")
+ print("\n".join(p.name for p in bad_suffix))
+ print()
+ if bad_issue_no:
+ print("Changelog entries without integer issue number")
+ print("----------------------------------------------")
+ print("\n".join(p.name for p in bad_issue_no))
+ print()
+ if bad_type:
+ print("Changelog entries without valid type")
+ print("------------------------------------")
+ print("\n".join(p.name for p in bad_type))
+ print(f"Valid types are: {VALID_CHANGELOG_TYPES}")
+ print()
+ return 1
+ return 0
+
+
+if __name__ == "__main__":
+ directory = Path(sys.argv[1]).resolve() if len(sys.argv) > 1 else DEFAULT_DIRECTORY
+ sys.exit(check(directory))
diff --git a/ci/check_documented_exports.py b/ci/check_documented_exports.py
new file mode 100644
index 0000000000..0772954399
--- /dev/null
+++ b/ci/check_documented_exports.py
@@ -0,0 +1,161 @@
+"""Check that every public top-level export is in the API reference.
+
+The API reference is authored as explicit mkdocstrings directives (``::: target``)
+under ``docs/api/`` -- one per documented symbol -- rather than autodoc, so a newly
+added ``zarr.__all__`` entry will not appear in the docs until someone writes a page
+for it (or it becomes a rendered member of an already-documented module). This script
+catches that gap: it resolves every ``:::`` target, expands module directives into the
+members they render (honoring ``members: false``), and asserts each name in
+``zarr.__all__`` resolves to a documented object.
+
+Usage:
+ python ci/check_documented_exports.py [API_DOCS_DIR]
+
+API_DOCS_DIR defaults to the repo-root ``docs/api``. Exits non-zero (and prints the
+undocumented exports to stderr) if any public export is missing from the reference.
+"""
+
+from __future__ import annotations
+
+import importlib
+import re
+import sys
+from pathlib import Path
+from types import ModuleType
+from typing import TYPE_CHECKING, Any
+
+import zarr
+
+if TYPE_CHECKING:
+ from collections.abc import Iterator
+
+REPO_ROOT = Path(__file__).parent.parent.resolve()
+DEFAULT_API_DOCS_ROOT = REPO_ROOT / "docs" / "api"
+
+# Names in zarr.__all__ that are intentionally absent from the API reference.
+# Keep this list short and justified -- it is the only escape hatch from the guard.
+EXEMPT_EXPORTS = {
+ "__version__", # version string, not an API symbol
+ "print_debug_info", # debugging helper, deliberately not in the reference
+}
+
+# A mkdocstrings autodoc directive: `::: some.dotted.target` at the start of a line.
+DIRECTIVE_RE = re.compile(r"^:::[ \t]+(?P\S+)")
+# `members: false` (or `members: []`) within a directive's option block disables
+# rendering of a module's members.
+MEMBERS_DISABLED_RE = re.compile(r"^\s+members:\s*(false|\[\s*\])\s*$")
+
+
+def resolve(target: str) -> Any:
+ """Resolve a `:::` target (a dotted path) to the Python object it documents."""
+ try:
+ return importlib.import_module(target)
+ except ImportError:
+ pass
+ module_path, _, attr = target.rpartition(".")
+ try:
+ return getattr(importlib.import_module(module_path), attr)
+ except (ImportError, AttributeError):
+ return None
+
+
+def iter_directives(text: str) -> Iterator[tuple[str, bool]]:
+ """Yield ``(target, members_enabled)`` for each ``:::`` directive in ``text``.
+
+ The file is split into lines once; for each directive we scan its indented option
+ block -- stopping at the first non-indented line, which ends the block -- so options
+ belonging to a later directive are never consulted. ``members_enabled`` is False when
+ that block sets ``members: false`` (or ``members: []``)."""
+ lines = text.splitlines()
+ i = 0
+ while i < len(lines):
+ match = DIRECTIVE_RE.match(lines[i])
+ if match is None:
+ i += 1
+ continue
+ members_enabled = True
+ i += 1
+ while i < len(lines):
+ line = lines[i]
+ if line.strip() == "":
+ i += 1
+ continue
+ if not line.startswith((" ", "\t")):
+ break # non-indented line: end of this directive's option block
+ if MEMBERS_DISABLED_RE.match(line):
+ members_enabled = False
+ i += 1
+ yield match.group("target"), members_enabled
+
+
+def module_member_ids(module: ModuleType) -> Iterator[int]:
+ """Yield the id() of each public member a module directive renders.
+
+ The rendered members are the module's ``__all__`` if defined, else its public
+ (non-underscore) attributes."""
+ member_names = getattr(module, "__all__", None) or [
+ name for name in dir(module) if not name.startswith("_")
+ ]
+ for name in member_names:
+ member = getattr(module, name, None)
+ if member is not None:
+ yield id(member)
+
+
+def documented_object_ids(api_docs_root: Path) -> set[int]:
+ """Collect the id()s of every object rendered by a `:::` directive under api_docs_root.
+
+ A directive pointing at an object documents that object. A directive pointing at a
+ module documents the module's public members unless the directive sets
+ ``members: false``."""
+ documented: set[int] = set()
+ for md_file in sorted(api_docs_root.rglob("*.md")):
+ for target, members_enabled in iter_directives(md_file.read_text(encoding="utf-8")):
+ obj = resolve(target)
+ if obj is None:
+ continue
+ documented.add(id(obj))
+ if isinstance(obj, ModuleType) and members_enabled:
+ documented.update(module_member_ids(obj))
+ return documented
+
+
+def find_undocumented_exports(api_docs_root: Path) -> list[str]:
+ documented = documented_object_ids(api_docs_root)
+ return sorted(
+ name
+ for name in zarr.__all__
+ if name not in EXEMPT_EXPORTS and id(getattr(zarr, name)) not in documented
+ )
+
+
+def main() -> int:
+ args = sys.argv[1:]
+ api_docs_root = Path(args[0]).resolve() if args else DEFAULT_API_DOCS_ROOT
+ if not api_docs_root.exists():
+ print(f"{api_docs_root} does not exist.", file=sys.stderr)
+ return 1
+
+ missing = find_undocumented_exports(api_docs_root)
+ if not missing:
+ print(f"All {len(zarr.__all__)} public exports are documented.")
+ return 0
+
+ print(
+ f"Found {len(missing)} public export(s) in zarr.__all__ missing from the API "
+ "reference (docs/api/):\n",
+ file=sys.stderr,
+ )
+ for name in missing:
+ print(f" - zarr.{name}", file=sys.stderr)
+ print(
+ "\nAdd a `::: zarr.` page under docs/api/zarr/ (and register it in "
+ "mkdocs.yml and docs/api/zarr/index.md), or -- if the export is intentionally "
+ "undocumented -- add it to EXEMPT_EXPORTS in this script with a reason.",
+ file=sys.stderr,
+ )
+ return 1
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/ci/check_unlinked_types.py b/ci/check_unlinked_types.py
new file mode 100644
index 0000000000..3ccfaab397
--- /dev/null
+++ b/ci/check_unlinked_types.py
@@ -0,0 +1,88 @@
+"""Check for unlinked type annotations in built documentation.
+
+mkdocstrings renders resolved types as links and unresolved
+types as Name without an anchor.
+This script finds all such unlinked types in the built HTML and reports them.
+
+Usage:
+ python ci/check_unlinked_types.py [site_dir]
+
+Raises ValueError if unlinked types are found.
+"""
+
+from __future__ import annotations
+
+import re
+import sys
+from pathlib import Path
+
+# Matches the griffe/mkdocstrings pattern for unlinked cross-references:
+# Name
+UNLINKED_PATTERN = re.compile(
+ r'(?P[^<]+)'
+)
+
+# Patterns to exclude from the report
+EXCLUDE_PATTERNS = [
+ # TypeVars and type parameters (single brackets like Foo[T])
+ re.compile(r"\[.+\]$"),
+ # Dataclass field / namedtuple field references (contain parens)
+ re.compile(r"\("),
+ # Private names
+ re.compile(r"\._"),
+ # Dunder attributes
+ re.compile(r"\.__\w+__$"),
+ # Testing utilities
+ re.compile(r"^zarr\.testing\."),
+ # Third-party types (hypothesis, pytest, etc.)
+ re.compile(r"^(hypothesis|pytest|typing_extensions|builtins|dataclasses)\."),
+]
+
+
+def should_exclude(qualname: str) -> bool:
+ return any(p.search(qualname) for p in EXCLUDE_PATTERNS)
+
+
+def find_unlinked_types(site_dir: Path) -> dict[str, set[str]]:
+ """Find all unlinked types in built HTML files.
+
+ Returns a dict mapping qualified type names to the set of pages where they appear.
+ """
+ api_dir = site_dir / "api"
+ if not api_dir.exists():
+ raise FileNotFoundError(f"{api_dir} does not exist. Run 'mkdocs build' first.")
+
+ unlinked: dict[str, set[str]] = {}
+ for html_file in api_dir.rglob("*.html"):
+ content = html_file.read_text(errors="replace")
+ rel_path = str(html_file.relative_to(site_dir))
+ for match in UNLINKED_PATTERN.finditer(content):
+ qualname = match.group("qualname")
+ if not should_exclude(qualname):
+ unlinked.setdefault(qualname, set()).add(rel_path)
+
+ return unlinked
+
+
+def main() -> None:
+ site_dir = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("site")
+ unlinked = find_unlinked_types(site_dir)
+
+ if not unlinked:
+ print("No unlinked types found.")
+ return
+
+ lines = [f"Found {len(unlinked)} unlinked types:\n"]
+ for qualname in sorted(unlinked):
+ pages = sorted(unlinked[qualname])
+ lines.append(f" {qualname}")
+ lines.extend(f" - {page}" for page in pages)
+
+ all_pages = {p for ps in unlinked.values() for p in ps}
+ lines.append(f"\nTotal: {len(unlinked)} unlinked types across {len(all_pages)} pages")
+ report = "\n".join(lines)
+ raise ValueError(report)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ci/lint_docs.py b/ci/lint_docs.py
new file mode 100644
index 0000000000..a847e8aa9d
--- /dev/null
+++ b/ci/lint_docs.py
@@ -0,0 +1,341 @@
+"""Lint docstrings and Markdown for reStructuredText markup that won't render.
+
+This project renders API docs with mkdocstrings (``docstring_style: numpy``) and prose
+with MkDocs + Markdown -- not Sphinx/reStructuredText. RST constructs that survive from
+older docstrings (or muscle memory) are not interpreted: a Sphinx role passes through as
+literal text instead of becoming a link, an ``.. note::`` directive renders as a stray
+line, and a ``:param:`` field list never becomes a documented parameter.
+
+Crucially, none of this is caught by the rest of the docs CI. ``mkdocs build --strict``
+sees the residue as ordinary prose (no warning), and ``ci/check_unlinked_types.py`` only
+finds cross-references mkdocstrings *attempted* to resolve -- a raw ``:class:`` role is
+never attempted, so it leaves no unlinked-type span. This linter fills that gap with a
+fast, source-level check that needs no docs build.
+
+Checks fall into two groups -- RST markup that silently fails under MkDocs/mkdocstrings,
+and Markdown structural problems that render as valid-but-wrong HTML (so `mkdocs build`
+emits no warning):
+
+ sphinx-role :class:`X`, :func:`X`, :py:meth:`X` -> [`X`][zarr.X]
+ rst-directive .. note:: / .. code-block:: python -> MkDocs admonition / fenced code
+ rst-field :param x:, :returns:, :rtype: -> numpydoc Parameters/Returns/Raises
+ rst-link `text `_ -> [text](https://example)
+ list-break unindented code fence between list items -> indent the fence under its item
+ list-indent continuation block indented < 4 spaces -> indent it 4 spaces
+ list-blank list item directly after indented block -> blank line before the item
+
+The ``list-break`` check catches a fenced code block at column 0 placed *between* two list
+items: because the fence is not indented into the preceding item, Markdown ends the list at
+the fence and the following item starts a fresh list -- renumbering an ordered list (1, 1, 2
+instead of 1, 2, 3) or breaking the grouping/spacing of any list. markdownlint's MD029 only
+notices this for sequentially-numbered ordered lists; lazily-numbered (1., 1.) and unordered
+lists slip past it, so this structural check covers the gap.
+
+The ``list-indent`` and ``list-blank`` checks catch the two halves of Python-Markdown's
+strict list-continuation rules, which differ from CommonMark. A blank-line-separated
+block (paragraph, nested list, table) belongs to a list item only when indented at least
+4 spaces; at the 2-space indent other renderers accept, Python-Markdown ends the list and
+the block escapes to the top level (``list-indent``). And a new list item can not start
+directly after an indented continuation block: without a blank line first, the ``- `` line
+is lazily absorbed into the preceding paragraph as literal text (``list-blank``). Both
+produced silently-broken changelog rendering in ``docs/release-notes.md``.
+
+Usage:
+ python ci/lint_docs.py [PATH ...]
+
+PATH defaults to the repo-root ``src/zarr`` and ``docs``. Each PATH may be a file or a
+directory (directories are searched for ``*.py`` and ``*.md``). Exits non-zero if any
+issues are found.
+"""
+
+from __future__ import annotations
+
+import ast
+import re
+import sys
+from dataclasses import dataclass
+from pathlib import Path
+from typing import NamedTuple
+
+REPO_ROOT = Path(__file__).parent.parent.resolve()
+DEFAULT_PATHS = (REPO_ROOT / "src" / "zarr", REPO_ROOT / "docs")
+
+# A Sphinx interpreted-text role: an optional domain, a role name, then a backtick
+# target -- e.g. :class:`Foo` or :py:meth:`Foo.bar`. Requires the trailing backtick so
+# plain "::" (RST literal markers, time strings, mkdocs-material :icon: shortcodes) and
+# URLs ("https://") never match.
+SPHINX_ROLE = re.compile(r":[a-zA-Z_]\w*(?::[a-zA-Z_]\w*)?:`[^`\n]+`")
+
+# An RST directive line: ".. name::" (with or without an argument after it). RST hyperlink
+# targets (".. _label:") and comments (".. text") lack the "::" and are not flagged.
+RST_DIRECTIVE = re.compile(r"^\s*\.\.[ \t]+[\w-]+::")
+
+# An RST field-list entry used for docstring fields. The role names above (class, func,
+# ...) are deliberately excluded so a role is reported as a role, not a field.
+RST_FIELD = re.compile(
+ r"^\s*:(param|parameter|arg|argument|key|keyword|kwarg|type|returns?|rtype"
+ r"|raises?|except|exception|yields?|ytype|var|cvar|ivar)\b[^:]*:"
+)
+
+# An RST external hyperlink: `text `_
+RST_LINK = re.compile(r"`[^`\n]+\n]+>`_")
+
+# A list item at column 0: an ordered marker (1. / 1)) or a bullet (-, *, +) followed by
+# whitespace and content. Leading-whitespace (nested/continuation) lines are intentionally
+# not matched -- the list-break check only fires on top-level items.
+LIST_ITEM = re.compile(r"^(?:\d+[.)]|[-*+])\s+\S")
+
+
+class Check(NamedTuple):
+ """One docs-residue check: its category, the line pattern that flags it (None for a
+ structural check matched outside ``_scan_line``), and the user-facing remediation
+ shown by ``main()``. Keeping ``example``/``fix`` here makes this the single source for
+ the help text, so adding a check can't leave the help out of date."""
+
+ category: str
+ pattern: re.Pattern[str] | None
+ example: str
+ fix: str
+
+
+# The ``list-*`` checks carry no pattern -- they are detected structurally, not by scanning
+# a single line -- but they appear here so they share the remediation help.
+CHECKS = (
+ Check("sphinx-role", SPHINX_ROLE, ":class:`X`", "[`X`][zarr.X]"),
+ Check("rst-directive", RST_DIRECTIVE, ".. note::", "MkDocs admonition (!!! note)"),
+ Check("rst-field", RST_FIELD, ":param x:", "numpydoc Parameters/Returns/Raises section"),
+ Check("rst-link", RST_LINK, "`text `_", "[text](url)"),
+ Check("list-break", None, "fence between items", "indent the fence 4 spaces to nest it"),
+ Check("list-indent", None, "2-space continuation", "indent the block 4 spaces under its item"),
+ Check("list-blank", None, "item after indented block", "add a blank line before the item"),
+)
+
+
+@dataclass(frozen=True)
+class Finding:
+ path: Path
+ line: int
+ category: str
+ snippet: str
+
+ def format(self) -> str:
+ try:
+ location: Path | str = self.path.relative_to(REPO_ROOT)
+ except ValueError:
+ location = self.path
+ return f" {location}:{self.line}: [{self.category}] {self.snippet.strip()}"
+
+
+def _scan_line(text: str) -> list[str]:
+ """Return every RST-residue category found in a single line (a line can carry more
+ than one, e.g. a role and an external link)."""
+ return [c.category for c in CHECKS if c.pattern is not None and c.pattern.search(text)]
+
+
+def lint_python(path: Path) -> list[Finding]:
+ """Scan the docstrings (module, classes, functions) of a Python file.
+
+ Only docstrings are checked -- they are what mkdocstrings renders -- so RST-looking
+ text inside ordinary code or string literals is never misreported."""
+ source = path.read_text(encoding="utf-8")
+ try:
+ tree = ast.parse(source)
+ except SyntaxError as exc: # pragma: no cover - surfaced, not silently skipped
+ return [Finding(path, exc.lineno or 0, "syntax-error", str(exc.msg))]
+
+ doc_nodes = (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)
+ # node.body[0].value is the docstring literal; its lineno is the line the string opens
+ # on, so content line i maps to source line (start + i).
+ docstrings = [
+ (docstring, node.body[0].value.lineno) # type: ignore[attr-defined]
+ for node in ast.walk(tree)
+ if isinstance(node, doc_nodes)
+ if (docstring := ast.get_docstring(node, clean=False))
+ ]
+ return [
+ Finding(path, start + offset, category, line)
+ for docstring, start in docstrings
+ for offset, line in enumerate(docstring.splitlines())
+ for category in _scan_line(line)
+ ]
+
+
+class Fence(NamedTuple):
+ """A fenced code block, by 0-based line index. ``terminated`` is False when the fence
+ has no closing delimiter before EOF, in which case ``close`` is the last line."""
+
+ open: int
+ close: int
+ terminated: bool
+
+
+def fenced_blocks(lines: list[str]) -> list[Fence]:
+ """Index every fenced code block in ``lines``.
+
+ An unterminated fence is malformed Markdown that `mkdocs build` surfaces anyway; it is
+ still returned (with ``terminated=False``, ``close`` at the last line) so callers that
+ skip code can skip to EOF."""
+ blocks: list[Fence] = []
+ fence: str | None = None
+ open_idx = -1
+ for i, line in enumerate(lines):
+ stripped = line.lstrip()
+ if fence is None:
+ if stripped.startswith(("```", "~~~")):
+ fence, open_idx = stripped[:3], i
+ elif stripped.startswith(fence):
+ blocks.append(Fence(open_idx, i, terminated=True))
+ fence = None
+ if fence is not None:
+ blocks.append(Fence(open_idx, len(lines) - 1, terminated=False))
+ return blocks
+
+
+def find_list_breaking_fences(lines: list[str], blocks: list[Fence]) -> list[tuple[int, str]]:
+ """Return ``(lineno, snippet)`` for each fenced code block at column 0 that splits a
+ list -- i.e. one whose nearest non-blank neighbours on both sides are top-level list
+ items. Such a fence is not indented into the preceding item, so Markdown closes the
+ list at the fence and the following item starts a new one. The fix is to indent the
+ fence (4 spaces) so it nests inside its list item. See the module docstring.
+
+ Conservative on purpose: it requires a list item *directly* before and after (a
+ continuation line or paragraph in between is not matched), keeping false positives low
+ for a check that fails CI. Unterminated fences are ignored."""
+
+ def neighbour(start: int, step: int) -> str | None:
+ j = start + step
+ while 0 <= j < len(lines):
+ if lines[j].strip():
+ return lines[j]
+ j += step
+ return None
+
+ def splits_a_list(open_i: int, close_i: int) -> bool:
+ if lines[open_i][:1].isspace():
+ return False # indented fence: already nested in the list item, not a break
+ before = neighbour(open_i, -1)
+ after = neighbour(close_i, +1)
+ return bool(before and after and LIST_ITEM.match(before) and LIST_ITEM.match(after))
+
+ return [
+ (fence.open + 1, lines[fence.open])
+ for fence in blocks
+ if fence.terminated and splits_a_list(fence.open, fence.close)
+ ]
+
+
+def find_list_continuation_issues(
+ lines: list[str], in_code: set[int]
+) -> list[tuple[int, str, str]]:
+ """Return ``(lineno, category, snippet)`` for list continuations Python-Markdown will
+ mis-render (see the module docstring):
+
+ - ``list-indent``: a blank-line-separated block inside a list item indented 1-3
+ spaces. Python-Markdown requires 4; at less, the block escapes the list.
+ - ``list-blank``: a top-level list item directly after a line indented 4+ spaces.
+ Without a blank line in between, the item is absorbed into the preceding paragraph
+ as literal ``- `` text.
+
+ Lazy continuations (an indented line with no blank line before it) are valid at any
+ indent and are not flagged. Fenced-code lines are opaque: never flagged themselves,
+ but they keep the item scope open and their indent feeds the ``list-blank`` check so
+ an item directly after an indented fence is still caught."""
+ findings: list[tuple[int, str, str]] = []
+ in_item = False # inside a top-level list item's scope
+ prev_blank = True
+ prev_indent = 0
+ for i, line in enumerate(lines):
+ stripped = line.strip()
+ if not stripped:
+ prev_blank = True
+ continue
+ indent = len(line) - len(line.lstrip(" "))
+ if i not in in_code:
+ if indent == 0:
+ is_item = bool(LIST_ITEM.match(line))
+ if is_item and in_item and not prev_blank and prev_indent >= 4:
+ findings.append((i + 1, "list-blank", line))
+ in_item = is_item
+ elif in_item and prev_blank and indent < 4:
+ findings.append((i + 1, "list-indent", line))
+ prev_blank = False
+ prev_indent = indent
+ return findings
+
+
+def lint_markdown(path: Path) -> list[Finding]:
+ """Scan a Markdown file: RST residue in prose (skipping fenced code blocks), plus
+ list-structure problems (see find_list_breaking_fences and
+ find_list_continuation_issues)."""
+ lines = path.read_text(encoding="utf-8").splitlines()
+ blocks = fenced_blocks(lines)
+ in_code = {i for fence in blocks for i in range(fence.open, fence.close + 1)}
+
+ prose = [
+ Finding(path, lineno, category, line)
+ for lineno, line in enumerate(lines, start=1)
+ if lineno - 1 not in in_code
+ for category in _scan_line(line)
+ ]
+ breaks = [
+ Finding(path, lineno, "list-break", snippet)
+ for lineno, snippet in find_list_breaking_fences(lines, blocks)
+ ]
+ continuations = [
+ Finding(path, lineno, category, snippet)
+ for lineno, category, snippet in find_list_continuation_issues(lines, in_code)
+ ]
+ return prose + breaks + continuations
+
+
+def iter_files(paths: tuple[Path, ...]) -> list[Path]:
+ files: list[Path] = []
+ for path in paths:
+ if path.is_file():
+ files.append(path)
+ elif path.is_dir():
+ files.extend(sorted(path.rglob("*.py")))
+ files.extend(sorted(path.rglob("*.md")))
+ else:
+ raise FileNotFoundError(f"{path} does not exist")
+ return files
+
+
+LINTERS = {".py": lint_python, ".md": lint_markdown}
+
+
+def lint(paths: tuple[Path, ...]) -> list[Finding]:
+ return [
+ finding
+ for file in iter_files(paths)
+ if file.suffix in LINTERS
+ for finding in LINTERS[file.suffix](file)
+ ]
+
+
+def main() -> int:
+ args = sys.argv[1:]
+ paths = tuple(Path(a).resolve() for a in args) if args else DEFAULT_PATHS
+ findings = lint(paths)
+
+ if not findings:
+ print("No reStructuredText residue or list-breaking fences found in docs.")
+ return 0
+
+ print(
+ f"Found {len(findings)} docs issue(s) -- RST markup that will not render under "
+ "MkDocs/mkdocstrings, or Markdown that renders as valid-but-wrong HTML:\n",
+ file=sys.stderr,
+ )
+ for finding in findings:
+ print(finding.format(), file=sys.stderr)
+ remediation = "\n".join(f" {c.category:<13} {c.example:<19} -> {c.fix}" for c in CHECKS)
+ print(
+ f"\nFix each issue (see ci/lint_docs.py header):\n{remediation}",
+ file=sys.stderr,
+ )
+ return 1
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/codecov.yml b/codecov.yml
new file mode 100644
index 0000000000..a3783cc39a
--- /dev/null
+++ b/codecov.yml
@@ -0,0 +1,34 @@
+coverage:
+ status:
+ patch:
+ default:
+ target: auto
+ informational: true
+ project:
+ default:
+ target: auto
+ threshold: 0.1
+ flags:
+ - tests
+flags:
+ tests:
+ paths:
+ - src/
+ carryforward: true
+ gpu:
+ paths:
+ - src/
+ carryforward: true
+codecov:
+ notify:
+ # 6 = test.yml: 3 (optional+ubuntu) + 2 (upstream + min_deps), hypothesis: 1
+ after_n_builds: 6
+ wait_for_ci: yes
+comment:
+ layout: "diff, files"
+ behavior: default
+ require_changes: true # if true: only post the comment if coverage changes
+ branches: # branch names that can post comment
+ - "main"
+github_checks:
+ annotations: false
diff --git a/design/chunk-grid.md b/design/chunk-grid.md
new file mode 100644
index 0000000000..0f12e35c4b
--- /dev/null
+++ b/design/chunk-grid.md
@@ -0,0 +1,720 @@
+# Unified Chunk Grid
+
+Version: 6
+
+Design document for adding rectilinear (variable) chunk grid support to **zarr-python**, conforming to the [rectilinear chunk grid extension spec](https://github.com/zarr-developers/zarr-extensions/pull/25).
+
+**Related:**
+
+- [#3750](https://github.com/zarr-developers/zarr-python/issues/3750) (single ChunkGrid proposal)
+- [#3534](https://github.com/zarr-developers/zarr-python/pull/3534) (rectilinear implementation)
+- [#3735](https://github.com/zarr-developers/zarr-python/pull/3735) (chunk grid module/registry)
+- [ZEP0003](https://github.com/zarr-developers/zeps/blob/main/draft/ZEP0003.md) (variable chunking spec)
+- [zarr-specs#370](https://github.com/zarr-developers/zarr-specs/pull/370) (sharding v1.1: non-divisible subchunks)
+- [zarr-extensions#25](https://github.com/zarr-developers/zarr-extensions/pull/25) (rectilinear extension)
+- [zarr-extensions#34](https://github.com/zarr-developers/zarr-extensions/issues/34) (sharding + rectilinear)
+
+## Problem
+
+Chunk grids form a hierarchy — the rectilinear grid is strictly more general than the regular grid. Any regular grid is expressible as a rectilinear grid. There is no known chunk grid that is both (a) more general than rectilinear and (b) retains the axis-aligned tessellation properties Zarr assumes. All known grids are special cases:
+
+| Grid type | Description | Example |
+|---|---|---|
+| Regular | Uniform chunk size, boundary chunks padded with fill_value | `[10, 10, 10, 10]` |
+| Regular-bounded (zarrs) | Uniform chunk size, boundary chunks trimmed to array extent | `[10, 10, 10, 5]` |
+| HPC boundary-padded | Regular interior, larger boundary chunks ([VirtualiZarr#217](https://github.com/zarr-developers/VirtualiZarr/issues/217)) | `[10, 8, 8, 8, 10]` |
+| Fully variable | Arbitrary per-chunk sizes | `[5, 12, 3, 20]` |
+
+Prior iterations on the chunk grid design were based on the Zarr V3 spec's definition of chunk grids as an extension point alongside codecs, dtypes, etc. Therefore, we started designing the chunk grid implementation following a similar registry-based approach. However, in practice chunk grids are fundamentally different than codecs. Codecs are independent; supporting `zstd` tells you nothing about `gzip`. Chunk grids are not: every regular grid is a valid rectilinear grid. A registry-based plugin system makes sense for codecs but adds complexity without clear benefit for chunk grids. Here we start from some basic goals and propose a more fitting design for supporting different chunk grids in zarr-python.
+
+## Goals
+
+1. **Follow the zarr extension proposal.** The implementation should conform to the [rectilinear chunk grid spec](https://github.com/zarr-developers/zarr-extensions/tree/main/chunk-grids/rectilinear), not innovate on the metadata format.
+2. **Minimize changes to the public API.** Users creating regular arrays should see no difference. Rectilinear is additive.
+3. **Maintain backwards compatibility.** Existing code using `.chunks`, `isinstance` checks, or importing `RegularChunkGrid`/`RectilinearChunkGrid` from `zarr.core.chunk_grids` should continue to work where practical (with deprecation warnings where appropriate). Internal code paths/imports may be broken with justification.
+4. **Design for future iteration.** The internal architecture should allow refactoring (e.g., metadata/array separation, new dimension types) without breaking the public API.
+5. **Minimize downstream changes.** xarray, VirtualiZarr, Icechunk, Cubed, etc. should need minimal updates.
+6. **Minimize time to stable release.** Ship behind a feature flag, stabilize through real-world usage, promote to stable API.
+7. **The new API should be useful.** `read_chunk_sizes`/`write_chunk_sizes`, `ChunkGrid.__getitem__`, `is_regular` — these should solve real problems, not just expose internals.
+8. **Extensible for other serialization structures.** The per-dimension design should support future encodings (tile, temporal) without changes to indexing or codecs.
+
+## Design
+
+### Design choices
+
+1. **A chunk grid is a concrete arrangement of chunks.** Not an abstract tiling pattern. This means that the chunk grid is bound to specific array dimensions, which enables the chunk grid to answer any question about any chunk (offset, size, count) without external parameters.
+2. **One implementation, multiple serialization forms.** A single `ChunkGrid` class handles all chunking logic. The serialization format (`"regular"` vs `"rectilinear"`) is chosen by the metadata layer, not the grid.
+3. **No chunk grid registry.** Simple name-based dispatch in the metadata layer's `parse_chunk_grid()`.
+4. **Fixed vs Varying per dimension.** `FixedDimension(size, extent)` for uniform chunks; `VaryingDimension(edges, extent)` for per-chunk edge lengths with precomputed prefix sums. Avoids expanding regular dimensions into lists of identical values.
+5. **Transparent transitions.** Operations like `resize()` can move an array from regular to rectilinear chunking.
+
+### Internal representation
+
+```python
+@dataclass(frozen=True)
+class FixedDimension:
+ """Uniform chunk size. Boundary chunks contain less data but are
+ encoded at full size by the codec pipeline."""
+ size: int # chunk edge length (>= 0)
+ extent: int # array dimension length
+
+ def __post_init__(self) -> None:
+ # validates size >= 0 and extent >= 0
+
+ @property
+ def nchunks(self) -> int:
+ if self.size == 0:
+ return 0
+ return ceildiv(self.extent, self.size)
+
+ def index_to_chunk(self, idx: int) -> int:
+ return idx // self.size # raises IndexError if OOB
+ def chunk_offset(self, chunk_ix: int) -> int:
+ return chunk_ix * self.size # raises IndexError if OOB
+ def chunk_size(self, chunk_ix: int) -> int:
+ return self.size # always uniform; raises IndexError if OOB
+ def data_size(self, chunk_ix: int) -> int:
+ return max(0, min(self.size, self.extent - chunk_ix * self.size)) # raises IndexError if OOB
+ @property
+ def unique_edge_lengths(self) -> Iterable[int]:
+ return (self.size,) # O(1)
+ def indices_to_chunks(self, indices: NDArray) -> NDArray:
+ return indices // self.size
+ def with_extent(self, new_extent: int) -> FixedDimension:
+ return FixedDimension(size=self.size, extent=new_extent)
+ def resize(self, new_extent: int) -> FixedDimension:
+ return FixedDimension(size=self.size, extent=new_extent)
+
+@dataclass(frozen=True)
+class VaryingDimension:
+ """Explicit per-chunk sizes. The last chunk may extend past the array
+ extent (extent < sum(edges)), in which case data_size clips to the
+ valid region while chunk_size returns the full edge length for codec
+ processing. This underflow is allowed to match how regular grids
+ handle boundary chunks, and to support shrinking an array without
+ rewriting chunk edges (the spec allows trailing edges beyond the extent)."""
+ edges: tuple[int, ...] # per-chunk edge lengths (all > 0)
+ cumulative: tuple[int, ...] # prefix sums for O(log n) lookup
+ extent: int # array dimension length (may be < sum(edges))
+
+ def __init__(self, edges: Sequence[int], extent: int) -> None:
+ # validates edges non-empty, all > 0, extent >= 0, extent <= sum(edges)
+ # computes cumulative via itertools.accumulate
+ # uses object.__setattr__ for frozen dataclass
+
+ @property
+ def nchunks(self) -> int:
+ # number of chunks that overlap [0, extent)
+ if extent == 0:
+ return 0
+ return bisect.bisect_left(self.cumulative, extent) + 1
+
+ @property
+ def ngridcells(self) -> int:
+ return len(self.edges)
+
+ def index_to_chunk(self, idx: int) -> int:
+ return bisect.bisect_right(self.cumulative, idx) # raises IndexError if OOB
+ def chunk_offset(self, chunk_ix: int) -> int:
+ return self.cumulative[chunk_ix - 1] if chunk_ix > 0 else 0 # raises IndexError if OOB
+ def chunk_size(self, chunk_ix: int) -> int:
+ return self.edges[chunk_ix] # raises IndexError if OOB
+ def data_size(self, chunk_ix: int) -> int:
+ offset = self.chunk_offset(chunk_ix)
+ return max(0, min(self.edges[chunk_ix], self.extent - offset)) # raises IndexError if OOB
+ @property
+ def unique_edge_lengths(self) -> Iterable[int]:
+ # lazy generator: yields unseen values, short-circuits deduplication
+ def indices_to_chunks(self, indices: NDArray) -> NDArray:
+ return np.searchsorted(self.cumulative, indices, side='right')
+ def with_extent(self, new_extent: int) -> VaryingDimension:
+ # validates cumulative[-1] >= new_extent (O(1)), re-binds extent
+ return VaryingDimension(self.edges, extent=new_extent)
+ def resize(self, new_extent: int) -> VaryingDimension:
+ # grow past edge sum: append chunk of size (new_extent - sum(edges))
+ # shrink or grow within edge sum: preserve all edges, re-bind extent
+```
+
+Both types implement the `DimensionGrid` protocol: `nchunks`, `extent`, `index_to_chunk`, `chunk_offset`, `chunk_size`, `data_size`, `indices_to_chunks`, `unique_edge_lengths`, `with_extent`, `resize`. Memory usage scales with the number of *varying* dimensions, not total chunks.
+
+All per-chunk methods (`chunk_offset`, `chunk_size`, `data_size`) raise `IndexError` for out-of-bounds chunk indices, providing consistent fail-fast behavior across both dimension types.
+
+The two size methods serve different consumers:
+
+| Method | Returns | Consumer |
+|---|---|---|
+| `chunk_size` | Buffer size for codec processing | Codec pipeline (`ArraySpec.shape`) |
+| `data_size` | Valid data region within the buffer | Indexing pipeline (`chunk_selection` slicing) |
+
+For `FixedDimension`, these differ only at the boundary. For `VaryingDimension`, these differ only when the last chunk extends past the extent (i.e., `extent < sum(edges)`). This matches current zarr-python behavior: `get_chunk_spec` passes the full `chunk_shape` to the codec for all chunks, and the indexer generates a `chunk_selection` that clips the decoded buffer.
+
+### DimensionGrid Protocol
+
+```python
+@runtime_checkable
+class DimensionGrid(Protocol):
+ """Structural interface shared by FixedDimension and VaryingDimension."""
+
+ @property
+ def nchunks(self) -> int: ...
+ @property
+ def ngridcells(self) -> int: ...
+ @property
+ def extent(self) -> int: ...
+ def index_to_chunk(self, idx: int) -> int: ...
+ def chunk_offset(self, chunk_ix: int) -> int: ... # raises IndexError if OOB
+ def chunk_size(self, chunk_ix: int) -> int: ... # raises IndexError if OOB
+ def data_size(self, chunk_ix: int) -> int: ... # raises IndexError if OOB
+ def indices_to_chunks(self, indices: NDArray[np.intp]) -> NDArray[np.intp]: ...
+ @property
+ def unique_edge_lengths(self) -> Iterable[int]: ...
+ def with_extent(self, new_extent: int) -> DimensionGrid: ...
+ def resize(self, new_extent: int) -> DimensionGrid: ...
+```
+
+The protocol is `@runtime_checkable`, enabling polymorphic handling of both dimension types without `isinstance` checks.
+
+`nchunks` and `ngridcells` differ when `extent < sum(edges)`: `nchunks` counts only chunks that overlap `[0, extent)`, while `ngridcells` counts total defined grid cells (i.e., `len(edges)`). For `FixedDimension`, both are equal. For `VaryingDimension`, they differ after a resize that shrinks the extent below the edge sum.
+
+### ChunkSpec
+
+```python
+@dataclass(frozen=True)
+class ChunkSpec:
+ slices: tuple[slice, ...] # valid data region in array coordinates
+ codec_shape: tuple[int, ...] # buffer shape for codec processing
+
+ @property
+ def shape(self) -> tuple[int, ...]:
+ return tuple(s.stop - s.start for s in self.slices)
+
+ @property
+ def is_boundary(self) -> bool:
+ return self.shape != self.codec_shape
+```
+
+For interior chunks, `shape == codec_shape`. For boundary chunks of a regular grid, `codec_shape` is the full declared chunk size while `shape` is clipped. For rectilinear grids, `shape == codec_shape` unless the last chunk extends past the extent.
+
+### API
+
+```python
+# Creating arrays
+arr = zarr.create_array(shape=(100, 200), chunks=(10, 20)) # regular
+arr = zarr.create_array(shape=(60, 100), chunks=[[10, 20, 30], [25, 25, 25, 25]]) # rectilinear
+
+# ChunkGrid as a collection
+grid = arr._chunk_grid # ChunkGrid (bound to array shape)
+grid.grid_shape # (10, 10) — number of chunks per dimension
+grid.ndim # 2
+grid.is_regular # True if all dimensions are Fixed
+
+spec = grid[0, 1] # ChunkSpec for chunk at grid position (0, 1)
+spec.slices # (slice(0, 10), slice(20, 40))
+spec.shape # (10, 20) — data shape
+spec.codec_shape # (10, 20) — same for interior chunks
+
+boundary = grid[9, 0] # boundary chunk (extent=100, size=10)
+boundary.shape # (10, 20) — data shape
+boundary.codec_shape # (10, 20) — codec sees full buffer
+
+grid[99, 99] # None — out of bounds
+
+for spec in grid: # iterate all chunks
+ ...
+
+# .chunks property: retained for regular grids, raises NotImplementedError for rectilinear
+arr.chunks # (10, 20)
+
+# .read_chunk_sizes / .write_chunk_sizes: works for all grids (dask-style)
+arr.write_chunk_sizes # ((10, 10, ..., 10), (20, 20, ..., 20))
+```
+
+`ChunkGrid.__getitem__` constructs `ChunkSpec` using `chunk_size` for `codec_shape` and `data_size` for `slices`:
+
+```python
+def __getitem__(self, coords: int | tuple[int, ...]) -> ChunkSpec | None:
+ if isinstance(coords, int):
+ coords = (coords,)
+ slices = []
+ codec_shape = []
+ for dim, ix in zip(self.dimensions, coords):
+ if ix < 0 or ix >= dim.nchunks:
+ return None
+ offset = dim.chunk_offset(ix)
+ slices.append(slice(offset, offset + dim.data_size(ix)))
+ codec_shape.append(dim.chunk_size(ix))
+ return ChunkSpec(tuple(slices), tuple(codec_shape))
+```
+
+#### Construction
+
+`from_sizes` requires `array_shape`, binding the extent per dimension at construction time. This is a core design choice: a chunk grid is a concrete arrangement for a specific array, not an abstract tiling pattern.
+
+```python
+# Regular grid — all FixedDimension
+grid = ChunkGrid.from_sizes(array_shape=(100, 200), chunk_sizes=(10, 20))
+
+# Rectilinear grid — extent = sum(edges) when shape matches
+grid = ChunkGrid.from_sizes(array_shape=(60, 100), chunk_sizes=[[10, 20, 30], [25, 25, 25, 25]])
+
+# Rectilinear grid with boundary clipping — last chunk extends past array extent
+# e.g., shape=(55, 90) but edges sum to (60, 100): data_size clips at extent
+grid = ChunkGrid.from_sizes(array_shape=(55, 90), chunk_sizes=[[10, 20, 30], [25, 25, 25, 25]])
+
+# Direct construction
+grid = ChunkGrid(dimensions=(FixedDimension(10, 100), VaryingDimension([10, 20, 30], 55)))
+```
+
+When `extent < sum(edges)`, the dimension is always stored as `VaryingDimension` (even if all edges are identical) to preserve the explicit edge count. The last chunk's `chunk_size` returns the full declared edge (codec buffer) while `data_size` clips to the extent. This mirrors how `FixedDimension` handles boundary chunks in regular grids.
+
+#### Serialization
+
+```python
+# Regular grid:
+{"name": "regular", "configuration": {"chunk_shape": [10, 20]}}
+
+# Rectilinear grid (with RLE compression and "kind" field):
+{
+ "name": "rectilinear",
+ "configuration": {"kind": "inline", "chunk_shapes": [[10, 20, 30], [[25, 4]]]},
+}
+```
+
+Both names deserialize to the same `ChunkGrid` class. The serialized form does not include the array extent — that comes from `shape` in array metadata and is combined with the chunk grid when constructing a `ChunkGrid` via `ChunkGrid.from_metadata()`.
+
+**The `ChunkGrid` does not serialize itself.** The format choice (`"regular"` vs `"rectilinear"`) belongs to `ArrayV3Metadata`. Serialization and deserialization are handled by the metadata-layer chunk grid classes (`RegularChunkGridMetadata` and `RectilinearChunkGridMetadata` in `metadata/v3.py`), which provide `to_dict()` and `from_dict()` methods.
+
+For `create_array`, the format is inferred from the `chunks` argument: a flat tuple produces `"regular"`, a nested list produces `"rectilinear"`. The `_is_rectilinear_chunks()` helper detects nested sequences like `[[10, 20], [5, 5]]`.
+
+##### Rectilinear spec compliance
+
+The rectilinear format requires `"kind": "inline"` (validated by `validate_rectilinear_kind()`). Per the spec, each element of `chunk_shapes` can be:
+
+- A bare integer `m`: repeated until `sum >= array_extent`
+- A list of bare integers: explicit per-chunk sizes
+- A mixed array of bare integers and `[value, count]` RLE pairs
+
+RLE compression is used when serializing: runs of identical sizes become `[value, count]` pairs, singletons stay as bare integers.
+
+```python
+# compress_rle([10, 10, 10, 5]) -> [[10, 3], 5]
+# expand_rle([[10, 3], 5]) -> [10, 10, 10, 5]
+```
+
+For a single-element `chunk_shapes` tuple like `(10,)`, `RectilinearChunkGridMetadata.to_dict()` serializes it as a bare integer `10`. Per the rectilinear spec, a bare integer is repeated until the sum >= extent, preserving the full codec buffer size for boundary chunks.
+
+**Zero-extent handling:** Regular grids serialize zero-extent dimensions without issue (the format encodes only `chunk_shape`, no edges). Rectilinear grids cannot represent zero-extent dimensions because the spec requires at least one positive-integer edge length per axis.
+
+#### read_chunk_sizes / write_chunk_sizes
+
+The `read_chunk_sizes` and `write_chunk_sizes` properties provide universal access to per-dimension chunk data sizes, matching the dask `Array.chunks` convention. They work for both regular and rectilinear grids:
+
+- `write_chunk_sizes`: always returns outer (storage) chunk sizes
+- `read_chunk_sizes`: returns inner chunk sizes when sharding is used, otherwise same as `write_chunk_sizes`
+
+```python
+>>> arr = zarr.create_array(store, shape=(100, 80), chunks=(30, 40))
+>>> arr.write_chunk_sizes
+((30, 30, 30, 10), (40, 40))
+
+>>> arr = zarr.create_array(store, shape=(60, 100), chunks=[[10, 20, 30], [50, 50]])
+>>> arr.write_chunk_sizes
+((10, 20, 30), (50, 50))
+```
+
+The underlying `ChunkGrid.chunk_sizes` property (on the grid, not the array) returns the same as `write_chunk_sizes`.
+
+#### Resize
+
+```python
+arr.resize((80, 100)) # re-binds extent; FixedDimension stays fixed
+arr.resize((200, 100)) # VaryingDimension grows by appending a new chunk
+arr.resize((30, 100)) # VaryingDimension shrinks: preserves all edges, re-binds extent
+```
+
+Resize uses `ChunkGrid.update_shape(new_shape)`, which delegates to each dimension's `.resize()` method:
+- `FixedDimension.resize()`: simply re-binds the extent (identical to `with_extent`)
+- `VaryingDimension.resize()`: grow past `sum(edges)` appends a chunk covering the gap; shrink or grow within `sum(edges)` preserves all edges and re-binds the extent (the spec allows trailing edges beyond the array extent)
+
+**Known limitation (deferred):** When growing a `VaryingDimension`, the current implementation always appends a single chunk covering the new region. For example, `[10, 10, 10]` resized from 30 to 45 produces `[10, 10, 10, 15]` instead of the more natural `[10, 10, 10, 10, 10]`. A future improvement should add an optional `chunks` parameter to `resize()` that controls how the new region is partitioned, with a sane default (e.g., repeating the last chunk size). This is safely deferrable because:
+- `FixedDimension` already handles resize correctly (regular grids stay regular)
+- The single-chunk default produces valid state, just suboptimal chunk layout
+- Rectilinear arrays are behind an experimental feature flag
+- Adding an optional parameter is backwards-compatible
+
+Open design questions for the `chunks` parameter:
+- Does it describe the new region only, or the entire post-resize array?
+- Must the overlapping portion agree with existing chunks (no rechunking)?
+- What is the type? Same as `chunks` in `create_array`?
+
+#### from_array
+
+The `from_array()` function handles both regular and rectilinear source arrays:
+
+```python
+src = zarr.create_array(store, shape=(60, 100), chunks=[[10, 20, 30], [50, 50]])
+new = zarr.from_array(data=src, store=new_store, chunks="keep")
+# Preserves rectilinear structure: new.write_chunk_sizes == ((10, 20, 30), (50, 50))
+```
+
+When `chunks="keep"`, the logic checks `data._chunk_grid.is_regular`:
+- Regular: extracts `data.chunks` (flat tuple) and preserves shards
+- Rectilinear: extracts `data.write_chunk_sizes` (nested tuples) and forces shards to None
+
+### Indexing
+
+The indexing pipeline is coupled to regular grid assumptions — every per-dimension indexer takes a scalar `dim_chunk_len: int` and uses `//` and `*`:
+
+```python
+dim_chunk_ix = self.dim_sel // self.dim_chunk_len # IntDimIndexer
+dim_offset = dim_chunk_ix * self.dim_chunk_len # SliceDimIndexer
+```
+
+Replace `dim_chunk_len: int` with the dimension object (`FixedDimension | VaryingDimension`). The shared interface means the indexer code structure stays the same — `dim_sel // dim_chunk_len` becomes `dim_grid.index_to_chunk(dim_sel)`. O(1) for regular, binary search for varying.
+
+### Codec pipeline
+
+Today, `get_chunk_spec()` returns the same `ArraySpec(shape=chunk_grid.chunk_shape)` for every chunk. For rectilinear grids, each chunk has a different codec shape:
+
+```python
+def get_chunk_spec(self, chunk_coords, array_config, prototype) -> ArraySpec:
+ spec = self._chunk_grid[chunk_coords]
+ return ArraySpec(shape=spec.codec_shape, ...)
+```
+
+Note `spec.codec_shape`, not `spec.shape`. For regular grids, `codec_shape` is uniform (preserving current behavior). The boundary clipping flow is unchanged:
+
+```
+Write: user data → pad to codec_shape with fill_value → encode → store
+Read: store → decode to codec_shape → slice via chunk_selection → user data
+```
+
+### Sharding
+
+The `ShardingCodec` constructs a `ChunkGrid` per shard using the shard shape as extent and the subchunk shape as `FixedDimension`. Each shard is self-contained — it doesn't need to know whether the outer grid is regular or rectilinear. Validation checks that every unique edge length per dimension is divisible by the inner chunk size, using `dim.unique_edge_lengths` for efficient polymorphic iteration (O(1) for fixed dimensions, lazy-deduplicated for varying).
+
+```
+Level 1 — Outer chunk grid (shard boundaries): regular or rectilinear
+Level 2 — Inner subchunk grid (within each shard): always regular
+Level 3 — Shard index: ceil(shard_dim / subchunk_dim) entries per dimension
+```
+
+[zarr-specs#370](https://github.com/zarr-developers/zarr-specs/pull/370) lifts the requirement that subchunk shapes evenly divide the shard shape. With the proposed `ChunkGrid`, this just means removing the `shard_shape % subchunk_shape == 0` validation — `FixedDimension` already handles boundary clipping via `data_size`.
+
+| Outer grid | Subchunk divisibility | Required change |
+|---|---|---|
+| Regular | Evenly divides (v1.0) | None |
+| Regular | Non-divisible (v1.1) | Remove divisibility validation |
+| Rectilinear | Evenly divides | Remove "sharding incompatible" guard |
+| Rectilinear | Non-divisible | Both changes |
+
+### What this replaces
+
+| Current | Proposed |
+|---|---|
+| `ChunkGrid` ABC + `RegularChunkGrid` subclass | Single concrete `ChunkGrid` with `is_regular` |
+| `RectilinearChunkGrid` (#3534) | Same `ChunkGrid` class |
+| Chunk grid registry + entrypoints (#3735) | Direct name dispatch |
+| `arr.chunks` | Retained for regular; `arr.read_chunk_sizes`/`arr.write_chunk_sizes` for general use |
+| `get_chunk_shape(shape, coord)` | `grid[coord].codec_shape` or `grid[coord].shape` |
+
+## Design decisions
+
+### Why store the extent in ChunkGrid?
+
+The chunk grid is a concrete arrangement, not an abstract tiling pattern. A finite collection naturally has an extent. Storing it enables `__getitem__`, eliminates `dim_len` parameters from every method, and makes the grid self-describing.
+
+This does *not* mean `ArrayV3Metadata.shape` should delegate to the grid. The array shape remains an independent field in metadata. The extent is passed into the grid at construction time so it can answer boundary questions without external parameters. It is **not** serialized as part of the chunk grid JSON — it comes from the `shape` field in array metadata and is combined with the chunk grid configuration in `ChunkGrid.from_metadata()`.
+
+### Why distinguish chunk_size from data_size?
+
+A chunk in a regular grid has two sizes. `chunk_size` is the buffer size the codec processes — always `size` for `FixedDimension`, even at the boundary (padded with `fill_value`). `data_size` is the valid data region — clipped to `extent % size` at the boundary. The indexing layer uses `data_size` to generate `chunk_selection` slices.
+
+This matches current zarr-python behavior and matters for:
+1. **Backward compatibility.** Existing stores have boundary chunks encoded at full `chunk_shape`.
+2. **Codec simplicity.** Codecs assume uniform input shapes for regular grids.
+3. **Shard index correctness.** The index assumes `subchunk_dim`-sized entries.
+
+For `VaryingDimension`, `chunk_size == data_size` when `extent == sum(edges)`. When `extent < sum(edges)` (e.g., after a resize that keeps the last chunk oversized), `data_size` clips the last chunk. This is the fundamental difference: `FixedDimension` has a declared size plus an extent that clips data; `VaryingDimension` has explicit sizes that normally *are* the extent but can also extend past it.
+
+### Why not a chunk grid registry?
+
+There is no known chunk grid outside the rectilinear family that retains the tessellation properties zarr-python assumes. A `match` on the grid name is sufficient.
+
+### Why a single ChunkGrid class instead of RegularChunkGrid + RectilinearChunkGrid?
+
+[Discussed in #3534.](https://github.com/zarr-developers/zarr-python/pull/3534) @d-v-b argued that `RegularChunkGrid` is unnecessary since rectilinear is more general; @dcherian argued that downstream libraries need a fast way to detect regular grids without inspecting potentially millions of chunk edges (see [xarray#9808](https://github.com/pydata/xarray/pull/9808)).
+
+The resolution: a single `ChunkGrid` class with an `is_regular` property (O(1), cached at construction). This gives downstream code the fast-path detection @dcherian needed without the class hierarchy complexity @d-v-b wanted to avoid. The metadata document's `name` field (`"regular"` vs `"rectilinear"`) is also available for clients who inspect JSON directly.
+
+A backwards-compatibility shim in `chunk_grids.py` preserves the old `RegularChunkGrid` / `RectilinearChunkGrid` import paths with deprecation warnings — see [Backwards compatibility](#backwards-compatibility).
+
+### Why is ChunkGrid a concrete class instead of a Protocol/ABC?
+
+The old design had `ChunkGrid` as an ABC with `RegularChunkGrid` as its only subclass. #3534 added `RectilinearChunkGrid` as a second subclass. This branch makes `ChunkGrid` a single concrete class instead, with separate metadata DTOs (`RegularChunkGridMetadata` and `RectilinearChunkGridMetadata` in `metadata/v3.py`) for serialization.
+
+All known grids are special cases of rectilinear, so there's no need for a class hierarchy at the grid level. A `ChunkGrid` Protocol/ABC would mean every caller programs against an abstract interface and adding a grid type requires implementing ~15 methods. A single class is simpler.
+
+Note: the *dimension* types (`FixedDimension`, `VaryingDimension`) do use a `DimensionGrid` Protocol — that's where the polymorphism lives. The grid-level class is concrete; the dimension-level types are polymorphic. If a genuinely novel grid type emerges that can't be expressed as a combination of per-dimension types, a grid-level Protocol can be extracted.
+
+### Why `.chunks` raises for rectilinear grids
+
+[Debated in #3534.](https://github.com/zarr-developers/zarr-python/pull/3534) @d-v-b suggested making `.chunks` return `tuple[tuple[int, ...], ...]` (dask-style) for all grids. @dcherian strongly objected: every downstream consumer expects `tuple[int, ...]`, and silently returning a different type would be worse than raising. Materializing O(10M) chunk edges into a Python tuple is also a real performance risk ([xarray#8902](https://github.com/pydata/xarray/issues/8902#issuecomment-2546127373)).
+
+The resolution:
+- `.chunks` is retained for regular grids (returns `tuple[int, ...]` as before)
+- `.chunks` raises `NotImplementedError` for rectilinear grids with a message pointing to `.read_chunk_sizes`/`.write_chunk_sizes`
+- `.read_chunk_sizes` and `.write_chunk_sizes` return `tuple[tuple[int, ...], ...]` (dask convention) for all grids
+
+@maxrjones noted in review that deprecating `.chunks` for regular grids was not desirable. The current branch does not deprecate it.
+
+### User control over grid serialization format
+
+@d-v-b raised in #3534 that users need a way to say "these chunks are regular, but serialize as rectilinear" (e.g., to allow future append/extend workflows without format changes). @jhamman initially made nested-list input always produce `RectilinearChunkGridMetadata`.
+
+The current branch resolves this via the metadata-layer chunk grid classes. When metadata is deserialized, the original name (from `{"name": "regular"}` or `{"name": "rectilinear"}`) determines which metadata class is instantiated (`RegularChunkGridMetadata` or `RectilinearChunkGridMetadata`), and that class handles serialization via `to_dict()`. Current inference behavior for `create_array`:
+- `chunks=(10, 20)` (flat tuple) → infers `"regular"`
+- `chunks=[[10, 20], [5, 5]]` (nested lists with varying sizes) → infers `"rectilinear"`
+- `chunks=[[10, 10], [20, 20]]` (nested lists with uniform sizes) → `from_sizes` collapses to `FixedDimension`, so `is_regular=True` and infers `"regular"`
+
+**Open question:** Should uniform nested lists preserve `"rectilinear"` to support future append workflows without a format change? This could be addressed by checking the input form before collapsing, or by allowing users to pass `chunk_grid_name` explicitly through the `create_array` API.
+
+### Deferred: Tiled/periodic chunk patterns
+
+[#3750 discussion](https://github.com/zarr-developers/zarr-python/issues/3750) identified periodic chunk patterns as a use case not efficiently served by RLE alone. RLE compresses runs of identical values (`np.repeat`), but periodic patterns like days-per-month (`[31, 28, 31, 30, ...]` repeated 30 years) need a tile encoding (`np.tile`). Real-world examples include:
+
+- **Oceanographic models** (ROMS): HPC boundary-padded chunks like `[10, 8, 8, 8, 10]` — handled by RLE
+- **Temporal axes**: days-per-month, hours-per-day — need tile encoding for compact metadata
+- **Temporal-aware grids**: date/time-aware chunk grids that layer over other axes (raised by @LDeakin)
+
+A `TiledDimension` prototype was built ([commit 9c0f582](https://github.com/maxrjones/zarr-python/commit/9c0f582f)) demonstrating that the per-dimension design supports this without changes to indexing or the codec pipeline. However, it was intentionally excluded from this release because:
+
+1. **Metadata format must come first.** Tile encoding requires a new `kind` value in the rectilinear spec (currently only `"inline"` is defined). This should go through [zarr-extensions#25](https://github.com/zarr-developers/zarr-extensions/pull/25), not zarr-python unilaterally.
+2. **The per-dimension architecture doesn't preclude it.** A future `TiledDimension` can implement the `DimensionGrid` protocol alongside `FixedDimension` and `VaryingDimension` with no changes to indexing, codecs, or the `ChunkGrid` class.
+3. **RLE covers the MVP.** Most real-world variable chunk patterns (HPC boundaries, irregular partitions) are efficiently encoded with RLE. Tile encoding is an optimization for a specific (temporal) subset.
+
+### Metadata / Array separation (partially implemented)
+
+An earlier design doc proposed decoupling `ChunkGrid` (runtime) from `ArrayV3Metadata` (serialization), so that metadata would store only a plain dict and the array layer would construct the `ChunkGrid`.
+
+The current implementation partially realizes this separation:
+
+- **Metadata DTOs** (`RegularChunkGridMetadata`, `RectilinearChunkGridMetadata` in `metadata/v3.py`): Pure data, frozen dataclasses, no array shape. These live on `ArrayV3Metadata.chunk_grid` and represent only what goes into `zarr.json`.
+- **`ChunkGrid`** (`chunk_grids.py`): Shape-bound, supports indexing, iteration, and chunk specs. Lives on `AsyncArray._chunk_grid`, constructed from metadata + `shape` via `ChunkGrid.from_metadata()`.
+
+This means `ArrayV3Metadata.chunk_grid` is now a `ChunkGridMetadata` (the DTO union type), **not** the runtime `ChunkGrid`. Code that previously accessed runtime methods on `metadata.chunk_grid` (e.g., `all_chunk_coords()`, `__getitem__`) must now use the grid from the array layer instead.
+
+The name controls serialization format; each metadata DTO class provides its own `to_dict()` method for serialization. The `ChunkGrid` handles all runtime queries.
+
+## Prior art
+
+**zarrs (Rust):** Three independent grid types behind a `ChunkGridTraits` trait. Key patterns adopted: Fixed vs Varying per dimension, prefix sums + binary search, `Option` for out-of-bounds, `NonZeroU64` for chunk dimensions, separate subchunk grid per shard, array shape at construction.
+
+**TensorStore (C++):** Stores only `chunk_shape` — boundary clipping via `valid_data_bounds` at query time. Both `RegularGridRef` and `IrregularGrid` internally. No registry.
+
+## Migration
+
+### Public API compatibility
+
+The user-facing API is fully backward-compatible. Existing code that creates, opens, reads, and writes zarr arrays continues to work without changes:
+
+- `zarr.create_array`, `zarr.open`, `zarr.open_array`, `zarr.open_group` -- unchanged signatures. The `chunks` parameter type is *widened* (now also accepts nested sequences for rectilinear grids), but all existing call patterns still work.
+- `arr.chunks` -- returns `tuple[int, ...]` for regular arrays, same as before.
+- `arr.shape`, `arr.dtype`, `arr.ndim`, `arr.shards` -- unchanged.
+- Top-level `zarr` exports -- unchanged.
+- Rectilinear chunks are gated behind `zarr.config.set({'array.rectilinear_chunks': True})`, so they cannot be created accidentally.
+
+New additions (purely additive): `arr.read_chunk_sizes`, `arr.write_chunk_sizes`, `zarr.experimental.ChunkGrid`, `zarr.experimental.ChunkSpec`.
+
+The breaking changes discussed below are confined to **internal modules** (`zarr.core.chunk_grids`, `zarr.core.metadata.v3`, `zarr.core.indexing`) that downstream libraries like cubed and VirtualiZarr access directly.
+
+### Internal API compatibility trade-off analysis
+
+This section analyzes the internal breaking changes from the metadata/array separation and evaluates two strategies: (A) add backward-compatibility shims in zarr-python, vs. (B) require downstream packages to update. The baseline is **no shims at all**.
+
+#### What breaks without any shims
+
+Three API changes affect downstream code:
+
+1. **`RegularChunkGrid` class removed from `zarr.core.chunk_grids`.** On `main`, `RegularChunkGrid` is defined in `chunk_grids.py` as a `Metadata` subclass. This branch replaces it with `RegularChunkGridMetadata` in `metadata/v3.py`. Without a shim, `from zarr.core.chunk_grids import RegularChunkGrid` raises `ImportError`.
+
+2. **`RegularChunkGrid` no longer available from `zarr.core.metadata.v3`.** On `main`, `v3.py` imports `RegularChunkGrid` from `chunk_grids.py` for internal use. VirtualiZarr imports it from this location (`from zarr.core.metadata.v3 import RegularChunkGrid`). Without the internal import, this raises `ImportError`.
+
+3. **`OrthogonalIndexer` constructor expects `ChunkGrid`, not `RegularChunkGrid`/`RegularChunkGridMetadata`.** Even if the import shims above resolve to `RegularChunkGridMetadata`, the indexer constructors access `chunk_grid._dimensions`, which only exists on the runtime `ChunkGrid` class. Cubed constructs `OrthogonalIndexer(selection, shape, RegularChunkGrid(chunk_shape=chunks))` directly.
+
+#### Downstream impact without shims
+
+**VirtualiZarr** (5 line changes across 2 files):
+
+```python
+# manifests/array.py (line 6): import
+- from zarr.core.metadata.v3 import ArrayV3Metadata, RegularChunkGrid
++ from zarr.core.metadata.v3 import ArrayV3Metadata, RegularChunkGridMetadata
+
+# manifests/array.py (line 53): isinstance check
+- if not isinstance(_metadata.chunk_grid, RegularChunkGrid):
++ if not isinstance(_metadata.chunk_grid, RegularChunkGridMetadata):
+
+# parsers/zarr.py (line 16): import
+- from zarr.core.chunk_grids import RegularChunkGrid
++ from zarr.core.metadata.v3 import RegularChunkGridMetadata
+
+# parsers/zarr.py (line 270): isinstance check
+- if not isinstance(array_v3_metadata.chunk_grid, RegularChunkGrid):
++ if not isinstance(array_v3_metadata.chunk_grid, RegularChunkGridMetadata):
+
+# parsers/zarr.py (line 390): cast
+- cast(RegularChunkGrid, metadata.chunk_grid).chunk_shape
++ cast(RegularChunkGridMetadata, metadata.chunk_grid).chunk_shape
+```
+
+The `manifests/array.py` import is from `zarr.core.metadata.v3` (never a documented export; VirtualiZarr relied on a transitive import). The `parsers/zarr.py` import is from `zarr.core.chunk_grids` (the canonical location on `main`). Both are straightforward renames. The `.chunk_shape` attribute is unchanged on the new class.
+
+If VirtualiZarr needs to support both old and new zarr-python, a version-conditional import adds ~5 more lines.
+
+**Cubed** (3 line changes in 1 file):
+
+```python
+# core/ops.py (lines 626-631)
+def _create_zarr_indexer(selection, shape, chunks):
+ if zarr.__version__[0] == "3":
+- from zarr.core.chunk_grids import RegularChunkGrid
++ from zarr.core.chunk_grids import ChunkGrid
+ from zarr.core.indexing import OrthogonalIndexer
+- return OrthogonalIndexer(selection, shape, RegularChunkGrid(chunk_shape=chunks))
++ return OrthogonalIndexer(selection, shape, ChunkGrid.from_sizes(shape, chunks))
+```
+
+Note that `ChunkGrid` is *not* a renamed class. `RegularChunkGrid(chunk_shape=chunks)` took only chunk sizes; `ChunkGrid.from_sizes(shape, chunks)` also requires the array shape. The `shape` parameter is already available at this call site.
+
+If cubed needs to support both old and new zarr-python:
+
+```python
+def _create_zarr_indexer(selection, shape, chunks):
+ if zarr.__version__[0] == "3":
+ from zarr.core.indexing import OrthogonalIndexer
+
+ try:
+ from zarr.core.chunk_grids import ChunkGrid
+
+ return OrthogonalIndexer(selection, shape, ChunkGrid.from_sizes(shape, chunks))
+ except ImportError:
+ from zarr.core.chunk_grids import RegularChunkGrid
+
+ return OrthogonalIndexer(selection, shape, RegularChunkGrid(chunk_shape=chunks))
+ else:
+ from zarr.indexing import OrthogonalIndexer
+
+ return OrthogonalIndexer(selection, ZarrArrayIndexingAdaptor(shape, chunks))
+```
+
+#### What shims can cover
+
+**Shim 1: `__getattr__` in `chunk_grids.py`** (~15 lines)
+
+Maps `RegularChunkGrid` to `RegularChunkGridMetadata` with a deprecation warning. Covers:
+- The `from zarr.core.chunk_grids import RegularChunkGrid` import pattern (used by cubed and VirtualiZarr's `parsers/zarr.py`)
+- `isinstance(x, RegularChunkGrid)` checks (because the name resolves to the actual class)
+- `RegularChunkGrid(chunk_shape=(...))` construction (because `RegularChunkGridMetadata` accepts the same arguments)
+
+Does **not** cover: passing the result to `OrthogonalIndexer`, because `RegularChunkGridMetadata` lacks `._dimensions`.
+
+**Shim 2: `__getattr__` in `metadata/v3.py`** (~12 lines)
+
+Same pattern, covers VirtualiZarr's import from `zarr.core.metadata.v3`. Mirrors Shim 1 for a different import path.
+
+**Shim 3: Auto-coerce `ChunkGridMetadata` in indexer constructors** (~30 lines)
+
+A helper function + 1-line insertion in each of `BasicIndexer`, `OrthogonalIndexer`, `CoordinateIndexer`, and `MaskIndexer`:
+
+```python
+def _resolve_chunk_grid(chunk_grid, shape):
+ """Coerce ChunkGridMetadata to runtime ChunkGrid if needed."""
+ from zarr.core.chunk_grids import ChunkGrid as _ChunkGrid
+ from zarr.core.metadata.v3 import ChunkGridMetadata
+
+ if isinstance(chunk_grid, _ChunkGrid):
+ return chunk_grid
+ if isinstance(chunk_grid, ChunkGridMetadata):
+ warnings.warn(
+ "Passing ChunkGridMetadata to indexers is deprecated. "
+ "Use ChunkGrid.from_sizes() instead.",
+ DeprecationWarning,
+ stacklevel=2,
+ )
+ if hasattr(chunk_grid, "chunk_shape"):
+ return _ChunkGrid.from_sizes(shape, tuple(chunk_grid.chunk_shape))
+ return _ChunkGrid.from_sizes(shape, chunk_grid.chunk_shapes)
+ raise TypeError(f"Expected ChunkGrid or ChunkGridMetadata, got {type(chunk_grid)}")
+```
+
+This covers cubed's `OrthogonalIndexer(selection, shape, RegularChunkGrid(...))` pattern end-to-end (combined with Shim 1).
+
+#### Comparison
+
+| | No shims | Shims 1+2 only | Shims 1+2+3 |
+|---|---|---|---|
+| **zarr-python additions** | 0 lines | ~27 lines | ~57 lines |
+| **VirtualiZarr changes** | 5 lines | 0 lines | 0 lines |
+| **Cubed changes** | 3 lines | 3 lines | 0 lines |
+| **Maintenance burden** | None | Low (deprecation shims are well-understood) | Medium (indexer coercion blurs metadata/runtime boundary) |
+| **API clarity** | Clean (metadata DTOs and runtime types are distinct) | Good (old names redirect to new names) | Weaker (indexers implicitly accept two type families) |
+
+With Shims 1+2 only, VirtualiZarr's `manifests/array.py` import from `zarr.core.metadata.v3` is covered by Shim 2, and the `parsers/zarr.py` import from `zarr.core.chunk_grids` is covered by Shim 1. The `isinstance` checks work because both shims resolve to `RegularChunkGridMetadata`. The `cast` works because `.chunk_shape` is unchanged. So VirtualiZarr needs 0 changes with Shims 1+2. The 3 lines for cubed remain because Shim 1 resolves the import but `OrthogonalIndexer` still needs a runtime `ChunkGrid`.
+
+### Downstream migration
+
+Migration from `main` (where only `RegularChunkGrid` and the abstract `ChunkGrid` ABC exist):
+
+| Old pattern (on `main`) | New pattern |
+|---|---|
+| `from zarr.core.chunk_grids import RegularChunkGrid` | `from zarr.core.metadata.v3 import RegularChunkGridMetadata` |
+| `from zarr.core.chunk_grids import ChunkGrid` (ABC) | `from zarr.core.chunk_grids import ChunkGrid` (concrete class, different API) |
+| `isinstance(cg, RegularChunkGrid)` | `isinstance(cg, RegularChunkGridMetadata)` or `grid.is_regular` on the runtime `ChunkGrid` |
+| `cg.chunk_shape` on `RegularChunkGrid` | `cg.chunk_shape` on `RegularChunkGridMetadata` (unchanged) |
+| `ChunkGrid.from_dict(data)` | `parse_chunk_grid(data)` from `zarr.core.metadata.v3` |
+| `chunk_grid.all_chunk_coords(array_shape)` | `chunk_grid.all_chunk_coords()` (shape now stored in grid) |
+| `chunk_grid.get_nchunks(array_shape)` | `chunk_grid.get_nchunks()` (shape now stored in grid) |
+
+During the earlier [#3534](https://github.com/zarr-developers/zarr-python/pull/3534) effort (which used separate `RegularChunkGrid`/`RectilinearChunkGrid` classes), downstream PRs and issues were opened to explore compatibility:
+
+- xarray ([#10880](https://github.com/pydata/xarray/pull/10880)), VirtualiZarr ([#877](https://github.com/zarr-developers/VirtualiZarr/pull/877)), Icechunk ([#1338](https://github.com/earth-mover/icechunk/issues/1338)), cubed ([#876](https://github.com/cubed-dev/cubed/issues/876))
+
+These target #3534's API, not this branch's unified `ChunkGrid` design. New downstream POC branches for this design are linked in [Proofs of concepts](#proofs-of-concepts).
+
+### Credits
+
+This implementation builds on prior work:
+
+- **[#3534](https://github.com/zarr-developers/zarr-python/pull/3534)** (@jhamman) — RLE helpers, validation logic, test cases, and the review discussion that shaped the architecture.
+- **[#3737](https://github.com/zarr-developers/zarr-python/pull/3737)** — extent-in-grid idea (adopted per-dimension).
+- **[#1483](https://github.com/zarr-developers/zarr-python/pull/1483)** — original variable chunking POC.
+- **[#3736](https://github.com/zarr-developers/zarr-python/pull/3736)** — resolved by storing extent per-dimension.
+
+
+## Open questions
+
+1. **Resize defaults (deferred):** When growing a rectilinear array, should `resize()` accept an optional `chunks` parameter? See the [Resize section](#resize) for details and open design questions. Regular arrays already stay regular on resize.
+2. **`ChunkSpec` complexity:** `ChunkSpec` carries both `slices` and `codec_shape`. Should the grid expose separate methods for codec vs data queries instead?
+3. **`__getitem__` with slices:** Should `grid[0, :]` or `grid[0:3, :]` return a sub-grid or an iterator of `ChunkSpec`s?
+4. **Uniform nested lists:** Should `chunks=[[10, 10], [20, 20]]` serialize as `"rectilinear"` (preserving user intent for future append) or `"regular"` (current behavior, collapses uniform edges)? See [User control over grid serialization format](#user-control-over-grid-serialization-format).
+5. **`zarr.open` with rectilinear:** @tomwhite noted in #3534 that `zarr.open(mode="w")` doesn't support rectilinear chunks directly. This could be addressed in a follow-up.
+
+## Proofs of concepts
+
+- Zarr-Python:
+ - branch - https://github.com/maxrjones/zarr-python/tree/poc/unified-chunk-grid
+ - diff - https://github.com/zarr-developers/zarr-python/compare/main...maxrjones:zarr-python:poc/unified-chunk-grid?expand=1
+- Xarray:
+ - branch - https://github.com/maxrjones/xarray/tree/poc/unified-zarr-chunk-grid
+ - diff - https://github.com/pydata/xarray/compare/main...maxrjones:xarray:poc/unified-zarr-chunk-grid?expand=1
+- VirtualiZarr:
+ - branch - https://github.com/maxrjones/VirtualiZarr/tree/poc/unified-chunk-grid
+ - diff - https://github.com/zarr-developers/VirtualiZarr/compare/main...maxrjones:VirtualiZarr:poc/unified-chunk-grid?expand=1
+- Virtual TIFF:
+ - branch - https://github.com/virtual-zarr/virtual-tiff/tree/poc/unified-chunk-grid
+ - diff - https://github.com/virtual-zarr/virtual-tiff/compare/main...poc/unified-chunk-grid?expand=1
+- Cubed:
+ - branch - https://github.com/maxrjones/cubed/tree/poc/unified-chunk-grid
+- Microbenchmarks:
+ - https://github.com/maxrjones/zarr-chunk-grid-tests/tree/unified-chunk-grid
diff --git a/docs/Makefile b/docs/Makefile
deleted file mode 100644
index fc8fa12915..0000000000
--- a/docs/Makefile
+++ /dev/null
@@ -1,231 +0,0 @@
-# Makefile for Sphinx documentation
-#
-
-# You can set these variables from the command line.
-SPHINXOPTS = -W --keep-going
-SPHINXBUILD = sphinx-build
-PAPER =
-BUILDDIR = _build
-
-# User-friendly check for sphinx-build
-ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
- $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don\'t have Sphinx installed, grab it from https://www.sphinx-doc.org/)
-endif
-
-# Internal variables.
-PAPEROPT_a4 = -D latex_paper_size=a4
-PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
-# the i18n builder cannot share the environment and doctrees with the others
-I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
-
-.PHONY: help
-help:
- @echo "Please use \`make ' where is one of"
- @echo " html to make standalone HTML files"
- @echo " dirhtml to make HTML files named index.html in directories"
- @echo " singlehtml to make a single large HTML file"
- @echo " pickle to make pickle files"
- @echo " json to make JSON files"
- @echo " htmlhelp to make HTML files and a HTML help project"
- @echo " qthelp to make HTML files and a qthelp project"
- @echo " applehelp to make an Apple Help Book"
- @echo " devhelp to make HTML files and a Devhelp project"
- @echo " epub to make an epub"
- @echo " epub3 to make an epub3"
- @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
- @echo " latexpdf to make LaTeX files and run them through pdflatex"
- @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
- @echo " text to make text files"
- @echo " man to make manual pages"
- @echo " texinfo to make Texinfo files"
- @echo " info to make Texinfo files and run them through makeinfo"
- @echo " gettext to make PO message catalogs"
- @echo " changes to make an overview of all changed/added/deprecated items"
- @echo " xml to make Docutils-native XML files"
- @echo " pseudoxml to make pseudoxml-XML files for display purposes"
- @echo " linkcheck to check all external links for integrity"
- @echo " doctest to run all doctests embedded in the documentation (if enabled)"
- @echo " coverage to run coverage check of the documentation (if enabled)"
- @echo " dummy to check syntax errors of document sources"
-
-.PHONY: clean
-clean:
- rm -rf $(BUILDDIR)/*
- rm -rf $(BUILDDIR)/../_autoapi
-
-.PHONY: html
-html:
- $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
- @echo
- @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
-
-.PHONY: dirhtml
-dirhtml:
- $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
- @echo
- @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
-
-.PHONY: singlehtml
-singlehtml:
- $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
- @echo
- @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
-
-.PHONY: pickle
-pickle:
- $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
- @echo
- @echo "Build finished; now you can process the pickle files."
-
-.PHONY: json
-json:
- $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
- @echo
- @echo "Build finished; now you can process the JSON files."
-
-.PHONY: htmlhelp
-htmlhelp:
- $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
- @echo
- @echo "Build finished; now you can run HTML Help Workshop with the" \
- ".hhp project file in $(BUILDDIR)/htmlhelp."
-
-.PHONY: qthelp
-qthelp:
- $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
- @echo
- @echo "Build finished; now you can run "qcollectiongenerator" with the" \
- ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
- @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/zarr.qhcp"
- @echo "To view the help file:"
- @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/zarr.qhc"
-
-.PHONY: applehelp
-applehelp:
- $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
- @echo
- @echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
- @echo "N.B. You won't be able to view it unless you put it in" \
- "~/Library/Documentation/Help or install it in your application" \
- "bundle."
-
-.PHONY: devhelp
-devhelp:
- $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
- @echo
- @echo "Build finished."
- @echo "To view the help file:"
- @echo "# mkdir -p $$HOME/.local/share/devhelp/zarr"
- @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/zarr"
- @echo "# devhelp"
-
-.PHONY: epub
-epub:
- $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
- @echo
- @echo "Build finished. The epub file is in $(BUILDDIR)/epub."
-
-.PHONY: epub3
-epub3:
- $(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3
- @echo
- @echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3."
-
-.PHONY: latex
-latex:
- $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
- @echo
- @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
- @echo "Run \`make' in that directory to run these through (pdf)latex" \
- "(use \`make latexpdf' here to do that automatically)."
-
-.PHONY: latexpdf
-latexpdf:
- $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
- @echo "Running LaTeX files through pdflatex..."
- $(MAKE) -C $(BUILDDIR)/latex all-pdf
- @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
-
-.PHONY: latexpdfja
-latexpdfja:
- $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
- @echo "Running LaTeX files through platex and dvipdfmx..."
- $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
- @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
-
-.PHONY: text
-text:
- $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
- @echo
- @echo "Build finished. The text files are in $(BUILDDIR)/text."
-
-.PHONY: man
-man:
- $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
- @echo
- @echo "Build finished. The manual pages are in $(BUILDDIR)/man."
-
-.PHONY: texinfo
-texinfo:
- $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
- @echo
- @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
- @echo "Run \`make' in that directory to run these through makeinfo" \
- "(use \`make info' here to do that automatically)."
-
-.PHONY: info
-info:
- $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
- @echo "Running Texinfo files through makeinfo..."
- make -C $(BUILDDIR)/texinfo info
- @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
-
-.PHONY: gettext
-gettext:
- $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
- @echo
- @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
-
-.PHONY: changes
-changes:
- $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
- @echo
- @echo "The overview file is in $(BUILDDIR)/changes."
-
-.PHONY: linkcheck
-linkcheck:
- $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
- @echo
- @echo "Link check complete; look for any errors in the above output " \
- "or in $(BUILDDIR)/linkcheck/output.txt."
-
-.PHONY: doctest
-doctest:
- $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
- @echo "Testing of doctests in the sources finished, look at the " \
- "results in $(BUILDDIR)/doctest/output.txt."
-
-.PHONY: coverage
-coverage:
- $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
- @echo "Testing of coverage in the sources finished, look at the " \
- "results in $(BUILDDIR)/coverage/python.txt."
-
-.PHONY: xml
-xml:
- $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
- @echo
- @echo "Build finished. The XML files are in $(BUILDDIR)/xml."
-
-.PHONY: pseudoxml
-pseudoxml:
- $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
- @echo
- @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
-
-.PHONY: dummy
-dummy:
- $(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy
- @echo
- @echo "Build finished. Dummy builder generates no files."
diff --git a/docs/_static/custom.css b/docs/_static/custom.css
deleted file mode 100644
index 1d32606f9a..0000000000
--- a/docs/_static/custom.css
+++ /dev/null
@@ -1,110 +0,0 @@
-@import url('https://fonts.googleapis.com/css2?family=Lato:ital,wght@0,400;0,700;0,900;1,400;1,700;1,900&family=Open+Sans:ital,wght@0,400;0,600;1,400;1,600&display=swap');
-
-body {
- font-family: 'Open Sans', sans-serif;
-}
-
-pre, code {
- font-size: 100%;
- line-height: 155%;
-}
-
-/* Style the active version button.
-
-- dev: orange
-- stable: green
-- old, PR: red
-
-Colors from:
-
-Wong, B. Points of view: Color blindness.
-Nat Methods 8, 441 (2011). https://doi.org/10.1038/nmeth.1618
-*/
-
-/* If the active version has the name "dev", style it orange */
-#version_switcher_button[data-active-version-name*="dev"] {
- background-color: #E69F00;
- border-color: #E69F00;
- color:#000000;
-}
-
-/* green for `stable` */
-#version_switcher_button[data-active-version-name*="stable"] {
- background-color: #009E73;
- border-color: #009E73;
-}
-
-/* red for `old` */
-#version_switcher_button:not([data-active-version-name*="stable"], [data-active-version-name*="dev"], [data-active-version-name=""]) {
- background-color: #980F0F;
- border-color: #980F0F;
-}
-
-/* Main page overview cards */
-
-.sd-card {
- background: #fff;
- border-radius: 0;
- padding: 30px 10px 20px 10px;
- margin: 10px 0px;
-}
-
-.sd-card .sd-card-header {
- text-align: center;
-}
-
-.sd-card .sd-card-header .sd-card-text {
- margin: 0px;
-}
-
-.sd-card .sd-card-img-top {
- height: 52px;
- width: 52px;
- margin-left: auto;
- margin-right: auto;
-}
-
-.sd-card .sd-card-header {
- border: none;
- background-color: white;
- font-size: var(--pst-font-size-h5);
- font-weight: bold;
- padding: 2.5rem 0rem 0.5rem 0rem;
-}
-
-.sd-card .sd-card-footer {
- border: none;
- background-color: white;
-}
-
-.sd-card .sd-card-footer .sd-card-text {
- max-width: 220px;
- margin-left: auto;
- margin-right: auto;
-}
-
-/* Dark theme tweaking */
-html[data-theme=dark] .sd-card img[src*='.svg'] {
- filter: invert(0.82) brightness(0.8) contrast(1.2);
-}
-
-/* Main index page overview cards */
-html[data-theme=dark] .sd-card {
- background-color:var(--pst-color-background);
-}
-
-html[data-theme=dark] .sd-shadow-sm {
- box-shadow: 0 .1rem 1rem rgba(250, 250, 250, .6) !important
-}
-
-html[data-theme=dark] .sd-card .sd-card-header {
- background-color:var(--pst-color-background);
-}
-
-html[data-theme=dark] .sd-card .sd-card-footer {
- background-color:var(--pst-color-background);
-}
-
-html[data-theme=dark] h1 {
- color: var(--pst-color-primary);
-}
diff --git a/docs/_static/custom.js b/docs/_static/custom.js
deleted file mode 100644
index 52f1cba9e0..0000000000
--- a/docs/_static/custom.js
+++ /dev/null
@@ -1,17 +0,0 @@
-// handle redirects
-(() => {
- let anchorMap = {
- "installation": "installation.html",
- "getting-started": "getting_started.html#getting-started",
- "highlights": "getting_started.html#highlights",
- "contributing": "contributing.html",
- "projects-using-zarr": "getting_started.html#projects-using-zarr",
- "contents": "getting_started.html#contents",
- "indices-and-tables": "api.html#indices-and-tables"
- }
-
- let hash = window.location.hash.substring(1);
- if (hash && hash in anchorMap) {
- window.location.replace(anchorMap[hash]);
- }
-})();
diff --git a/docs/_static/favicon-96x96.png b/docs/_static/favicon-96x96.png
new file mode 100644
index 0000000000..e77977ccf4
Binary files /dev/null and b/docs/_static/favicon-96x96.png differ
diff --git a/docs/_static/index_api.svg b/docs/_static/index_api.svg
deleted file mode 100644
index 69f7ba1d2d..0000000000
--- a/docs/_static/index_api.svg
+++ /dev/null
@@ -1,97 +0,0 @@
-
-
-
-
diff --git a/docs/_static/index_contribute.svg b/docs/_static/index_contribute.svg
deleted file mode 100644
index de3d902379..0000000000
--- a/docs/_static/index_contribute.svg
+++ /dev/null
@@ -1,76 +0,0 @@
-
-
-
-
diff --git a/docs/_static/index_getting_started.svg b/docs/_static/index_getting_started.svg
deleted file mode 100644
index 2d36622cb7..0000000000
--- a/docs/_static/index_getting_started.svg
+++ /dev/null
@@ -1,66 +0,0 @@
-
-
-
-
diff --git a/docs/_static/index_user_guide.svg b/docs/_static/index_user_guide.svg
deleted file mode 100644
index bd17053517..0000000000
--- a/docs/_static/index_user_guide.svg
+++ /dev/null
@@ -1,67 +0,0 @@
-
-
-
-
diff --git a/docs/_static/logo_bw.png b/docs/_static/logo_bw.png
new file mode 100644
index 0000000000..df1979d3cc
Binary files /dev/null and b/docs/_static/logo_bw.png differ
diff --git a/docs/about.rst b/docs/about.rst
deleted file mode 100644
index 7a0af998c0..0000000000
--- a/docs/about.rst
+++ /dev/null
@@ -1,24 +0,0 @@
-About
-=====
-
-Zarr is a format for the storage of chunked, compressed, N-dimensional arrays
-inspired by `HDF5 `_, `h5py
-`_ and `bcolz `_.
-
-These documents describe the Zarr-Python implementation. More information
-about the Zarr format can be found on the `main website `_.
-
-Projects using Zarr
--------------------
-
-If you are using Zarr-Python, we would `love to hear about it
-`_.
-
-Funding
--------
-The project is fiscally sponsored by `NumFOCUS `_, a US
-501(c)(3) public charity, and development is supported by the
-`MRC Centre for Genomics and Global Health `_
-and the `Chan Zuckerberg Initiative `_.
-
-.. _NumCodecs: https://numcodecs.readthedocs.io/
diff --git a/docs/api/index.rst b/docs/api/index.rst
deleted file mode 100644
index 26d7ce0224..0000000000
--- a/docs/api/index.rst
+++ /dev/null
@@ -1,7 +0,0 @@
-API reference
-=============
-
-.. toctree::
- :maxdepth: 1
-
- ../_autoapi/zarr/index
diff --git a/docs/api/zarr/abc/buffer.md b/docs/api/zarr/abc/buffer.md
new file mode 100644
index 0000000000..d1ace2c899
--- /dev/null
+++ b/docs/api/zarr/abc/buffer.md
@@ -0,0 +1,12 @@
+---
+title: buffer
+---
+
+::: zarr.abc
+ options:
+ show_root_heading: true
+ show_root_toc_entry: true
+ members: false
+
+
+::: zarr.abc.buffer
diff --git a/docs/api/zarr/abc/codec.md b/docs/api/zarr/abc/codec.md
new file mode 100644
index 0000000000..d4eaecabe9
--- /dev/null
+++ b/docs/api/zarr/abc/codec.md
@@ -0,0 +1,5 @@
+---
+title: codec
+---
+
+::: zarr.abc.codec
diff --git a/docs/api/zarr/abc/index.md b/docs/api/zarr/abc/index.md
new file mode 100644
index 0000000000..7e15cb2a51
--- /dev/null
+++ b/docs/api/zarr/abc/index.md
@@ -0,0 +1,13 @@
+---
+title: zarr.abc
+---
+
+# zarr.abc
+
+Abstract base classes for extending Zarr-Python.
+
+- **[zarr.abc.buffer](./buffer.md)** - Providing access to underlying memory via [buffers](https://docs.python.org/3/c-api/buffer.html)
+- **[zarr.abc.codec](./codec.md)** - Expressing [zarr codecs](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#chunk-encoding)
+- **[zarr.abc.metadata](./metadata.md)** - Creating metadata classes compatible with the Zarr API
+- **[zarr.abc.numcodec](./numcodec.md)** - Protocols and classes for modeling codec interface used by numcodecs
+- **[zarr.abc.store](./store.md)** - ABC for implementing Zarr stores and managing getting and setting bytes in a store
diff --git a/docs/api/zarr/abc/metadata.md b/docs/api/zarr/abc/metadata.md
new file mode 100644
index 0000000000..7cc1e00662
--- /dev/null
+++ b/docs/api/zarr/abc/metadata.md
@@ -0,0 +1,5 @@
+---
+title: metadata
+---
+
+::: zarr.abc.metadata
diff --git a/docs/api/zarr/abc/numcodec.md b/docs/api/zarr/abc/numcodec.md
new file mode 100644
index 0000000000..ffbca600cc
--- /dev/null
+++ b/docs/api/zarr/abc/numcodec.md
@@ -0,0 +1,5 @@
+---
+title: numcodec
+---
+
+::: zarr.abc.numcodec
diff --git a/docs/api/zarr/abc/store.md b/docs/api/zarr/abc/store.md
new file mode 100644
index 0000000000..f711448541
--- /dev/null
+++ b/docs/api/zarr/abc/store.md
@@ -0,0 +1,5 @@
+---
+title: store
+---
+
+::: zarr.abc.store
diff --git a/docs/api/zarr/api/asynchronous.md b/docs/api/zarr/api/asynchronous.md
new file mode 100644
index 0000000000..f5df894134
--- /dev/null
+++ b/docs/api/zarr/api/asynchronous.md
@@ -0,0 +1,5 @@
+---
+title: asynchronous
+---
+
+::: zarr.api.asynchronous
\ No newline at end of file
diff --git a/docs/api/zarr/api/index.md b/docs/api/zarr/api/index.md
new file mode 100644
index 0000000000..7fac5e766a
--- /dev/null
+++ b/docs/api/zarr/api/index.md
@@ -0,0 +1,7 @@
+---
+title: zarr.api
+---
+
+# zarr.api
+
+Zarr provides both an [async](./asynchronous.md) and a [sync](./synchronous.md) API. See those pages for more details.
diff --git a/docs/api/zarr/api/synchronous.md b/docs/api/zarr/api/synchronous.md
new file mode 100644
index 0000000000..63a4aec537
--- /dev/null
+++ b/docs/api/zarr/api/synchronous.md
@@ -0,0 +1,11 @@
+---
+title: synchronous
+---
+
+::: zarr.api
+ options:
+ show_root_heading: true
+ show_root_toc_entry: true
+ members: false
+
+::: zarr.api.synchronous
\ No newline at end of file
diff --git a/docs/api/zarr/array.md b/docs/api/zarr/array.md
new file mode 100644
index 0000000000..ff61cb1fe2
--- /dev/null
+++ b/docs/api/zarr/array.md
@@ -0,0 +1,2 @@
+::: zarr.Array
+::: zarr.AsyncArray
diff --git a/docs/api/zarr/buffer/cpu.md b/docs/api/zarr/buffer/cpu.md
new file mode 100644
index 0000000000..9d4726d3ea
--- /dev/null
+++ b/docs/api/zarr/buffer/cpu.md
@@ -0,0 +1 @@
+::: zarr.buffer.cpu
diff --git a/docs/api/zarr/buffer/gpu.md b/docs/api/zarr/buffer/gpu.md
new file mode 100644
index 0000000000..e2276d8d82
--- /dev/null
+++ b/docs/api/zarr/buffer/gpu.md
@@ -0,0 +1 @@
+::: zarr.buffer.gpu
diff --git a/docs/api/zarr/buffer/index.md b/docs/api/zarr/buffer/index.md
new file mode 100644
index 0000000000..ebbb9e1c99
--- /dev/null
+++ b/docs/api/zarr/buffer/index.md
@@ -0,0 +1,7 @@
+---
+title: zarr.buffer
+---
+
+Zarr provides buffer classes for both the [cpu](./cpu.md) and [gpu](./gpu.md). Generic buffer functionality is also detailed below.
+
+::: zarr.buffer
diff --git a/docs/api/zarr/codecs.md b/docs/api/zarr/codecs.md
new file mode 100644
index 0000000000..c5a0f046ed
--- /dev/null
+++ b/docs/api/zarr/codecs.md
@@ -0,0 +1,5 @@
+---
+title: codecs
+---
+
+::: zarr.codecs
diff --git a/docs/api/zarr/codecs/numcodecs.md b/docs/api/zarr/codecs/numcodecs.md
new file mode 100644
index 0000000000..ce2a7de145
--- /dev/null
+++ b/docs/api/zarr/codecs/numcodecs.md
@@ -0,0 +1,5 @@
+---
+title: numcodecs
+---
+
+::: zarr.codecs.numcodecs
diff --git a/docs/api/zarr/config.md b/docs/api/zarr/config.md
new file mode 100644
index 0000000000..30803918f5
--- /dev/null
+++ b/docs/api/zarr/config.md
@@ -0,0 +1,5 @@
+---
+title: config
+---
+
+::: zarr.config
diff --git a/docs/api/zarr/dtype.md b/docs/api/zarr/dtype.md
new file mode 100644
index 0000000000..c08910b97f
--- /dev/null
+++ b/docs/api/zarr/dtype.md
@@ -0,0 +1,5 @@
+---
+title: dtype
+---
+
+::: zarr.dtype
diff --git a/docs/api/zarr/errors.md b/docs/api/zarr/errors.md
new file mode 100644
index 0000000000..2ba2213071
--- /dev/null
+++ b/docs/api/zarr/errors.md
@@ -0,0 +1,5 @@
+---
+title: errors
+---
+
+::: zarr.errors
\ No newline at end of file
diff --git a/docs/api/zarr/experimental.md b/docs/api/zarr/experimental.md
new file mode 100644
index 0000000000..60f1f987b5
--- /dev/null
+++ b/docs/api/zarr/experimental.md
@@ -0,0 +1,9 @@
+---
+title: experimental
+---
+
+Experimental functionality is not stable and may change or be removed at any point.
+
+## Classes
+
+::: zarr.experimental.cache_store
diff --git a/docs/api/zarr/functions/array.md b/docs/api/zarr/functions/array.md
new file mode 100644
index 0000000000..ff7242005c
--- /dev/null
+++ b/docs/api/zarr/functions/array.md
@@ -0,0 +1,5 @@
+---
+title: zarr.array
+---
+
+::: zarr.array
diff --git a/docs/api/zarr/functions/consolidate_metadata.md b/docs/api/zarr/functions/consolidate_metadata.md
new file mode 100644
index 0000000000..946531f028
--- /dev/null
+++ b/docs/api/zarr/functions/consolidate_metadata.md
@@ -0,0 +1,5 @@
+---
+title: zarr.consolidate_metadata
+---
+
+::: zarr.consolidate_metadata
diff --git a/docs/api/zarr/functions/create.md b/docs/api/zarr/functions/create.md
new file mode 100644
index 0000000000..b43094eaba
--- /dev/null
+++ b/docs/api/zarr/functions/create.md
@@ -0,0 +1,5 @@
+---
+title: zarr.create
+---
+
+::: zarr.create
diff --git a/docs/api/zarr/functions/create_array.md b/docs/api/zarr/functions/create_array.md
new file mode 100644
index 0000000000..a9f4a24bd0
--- /dev/null
+++ b/docs/api/zarr/functions/create_array.md
@@ -0,0 +1,5 @@
+---
+title: zarr.create_array
+---
+
+::: zarr.create_array
diff --git a/docs/api/zarr/functions/create_group.md b/docs/api/zarr/functions/create_group.md
new file mode 100644
index 0000000000..50beb0674c
--- /dev/null
+++ b/docs/api/zarr/functions/create_group.md
@@ -0,0 +1,5 @@
+---
+title: zarr.create_group
+---
+
+::: zarr.create_group
diff --git a/docs/api/zarr/functions/create_hierarchy.md b/docs/api/zarr/functions/create_hierarchy.md
new file mode 100644
index 0000000000..38938ffee2
--- /dev/null
+++ b/docs/api/zarr/functions/create_hierarchy.md
@@ -0,0 +1,5 @@
+---
+title: zarr.create_hierarchy
+---
+
+::: zarr.create_hierarchy
diff --git a/docs/api/zarr/functions/empty.md b/docs/api/zarr/functions/empty.md
new file mode 100644
index 0000000000..aff67bb9ed
--- /dev/null
+++ b/docs/api/zarr/functions/empty.md
@@ -0,0 +1,5 @@
+---
+title: zarr.empty
+---
+
+::: zarr.empty
diff --git a/docs/api/zarr/functions/empty_like.md b/docs/api/zarr/functions/empty_like.md
new file mode 100644
index 0000000000..9e2fbd26a5
--- /dev/null
+++ b/docs/api/zarr/functions/empty_like.md
@@ -0,0 +1,5 @@
+---
+title: zarr.empty_like
+---
+
+::: zarr.empty_like
diff --git a/docs/api/zarr/functions/from_array.md b/docs/api/zarr/functions/from_array.md
new file mode 100644
index 0000000000..7ab8179b05
--- /dev/null
+++ b/docs/api/zarr/functions/from_array.md
@@ -0,0 +1,5 @@
+---
+title: zarr.from_array
+---
+
+::: zarr.from_array
diff --git a/docs/api/zarr/functions/full.md b/docs/api/zarr/functions/full.md
new file mode 100644
index 0000000000..d6c60de2d5
--- /dev/null
+++ b/docs/api/zarr/functions/full.md
@@ -0,0 +1,5 @@
+---
+title: zarr.full
+---
+
+::: zarr.full
diff --git a/docs/api/zarr/functions/full_like.md b/docs/api/zarr/functions/full_like.md
new file mode 100644
index 0000000000..eb5c162f76
--- /dev/null
+++ b/docs/api/zarr/functions/full_like.md
@@ -0,0 +1,5 @@
+---
+title: zarr.full_like
+---
+
+::: zarr.full_like
diff --git a/docs/api/zarr/functions/group.md b/docs/api/zarr/functions/group.md
new file mode 100644
index 0000000000..3048218f6b
--- /dev/null
+++ b/docs/api/zarr/functions/group.md
@@ -0,0 +1,5 @@
+---
+title: zarr.group
+---
+
+::: zarr.group
diff --git a/docs/api/zarr/functions/load.md b/docs/api/zarr/functions/load.md
new file mode 100644
index 0000000000..aa004076ab
--- /dev/null
+++ b/docs/api/zarr/functions/load.md
@@ -0,0 +1,5 @@
+---
+title: zarr.load
+---
+
+::: zarr.load
diff --git a/docs/api/zarr/functions/ones.md b/docs/api/zarr/functions/ones.md
new file mode 100644
index 0000000000..b7757da1bc
--- /dev/null
+++ b/docs/api/zarr/functions/ones.md
@@ -0,0 +1,5 @@
+---
+title: zarr.ones
+---
+
+::: zarr.ones
diff --git a/docs/api/zarr/functions/ones_like.md b/docs/api/zarr/functions/ones_like.md
new file mode 100644
index 0000000000..cffccb10ef
--- /dev/null
+++ b/docs/api/zarr/functions/ones_like.md
@@ -0,0 +1,5 @@
+---
+title: zarr.ones_like
+---
+
+::: zarr.ones_like
diff --git a/docs/api/zarr/functions/open.md b/docs/api/zarr/functions/open.md
new file mode 100644
index 0000000000..3d75977395
--- /dev/null
+++ b/docs/api/zarr/functions/open.md
@@ -0,0 +1,5 @@
+---
+title: zarr.open
+---
+
+::: zarr.open
diff --git a/docs/api/zarr/functions/open_array.md b/docs/api/zarr/functions/open_array.md
new file mode 100644
index 0000000000..f40da1bd5d
--- /dev/null
+++ b/docs/api/zarr/functions/open_array.md
@@ -0,0 +1,5 @@
+---
+title: zarr.open_array
+---
+
+::: zarr.open_array
diff --git a/docs/api/zarr/functions/open_consolidated.md b/docs/api/zarr/functions/open_consolidated.md
new file mode 100644
index 0000000000..de71cf7662
--- /dev/null
+++ b/docs/api/zarr/functions/open_consolidated.md
@@ -0,0 +1,5 @@
+---
+title: zarr.open_consolidated
+---
+
+::: zarr.open_consolidated
diff --git a/docs/api/zarr/functions/open_group.md b/docs/api/zarr/functions/open_group.md
new file mode 100644
index 0000000000..4944e94e06
--- /dev/null
+++ b/docs/api/zarr/functions/open_group.md
@@ -0,0 +1,5 @@
+---
+title: zarr.open_group
+---
+
+::: zarr.open_group
diff --git a/docs/api/zarr/functions/open_like.md b/docs/api/zarr/functions/open_like.md
new file mode 100644
index 0000000000..1aea075a81
--- /dev/null
+++ b/docs/api/zarr/functions/open_like.md
@@ -0,0 +1,5 @@
+---
+title: zarr.open_like
+---
+
+::: zarr.open_like
diff --git a/docs/api/zarr/functions/print_debug_info.md b/docs/api/zarr/functions/print_debug_info.md
new file mode 100644
index 0000000000..c98329f893
--- /dev/null
+++ b/docs/api/zarr/functions/print_debug_info.md
@@ -0,0 +1,5 @@
+---
+title: zarr.print_debug_info
+---
+
+::: zarr.print_debug_info
diff --git a/docs/api/zarr/functions/save.md b/docs/api/zarr/functions/save.md
new file mode 100644
index 0000000000..6c8eae410f
--- /dev/null
+++ b/docs/api/zarr/functions/save.md
@@ -0,0 +1,5 @@
+---
+title: zarr.save
+---
+
+::: zarr.save
diff --git a/docs/api/zarr/functions/save_array.md b/docs/api/zarr/functions/save_array.md
new file mode 100644
index 0000000000..58a6d5143d
--- /dev/null
+++ b/docs/api/zarr/functions/save_array.md
@@ -0,0 +1,5 @@
+---
+title: zarr.save_array
+---
+
+::: zarr.save_array
diff --git a/docs/api/zarr/functions/save_group.md b/docs/api/zarr/functions/save_group.md
new file mode 100644
index 0000000000..ba66a70563
--- /dev/null
+++ b/docs/api/zarr/functions/save_group.md
@@ -0,0 +1,5 @@
+---
+title: zarr.save_group
+---
+
+::: zarr.save_group
diff --git a/docs/api/zarr/functions/zeros.md b/docs/api/zarr/functions/zeros.md
new file mode 100644
index 0000000000..d43e8d913b
--- /dev/null
+++ b/docs/api/zarr/functions/zeros.md
@@ -0,0 +1,5 @@
+---
+title: zarr.zeros
+---
+
+::: zarr.zeros
diff --git a/docs/api/zarr/functions/zeros_like.md b/docs/api/zarr/functions/zeros_like.md
new file mode 100644
index 0000000000..5adf1a23b3
--- /dev/null
+++ b/docs/api/zarr/functions/zeros_like.md
@@ -0,0 +1,5 @@
+---
+title: zarr.zeros_like
+---
+
+::: zarr.zeros_like
diff --git a/docs/api/zarr/group.md b/docs/api/zarr/group.md
new file mode 100644
index 0000000000..0cf9372de2
--- /dev/null
+++ b/docs/api/zarr/group.md
@@ -0,0 +1,2 @@
+::: zarr.Group
+::: zarr.AsyncGroup
diff --git a/docs/api/zarr/index.md b/docs/api/zarr/index.md
new file mode 100644
index 0000000000..f691c4599a
--- /dev/null
+++ b/docs/api/zarr/index.md
@@ -0,0 +1,62 @@
+# API Reference
+
+Complete reference documentation for the Zarr-Python API.
+
+::: zarr
+ options:
+ show_root_heading: true
+ show_root_toc_entry: true
+ members: false
+
+## Core API
+
+### Essential Classes and Functions
+
+- **[Array](array.md)** - The main Zarr array class for N-dimensional data
+- **[Group](group.md)** - Hierarchical organization of arrays and subgroups
+- **[create_array](functions/create_array.md)** and **[create_group](functions/create_group.md)** - Creating new arrays and groups
+- **[open](functions/open.md)**, **[open_array](functions/open_array.md)**, and **[open_group](functions/open_group.md)** - Opening existing Zarr stores, arrays, and groups
+
+### Data Operations
+
+- **[load](functions/load.md)** - Loading data from Zarr stores
+- **[save](functions/save.md)** - Saving data to Zarr format
+
+### Data Types and Configuration
+
+- **[Data Types](dtype.md)** - Supported NumPy data types and type handling
+- **[Configuration](config.md)** - Runtime configuration and settings
+
+## Storage and Compression
+
+- **[Codecs](codecs.md)** - Compression and filtering codecs
+- **[Storage](storage.md)** - Storage backend implementations and interfaces
+- **[Registry](registry.md)** - Codec and storage backend registry
+
+## API Variants
+
+Zarr-Python provides both synchronous and asynchronous APIs:
+
+- **[Async API](./api/asynchronous.md)** - Asynchronous operations for concurrent access
+- **[Sync API](./api/synchronous.md)** - Synchronous operations for simple usage
+
+## Abstract Base Classes
+
+The ABC module defines interfaces for extending Zarr:
+
+- **[Codec ABC](abc/codec.md)** - Interface for custom compression codecs
+- **[Metadata ABC](abc/metadata.md)** - Interface for metadata handling
+- **[Store ABC](abc/store.md)** - Interface for custom storage backends
+
+## Utilities
+
+- **[Errors](errors.md)** - Exception classes and error handling
+- **[Testing](testing/index.md)** - Utilities for testing Zarr-based code
+
+
+## Getting Help
+
+- Check the [User Guide](../../user-guide/index.md) for tutorials and examples
+- Browse function signatures and docstrings in the API reference
+- Report issues on [GitHub](https://github.com/zarr-developers/zarr-python)
+- Join discussions on the [Zarr community forum](https://github.com/zarr-developers/community)
diff --git a/docs/api/zarr/metadata.md b/docs/api/zarr/metadata.md
new file mode 100644
index 0000000000..12eb909086
--- /dev/null
+++ b/docs/api/zarr/metadata.md
@@ -0,0 +1,6 @@
+---
+title: metadata
+---
+
+::: zarr.metadata
+::: zarr.metadata.migrate_v3
diff --git a/docs/api/zarr/registry.md b/docs/api/zarr/registry.md
new file mode 100644
index 0000000000..d2c3769596
--- /dev/null
+++ b/docs/api/zarr/registry.md
@@ -0,0 +1,5 @@
+---
+title: registry
+---
+
+::: zarr.registry
\ No newline at end of file
diff --git a/docs/api/zarr/storage.md b/docs/api/zarr/storage.md
new file mode 100644
index 0000000000..33580d1d8a
--- /dev/null
+++ b/docs/api/zarr/storage.md
@@ -0,0 +1,11 @@
+---
+title: storage
+---
+
+## Attributes
+
+::: zarr.storage.StoreLike
+
+## Classes
+
+::: zarr.storage
diff --git a/docs/api/zarr/testing/buffer.md b/docs/api/zarr/testing/buffer.md
new file mode 100644
index 0000000000..e0ae5e5dfd
--- /dev/null
+++ b/docs/api/zarr/testing/buffer.md
@@ -0,0 +1,3 @@
+## Buffer
+
+::: zarr.testing.buffer
diff --git a/docs/api/zarr/testing/index.md b/docs/api/zarr/testing/index.md
new file mode 100644
index 0000000000..2b48ad349f
--- /dev/null
+++ b/docs/api/zarr/testing/index.md
@@ -0,0 +1,13 @@
+---
+title: zarr.testing
+---
+
+# zarr.testing
+
+See the following sub-modules:
+
+- [zarr.testing.buffer](./buffer.md)
+- [zarr.testing.stateful](./stateful.md)
+- [zarr.testing.store](./store.md)
+- [zarr.testing.strategies](./strategies.md)
+- [zarr.testing.utils](./utils.md)
diff --git a/docs/api/zarr/testing/stateful.md b/docs/api/zarr/testing/stateful.md
new file mode 100644
index 0000000000..53c51b11ec
--- /dev/null
+++ b/docs/api/zarr/testing/stateful.md
@@ -0,0 +1,3 @@
+## Stateful
+
+::: zarr.testing.stateful
diff --git a/docs/api/zarr/testing/store.md b/docs/api/zarr/testing/store.md
new file mode 100644
index 0000000000..f190c65f95
--- /dev/null
+++ b/docs/api/zarr/testing/store.md
@@ -0,0 +1,4 @@
+
+## Store
+
+::: zarr.testing.store
diff --git a/docs/api/zarr/testing/strategies.md b/docs/api/zarr/testing/strategies.md
new file mode 100644
index 0000000000..dd6d546165
--- /dev/null
+++ b/docs/api/zarr/testing/strategies.md
@@ -0,0 +1,4 @@
+
+## Strategies
+
+::: zarr.testing.strategies
diff --git a/docs/api/zarr/testing/utils.md b/docs/api/zarr/testing/utils.md
new file mode 100644
index 0000000000..61202ac4b2
--- /dev/null
+++ b/docs/api/zarr/testing/utils.md
@@ -0,0 +1,3 @@
+## Utils
+
+::: zarr.testing.utils
diff --git a/docs/blog/.authors.yml b/docs/blog/.authors.yml
new file mode 100644
index 0000000000..10ce423cfc
--- /dev/null
+++ b/docs/blog/.authors.yml
@@ -0,0 +1,6 @@
+authors:
+ d-v-b:
+ name: Davis Bennett
+ description: Core developer
+ avatar: https://github.com/d-v-b.png
+ url: https://github.com/d-v-b
diff --git a/docs/blog/index.md b/docs/blog/index.md
new file mode 100644
index 0000000000..fca29e2578
--- /dev/null
+++ b/docs/blog/index.md
@@ -0,0 +1,3 @@
+# Blog
+
+News, release highlights, and design notes from the Zarr-Python developers.
diff --git a/docs/blog/posts/3.3.0-release.md b/docs/blog/posts/3.3.0-release.md
new file mode 100644
index 0000000000..13368848ae
--- /dev/null
+++ b/docs/blog/posts/3.3.0-release.md
@@ -0,0 +1,169 @@
+---
+date: 2026-07-30
+authors:
+ - d-v-b
+categories:
+ - Release
+---
+
+# Zarr-Python 3.3.0
+
+We're happy to announce the release of version 3.3.0 of Zarr-Python. It's been a while since our last release ([3.2.1](https://github.com/zarr-developers/zarr-python/releases/tag/v3.2.1) dropped in May of this year),
+and we're bringing some exciting additions to the latest version. For the full release notes, see the [3.3.0 release notes](../../release-notes.md), otherwise stick around for an overview of two performance-centric highlights of this release.
+
+
+
+## Faster low-latency storage
+
+Relevant issues and pull requests:
+
+- [#3524](https://github.com/zarr-developers/zarr-python/issues/3524) -- the performance report that started this work
+- [#3885](https://github.com/zarr-developers/zarr-python/pull/3885) -- synchronous codec APIs and the `FusedCodecPipeline`
+
+### The cost of async overhead
+
+Zarr-Python 3.x uses async routines for fetching data and decoding chunks. In terms of code, this means our store (data fetching) and codec (chunk decoding) APIs are both async. This makes
+I/O against high-latency storage backends like cloud object storage efficient. But for *low-latency* storage, like in-process memory or the file system, async routines add measurable overhead and offer no benefit. Async only adds value when there's work to be done while waiting for I/O to complete, but when I/O latency is low, it completes too quickly to run anything while waiting, and we are left paying the performance bill for obligatory async task scheduling that offered no value.
+
+This performance problem became acute when Zarr-Python users reported that in-memory array indexing workloads ran *slower* in Zarr-Python 3.1.3 relative to Zarr-Python 2.18.7 ([#3524](https://github.com/zarr-developers/zarr-python/issues/3524)). Fortunately this performance regression had a straightforward fix (I don't say "easy" because it was a lot of work).
+
+### Synchronous execution restores performance
+
+If async overhead makes low-latency storage slow, does *removing* that overhead restore performance? Yes, it does!
+
+In [#3885](https://github.com/zarr-developers/zarr-python/pull/3885) we defined synchronous versions of our storage and codec APIs -- the `SyncByteGetter` and `SyncByteSetter` protocols, plus a `get_ranges_sync` method on the `Store` ABC -- and then combined them in a new codec orchestration class called `FusedCodecPipeline`. The `FusedCodecPipeline` is an opt-in alternative to the default (the `BatchedCodecPipeline`) that gives large speedups for low-latency storage. It is currently marked [experimental](../../user-guide/experimental.md), so we may change it as we learn more; the default pipeline is untouched, and existing code keeps working unless you opt in.
+
+The win here is *not* a faster compressor. It is the removal of async scheduling overhead (including some [nasty `asyncio.to_thread` overhead](https://github.com/python/cpython/issues/136084)), plus a few vectorized fast paths for dense, uncompressed shards. And we only expect this new pipeline to accelerate workloads targeting a subset of storage backends, namely any store with methods that advertise low latency.
+
+On this author's 10-core Apple M4 laptop, the `FusedCodecPipeline` delivers the following results against memory-backed arrays:
+
+- uncompressed writes are *~4 times faster*
+- uncompressed reads are *~5 times faster*
+- compressed writes are *~2 times faster*
+- compressed reads are *~2 times faster*
+
+These numbers came from a [runnable example](../../user-guide/examples/codec_pipeline_performance.md) that ships with the documentation. Run it yourself to get a sense of how the `FusedCodecPipeline` behaves on your system -- when and how you use it depends on your hardware, your array layout, and how your chunks are compressed. What's certain is that for in-memory arrays, and arrays saved to the local file system, the `FusedCodecPipeline` is worth a try.
+
+Getting good numbers requires choosing the right level of thread-based parallelism for your workload, which is part of the configuration of the `FusedCodecPipeline`. For uncompressed chunks there's no CPU-bound work to do after fetching a chunk and so
+thread-based parallelism is worse than useless and slows things down. But for compressed chunks, threading offers a substantial payoff.
+
+### How to use it
+
+Select the pipeline through the [runtime configuration](../../user-guide/config.md) by setting `codec_pipeline.path`. Set it globally to affect every array created or opened afterwards:
+
+```python exec="true" session="blog-330" source="above"
+import zarr
+
+zarr.config.set(
+ {"codec_pipeline.path": "zarr.core.codec_pipeline.FusedCodecPipeline"}
+)
+```
+
+Or scope it to a block of code by using `zarr.config.set` as a context manager, which is the safer choice if you only want the new pipeline for part of your program:
+
+```python exec="true" session="blog-330" source="above" result="ansi"
+import numpy as np
+import zarr
+from zarr.storage import MemoryStore
+
+with zarr.config.set(
+ {"codec_pipeline.path": "zarr.core.codec_pipeline.FusedCodecPipeline"}
+):
+ arr = zarr.create_array(
+ store=MemoryStore(),
+ shape=(1000, 1000),
+ chunks=(100, 100),
+ shards=(1000, 1000),
+ dtype="float32",
+ )
+ arr[:] = np.random.random((1000, 1000)).astype("float32")
+ result = arr[:]
+
+print(result.shape)
+```
+
+Thread-based parallelism is configured separately, via `codec_pipeline.max_workers`. It defaults to `None`, meaning a pool sized to `os.cpu_count()`. Note that this setting is read *only* by the `FusedCodecPipeline` -- the default `BatchedCodecPipeline` ignores it, so tuning it without opting in above does nothing.
+
+As noted, memory-backed and uncompressed workloads often do better with a single worker, which runs everything inline on the calling thread:
+
+```python exec="true" session="blog-330" source="above"
+import zarr
+
+# No thread pool: run codec compute inline. Often best for uncompressed,
+# memory-backed arrays, where there's no CPU-bound work to overlap.
+zarr.config.set({"codec_pipeline.max_workers": 1})
+
+# A fixed-size thread pool, which pays off once compression is in play.
+zarr.config.set({"codec_pipeline.max_workers": 8})
+
+# Or back to the default, sized to the number of CPUs.
+zarr.config.set({"codec_pipeline.max_workers": None})
+```
+
+To return to the default pipeline, set `codec_pipeline.path` back to the batched implementation:
+
+```python exec="true" session="blog-330" source="above"
+import zarr
+
+zarr.config.set(
+ {"codec_pipeline.path": "zarr.core.codec_pipeline.BatchedCodecPipeline"}
+)
+```
+
+## Faster sharded reads
+
+Relevant issues and pull requests:
+
+- [#3004](https://github.com/zarr-developers/zarr-python/pull/3004) -- optimize partial shard reads
+- [#3925](https://github.com/zarr-developers/zarr-python/pull/3925) -- `Store.get_ranges` for concurrent, coalesced multi-range reads
+- [#3987](https://github.com/zarr-developers/zarr-python/pull/3987) -- control coalescing through `ArrayConfig` and the runtime config
+
+### How sharding works
+
+Chunks encoded with the `sharding_indexed` codec contain a secondary level of chunking, called subchunks. For example, if the `chunk_grid` field of the array metadata declares an "outer chunk" size of, say `(10, 10)`, a `sharding_indexed` codec in the `codecs` field could declare an "inner chunk" size of `(5, 5)`. Readers accessing such a chunk will observe a stored object (a stream of bytes) that decodes to an array with size `(10, 10)` (the "outer chunk"), which is comprised of four separate, contiguous byte ranges that each decode to a `(5, 5)` inner chunk. Each inner chunk occupies its own byte range in the outer chunk.
+
+A reader can satisfy a request for all four inner chunks by issuing four separate byte-range requests, or by making a *single* request for a byte range that spans all four inner chunks. The latter option is nice because it cuts down on the number of requests we need. Historically Zarr-Python used this optimization when reading entire outer chunks; in 3.3.0, we use this optimization in more cases, resulting in more efficient I/O patterns for sharded reads.
+
+### Interval equivalence
+
+Byte ranges, being intervals, obey some combination rules: the values in two half-open intervals `[a, b), [b, c)` can be captured by the single interval `[a, c)`. That means a reader can get multiple inner chunks with *one* byte-range request by requesting a range of bytes starting with the first byte of the first subchunk and ending with the last byte of the last subchunk. When individual requests are expensive, this kind of optimization is worth a lot.
+
+The requested inner chunks are not necessarily contiguous -- there might be a byte range gap between them. As long as that gap is not too big, its often efficient to fetch the entire byte range, gap included, and pick out the inner chunk byte ranges after I/O is done.
+
+### Byte range coalescing
+
+We call this procedure -- merging adjacent byte ranges -- "byte range coalescing", and it's a new performance optimization shipping in Zarr-Python 3.3.0. Unlike the `FusedCodecPipeline`, this one is on by default with base settings we think are good, so most users won't need to tune anything.
+
+Two knobs control it, both documented in the [runtime configuration guide](../../user-guide/config.md). Nearby byte ranges in the same shard are merged into a single request when the gap between them is no larger than `array.sharding_coalesce_max_gap_bytes` (default 1 MiB) and the merged read stays within `array.sharding_coalesce_max_bytes` (default 16 MiB). The gap threshold is what trades wasted bytes against saved requests: raising it reads more data you didn't ask for, in exchange for fewer requests.
+
+For a runnable demonstration -- counting the store requests saved and timing them against a store with simulated latency -- see the [sharded read coalescing example](../../user-guide/examples/sharding_coalescing.md).
+
+You can set them globally, or per array by passing `config={...}` to [`zarr.create_array`][]:
+
+```python exec="true" session="blog-330" source="above" result="ansi"
+import zarr
+from zarr.storage import MemoryStore
+
+arr = zarr.create_array(
+ store=MemoryStore(),
+ shape=(1000, 1000),
+ chunks=(100, 100),
+ shards=(1000, 1000),
+ dtype="float32",
+ config={
+ "sharding_coalesce_max_gap_bytes": 4 * 1024**2, # 4 MiB
+ "sharding_coalesce_max_bytes": 64 * 1024**2, # 64 MiB
+ },
+)
+print(arr.shape)
+```
+
+## Tell us what you think
+
+We hope these new features are helpful, and we would appreciate any feedback that helps us improve them, or any other aspect of Zarr-Python.
+
+## Going faster
+
+The updates in this release are just the first step of a larger performance-oriented direction for Zarr-Python. Landing these two enhancements taught us a *lot* about the performance-sensitive areas of the library. We can and will invest more time in performance tuning, e.g. by adding or changing abstractions, writing code for special cases, etc.
+
+We plan to consider including compiled code that should enable significant performance improvements. The [`zarrs`](https://zarrs.dev/) project is an ecosystem of Zarr tools written in Rust, with [extremely high performance](https://book.zarrs.dev/#-zarrs-is-fast-). Is there a `zarrs` binding in Zarr-Python's future? I hope so! We are keenly observing development of [`zarrista`](https://developmentseed.org/zarrista/latest/) as a proof-of-concept for what a Python-`zarrs` binding layer might look like. Stay tuned!
diff --git a/docs/conf.py b/docs/conf.py
deleted file mode 100644
index 3389c16549..0000000000
--- a/docs/conf.py
+++ /dev/null
@@ -1,379 +0,0 @@
-#!/usr/bin/env python3
-#
-# zarr documentation build configuration file, created by
-# sphinx-quickstart on Mon May 2 21:40:09 2016.
-#
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
-
-
-import os
-import sys
-from typing import Any
-
-import sphinx
-import sphinx.application
-
-from importlib.metadata import version as get_version
-
-import sphinx
-
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
-sys.path.append(os.path.abspath(".."))
-
-
-# -- General configuration ------------------------------------------------
-
-# If your documentation needs a minimal Sphinx version, state it here.
-# needs_sphinx = '1.0'
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
-# ones.
-extensions = [
- "sphinx.ext.autodoc",
- "sphinx.ext.autosummary",
- "sphinx.ext.viewcode",
- "sphinx.ext.intersphinx",
- 'autoapi.extension',
- "numpydoc",
- "sphinx_issues",
- "sphinx_copybutton",
- "sphinx_design",
- 'sphinx_reredirects',
-]
-
-issues_github_path = "zarr-developers/zarr-python"
-
-autoapi_dirs = ['../src/zarr']
-autoapi_add_toctree_entry = False
-autoapi_generate_api_docs = True
-autoapi_member_order = "groupwise"
-autoapi_root = "_autoapi"
-autoapi_keep_files = True
-autoapi_options = [ 'members', 'undoc-members', 'show-inheritance', 'show-module-summary', 'imported-members', ]
-
-def skip_submodules(
- app: sphinx.application.Sphinx,
- what: str,
- name: str,
- obj: object,
- skip: bool,
- options: dict[str, Any]
- ) -> bool:
- # Skip documenting zarr.codecs submodules
- # codecs are documented in the main zarr.codecs namespace
- if what == "module" and name.startswith("zarr.codecs."):
- skip = True
- return skip
-
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ["_templates"]
-
-# The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-# source_suffix = ['.rst', '.md']
-source_suffix = ".rst"
-
-# The encoding of source files.
-# source_encoding = 'utf-8-sig'
-
-# The main toctree document.
-main_doc = "index"
-
-# General information about the project.
-project = "zarr"
-copyright = "2024, Zarr Developers"
-author = "Zarr Developers"
-
-version = get_version("zarr")
-release = get_version("zarr")
-
-redirects = {
- "spec": "https://zarr-specs.readthedocs.io",
- "spec/v1": 'https://zarr-specs.readthedocs.io/en/latest/v1/v1.0.html',
- "spec/v2": "https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html",
- "spec/v3": "https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html",
- "license": "https://github.com/zarr-developers/zarr-python/blob/main/LICENSE.txt",
- "tutorial": "user-guide",
- "getting-started": "quickstart",
- "release": "developers/release.html",
- "roadmap": "developers/roadmap.html",
- "installation": "user-guide/installation.html",
-}
-
-# The language for content autogenerated by Sphinx. Refer to documentation
-# for a list of supported languages.
-#
-# This is also used if you do content translation via gettext catalogs.
-# Usually you set "language" from the command line for these cases.
-language = "en"
-
-# There are two options for replacing |today|: either, you set today to some
-# non-false value, then it is used:
-# today = ''
-# Else, today_fmt is used as the format for a strftime call.
-# today_fmt = '%B %d, %Y'
-
-# List of patterns, relative to source directory, that match files and
-# directories to ignore when looking for source files.
-# This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "talks"]
-
-# The reST default role (used for this markup: `text`) to use for all
-# documents.
-# default_role = None
-
-# If true, '()' will be appended to :func: etc. cross-reference text.
-# add_function_parentheses = True
-
-# If true, the current module name will be prepended to all description
-# unit titles (such as .. function::).
-# add_module_names = True
-
-# If true, sectionauthor and moduleauthor directives will be shown in the
-# output. They are ignored by default.
-# show_authors = False
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = "sphinx"
-
-# A list of ignored prefixes for module index sorting.
-# modindex_common_prefix = []
-
-# If true, keep warnings as "system message" paragraphs in the built documents.
-# keep_warnings = False
-
-# If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = False
-
-
-# -- Options for HTML output ----------------------------------------------
-
-# The theme to use for HTML and HTML Help pages. See the documentation for
-# a list of builtin themes.
-html_theme = "pydata_sphinx_theme"
-
-html_favicon = "_static/logo1.png"
-
-# Theme options are theme-specific and customize the look and feel of a theme
-# further. For a list of options available for each theme, see the
-# documentation.
-html_theme_options = {
- "github_url": "https://github.com/zarr-developers/zarr-python",
- "twitter_url": "https://twitter.com/zarr_dev",
- "icon_links": [
- {
- "name": "Zarr Dev",
- "url": "https://zarr.dev/",
- "icon": "_static/logo1.png",
- "type": "local",
- },
- ],
- "collapse_navigation": True,
- "navigation_with_keys": False,
-}
-
-# Add any paths that contain custom themes here, relative to this directory.
-# html_theme_path = []
-
-# The name for this set of Sphinx documents.
-# " v documentation" by default.
-# html_title = 'zarr v@@'
-
-# A shorter title for the navigation bar. Default is the same as html_title.
-# html_short_title = None
-
-# The name of an image file (relative to this directory) to place at the top
-# of the sidebar.
-html_logo = "_static/logo_horizontal.svg"
-
-
-def setup(app: sphinx.application.Sphinx) -> None:
- app.add_css_file("custom.css")
- app.connect("autoapi-skip-member", skip_submodules)
-
-
-# The name of an image file (relative to this directory) to use as a favicon of
-# the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
-# pixels large.
-# html_favicon = None
-
-# Add any paths that contain custom static files (such as style sheets) here,
-# relative to this directory. They are copied after the builtin static files,
-# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ["_static"]
-html_js_files = [
- "custom.js",
-]
-
-# Add any extra paths that contain custom files (such as robots.txt or
-# .htaccess) here, relative to this directory. These files are copied
-# directly to the root of the documentation.
-# html_extra_path = []
-
-# If not None, a 'Last updated on:' timestamp is inserted at every page
-# bottom, using the given strftime format.
-# The empty string is equivalent to '%b %d, %Y'.
-# html_last_updated_fmt = None
-
-# If true, SmartyPants will be used to convert quotes and dashes to
-# typographically correct entities.
-# html_use_smartypants = True
-
-# Custom sidebar templates, maps document names to template names.
-html_sidebars = {"tutorial": []}
-
-# Additional templates that should be rendered to pages, maps page names to
-# template names.
-# html_additional_pages = {}
-
-# If false, no module index is generated.
-# html_domain_indices = True
-
-# If false, no index is generated.
-# html_use_index = True
-
-# If true, the index is split into individual pages for each letter.
-# html_split_index = False
-
-# If true, links to the reST sources are added to the pages.
-# html_show_sourcelink = True
-
-# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
-# html_show_sphinx = True
-
-# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
-# html_show_copyright = True
-
-# If true, an OpenSearch description file will be output, and all pages will
-# contain a tag referring to it. The value of this option must be the
-# base URL from which the finished HTML is served.
-# html_use_opensearch = ''
-
-# This is the file name suffix for HTML files (e.g. ".xhtml").
-# html_file_suffix = None
-
-# Language to be used for generating the HTML full-text search index.
-# Sphinx supports the following languages:
-# 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
-# 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr', 'zh'
-# html_search_language = 'en'
-
-# A dictionary with options for the search language support, empty by default.
-# 'ja' uses this config value.
-# 'zh' user can custom change `jieba` dictionary path.
-# html_search_options = {'type': 'default'}
-
-# The name of a javascript file (relative to the configuration directory) that
-# implements a search results scorer. If empty, the default will be used.
-# html_search_scorer = 'scorer.js'
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = "zarrdoc"
-
-maximum_signature_line_length = 80
-
-# -- Options for LaTeX output ---------------------------------------------
-
-latex_elements = {
- # The paper size ('letterpaper' or 'a4paper').
- #'papersize': 'letterpaper',
- # The font size ('10pt', '11pt' or '12pt').
- #'pointsize': '10pt',
- # Additional stuff for the LaTeX preamble.
- #'preamble': '',
- # Latex figure (float) alignment
- #'figure_align': 'htbp',
-}
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title,
-# author, documentclass [howto, manual, or own class]).
-latex_documents = [
- (main_doc, "zarr.tex", "Zarr-Python", author, "manual"),
-]
-
-# The name of an image file (relative to this directory) to place at the top of
-# the title page.
-# latex_logo = None
-
-# For "manual" documents, if this is true, then toplevel headings are parts,
-# not chapters.
-# latex_use_parts = False
-
-# If true, show page references after internal links.
-# latex_show_pagerefs = False
-
-# If true, show URL addresses after external links.
-# latex_show_urls = False
-
-# Documents to append as an appendix to all manuals.
-# latex_appendices = []
-
-# If false, no module index is generated.
-# latex_domain_indices = True
-
-
-# -- Options for manual page output ---------------------------------------
-
-# One entry per manual page. List of tuples
-# (source start file, name, description, authors, manual section).
-man_pages = [(main_doc, "zarr", "Zarr-Python", [author], 1)]
-
-# If true, show URL addresses after external links.
-# man_show_urls = False
-
-
-# -- Options for Texinfo output -------------------------------------------
-
-# Grouping the document tree into Texinfo files. List of tuples
-# (source start file, target name, title, author,
-# dir menu entry, description, category)
-texinfo_documents = [
- (
- main_doc,
- "zarr",
- "Zarr-Python",
- author,
- "zarr",
- "One line description of project.",
- "Miscellaneous",
- ),
-]
-
-# Documents to append as an appendix to all manuals.
-# texinfo_appendices = []
-
-# If false, no module index is generated.
-# texinfo_domain_indices = True
-
-# How to display URL addresses: 'footnote', 'no', or 'inline'.
-# texinfo_show_urls = 'footnote'
-
-# If true, do not generate a @detailmenu in the "Top" node's menu.
-# texinfo_no_detailmenu = False
-
-
-# Example configuration for intersphinx: refer to the Python standard library.
-# use in refs e.g:
-# :ref:`comparison manual `
-intersphinx_mapping = {
- "python": ("https://docs.python.org/3/", None),
- "numpy": ("https://numpy.org/doc/stable/", None),
- "numcodecs": ("https://numcodecs.readthedocs.io/en/stable/", None),
-}
-
-
-# sphinx-copybutton configuration
-copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | {2,5}\.\.\.: | {5,8}: "
-copybutton_line_continuation_character = "\\"
-copybutton_prompt_is_regexp = True
diff --git a/docs/contributing.md b/docs/contributing.md
new file mode 100644
index 0000000000..dea7256c36
--- /dev/null
+++ b/docs/contributing.md
@@ -0,0 +1,422 @@
+# Contributing
+
+Zarr is a community maintained project. We welcome contributions in the form of bug reports, bug fixes, documentation, enhancement proposals and more. This page provides information on how best to contribute.
+
+## Asking for help
+
+If you have a question about how to use Zarr, please post your question on StackOverflow using the ["zarr" tag](https://stackoverflow.com/questions/tagged/zarr). If you don't get a response within a day or two, feel free to raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new) including a link to your StackOverflow question. We will try to respond to questions as quickly as possible, but please bear in mind that there may be periods where we have limited time to answer questions due to other commitments.
+
+## Bug reports
+
+If you find a bug, please raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new). Please include the following items in a bug report:
+
+1. A minimal, self-contained snippet of Python code reproducing the problem. You can format the code nicely using markdown, e.g.:
+
+ ```python exec="false" reason="illustrative pseudocode with a '# etc.' placeholder, not runnable"
+ import zarr
+ g = zarr.group()
+ # etc.
+ ```
+
+2. An explanation of why the current behavior is wrong/not desired, and what you expect instead.
+
+3. Information about the version of Zarr, along with versions of dependencies and the Python interpreter, and installation information. The version of Zarr can be obtained from the `zarr.__version__` attribute. Please also state how Zarr was installed, e.g., "installed via pip into a virtual environment", or "installed using conda". Information about other packages installed can be obtained by executing `pip freeze` (if using pip to install packages) or `conda env export` (if using conda to install packages) from the operating system command prompt. The version of the Python interpreter can be obtained by running a Python interactive session, e.g.:
+
+ ```console
+ python
+ ```
+
+ ```ansi
+ Python 3.12.7 | packaged by conda-forge | (main, Oct 4 2024, 15:57:01) [Clang 17.0.6 ] on darwin
+ ```
+
+## Enhancement proposals
+
+If you have an idea about a new feature or some other improvement to Zarr, please raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new) first to discuss.
+
+We very much welcome ideas and suggestions for how to improve Zarr, but please bear in mind that we are likely to be conservative in accepting proposals for new features. The reasons for this are that we would like to keep the Zarr code base lean and focused on a core set of functionalities, and available time for development, review and maintenance of new features is limited. But if you have a great idea, please don't let that stop you from posting it on GitHub, just please don't be offended if we respond cautiously.
+
+## AI-assisted contributions
+
+AI coding tools are increasingly common in open source development. These tools are welcome in Zarr-Python, but the same standards apply to all contributions regardless of how they were produced — whether written by hand, with AI assistance, or generated entirely by an AI tool.
+
+### You are responsible for your changes
+
+If you submit a pull request, you are responsible for understanding and having fully reviewed the changes. You must be able to explain why each change is correct and how it fits into the project.
+
+### Communication must be your own
+
+PR descriptions, issue comments, and review responses must be in your own words. The substance and reasoning must come from you. Using AI to polish grammar or phrasing is fine, but do not paste AI-generated text as comments or review responses.
+
+### Review every line
+
+You must have personally reviewed and understood all changes before submitting. If you used AI to generate code, you are expected to have read it critically and tested it. The PR description should explain the approach and reasoning — do not leave it to reviewers to figure out what the code does and why.
+
+### Keep PRs reviewable
+
+Generating code with AI is fast; reviewing it is not. A large diff shifts the burden from the contributor to the reviewer. PRs that cannot be reviewed in reasonable time with reasonable effort may be closed, regardless of their potential usefulness or correctness. Use AI tools not only to write code but to prepare better, more reviewable PRs — well-structured commits, clear descriptions, and minimal scope.
+
+If you are planning a large AI-assisted contribution (e.g., a significant refactor or a new subsystem), **open an issue first** to discuss the scope and approach with maintainers. Maintainers may also request that large changes be broken into smaller, reviewable pieces.
+
+### Documentation
+
+The same principles apply to documentation. Zarr has domain-specific semantics (chunked storage, codec pipelines, Zarr v2/v3 format details) that AI tools frequently get wrong. Do not submit documentation that you haven't carefully read and verified.
+
+## Contributing code and/or documentation
+
+### Forking the repository
+
+The Zarr source code is hosted on GitHub at the following location:
+
+* [https://github.com/zarr-developers/zarr-python](https://github.com/zarr-developers/zarr-python)
+
+You will need your own fork to work on the code. Go to the link above and hit the ["Fork"](https://github.com/zarr-developers/zarr-python/fork) button. Then clone your fork to your local machine:
+
+```bash
+git clone git@github.com:your-user-name/zarr-python.git
+cd zarr-python
+git remote add upstream git@github.com:zarr-developers/zarr-python.git
+```
+
+### Creating a development environment
+
+To work with the Zarr source code, it is recommended to use [hatch](https://hatch.pypa.io/latest/index.html) to create and manage development environments. Hatch will automatically install all Zarr dependencies using the same versions as are used by the core developers and continuous integration services. Assuming you have a Python 3 interpreter already installed, and you have cloned the Zarr source code and your current working directory is the root of the repository, you can do something like the following:
+
+```bash
+pip install hatch
+hatch env show # list all available environments
+```
+
+To verify that your development environment is working, you can run the unit tests for one of the test environments, e.g.:
+
+```bash
+hatch env run --env test.py3.12-optional run
+```
+
+### Creating a branch
+
+Before you do any new work or submit a pull request, please open an issue on GitHub to report the bug or propose the feature you'd like to add.
+
+It's best to synchronize your fork with the upstream repository, then create a new, separate branch for each piece of work you want to do. E.g.:
+
+```bash
+git checkout main
+git fetch upstream
+git checkout -b shiny-new-feature upstream/main
+git push -u origin shiny-new-feature
+```
+
+This changes your working directory to the 'shiny-new-feature' branch. Keep any changes in this branch specific to one bug or feature so it is clear what the branch brings to Zarr.
+
+To update this branch with latest code from Zarr, you can retrieve the changes from the main branch and perform a rebase:
+
+```bash
+git fetch upstream
+git rebase upstream/main
+```
+
+This will replay your commits on top of the latest Zarr git main. If this leads to merge conflicts, these need to be resolved before submitting a pull request. Alternatively, you can merge the changes in from upstream/main instead of rebasing, which can be simpler:
+
+```bash
+git pull upstream main
+```
+
+Again, any conflicts need to be resolved before submitting a pull request.
+
+### Running the test suite
+
+Zarr includes a suite of unit tests. The simplest way to run the unit tests is to activate your development environment (see [creating a development environment](#creating-a-development-environment) above) and invoke:
+
+```bash
+hatch env run --env test.py3.12-optional run
+```
+
+All tests are automatically run via GitHub Actions for every pull request and must pass before code can be accepted. Test coverage is also collected automatically via the Codecov service.
+
+### Code standards - using prek
+
+All code must conform to the PEP8 standard. Regarding line length, lines up to 100 characters are allowed, although please try to keep under 90 wherever possible.
+
+`Zarr` uses a set of git hooks managed by [`prek`](https://github.com/j178/prek), a fast, Rust-based pre-commit hook manager that is fully compatible with `.pre-commit-config.yaml` files. `prek` can be installed locally by running:
+
+```bash
+uv tool install prek
+```
+
+or:
+
+```bash
+pip install prek
+```
+
+The hooks can be installed locally by running:
+
+```bash
+prek install
+```
+
+This will run the checks every time a commit is created locally. The checks will by default only run on the files modified by a commit, but the checks can be triggered for all the files by running:
+
+```bash
+prek run --all-files
+```
+
+You can also run hooks only for files in a specific directory:
+
+```bash
+prek run --directory src/zarr
+```
+
+Or run hooks for files changed in the last commit:
+
+```bash
+prek run --last-commit
+```
+
+To list all available hooks:
+
+```bash
+prek list
+```
+
+If you would like to skip the failing checks and push the code for further discussion, use the `--no-verify` option with `git commit`.
+
+### Test coverage
+
+> **Note:** Test coverage for Zarr-Python 3 is currently not at 100%. This is a known issue and help is welcome to bring test coverage back to 100%. See issue #2613 for more details.
+
+Zarr strives to maintain 100% test coverage under the latest Python stable release. Both unit tests and docstring doctests are included when computing coverage. Running:
+
+```bash
+hatch env run --env test.py3.12-optional run-coverage
+```
+
+will automatically run the test suite with coverage and produce an XML coverage report. This should be 100% before code can be accepted into the main code base.
+
+You can also generate an HTML coverage report by running:
+
+```bash
+hatch env run --env test.py3.12-optional run-coverage-html
+```
+
+When submitting a pull request, coverage will also be collected across all supported Python versions via the Codecov service, and will be reported back within the pull request. Codecov coverage must also be 100% before code can be accepted.
+
+### Documentation
+
+Docstrings for user-facing classes and functions should follow the [numpydoc](https://numpydoc.readthedocs.io/en/stable/format.html#docstring-standard) standard, including sections for Parameters and Examples. All examples should run and pass as doctests under Python 3.12.
+
+Zarr uses mkdocs for documentation, hosted on readthedocs.org. Documentation is written in the Markdown markup language (.md files) in the `docs` folder. The documentation consists both of prose and API documentation. All user-facing classes and functions are included in the API documentation, under the `docs/api` folder using the [mkdocstrings](https://mkdocstrings.github.io/) extension. Add any new public functions or classes to the relevant markdown file in `docs/api/*.md`. Any new features or important usage information should be included in the user-guide (`docs/user-guide`). Any changes should also be included as a new file in the `changes` directory.
+
+The documentation can be built locally by running:
+
+```bash
+hatch --env docs run build
+```
+
+The resulting built documentation will be available in the `site` folder.
+
+Hatch can also be used to serve continuously updating version of the documentation during development at [http://127.0.0.1:8000/](http://127.0.0.1:8000/). This can be done by running:
+
+```bash
+hatch --env docs run serve
+```
+
+#### Adding executable code blocks in the documentation
+
+Zarr uses [Markdown Exec](https://pawamoy.github.io/markdown-exec/usage/) to execute code blocks in Markdown files. Add `exec="true"` to a code block header for it to be executed when the docs are built. For example:
+
+````md
+```python exec="true"
+print("Hello world")
+```
+````
+
+Below are other useful options that can be added to the code block. See [Markdown Exec's documentation](https://pawamoy.github.io/markdown-exec/usage/#options-summary) for a full list:
+
+ - `source="above"` makes sure the code within the code block is also rendered in the documentation (rather than just the output).
+ - `session=""` executes code blocks in a named session reusing previously defined variables.
+ - `result="ansi"` or `result="html"` to render the output. If the code does not produce output, you should leave off the `result` option to prevent an empty cell from rendering in the docs.
+
+For example:
+
+````md
+```python exec="true" session="contributing" source="above" result="ansi"
+print("Hello world")
+```
+````
+
+renders as:
+
+```python exec="true" session="contributing" source="above" result="ansi"
+print("Hello world")
+```
+
+#### Validating code blocks: `exec` vs `test`
+
+Every Python code block in the documentation is checked by a test
+(`tests/test_docs.py`) so that examples cannot quietly rot — the bug that motivated
+this was an example calling `zarr.create_array(..., mode="w")`, an argument that does
+not exist, which went unnoticed because nothing ran it. A block declares *how* it is
+validated using one of two independent attributes:
+
+ - **`exec="true"`** — Markdown Exec runs the block **at docs-build time to render its
+ output** into the page. This is the attribute described above; it is also what the
+ test suite executes. Use it for ordinary examples whose output should appear in the
+ docs.
+ - **`test="true"`** — the block is **run by the test suite only**, *not* at build time.
+ Use this for an example that should be validated but cannot run in the docs-build
+ environment — for example one that needs a GPU or a cloud backend. Markdown Exec
+ leaves a `test="true"` block as a static, syntax-highlighted snippet (it never
+ executes it), while the test suite still runs it (see the marker note below).
+
+A block may carry both (`exec="true" test="true"`), though in practice `exec="true"`
+already implies it is tested, so you rarely need `test="true"` alongside it.
+
+The two attributes are kept separate on purpose: `exec=` controls *build-time rendering*
+and `test=` controls *test-time validation*. Tagging a GPU/cloud example `exec="true"`
+would make `mkdocs build` try to run it on a machine without that infrastructure and fail
+the build; `test="true"` lets it be validated without being built.
+
+##### Opting a block out of validation
+
+A handful of blocks genuinely cannot run and are not executable Python — a REPL
+transcript, a deliberately-incorrect "before" snippet, a `--8<--` file include. Mark
+these explicitly by opening the fence with
+`exec="false" reason="REPL output transcript, not executable source"` (supply a reason
+that fits the block).
+
+`exec="false"` with a non-empty `reason` is an explicit, greppable opt-out. A test
+(`test_no_unvalidated_blocks`) requires **every** Python block to be either `exec="true"`,
+`test="true"`, or `exec="false"` with a reason — so a block can never silently skip
+validation. A bare ` ```python ` fence, or a typo like `exec="on"`, fails that test.
+
+Markdown Exec only renders `exec="true"` fences; the `mkdocs_hooks.py` hook at the
+repository root makes `test="true"` and `exec="false"` fences render as ordinary
+highlighted code blocks. Without it, these fences would fail superfences parsing and
+their contents would spill into the page as raw markdown.
+
+##### Marker-bound blocks (GPU, S3)
+
+A `test="true"` block that needs special infrastructure declares a pytest marker with
+`markers="..."`, which binds it to that infrastructure in the test suite:
+
+ - `markers="gpu"` — run only under `pytest -m gpu` (the GPU CI environment); skipped
+ elsewhere via `importorskip("cupy")`.
+ - `markers="s3"` — run against a mock S3 (moto) backend supplied by a test fixture, so
+ the example can use a bare `s3://…` URL with no test-only connection details on show.
+
+##### Placement of `test="true"` blocks
+
+Because Markdown Exec does not execute a `test="true"` (or `exec="false"`) block, placing
+one *before* an `exec="true"` block on the same page can disrupt the build-time execution
+of that later block. Put `test="true"` blocks **after** all `exec="true"` blocks on the
+page (or on a page where they are the only Python block). The `test_test_only_blocks_come_last`
+test enforces this, and the CI docs build runs with `--strict` so any such breakage fails
+the build rather than passing as a warning.
+
+#### Building documentation without executing code blocks
+
+Sometimes, you may want the documentation to build quicker. You can disable code block execution by commenting out the [markdown-exec plugin](https://github.com/zarr-developers/zarr-python/blob/884a8c91afcc3efe28b3da952be3b85125c453cb/mkdocs.yml#L132) in the mkdocs configuration file. This will make code blocks and cross references render incorrectly (i.e., expect build warnings), but also reduces build time by ~3x. Be sure to undo the commenting out before opening your pull request.
+
+### Changelog
+
+zarr-python uses [towncrier](https://towncrier.readthedocs.io/en/stable/tutorial.html) to manage release notes. Most pull requests should include at least one news fragment describing the changes. To add a release note, you'll need the GitHub issue or pull request number and the type of your change (`feature`, `bugfix`, `doc`, `removal`, `misc`). With that, run `towncrier create` with your development environment, which will prompt you for the issue number, change type, and the news text:
+
+```bash
+towncrier create
+```
+
+Alternatively, you can manually create the files in the `changes` directory using the naming convention `{issue-number}.{change-type}.md`.
+
+See the [towncrier](https://towncrier.readthedocs.io/en/stable/tutorial.html) docs for more.
+
+## Project governance
+
+This section documents the processes that core developers follow to maintain the project. The current core developers are listed in [`TEAM.md`](https://github.com/zarr-developers/zarr-python/blob/main/TEAM.md).
+
+### Merging pull requests
+
+Pull requests submitted by an external contributor should be reviewed and approved by at least one core developer before being merged. Ideally, pull requests submitted by a core developer should be reviewed and approved by at least one other core developer before being merged.
+
+Pull requests should not be merged until all CI checks have passed (GitHub Actions, Codecov) against code that has had the latest main merged in.
+
+Before merging, the milestone must be set to decide whether a PR will be in the next patch, minor, or major release. The next section explains which types of changes go in each release.
+
+### Self-merging pull requests
+
+The default is that a pull request opened by a core developer is reviewed and approved by at least one other core developer before it is merged. We trust core developers to use their judgment, though, and we would rather bias toward action than make routine changes wait on review they do not really need.
+
+So a core developer may merge their own pull request whenever they judge the change to be low-risk, provided the standard merge requirements are met — CI is green against code that has had the latest `main` merged in, a changelog fragment has been added, and the milestone is set — and other core developers have had a fair chance to weigh in. As a rule of thumb, leave the pull request open for a few days before self-merging, unless it is genuinely trivial or time-sensitive. If you are confident a change is fine, merge it; if you have real doubts, ask for a review. It is generally advisable to ping another developer in the PR description for awareness about the direction, even if you choose not to request a formal review.
+
+Some changes warrant more caution, and a second reviewer is usually worth seeking even when you could self-merge: changes to the public API, anything touching data-format or on-disk compatibility, and performance-sensitive code. These are the most expensive to get wrong and the hardest to reverse. Reverts, by contrast, are cheap — if a self-merged change turns out to be a mistake, reverting it is itself a low-risk change that any core developer can make, and the reworked version can go through normal review. When something recently merged is actively causing harm — a broken `main`, a release blocker, or data corruption — fix it fast and request review after the fact rather than waiting.
+
+This policy exists to lower the cost of routine work and to help newer core developers grow comfortable merging changes. It is not a license to merge past an unresolved objection: if another core developer asks to review a change, give them that chance.
+
+### Release procedure
+
+To give the release visibility and a single place to track progress, open an issue on GitHub announcing the release using the [release checklist template](https://github.com/zarr-developers/zarr-python/issues/new?template=release-checklist.md). The release checklist includes all steps necessary for the release.
+
+## Compatibility and versioning policies
+
+### Versioning
+
+Versions of this library are identified by a triplet of integers with the form `..`, for example `3.0.4`. A release of `zarr-python` is associated with a new version identifier. That new identifier is generated by incrementing exactly one of the components of the previous version identifier by 1. When incrementing the `major` component of the version identifier, the `minor` and `patch` components are reset to 0. When incrementing the minor component, the patch component is reset to 0.
+
+Releases are classified by the library changes contained in that release. This classification determines which component of the version identifier is incremented on release.
+
+* **major** releases (for example, `2.18.0` -> `3.0.0`) are for changes that will require extensive adaptation efforts from many users and downstream projects. For example, breaking changes to widely-used user-facing APIs should only be applied in a major release.
+
+ Users and downstream projects should carefully consider the impact of a major release before adopting it. In advance of a major release, developers should communicate the scope of the upcoming changes, and help users prepare for them.
+
+* **minor** releases (for example, `3.0.0` -> `3.1.0`) are for changes that do not require significant effort from most users or downstream projects to respond to. API changes are possible in minor releases if the burden on users imposed by those changes is sufficiently small.
+
+ For example, a recently released API may need fixes or refinements that are breaking, but low impact due to the recency of the feature. Such API changes are permitted in a minor release.
+
+ Minor releases are safe for most users and downstream projects to adopt.
+
+* **patch** releases (for example, `3.1.0` -> `3.1.1`) are for changes that contain no breaking or behavior changes for downstream projects or users. Examples of changes suitable for a patch release are bugfixes and documentation improvements.
+
+ Users should always feel safe upgrading to the latest patch release.
+
+Note that this versioning scheme is not consistent with [Semantic Versioning](https://semver.org/). Contrary to SemVer, the Zarr library may release breaking changes in `minor` releases, or even `patch` releases under exceptional circumstances. But we should strive to avoid doing so.
+
+A better model for our versioning scheme is [Intended Effort Versioning](https://jacobtomlinson.dev/effver/), or "EffVer". The guiding principle of EffVer is to categorize releases based on the *expected effort required to upgrade to that release*.
+
+Zarr developers should make changes as smooth as possible for users. This means making backwards-compatible changes wherever possible. When a backwards-incompatible change is necessary, users should be notified well in advance, e.g. via informative deprecation warnings.
+
+### Data format compatibility
+
+The Zarr library is an implementation of a file format standard defined externally -- see the [Zarr specifications website](https://zarr-specs.readthedocs.io) for the list of Zarr file format specifications.
+
+If an existing Zarr format version changes, or a new version of the Zarr format is released, then the Zarr library will generally require changes. It is very likely that a new Zarr format will require extensive breaking changes to the Zarr library, and so support for a new Zarr format in the Zarr library will almost certainly come in a new `major` release. When the Zarr library adds support for a new Zarr format, there may be a period of accelerated changes as developers refine newly added APIs and deprecate old APIs. In such a transitional phase breaking changes may be more frequent than usual.
+
+## Experimental API policy
+
+The `zarr.experimental` namespace contains features that are under active development and may change without notice. When contributing to or depending on experimental features, please keep the following in mind:
+
+### For contributors
+
+When adding a new feature to `zarr.experimental`:
+
+1. Place the feature under `src/zarr/experimental/` and export it from `src/zarr/experimental/__init__.py`.
+2. Document the feature in `docs/user-guide/experimental.md` and note clearly that it is experimental.
+3. Add a changelog entry categorized as `feature`.
+
+We aim to either **promote** or **remove** experimental features within **6 months** of their addition. To promote a feature to stable:
+
+1. Move it from `zarr.experimental` to the appropriate stable module.
+2. Keep a deprecated re-export in `zarr.experimental` for one minor release.
+3. Update the documentation to reflect the stable location.
+
+### For users
+
+Features in `zarr.experimental` carry no stability guarantees. They may be changed or removed in any release, including patch releases. If you depend on an experimental feature, pin your `zarr-python` version accordingly.
+
+## Benchmarks
+
+Zarr uses [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/latest/) for running
+performance benchmarks as part of our test suite. The benchmarks are found in `tests/benchmarks`.
+By default pytest is configured to run these benchmarks as plain tests (i.e., no benchmarking). To run
+a benchmark with timing measurements, use the `--benchmark-enable` when invoking `pytest`.
+
+The benchmarks are run as part of the continuous integration suite through [codspeed](https://app.codspeed.io/zarr-developers/zarr-python).
diff --git a/docs/developers/contributing.rst b/docs/developers/contributing.rst
deleted file mode 100644
index 4358230eff..0000000000
--- a/docs/developers/contributing.rst
+++ /dev/null
@@ -1,359 +0,0 @@
-.. _dev-guide-contributing:
-
-Contributing to Zarr
-====================
-
-Zarr is a community maintained project. We welcome contributions in the form of bug
-reports, bug fixes, documentation, enhancement proposals and more. This page provides
-information on how best to contribute.
-
-Asking for help
----------------
-
-If you have a question about how to use Zarr, please post your question on
-StackOverflow using the `"zarr" tag `_.
-If you don't get a response within a day or two, feel free to raise a `GitHub issue
-`_ including a link to your StackOverflow
-question. We will try to respond to questions as quickly as possible, but please bear
-in mind that there may be periods where we have limited time to answer questions
-due to other commitments.
-
-Bug reports
------------
-
-If you find a bug, please raise a `GitHub issue
-`_. Please include the following items in
-a bug report:
-
-1. A minimal, self-contained snippet of Python code reproducing the problem. You can
- format the code nicely using markdown, e.g.::
-
-
- ```python
- import zarr
- g = zarr.group()
- # etc.
- ```
-
-2. An explanation of why the current behaviour is wrong/not desired, and what you
- expect instead.
-
-3. Information about the version of Zarr, along with versions of dependencies and the
- Python interpreter, and installation information. The version of Zarr can be obtained
- from the ``zarr.__version__`` property. Please also state how Zarr was installed,
- e.g., "installed via pip into a virtual environment", or "installed using conda".
- Information about other packages installed can be obtained by executing ``pip freeze``
- (if using pip to install packages) or ``conda env export`` (if using conda to install
- packages) from the operating system command prompt. The version of the Python
- interpreter can be obtained by running a Python interactive session, e.g.::
-
- $ python
- Python 3.12.7 | packaged by conda-forge | (main, Oct 4 2024, 15:57:01) [Clang 17.0.6 ] on darwin
-
-Enhancement proposals
----------------------
-
-If you have an idea about a new feature or some other improvement to Zarr, please raise a
-`GitHub issue `_ first to discuss.
-
-We very much welcome ideas and suggestions for how to improve Zarr, but please bear in
-mind that we are likely to be conservative in accepting proposals for new features. The
-reasons for this are that we would like to keep the Zarr code base lean and focused on
-a core set of functionalities, and available time for development, review and maintenance
-of new features is limited. But if you have a great idea, please don't let that stop
-you from posting it on GitHub, just please don't be offended if we respond cautiously.
-
-Contributing code and/or documentation
---------------------------------------
-
-Forking the repository
-~~~~~~~~~~~~~~~~~~~~~~
-
-The Zarr source code is hosted on GitHub at the following location:
-
-* `https://github.com/zarr-developers/zarr-python `_
-
-You will need your own fork to work on the code. Go to the link above and hit
-the `"Fork" `_ button.
-Then clone your fork to your local machine::
-
- $ git clone git@github.com:your-user-name/zarr-python.git
- $ cd zarr-python
- $ git remote add upstream git@github.com:zarr-developers/zarr-python.git
-
-Creating a development environment
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To work with the Zarr source code, it is recommended to use
-`hatch `_ to create and manage development
-environments. Hatch will automatically install all Zarr dependencies using the same
-versions as are used by the core developers and continuous integration services.
-Assuming you have a Python 3 interpreter already installed, and you have cloned the
-Zarr source code and your current working directory is the root of the repository,
-you can do something like the following::
-
- $ pip install hatch
- $ hatch env show # list all available environments
-
-To verify that your development environment is working, you can run the unit tests
-for one of the test environments, e.g.::
-
- $ hatch env run --env test.py3.12-2.1-optional run
-
-Creating a branch
-~~~~~~~~~~~~~~~~~
-
-Before you do any new work or submit a pull request, please open an issue on GitHub to
-report the bug or propose the feature you'd like to add.
-
-It's best to synchronize your fork with the upstream repository, then create a
-new, separate branch for each piece of work you want to do. E.g.::
-
- git checkout main
- git fetch upstream
- git checkout -b shiny-new-feature upstream/main
- git push -u origin shiny-new-feature
-
-This changes your working directory to the 'shiny-new-feature' branch. Keep any changes in
-this branch specific to one bug or feature so it is clear what the branch brings to
-Zarr.
-
-To update this branch with latest code from Zarr, you can retrieve the changes from
-the main branch and perform a rebase::
-
- git fetch upstream
- git rebase upstream/main
-
-This will replay your commits on top of the latest Zarr git main. If this leads to
-merge conflicts, these need to be resolved before submitting a pull request.
-Alternatively, you can merge the changes in from upstream/main instead of rebasing,
-which can be simpler::
-
- git pull upstream main
-
-Again, any conflicts need to be resolved before submitting a pull request.
-
-Running the test suite
-~~~~~~~~~~~~~~~~~~~~~~
-
-Zarr includes a suite of unit tests. The simplest way to run the unit tests
-is to activate your development environment
-(see `creating a development environment`_ above) and invoke::
-
- $ hatch env run --env test.py3.12-2.1-optional run
-
-All tests are automatically run via GitHub Actions for every pull
-request and must pass before code can be accepted. Test coverage is
-also collected automatically via the Codecov service.
-
-.. note::
- Previous versions of Zarr-Python made extensive use of doctests. These tests were
- not maintained during the 3.0 refactor but may be brought back in the future.
- See :issue:`2614` for more details.
-
-Code standards - using pre-commit
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-All code must conform to the PEP8 standard. Regarding line length, lines up to 100
-characters are allowed, although please try to keep under 90 wherever possible.
-
-``Zarr`` uses a set of ``pre-commit`` hooks and the ``pre-commit`` bot to format,
-type-check, and prettify the codebase. ``pre-commit`` can be installed locally by
-running::
-
- $ python -m pip install pre-commit
-
-The hooks can be installed locally by running::
-
- $ pre-commit install
-
-This would run the checks every time a commit is created locally. These checks will also run
-on every commit pushed to an open PR, resulting in some automatic styling fixes by the
-``pre-commit`` bot. The checks will by default only run on the files modified by a commit,
-but the checks can be triggered for all the files by running::
-
- $ pre-commit run --all-files
-
-If you would like to skip the failing checks and push the code for further discussion, use
-the ``--no-verify`` option with ``git commit``.
-
-
-Test coverage
-~~~~~~~~~~~~~
-
-.. note::
- Test coverage for Zarr-Python 3 is currently not at 100%. This is a known issue and help
- is welcome to bring test coverage back to 100%. See :issue:`2613` for more details.
-
-Zarr strives to maintain 100% test coverage under the latest Python stable release
-Both unit tests and docstring doctests are included when computing coverage. Running::
-
- $ hatch env run --env test.py3.12-2.1-optional run-coverage
-
-will automatically run the test suite with coverage and produce a coverage report.
-This should be 100% before code can be accepted into the main code base.
-
-When submitting a pull request, coverage will also be collected across all supported
-Python versions via the Codecov service, and will be reported back within the pull
-request. Codecov coverage must also be 100% before code can be accepted.
-
-Documentation
-~~~~~~~~~~~~~
-
-Docstrings for user-facing classes and functions should follow the
-`numpydoc
-`_
-standard, including sections for Parameters and Examples. All examples
-should run and pass as doctests under Python 3.11.
-
-Zarr uses Sphinx for documentation, hosted on readthedocs.org. Documentation is
-written in the RestructuredText markup language (.rst files) in the ``docs`` folder.
-The documentation consists both of prose and API documentation. All user-facing classes
-and functions are included in the API documentation, under the ``docs/api`` folder
-using the `autodoc `_
-extension to sphinx. Any new features or important usage information should be included in the
-user-guide (``docs/user-guide``). Any changes should also be included in the release
-notes (``docs/developers/release.rst``).
-
-The documentation can be built locally by running::
-
- $ hatch --env docs run build
-
-The resulting built documentation will be available in the ``docs/_build/html`` folder.
-
-Hatch can also be used to serve continuously updating version of the documentation
-during development at `http://0.0.0.0:8000/ `_. This can be done by running::
-
- $ hatch --env docs run serve
-
-Development best practices, policies and procedures
----------------------------------------------------
-
-The following information is mainly for core developers, but may also be of interest to
-contributors.
-
-Merging pull requests
-~~~~~~~~~~~~~~~~~~~~~
-
-Pull requests submitted by an external contributor should be reviewed and approved by at least
-one core developers before being merged. Ideally, pull requests submitted by a core developer
-should be reviewed and approved by at least one other core developers before being merged.
-
-Pull requests should not be merged until all CI checks have passed (GitHub Actions
-Codecov) against code that has had the latest main merged in.
-
-Compatibility and versioning policies
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Because Zarr is a data storage library, there are two types of compatibility to
-consider: API compatibility and data format compatibility.
-
-API compatibility
-"""""""""""""""""
-
-All functions, classes and methods that are included in the API
-documentation (files under ``docs/api/*.rst``) are considered as part of the Zarr **public API**,
-except if they have been documented as an experimental feature, in which case they are part of
-the **experimental API**.
-
-Any change to the public API that does **not** break existing third party
-code importing Zarr, or cause third party code to behave in a different way, is a
-**backwards-compatible API change**. For example, adding a new function, class or method is usually
-a backwards-compatible change. However, removing a function, class or method; removing an argument
-to a function or method; adding a required argument to a function or method; or changing the
-behaviour of a function or method, are examples of **backwards-incompatible API changes**.
-
-If a release contains no changes to the public API (e.g., contains only bug fixes or
-other maintenance work), then the micro version number should be incremented (e.g.,
-2.2.0 -> 2.2.1). If a release contains public API changes, but all changes are
-backwards-compatible, then the minor version number should be incremented
-(e.g., 2.2.1 -> 2.3.0). If a release contains any backwards-incompatible public API changes,
-the major version number should be incremented (e.g., 2.3.0 -> 3.0.0).
-
-Backwards-incompatible changes to the experimental API can be included in a minor release,
-although this should be minimised if possible. I.e., it would be preferable to save up
-backwards-incompatible changes to the experimental API to be included in a major release, and to
-stabilise those features at the same time (i.e., move from experimental to public API), rather than
-frequently tinkering with the experimental API in minor releases.
-
-Data format compatibility
-"""""""""""""""""""""""""
-
-The data format used by Zarr is defined by a specification document, which should be
-platform-independent and contain sufficient detail to construct an interoperable
-software library to read and/or write Zarr data using any programming language. The
-latest version of the specification document is available on the
-`Zarr specifications website `_.
-
-Here, **data format compatibility** means that all software libraries that implement a
-particular version of the Zarr storage specification are interoperable, in the sense
-that data written by any one library can be read by all others. It is obviously
-desirable to maintain data format compatibility wherever possible. However, if a change
-is needed to the storage specification, and that change would break data format
-compatibility in any way, then the storage specification version number should be
-incremented (e.g., 2 -> 3).
-
-The versioning of the Zarr software library is related to the versioning of the storage
-specification as follows. A particular version of the Zarr library will
-implement a particular version of the storage specification. For example, Zarr version
-2.2.0 implements the Zarr storage specification version 2. If a release of the Zarr
-library implements a different version of the storage specification, then the major
-version number of the Zarr library should be incremented. E.g., if Zarr version 2.2.0
-implements the storage spec version 2, and the next release of the Zarr library
-implements storage spec version 3, then the next library release should have version
-number 3.0.0. Note however that the major version number of the Zarr library may not
-always correspond to the spec version number. For example, Zarr versions 2.x, 3.x, and
-4.x might all implement the same version of the storage spec and thus maintain data
-format compatibility, although they will not maintain API compatibility.
-
-When to make a release
-~~~~~~~~~~~~~~~~~~~~~~
-
-Ideally, any bug fixes that don't change the public API should be released as soon as
-possible. It is fine for a micro release to contain only a single bug fix.
-
-When to make a minor release is at the discretion of the core developers. There are no
-hard-and-fast rules, e.g., it is fine to make a minor release to make a single new
-feature available; equally, it is fine to make a minor release that includes a number of
-changes.
-
-Major releases obviously need to be given careful consideration, and should be done as
-infrequently as possible, as they will break existing code and/or affect data
-compatibility in some way.
-
-Release procedure
-~~~~~~~~~~~~~~~~~
-
-.. note::
-
- Most of the release process is now handled by GitHub workflow which should
- automatically push a release to PyPI if a tag is pushed.
-
-Before releasing, make sure that all pull requests which will be
-included in the release have been properly documented in
-`docs/release.rst`.
-
-To make a new release, go to
-https://github.com/zarr-developers/zarr-python/releases and
-click "Draft a new release". Choose a version number prefixed
-with a `v` (e.g. `v0.0.0`). For pre-releases, include the
-appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`).
-
-
-Set the description of the release to::
-
- See release notes https://zarr.readthedocs.io/en/stable/release.html#release-0-0-0
-
-replacing the correct version numbers. For pre-release versions,
-the URL should omit the pre-release suffix, e.g. "a1" or "rc1".
-
-Click on "Generate release notes" to auto-file the description.
-
-After creating the release, the documentation will be built on
-https://readthedocs.io. Full releases will be available under
-`/stable `_ while
-pre-releases will be available under
-`/latest `_.
-
-Also review and merge the https://github.com/conda-forge/zarr-feedstock
-pull request that will be automatically generated.
diff --git a/docs/developers/index.rst b/docs/developers/index.rst
deleted file mode 100644
index 3feb0aff71..0000000000
--- a/docs/developers/index.rst
+++ /dev/null
@@ -1,10 +0,0 @@
-
-Developer's Guide
------------------
-
-.. toctree::
- :maxdepth: 1
-
- contributing
- release
- roadmap
diff --git a/docs/developers/release.rst b/docs/developers/release.rst
deleted file mode 100644
index ce15c68f4a..0000000000
--- a/docs/developers/release.rst
+++ /dev/null
@@ -1,2334 +0,0 @@
-Release notes
-=============
-
-..
- # Copy the warning statement _under_ the latest release version
- # and unindent for pre-releases.
-
- .. warning::
- Pre-release! Use :command:`pip install --pre zarr` to evaluate this release.
-
-..
- # Unindent the section between releases in order
- # to document your changes. On releases it will be
- # re-indented so that it does not show up in the notes.
-
-.. note::
- Zarr-Python 2.18.* is expected be the final release in the 2.* series. Work on Zarr-Python 3.0 is underway.
- See `GH1777 `_ for more details on the upcoming
- 3.0 release.
-
-.. release_3.0.0-beta:
-
-3.0.0-beta series
------------------
-
-.. warning::
- Zarr-Python 3.0.0-beta is a pre-release of the upcoming 3.0 release. This release is not feature complete or
- expected to be ready for production applications.
-
-.. note::
- The complete release notes for 3.0 have not been added to this document yet. See the
- `3.0.0-beta `_ release on GitHub
- for a record of changes included in this release.
-
-Dependency Changes
-~~~~~~~~~~~~~~~~~~
-
-* fsspec was moved from a required dependency to an optional one. Users should install
- fsspec and any relevant implementations (e.g. s3fs) before using the ``RemoteStore``.
- By :user:`Joe Hamman ` :issue:`2391`.
-
-* ``RemoteStore`` was renamed to ``FsspecStore``.
- By :user:`Joe Hamman ` :issue:`2557`.
-
-.. release_3.0.0-alpha:
-
-3.0.0-alpha series
-------------------
-
-.. warning::
- Zarr-Python 3.0.0-alpha is a pre-release of the upcoming 3.0 release. This release is not feature complete or
- expected to be ready for production applications.
-
-.. note::
- The complete release notes for 3.0 have not been added to this document yet. See the
- `3.0.0-alpha `_ release on GitHub
- for a record of changes included in this release.
-
-Enhancements
-~~~~~~~~~~~~
-
-* Implement listing of the sub-arrays and sub-groups for a V3 ``Group``.
- By :user:`Davis Bennett ` :issue:`1726`.
-
-* Bootstrap v3 branch with zarrita.
- By :user:`Joe Hamman ` :issue:`1584`.
-
-* Extensible codecs for V3.
- By :user:`Norman Rzepka ` :issue:`1588`.
-
-* Don't import from tests.
- By :user:`Davis Bennett ` :issue:`1601`.
-
-* Listable V3 Stores.
- By :user:`Joe Hamman ` :issue:`1634`.
-
-* Codecs without array metadata.
- By :user:`Norman Rzepka ` :issue:`1632`.
-
-* fix sync group class methods.
- By :user:`Joe Hamman ` :issue:`1652`.
-
-* implement eq for LocalStore.
- By :user:`Charoula Kyriakides ` :issue:`1792`.
-
-* V3 reorg.
- By :user:`Joe Hamman ` :issue:`1809`.
-
-* [v3] Sync with futures.
- By :user:`Davis Bennett ` :issue:`1804`.
-
-* implement group.members.
- By :user:`Davis Bennett ` :issue:`1726`.
-
-* Remove implicit groups.
- By :user:`Joe Hamman ` :issue:`1827`.
-
-* feature(store): ``list_*`` -> AsyncGenerators.
- By :user:`Joe Hamman ` :issue:`1844`.
-
-* Test codec entrypoints.
- By :user:`Norman Rzepka ` :issue:`1835`.
-
-* Remove extra v3 sync module.
- By :user:`Max Jones ` :issue:`1856`.
-
-* Use donfig for V3 configuration.
- By :user:`Max Jones ` :issue:`1655`.
-
-* groundwork for V3 group tests.
- By :user:`Davis Bennett ` :issue:`1743`.
-
-* [v3] First step to generalizes ndarray and bytes.
- By :user:`Mads R. B. Kristensen ` :issue:`1826`.
-
-* Reworked codec pipelines.
- By :user:`Norman Rzepka ` :issue:`1670`.
-
-* Followup on codecs.
- By :user:`Norman Rzepka ` :issue:`1889`.
-
-* Protocols for Buffer and NDBuffer.
- By :user:`Mads R. B. Kristensen ` :issue:`1899`.
-
-* [V3] Expand store tests.
- By :user:`Davis Bennett ` :issue:`1900`.
-
-* [v3] Feature: Store open mode.
- By :user:`Joe Hamman ` :issue:`1911`.
-
-* fix(types): Group.info -> NotImplementedError.
- By :user:`Joe Hamman ` :issue:`1936`.
-
-* feature(typing): add py.typed file to package root.
- By :user:`Joe Hamman ` :issue:`1935`.
-
-* Support all indexing variants.
- By :user:`Norman Rzepka ` :issue:`1917`.
-
-* Feature: group and array name properties.
- By :user:`Joe Hamman ` :issue:`1940`.
-
-* implement .chunks on v3 arrays.
- By :user:`Ryan Abernathey ` :issue:`1929`.
-
-* Fixes bug in transpose.
- By :user:`Norman Rzepka ` :issue:`1949`.
-
-* Buffer Prototype Argument.
- By :user:`Mads R. B. Kristensen ` :issue:`1910`.
-
-* Feature: Top level V3 API.
- By :user:`Joe Hamman ` :issue:`1884`.
-
-* Basic working FsspecStore.
- By :user:`Martin Durant `; :issue:`1785`.
-
-Typing
-~~~~~~
-
-* Resolve Mypy errors in v3 branch.
- By :user:`Daniel Jahn ` :issue:`1692`.
-
-* Allow dmypy to be run on v3 branch.
- By :user:`David Stansby ` :issue:`1780`.
-
-* Remove unused typing ignore comments.
- By :user:`David Stansby ` :issue:`1781`.
-
-* Check untyped defs on v3.
- By :user:`David Stansby ` :issue:`1784`.
-
-* [v3] Enable some more strict mypy options.
- By :user:`David Stansby ` :issue:`1793`.
-
-* [v3] Disallow generic Any typing.
- By :user:`David Stansby ` :issue:`1794`.
-
-* Disallow incomplete type definitions.
- By :user:`David Stansby ` :issue:`1814`.
-
-* Disallow untyped calls.
- By :user:`David Stansby ` :issue:`1811`.
-
-* Fix some untyped calls.
- By :user:`David Stansby ` :issue:`1865`.
-
-* Disallow untyped defs.
- By :user:`David Stansby ` :issue:`1834`.
-
-* Add more typing to zarr.group.
- By :user:`David Stansby ` :issue:`1870`.
-
-* Fix any generics in zarr.array.
- By :user:`David Stansby ` :issue:`1861`.
-
-* Remove some unused mypy overrides.
- By :user:`David Stansby ` :issue:`1894`.
-
-* Finish typing zarr.metadata.
- By :user:`David Stansby ` :issue:`1880`.
-
-* Disallow implicit re-exports.
- By :user:`David Stansby ` :issue:`1908`.
-
-* Make typing strict.
- By :user:`David Stansby ` :issue:`1879`.
-
-* Enable extra mypy error codes.
- By :user:`David Stansby ` :issue:`1909`.
-
-* Enable warn_unreachable for mypy.
- By :user:`David Stansby ` :issue:`1937`.
-
-* Fix final typing errors.
- By :user:`David Stansby ` :issue:`1939`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Remedy a situation where ``zarr-python`` was importing ``DummyStorageTransformer`` from the test suite.
- The dependency relationship is now reversed: the test suite imports this class from ``zarr-python``.
- By :user:`Davis Bennett ` :issue:`1601`.
-
-* [V3] Update minimum supported Python and Numpy versions.
- By :user:`Joe Hamman ` :issue:`1638`
-
-* use src layout and use hatch for packaging.
- By :user:`Davis Bennett ` :issue:`1592`.
-
-* temporarily disable mypy in v3 directory.
- By :user:`Joe Hamman ` :issue:`1649`.
-
-* create hatch test env.
- By :user:`Ryan Abernathey ` :issue:`1650`.
-
-* removed unused environments and workflows.
- By :user:`Ryan Abernathey ` :issue:`1651`.
-
-* Add env variables to sprint setup instructions.
- By :user:`Max Jones ` :issue:`1654`.
-
-* Add test matrix for V3.
- By :user:`Max Jones ` :issue:`1656`.
-
-* Remove attrs.
- By :user:`Davis Bennett ` :issue:`1660`.
-
-* Specify hatch envs using GitHub actions matrix for v3 tests.
- By :user:`Max Jones ` :issue:`1728`.
-
-* black -> ruff format + cleanup.
- By :user:`Saransh Chopra ` :issue:`1639`.
-
-* Remove old v3.
- By :user:`Davis Bennett ` :issue:`1742`.
-
-* V3 update pre commit.
- By :user:`Joe Hamman ` :issue:`1808`.
-
-* remove windows testing on v3 branch.
- By :user:`Joe Hamman ` :issue:`1817`.
-
-* fix: add mypy to test dependencies.
- By :user:`Davis Bennett ` :issue:`1789`.
-
-* chore(ci): add numpy 2 release candidate to test matrix.
- By :user:`Joe Hamman ` :issue:`1828`.
-
-* fix dependencies.
- By :user:`Norman Rzepka ` :issue:`1840`.
-
-* Add pytest to mypy dependencies.
- By :user:`David Stansby ` :issue:`1846`.
-
-* chore(pre-commit): update pre-commit versions and remove attrs dep mypy section.
- By :user:`Joe Hamman ` :issue:`1848`.
-
-* Enable some ruff rules (RUF) and fix issues.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1869`.
-
-* Configure Ruff to apply flake8-bugbear/isort/pyupgrade.
- By :user:`Norman Rzepka ` :issue:`1890`.
-
-* chore(ci): remove mypy from test action in favor of pre-commit action.
- By :user:`Joe Hamman ` :issue:`1887`.
-
-* Enable ruff/flake8-raise rules (RSE) and fix issues.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1872`.
-
-* Apply assorted ruff/refurb rules (FURB).
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1873`.
-
-* Enable ruff/flake8-implicit-str-concat rules (ISC) and fix issues.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1868`.
-
-* Add numpy to mypy pre-commit check env.
- By :user:`David Stansby ` :issue:`1893`.
-
-* remove fixture files from src.
- By :user:`Davis Bennett ` :issue:`1897`.
-
-* Fix list of packages in mypy pre-commit environment.
- By :user:`David Stansby ` :issue:`1907`.
-
-* Run sphinx directly on readthedocs.
- By :user:`David Stansby ` :issue:`1919`.
-
-* Apply preview ruff rules.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1942`.
-
-* Enable and apply ruff rule RUF009.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1941`.
-
-Documentation
-~~~~~~~~~~~~~
-
-* Specify docs hatch env for v3 branch.
- By :user:`Max Jones ` :issue:`1655`.
-
-* Development installation/contributing docs updates.
- By :user:`Alden Keefe Sampson ` :issue:`1643`.
-
-* chore: update project settings per scientific python repo-review.
- By :user:`Joe Hamman ` :issue:`1863`.
-
-* doc: update release notes for 3.0.0.alpha.
- By :user:`Joe Hamman ` :issue:`1959`.
-
-.. _release_2.18.3:
-
-2.18.3
-------
-
-Enhancements
-~~~~~~~~~~~~
-* Added support for creating a copy of data when converting a `zarr.Array`
- to a numpy array.
- By :user:`David Stansby ` (:issue:`2106`) and
- :user:`Joe Hamman ` (:issue:`2123`).
-
-Maintenance
-~~~~~~~~~~~
-* Removed support for Python 3.9.
- By :user:`David Stansby ` (:issue:`2074`).
-
-* Fix a regression when using orthogonal indexing with a scalar.
- By :user:`Deepak Cherian ` :issue:`1931`
-
-* Added compatibility with NumPy 2.1.
- By :user:`David Stansby `
-
-* Bump minimum NumPy version to 1.24.
- :user:`Joe Hamman ` (:issue:`2127`).
-
-Deprecations
-~~~~~~~~~~~~
-
-* Deprecate :class:`zarr.n5.N5Store` and :class:`zarr.n5.N5FSStore`. These
- stores are slated to be removed in Zarr Python 3.0.
- By :user:`Joe Hamman ` :issue:`2085`.
-
-.. _release_2.18.2:
-
-2.18.2
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* Add Zstd codec to old V3 code path.
- By :user:`Ryan Abernathey `
-
-.. _release_2.18.1:
-
-2.18.1
-------
-
-Maintenance
-~~~~~~~~~~~
-* Fix a regression when getting or setting a single value from arrays with size-1 chunks.
- By :user:`Deepak Cherian ` :issue:`1874`
-
-.. _release_2.18.0:
-
-2.18.0
-------
-
-Enhancements
-~~~~~~~~~~~~
-* Performance improvement for reading and writing chunks if any of the dimensions is size 1.
- By :user:`Deepak Cherian ` :issue:`1730`.
-
-Maintenance
-~~~~~~~~~~~
-* Enable ruff/bugbear rules (B) and fix issues.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1702`.
-
-* Minor updates to use `np.inf` instead of `np.PINF` / `np.NINF` in preparation for NumPy 2.0.0 release.
- By :user:`Joe Hamman ` :issue:`1842`.
-
-Deprecations
-~~~~~~~~~~~~
-
-* Deprecate experimental v3 support by issuing a `FutureWarning`.
- Also updated docs to warn about using the experimental v3 version.
- By :user:`Joe Hamman ` :issue:`1802` and :issue:`1807`.
-
-* Deprecate the following stores: :class:`zarr.storage.DBMStore`, :class:`zarr.storage.LMDBStore`,
- :class:`zarr.storage.SQLiteStore`, :class:`zarr.storage.MongoDBStore`, :class:`zarr.storage.RedisStore`,
- and :class:`zarr.storage.ABSStore`. These stores are slated to be removed from Zarr-Python in version 3.0.
- By :user:`Joe Hamman ` :issue:`1801`.
-
-.. _release_2.17.2:
-
-2.17.2
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* [v3] Dramatically reduce number of ``__contains__`` requests in favor of optimistically calling `__getitem__`
- and handling any error that may arise.
- By :user:`Deepak Cherian ` :issue:`1741`.
-
-* [v3] Reuse the downloaded array metadata when creating an ``Array``.
- By :user:`Deepak Cherian ` :issue:`1734`.
-
-* Optimize ``Array.info`` so that it calls `getsize` only once.
- By :user:`Deepak Cherian ` :issue:`1733`.
-
-* Override IPython ``_repr_*_`` methods to avoid expensive lookups against object stores.
- By :user:`Deepak Cherian ` :issue:`1716`.
-
-* FSStore now raises rather than return bad data.
- By :user:`Martin Durant ` and :user:`Ian Carroll ` :issue:`1604`.
-
-* Avoid redundant ``__contains__``.
- By :user:`Deepak Cherian ` :issue:`1739`.
-
-Docs
-~~~~
-
-* Fix link to GCSMap in ``tutorial.rst``.
- By :user:`Daniel Jahn ` :issue:`1689`.
-
-* Endorse `SPEC0000 `_ and state version support policy in ``installation.rst``.
- By :user:`Sanket Verma ` :issue:`1665`.
-
-* Migrate v1 and v2 specification to `Zarr-Specs `_.
- By :user:`Sanket Verma ` :issue:`1582`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Add CI test environment for Python 3.12
- By :user:`Joe Hamman ` :issue:`1719`.
-
-* Bump minimum supported NumPy version to 1.23 (per spec 0000)
- By :user:`Joe Hamman ` :issue:`1719`.
-
-* Minor fixes: Using ``is`` instead of ``type`` and removing unnecessary ``None``.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1737`.
-
-* Fix tests failure related to Pytest 8.
- By :user:`David Stansby ` :issue:`1714`.
-
-.. _release_2.17.1:
-
-2.17.1
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* Change occurrences of % and format() to f-strings.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1423`.
-
-* Proper argument for numpy.reshape.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1425`.
-
-* Add typing to dimension separator arguments.
- By :user:`David Stansby ` :issue:`1620`.
-
-Docs
-~~~~
-
-* ZIP related tweaks.
- By :user:`Davis Bennett ` :issue:`1641`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Update config.yml with Zulip.
- By :user:`Josh Moore `.
-
-* Replace Gitter with the new Zulip Chat link.
- By :user:`Sanket Verma ` :issue:`1685`.
-
-* Fix RTD build.
- By :user:`Sanket Verma ` :issue:`1694`.
-
-.. _release_2.17.0:
-
-2.17.0
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* Added type hints to ``zarr.creation.create()``.
- By :user:`David Stansby ` :issue:`1536`.
-
-* Pyodide support: Don't require fasteners on Emscripten.
- By :user:`Hood Chatham ` :issue:`1663`.
-
-Docs
-~~~~
-
-* Minor correction and changes in documentation.
- By :user:`Sanket Verma ` :issue:`1509`.
-
-* Fix typo in documentation.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1554`
-
-* The documentation build now fails if there are any warnings.
- By :user:`David Stansby ` :issue:`1548`.
-
-* Add links to ``numcodecs`` docs in the tutorial.
- By :user:`David Stansby ` :issue:`1535`.
-
-* Enable offline formats for documentation builds.
- By :user:`Sanket Verma ` :issue:`1551`.
-
-* Minor tweak to advanced indexing tutorial examples.
- By :user:`Ross Barnowski ` :issue:`1550`.
-
-* Automatically document array members using sphinx-automodapi.
- By :user:`David Stansby ` :issue:`1547`.
-
-* Add a markdown file documenting the current and former core-developer team.
- By :user:`Joe Hamman ` :issue:`1628`.
-
-* Add Norman Rzepka to core-dev team.
- By :user:`Joe Hamman ` :issue:`1630`.
-
-* Added section about accessing ZIP archives on s3.
- By :user:`Jeff Peck ` :issue:`1613`, :issue:`1615`, and :user:`Davis Bennett ` :issue:`1641`.
-
-* Add V3 roadmap and design document.
- By :user:`Joe Hamman ` :issue:`1583`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Drop Python 3.8 and NumPy 1.20
- By :user:`Josh Moore `; :issue:`1557`.
-
-* Cache result of ``FSStore._fsspec_installed()``.
- By :user:`Janick Martinez Esturo ` :issue:`1581`.
-
-* Extend copyright notice to 2023.
- By :user:`Jack Kelly ` :issue:`1528`.
-
-* Change occurrence of ``io.open()`` into ``open()``.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1421`.
-
-* Preserve ``dimension_separator`` when resizing arrays.
- By :user:`Ziwen Liu ` :issue:`1533`.
-
-* Initialise some sets in tests with set literals instead of list literals.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1534`.
-
-* Allow ``black`` code formatter to be run with any Python version.
- By :user:`David Stansby ` :issue:`1549`.
-
-* Remove ``sphinx-rtd-theme`` dependency from ``pyproject.toml``.
- By :user:`Sanket Verma ` :issue:`1563`.
-
-* Remove ``CODE_OF_CONDUCT.md`` file from the Zarr-Python repository.
- By :user:`Sanket Verma ` :issue:`1572`.
-
-* Bump version of black in pre-commit.
- By :user:`David Stansby ` :issue:`1559`.
-
-* Use list comprehension where applicable.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1555`.
-
-* Use format specification mini-language to format string.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1558`.
-
-* Single startswith() call instead of multiple ones.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1556`.
-
-* Move codespell options around.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1196`.
-
-* Remove unused mypy ignore comments.
- By :user:`David Stansby ` :issue:`1602`.
-
-.. _release_2.16.1:
-
-2.16.1
-------
-
-Maintenance
-~~~~~~~~~~~
-
-* Require ``setuptools_scm`` version ``1.5.4``\+
- By :user:`John A. Kirkham ` :issue:`1477`.
-
-* Add ``docs`` requirements to ``pyproject.toml``
- By :user:`John A. Kirkham ` :issue:`1494`.
-
-* Fixed caching issue in ``LRUStoreCache``.
- By :user:`Mads R. B. Kristensen ` :issue:`1499`.
-
-.. _release_2.16.0:
-
-2.16.0
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* Allow for partial codec specification in V3 array metadata.
- By :user:`Joe Hamman ` :issue:`1443`.
-
-* Add ``__contains__`` method to ``KVStore``.
- By :user:`Christoph Gohlke ` :issue:`1454`.
-
-* **Block Indexing**: Implemented blockwise (chunk blocks) indexing to ``zarr.Array``.
- By :user:`Altay Sansal ` :issue:`1428`
-
-Maintenance
-~~~~~~~~~~~
-
-* Refactor the core array tests to reduce code duplication.
- By :user:`Davis Bennett ` :issue:`1462`.
-
-* Style the codebase with ``ruff`` and ``black``.
- By :user:`Davis Bennett ` :issue:`1459`
-
-* Ensure that chunks is tuple of ints upon array creation.
- By :user:`Philipp Hanslovsky ` :issue:`1461`
-
-.. _release_2.15.0:
-
-2.15.0
-------
-
-Enhancements
-~~~~~~~~~~~~
-
-* Implement more extensive fallback of getitem/setitem for orthogonal indexing.
- By :user:`Andreas Albert ` :issue:`1029`.
-
-* Getitems supports ``meta_array``.
- By :user:`Mads R. B. Kristensen ` :issue:`1131`.
-
-* ``open_array()`` now takes the ``meta_array`` argument.
- By :user:`Mads R. B. Kristensen ` :issue:`1396`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Remove ``codecov`` from GitHub actions.
- By :user:`John A. Kirkham ` :issue:`1391`.
-
-* Replace ``np.product`` with ``np.prod`` due to deprecation.
- By :user:`James Bourbeau ` :issue:`1405`.
-
-* Activate Py 3.11 builds.
- By :user:`Joe Hamman ` :issue:`1415`.
-
-Documentation
-~~~~~~~~~~~~~
-
-* Add API reference for V3 Implementation in the docs.
- By :user:`Sanket Verma ` :issue:`1345`.
-
-Bug fixes
-~~~~~~~~~
-
-* Fix the conda-forge error. Read :issue:`1347` for detailed info.
- By :user:`Josh Moore ` :issue:`1364` and :issue:`1367`.
-
-* Fix ``ReadOnlyError`` when opening V3 store via fsspec reference file system.
- By :user:`Joe Hamman ` :issue:`1383`.
-
-* Fix ``normalize_fill_value`` for structured arrays.
- By :user:`Alan Du ` :issue:`1397`.
-
-.. _release_2.14.2:
-
-2.14.2
-------
-
-Bug fixes
-~~~~~~~~~
-
-* Ensure ``zarr.group`` uses writeable mode to fix issue with :issue:`1304`.
- By :user:`Brandur Thorgrimsson ` :issue:`1354`.
-
-.. _release_2.14.1:
-
-2.14.1
-------
-
-Documentation
-~~~~~~~~~~~~~
-
-* Fix API links.
- By :user:`Josh Moore ` :issue:`1346`.
-
-* Fix unit tests which prevented the conda-forge release.
- By :user:`Josh Moore ` :issue:`1348`.
-
-.. _release_2.14.0:
-
-2.14.0
-------
-
-Major changes
-~~~~~~~~~~~~~
-
-* Improve Zarr V3 support, adding partial store read/write and storage transformers.
- Add new features from the `v3 spec `_:
-
- * storage transformers
- * `get_partial_values` and `set_partial_values`
- * efficient `get_partial_values` implementation for `FSStoreV3`
- * sharding storage transformer
-
- By :user:`Jonathan Striebel `; :issue:`1096`, :issue:`1111`.
-
-* N5 nows supports Blosc.
- Remove warnings emitted when using N5Store or N5FSStore with a blosc-compressed array.
- By :user:`Davis Bennett `; :issue:`1331`.
-
-Bug fixes
-~~~~~~~~~
-
-* Allow reading utf-8 encoded json files
- By :user:`Nathan Zimmerberg ` :issue:`1308`.
-
-* Ensure contiguous data is give to ``FSStore``. Only copying if needed.
- By :user:`Mads R. B. Kristensen ` :issue:`1285`.
-
-* NestedDirectoryStore.listdir now returns chunk keys with the correct '/' dimension_separator.
- By :user:`Brett Graham ` :issue:`1334`.
-
-* N5Store/N5FSStore dtype returns zarr Stores readable dtype.
- By :user:`Marwan Zouinkhi ` :issue:`1339`.
-
-.. _release_2.13.6:
-
-2.13.6
-------
-
-Maintenance
-~~~~~~~~~~~
-
-* Bump gh-action-pypi-publish to 1.6.4.
- By :user:`Josh Moore ` :issue:`1320`.
-
-.. _release_2.13.5:
-
-2.13.5
-------
-
-Bug fixes
-~~~~~~~~~
-
-* Ensure ``zarr.create`` uses writeable mode to fix issue with :issue:`1304`.
- By :user:`James Bourbeau ` :issue:`1309`.
-
-.. _release_2.13.4:
-
-2.13.4
-------
-
-Appreciation
-~~~~~~~~~~~~~
-
-Special thanks to Outreachy participants for contributing to most of the
-maintenance PRs. Please read the blog post summarising the contribution phase
-and welcoming new Outreachy interns:
-https://zarr.dev/blog/welcoming-outreachy-2022-interns/
-
-
-Enhancements
-~~~~~~~~~~~~
-
-* Handle fsspec.FSMap using FSStore store.
- By :user:`Rafal Wojdyla ` :issue:`1304`.
-
-Bug fixes
-~~~~~~~~~
-
-* Fix bug that caused double counting of groups in ``groups()`` and ``group_keys()`` methods with V3 stores.
- By :user:`Ryan Abernathey ` :issue:`1228`.
-
-* Remove unnecessary calling of `contains_array` for key that ended in `.array.json`.
- By :user:`Joe Hamman ` :issue:`1149`.
-
-* Fix bug that caused double counting of groups in ``groups()`` and ``group_keys()``
- methods with V3 stores.
- By :user:`Ryan Abernathey ` :issue:`1228`.
-
-Documentation
-~~~~~~~~~~~~~
-
-* Fix minor indexing errors in tutorial and specification examples of documentation.
- By :user:`Kola Babalola ` :issue:`1277`.
-
-* Add `requirements_rtfd.txt` in `contributing.rst`.
- By :user:`AWA BRANDON AWA ` :issue:`1243`.
-
-* Add documentation for find/findall using visit.
- By :user:`Weddy Gikunda ` :issue:`1241`.
-
-* Refresh of the main landing page.
- By :user:`Josh Moore ` :issue:`1173`.
-
-Maintenance
-~~~~~~~~~~~
-
-* Migrate to ``pyproject.toml`` and remove redundant infrastructure.
- By :user:`Saransh Chopra ` :issue:`1158`.
-
-* Require ``setuptools`` 64.0.0+
- By :user:`Saransh Chopra ` :issue:`1193`.
-
-* Pin action versions (pypi-publish, setup-miniconda) for dependabot
- By :user:`Saransh Chopra ` :issue:`1205`.
-
-* Remove ``tox`` support
- By :user:`Saransh Chopra ` :issue:`1219`.
-
-* Add workflow to label PRs with "needs release notes".
- By :user:`Saransh Chopra ` :issue:`1239`.
-
-* Simplify if/else statement.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1227`.
-
-* Get coverage up to 100%.
- By :user:`John Kirkham ` :issue:`1264`.
-
-* Migrate coverage to ``pyproject.toml``.
- By :user:`John Kirkham ` :issue:`1250`.
-
-* Use ``conda-incubator/setup-miniconda@v2.2.0``.
- By :user:`John Kirkham ` :issue:`1263`.
-
-* Delete unused files.
- By :user:`John Kirkham ` :issue:`1251`.
-
-* Skip labeller for bot PRs.
- By :user:`Saransh Chopra ` :issue:`1271`.
-
-* Restore Flake8 configuration.
- By :user:`John Kirkham ` :issue:`1249`.
-
-* Add missing newline at EOF.
- By :user:`Dimitri Papadopoulos` :issue:`1253`.
-
-* Add `license_files` to `pyproject.toml`.
- By :user:`John Kirkham ` :issue:`1247`.
-
-* Adding `pyupgrade` suggestions.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1225`.
-
-* Fixed some linting errors.
- By :user:`Weddy Gikunda ` :issue:`1226`.
-
-* Added the link to main website in readthedocs sidebar.
- By :user:`Stephanie_nkwatoh ` :issue:`1216`.
-
-* Remove redundant wheel dependency in `pyproject.toml`.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1233`.
-
-* Turned on `isloated_build` in `tox.ini` file.
- By :user:`AWA BRANDON AWA ` :issue:`1210`.
-
-* Fixed `flake8` alert and avoid duplication of `Zarr Developers`.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1203`.
-
-* Bump to NumPy 1.20+ in `environment.yml`.
- By :user:`John Kirkham ` :issue:`1201`.
-
-* Bump to NumPy 1.20 in `pyproject.toml`.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1192`.
-
-* Remove LGTM (`.lgtm.yml`) configuration file.
- By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1191`.
-
-* Codespell will skip `fixture` in pre-commit.
- By :user:`Dimitri Papadopoulos Orfanos