>> chunk
- array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
- 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
- 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
- 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
- 1, 1, 1, 1, 1, 1, 1, 1], dtype=int32)
-
-Modify the array attributes::
-
- >>> a.attrs['foo'] = 42
- >>> a.attrs['bar'] = 'apples'
- >>> a.attrs['baz'] = [1, 2, 3, 4]
- >>> sorted(os.listdir('data/example.zarr'))
- ['.zarray', '.zattrs', '0.0', '0.1', '1.0', '1.1']
- >>> print(open('data/example.zarr/.zattrs').read())
- {
- "bar": "apples",
- "baz": [
- 1,
- 2,
- 3,
- 4
- ],
- "foo": 42
- }
-
-Storing multiple arrays in a hierarchy
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Below is an example of storing multiple Zarr arrays organized into a group
-hierarchy, using a directory on the local file system as storage. This storage
-implementation maps logical paths onto directory paths on the file system,
-however this is an implementation choice and is not required.
-
-Setup the store::
-
- >>> import zarr
- >>> store = zarr.DirectoryStore('data/group.zarr')
-
-Create the root group::
-
- >>> root_grp = zarr.group(store, overwrite=True)
-
-The metadata resource for the root group has been created::
-
- >>> import os
- >>> sorted(os.listdir('data/group.zarr'))
- ['.zgroup']
-
-Inspect the group metadata::
-
- >>> print(open('data/group.zarr/.zgroup').read())
- {
- "zarr_format": 2
- }
-
-Create a sub-group::
-
- >>> sub_grp = root_grp.create_group('foo')
-
-What has been stored::
-
- >>> sorted(os.listdir('data/group.zarr'))
- ['.zgroup', 'foo']
- >>> sorted(os.listdir('data/group.zarr/foo'))
- ['.zgroup']
-
-Create an array within the sub-group::
-
- >>> a = sub_grp.create_dataset('bar', shape=(20, 20), chunks=(10, 10))
- >>> a[:] = 42
-
-Set a custom attributes::
-
- >>> a.attrs['comment'] = 'answer to life, the universe and everything'
-
-What has been stored::
-
- >>> sorted(os.listdir('data/group.zarr'))
- ['.zgroup', 'foo']
- >>> sorted(os.listdir('data/group.zarr/foo'))
- ['.zgroup', 'bar']
- >>> sorted(os.listdir('data/group.zarr/foo/bar'))
- ['.zarray', '.zattrs', '0.0', '0.1', '1.0', '1.1']
-
-Here is the same example using a Zip file as storage::
-
- >>> store = zarr.ZipStore('data/group.zip', mode='w')
- >>> root_grp = zarr.group(store)
- >>> sub_grp = root_grp.create_group('foo')
- >>> a = sub_grp.create_dataset('bar', shape=(20, 20), chunks=(10, 10))
- >>> a[:] = 42
- >>> a.attrs['comment'] = 'answer to life, the universe and everything'
- >>> store.close()
-
-What has been stored::
-
- >>> import zipfile
- >>> zf = zipfile.ZipFile('data/group.zip', mode='r')
- >>> for name in sorted(zf.namelist()):
- ... print(name)
- .zgroup
- foo/.zgroup
- foo/bar/.zarray
- foo/bar/.zattrs
- foo/bar/0.0
- foo/bar/0.1
- foo/bar/1.0
- foo/bar/1.1
-
-.. _spec_v2_changes:
-
-Changes
--------
-
-Version 2 clarifications
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-The following changes have been made to the version 2 specification since it was
-initially published to clarify ambiguities and add some missing information.
-
-* The specification now describes how bytes fill values should be encoded and
- decoded for arrays with a fixed-length byte string data type (:issue:`165`,
- :issue:`176`).
-
-* The specification now clarifies that units must be specified for datetime64 and
- timedelta64 data types (:issue:`85`, :issue:`215`).
-
-* The specification now clarifies that the '.zattrs' key does not have to be present for
- either arrays or groups, and if absent then custom attributes should be treated as
- empty.
-
-* The specification now describes how structured datatypes with
- subarray shapes and/or with nested structured data types are encoded
- in array metadata (:issue:`111`, :issue:`296`).
-
-* Clarified the key/value pairs of custom attributes as "arbitrary" rather than
- "simple".
-
-Changes from version 1 to version 2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The following changes were made between version 1 and version 2 of this specification:
-
-* Added support for storing multiple arrays in the same store and organising
- arrays into hierarchies using groups.
-* Array metadata is now stored under the ".zarray" key instead of the "meta"
- key.
-* Custom attributes are now stored under the ".zattrs" key instead of the
- "attrs" key.
-* Added support for filters.
-* Changed encoding of "fill_value" field within array metadata.
-* Changed encoding of compressor information within array metadata to be
- consistent with representation of filter information.
diff --git a/docs/subprojects.md b/docs/subprojects.md
new file mode 100644
index 0000000000..9f7951e836
--- /dev/null
+++ b/docs/subprojects.md
@@ -0,0 +1,45 @@
+# Subprojects
+
+Alongside `zarr` itself, the
+[zarr-python repository](https://github.com/zarr-developers/zarr-python) hosts a
+small number of companion packages. Each one is developed in the same repository
+but versioned, released, and documented independently, so you can depend on it
+without taking on `zarr` as a dependency.
+
+
+
+- [:material-code-json:{ .lg .middle } __zarr-metadata__](https://zarr.readthedocs.io/projects/zarr-metadata/)
+
+ ---
+
+ Spec-defined metadata types, models, and validators for Zarr v2 and v3, with
+ minimal dependencies. Useful if your software reads or writes Zarr metadata
+ documents but does not need a full Zarr implementation.
+
+ ```bash
+ pip install zarr-metadata
+ ```
+
+- [:material-vector-polyline:{ .lg .middle } __zarr-indexing__](https://zarr.readthedocs.io/projects/zarr-indexing/)
+
+ ---
+
+ Composable, lazy coordinate transforms for Zarr array indexing. Makes the
+ mapping from requested coordinates to stored coordinates a first-class,
+ composable value, and resolves which chunks a selection touches.
+
+ ```bash
+ pip install zarr-indexing
+ ```
+
+- [:material-server:{ .lg .middle } __zarr-http-server__](https://zarr.readthedocs.io/projects/zarr-http-server/)
+
+ ---
+
+ HTTP server for Zarr stores, arrays, and groups.
+
+ ```bash
+ pip install zarr-http-server
+ ```
+
+
diff --git a/docs/talks/scipy2019/submission.rst b/docs/talks/scipy2019/submission.rst
deleted file mode 100644
index a7d531c91e..0000000000
--- a/docs/talks/scipy2019/submission.rst
+++ /dev/null
@@ -1,144 +0,0 @@
-Zarr - scalable storage of tensor data for use in parallel and distributed computing
-====================================================================================
-
-SciPy 2019 submission.
-
-
-Short summary
--------------
-
-Many scientific problems involve computing over large N-dimensional
-typed arrays of data, and reading or writing data is often the major
-bottleneck limiting speed or scalability. The Zarr project is
-developing a simple, scalable approach to storage of such data in a
-way that is compatible with a range of approaches to distributed and
-parallel computing. We describe the Zarr protocol and data storage
-format, and the current state of implementations for various
-programming languages including Python. We also describe current uses
-of Zarr in malaria genomics, the Human Cell Atlas, and the Pangeo
-project.
-
-
-Abstract
---------
-
-Background
-~~~~~~~~~~
-
-Across a broad range of scientific disciplines, data are naturally
-represented and stored as N-dimensional typed arrays, also known as
-tensors. The volume of data being generated is outstripping our
-ability to analyse it, and scientific communities are looking for ways
-to leverage modern multi-core CPUs and distributed computing
-platforms, including cloud computing. Retrieval and storage of data is
-often the major bottleneck, and new approaches to data storage are
-needed to accelerate distributed computations and enable them to scale
-on a variety of platforms.
-
-Methods
-~~~~~~~
-
-We have designed a new storage format and protocol for tensor data
-[1_], and have released an open source Python implementation [2_,
-3_]. Our approach builds on data storage concepts from HDF5 [4_],
-particularly chunking and compression, and hierarchical organisation
-of datasets. Key design goals include: a simple protocol and format
-that can be implemented in other programming languages; support for
-multiple concurrent readers or writers; support for a variety of
-parallel computing environments, from multi-threaded execution on a
-single CPU to multi-process execution across a multi-node cluster;
-pluggable storage subsystem with support for file systems, key-value
-databases and cloud object stores; pluggable encoding subsystem with
-support for a variety of modern compressors.
-
-Results
-~~~~~~~
-
-We illustrate the use of Zarr with examples from several scientific
-domains. Zarr is being used within the Pangeo project [5_], which is
-building a community platform for big data geoscience. The Pangeo
-community have converted a number of existing climate modelling and
-satellite observation datasets to Zarr [6_], and have demonstrated
-their use in computations using HPC and cloud computing
-environments. Within the MalariaGEN project [7_], Zarr is used to
-store genome variation data from next-generation sequencing of natural
-populations of malaria parasites and mosquitoes [8_] and these data
-are used as input to analyses of the evolution of these organisms in
-response to selective pressure from anti-malarial drugs and
-insecticides. Zarr is being used within the Human Cell Atlas (HCA)
-project [9_], which is building a reference atlas of healthy human
-cell types. This project hopes to leverage this information to better
-understand the dysregulation of cellular states that underly human
-disease. The Human Cell Atlas uses Zarr as the output data format
-because it enables the project to easily generate matrices containing
-user-selected subsets of cells.
-
-Conclusions
-~~~~~~~~~~~
-
-Zarr is generating interest across a range of scientific domains, and
-work is ongoing to establish a community process to support further
-development of the specifications and implementations in other
-programming languages [10_, 11_, 12_] and building interoperability
-with a similar project called N5 [13_]. Other packages within the
-PyData ecosystem, notably Dask [14_], Xarray [15_] and Intake [16_],
-have added capability to read and write Zarr, and together these
-packages provide a compelling solution for large scale data science
-using Python [17_]. Zarr has recently been presented in several
-venues, including a webinar for the ESIP Federation tech dive series
-[18_], and a talk at the AGU Fall Meeting 2018 [19_].
-
-
-References
-~~~~~~~~~~
-
-.. _1: https://zarr.readthedocs.io/en/stable/spec/v2.html
-.. _2: https://github.com/zarr-developers/zarr-python
-.. _3: https://github.com/zarr-developers/numcodecs
-.. _4: https://www.hdfgroup.org/solutions/hdf5/
-.. _5: https://pangeo.io/
-.. _6: https://pangeo.io/catalog.html
-.. _7: https://www.malariagen.net/
-.. _8: http://alimanfoo.github.io/2016/09/21/genotype-compression-benchmark.html
-.. _9: https://www.humancellatlas.org/
-.. _10: https://github.com/constantinpape/z5
-.. _11: https://github.com/lasersonlab/ndarray.scala
-.. _12: https://github.com/meggart/ZarrNative.jl
-.. _13: https://github.com/saalfeldlab/n5
-.. _14: http://docs.dask.org/en/latest/array-creation.html
-.. _15: http://xarray.pydata.org/en/stable/io.html
-.. _16: https://github.com/ContinuumIO/intake-xarray
-.. _17: http://matthewrocklin.com/blog/work/2018/01/22/pangeo-2
-.. _18: http://wiki.esipfed.org/index.php/Interoperability_and_Technology/Tech_Dive_Webinar_Series#8_March.2C_2018:_.22Zarr:_A_simple.2C_open.2C_scalable_solution_for_big_NetCDF.2FHDF_data_on_the_Cloud.22:_Alistair_Miles.2C_University_of_Oxford.
-.. _19: https://agu.confex.com/agu/fm18/meetingapp.cgi/Paper/390015
-
-
-Authors
--------
-
-Project contributors are listed in alphabetical order by surname.
-
-* `Ryan Abernathey `_, Columbia University
-* `Stephan Balmer `_, Meteotest
-* `Ambrose Carr `_, Chan Zuckerberg Initiative
-* `Tim Crone `_, Columbia University
-* `Martin Durant `_, Anaconda, inc.
-* `Jan Funke