zarr.codecs#

Classes#

`BloscCname`	Enum for compression library used by blosc.
`BloscCodec`	blosc codec
`BloscShuffle`	Enum for shuffle filter used by blosc.
`BytesCodec`	bytes codec
`Crc32cCodec`	crc32c codec
`Endian`	Enum for endian type used by bytes codec.
`GzipCodec`	gzip codec
`ShardingCodec`	Sharding codec
`ShardingCodecIndexLocation`	Enum for index location used by the sharding codec.
`TransposeCodec`	Transpose codec
`VLenBytesCodec`	Base class for array-to-bytes codecs.
`VLenUTF8Codec`	Variable-length UTF8 codec
`ZstdCodec`	zstd codec

Package Contents#

class zarr.codecs.BloscCname(*args, **kwds)[source]#

Bases: enum.Enum

Enum for compression library used by blosc.

blosclz = 'blosclz'#

lz4 = 'lz4'#

lz4hc = 'lz4hc'#

snappy = 'snappy'#

zlib = 'zlib'#

zstd = 'zstd'#

class zarr.codecs.BloscCodec( *, typesize: int | None = None, cname: BloscCname | str = BloscCname.zstd, clevel: int = 5, shuffle: BloscShuffle | str | None = None, blocksize: int = 0, )[source]#

Bases: zarr.abc.codec.BytesBytesCodec

blosc codec

abstractmethod compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

blocksize: int = 0#

clevel: int = 5#

cname: BloscCname#

is_fixed_size = False#

shuffle: BloscShuffle | None#

typesize: int | None#

class zarr.codecs.BloscShuffle(*args, **kwds)[source]#

Bases: enum.Enum

Enum for shuffle filter used by blosc.

classmethod from_int(num: int) → BloscShuffle[source]#

bitshuffle = 'bitshuffle'#

noshuffle = 'noshuffle'#

shuffle = 'shuffle'#

class zarr.codecs.BytesCodec(*, endian: Endian | str | None = default_system_endian)[source]#

Bases: zarr.abc.codec.ArrayBytesCodec

bytes codec

compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

endian: Endian | None#

is_fixed_size = True#

class zarr.codecs.Crc32cCodec[source]#

Bases: zarr.abc.codec.BytesBytesCodec

crc32c codec

compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

is_fixed_size = True#

class zarr.codecs.Endian(*args, **kwds)[source]#

Bases: enum.Enum

Enum for endian type used by bytes codec.

big = 'big'#

little = 'little'#

class zarr.codecs.GzipCodec(*, level: int = 5)[source]#

Bases: zarr.abc.codec.BytesBytesCodec

gzip codec

abstractmethod compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

is_fixed_size = False#

level: int = 5#

class zarr.codecs.ShardingCodec( *, chunk_shape: zarr.core.common.ShapeLike, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] = (BytesCodec(),), index_codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] = (BytesCodec(), Crc32cCodec()), index_location: ShardingCodecIndexLocation | str = ShardingCodecIndexLocation.end, )[source]#

Bases: zarr.abc.codec.ArrayBytesCodec, zarr.abc.codec.ArrayBytesCodecPartialDecodeMixin, zarr.abc.codec.ArrayBytesCodecPartialEncodeMixin

Sharding codec

compute_encoded_size( input_byte_length: int, shard_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async decode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteGetter, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[zarr.core.buffer.NDBuffer | None]#

Partially decodes a batch of chunks. This method determines parts of a chunk from the slice selection, fetches these parts from the store (via ByteGetter) and decodes them.

Parameters:

batch_infoIterable[tuple[ByteGetter, SelectorTuple, ArraySpec]]: Ordered set of information about slices of encoded chunks. The slice selection determines which parts of the chunk will be fetched. The ByteGetter is used to fetch the necessary bytes. The chunk spec contains information about the construction of an array from the bytes.

Returns:

Iterable[NDBuffer | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

async encode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteSetter, zarr.core.buffer.NDBuffer, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) → None#

Partially encodes a batch of chunks. This method determines parts of a chunk from the slice selection, encodes them and writes these parts to the store (via ByteSetter). If merging with existing chunk data in the store is necessary, this method will read from the store first and perform the merge.

Parameters:

batch_infoIterable[tuple[ByteSetter, NDBuffer, SelectorTuple, ArraySpec]]: Ordered set of information about slices of to-be-encoded chunks. The slice selection determines which parts of the chunk will be encoded. The ByteSetter is used to write the necessary bytes and fetch bytes for existing chunk data. The chunk spec contains information about the chunk.

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None[source]#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

chunk_shape: tuple[int, Ellipsis]#

property codec_pipeline: zarr.abc.codec.CodecPipeline#

codecs: tuple[zarr.abc.codec.Codec, Ellipsis]#

index_codecs: tuple[zarr.abc.codec.Codec, Ellipsis]#

index_location: ShardingCodecIndexLocation#

is_fixed_size: bool#

class zarr.codecs.ShardingCodecIndexLocation(*args, **kwds)[source]#

Bases: enum.Enum

Enum for index location used by the sharding codec.

end = 'end'#

start = 'start'#

class zarr.codecs.TransposeCodec(*, order: collections.abc.Iterable[int])[source]#

Bases: zarr.abc.codec.ArrayArrayCodec

Transpose codec

compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec[source]#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None[source]#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

is_fixed_size = True#

order: tuple[int, Ellipsis]#

class zarr.codecs.VLenBytesCodec[source]#

Bases: zarr.abc.codec.ArrayBytesCodec

Base class for array-to-bytes codecs.

abstractmethod compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

is_fixed_size: bool#

class zarr.codecs.VLenUTF8Codec[source]#

Bases: zarr.abc.codec.ArrayBytesCodec

Variable-length UTF8 codec

abstractmethod compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self[source]#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

is_fixed_size: bool#

class zarr.codecs.ZstdCodec(*, level: int = 0, checksum: bool = False)[source]#

Bases: zarr.abc.codec.BytesBytesCodec

zstd codec

abstractmethod compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) → int[source]#

Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors).

Parameters:

input_byte_lengthint
chunk_specArraySpec

Returns:

int

async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecInput | None]#

Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]]: Ordered set of encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecInput | None]

async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) → collections.abc.Iterable[CodecOutput | None]#

Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec.

Parameters:

chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]]: Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns:

Iterable[CodecOutput | None]

evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) → Self#

Fills in codec configuration parameters that can be automatically inferred from the array metadata.

Parameters:

array_specArraySpec

Returns:

Self

classmethod from_dict(data: dict[str, zarr.core.common.JSON]) → Self[source]#: Create an instance of the model from a dictionary

resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) → zarr.core.array_spec.ArraySpec#

Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline.

Parameters:

chunk_specArraySpec

Returns:

ArraySpec

to_dict() → dict[str, zarr.core.common.JSON][source]#: Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list.

validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) → None#

Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible.

Parameters:

shapetuple[int, …]: The array shape
dtypenp.dtype[Any]: The array data type
chunk_gridChunkGrid: The array chunk grid

checksum: bool = False#

is_fixed_size = True#

level: int = 0#