zarr.testing.buffer
===================

.. py:module:: zarr.testing.buffer


Classes
-------

.. autoapisummary::

   zarr.testing.buffer.NDBufferUsingTestNDArrayLike
   zarr.testing.buffer.StoreExpectingTestBuffer
   zarr.testing.buffer.TestBuffer


Module Contents
---------------

.. py:class:: NDBufferUsingTestNDArrayLike(array: zarr.core.buffer.core.NDArrayLike)

   Bases: :py:obj:`zarr.core.buffer.cpu.NDBuffer`


   
   Example of a custom NDBuffer that handles MyNDArrayLike
















   ..
       !! processed by numpydoc !!

   .. py:method:: all_equal(other: Any, equal_nan: bool = True) -> bool

      
      Compare to `other` using np.array_equal.
















      ..
          !! processed by numpydoc !!


   .. py:method:: as_ndarray_like() -> NDArrayLike

      
      Returns the underlying array (host or device memory) of this buffer

      This will never copy data.




      :Returns:

          The underlying array such as a NumPy or CuPy array.
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: as_numpy_array() -> numpy.typing.NDArray[Any]

      
      Returns the buffer as a NumPy array (host memory).





      :Returns:

          NumPy array of this buffer (might be a data copy)
              ..






      .. warning::

          Might have to copy data, consider using `.as_ndarray_like()` instead.





      ..
          !! processed by numpydoc !!


   .. py:method:: as_scalar() -> ScalarType

      
      Returns the buffer as a scalar value
















      ..
          !! processed by numpydoc !!


   .. py:method:: astype(dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = 'K') -> Self


   .. py:method:: copy() -> Self


   .. py:method:: create(*, shape: collections.abc.Iterable[int], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', fill_value: Any | None = None) -> Self
      :classmethod:


      
      Overwrite `NDBuffer.create` to create an TestNDArrayLike instance
















      ..
          !! processed by numpydoc !!


   .. py:method:: empty(shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C') -> Self
      :classmethod:


      
      Create an empty buffer with the given shape, dtype, and order.

      This method can be faster than ``NDBuffer.create`` because it doesn't
      have to initialize the memory used by the underlying ndarray-like
      object.

      :Parameters:

          **shape**
              The shape of the buffer and its underlying ndarray-like object

          **dtype**
              The datatype of the buffer and its underlying ndarray-like object

          **order**
              Whether to store multi-dimensional data in row-major (C-style) or
              column-major (Fortran-style) order in memory.



      :Returns:

          buffer
              New buffer representing a new ndarray_like object with empty data.







      .. seealso::

          
          :obj:`NDBuffer.create`
              Create a new buffer with some initial fill value.
          
          



      ..
          !! processed by numpydoc !!


   .. py:method:: fill(value: Any) -> None


   .. py:method:: from_ndarray_like(ndarray_like: NDArrayLike) -> Self
      :classmethod:


      
      Create a new buffer of a ndarray-like object


      :Parameters:

          **ndarray_like**
              ndarray-like object



      :Returns:

          New buffer representing `ndarray_like`
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: from_numpy_array(array_like: numpy.typing.ArrayLike) -> Self
      :classmethod:


      
      Create a new buffer of Numpy array-like object


      :Parameters:

          **array_like**
              Object that can be coerced into a Numpy array



      :Returns:

          New buffer representing `array_like`
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: reshape(newshape: tuple[int, Ellipsis] | Literal[-1]) -> Self


   .. py:method:: squeeze(axis: tuple[int, Ellipsis]) -> Self


   .. py:method:: transpose(axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None) -> Self


   .. py:property:: byteorder
      :type: zarr.codecs.bytes.Endian



   .. py:property:: dtype
      :type: numpy.dtype[Any]



   .. py:property:: shape
      :type: tuple[int, Ellipsis]



.. py:class:: StoreExpectingTestBuffer(store_dict: collections.abc.MutableMapping[str, zarr.core.buffer.Buffer] | None = None, *, read_only: bool = False)

   Bases: :py:obj:`zarr.storage.MemoryStore`


   
   Example of a custom Store that expect MyBuffer for all its non-metadata

   We assume that keys containing "json" is metadata















   ..
       !! processed by numpydoc !!

   .. py:method:: clear() -> None
      :async:


      
      Clear the store.

      Remove all keys and values from the store.















      ..
          !! processed by numpydoc !!


   .. py:method:: close() -> None

      
      Close the store.
















      ..
          !! processed by numpydoc !!


   .. py:method:: delete(key: str) -> None
      :async:


      
      Remove a key from the store


      :Parameters:

          **key** : str
              ..














      ..
          !! processed by numpydoc !!


   .. py:method:: delete_dir(prefix: str) -> None
      :async:


      
      Remove all keys and prefixes in the store that begin with a given prefix.
















      ..
          !! processed by numpydoc !!


   .. py:method:: exists(key: str) -> bool
      :async:


      
      Check if a key exists in the store.


      :Parameters:

          **key** : str
              ..



      :Returns:

          bool
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: get(key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: tuple[int, int | None] | None = None) -> zarr.core.buffer.Buffer | None
      :async:


      
      Retrieve the value associated with a given key.


      :Parameters:

          **key** : str
              ..

          **prototype** : BufferPrototype
              The prototype of the output buffer. Stores may support a default buffer prototype.

          **byte_range** : ByteRequest, optional
              ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved.
              - RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned.
              - OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header.
              - SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header.



      :Returns:

          Buffer
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: get_partial_values(prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]]) -> list[zarr.core.buffer.Buffer | None]
      :async:


      
      Retrieve possibly partial values from given key_ranges.


      :Parameters:

          **prototype** : BufferPrototype
              The prototype of the output buffer. Stores may support a default buffer prototype.

          **key_ranges** : Iterable[tuple[str, tuple[int | None, int | None]]]
              Ordered set of key, range pairs, a key may occur multiple times with different ranges



      :Returns:

          list of values, in the order of the key_ranges, may contain null/none for missing keys
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: getsize(key: str) -> int
      :async:


      
      Return the size, in bytes, of a value in a Store.


      :Parameters:

          **key** : str
              ..



      :Returns:

          **nbytes** : int
              The size of the value (in bytes).




      :Raises:

          FileNotFoundError
              When the given key does not exist in the store.







      ..
          !! processed by numpydoc !!


   .. py:method:: getsize_prefix(prefix: str) -> int
      :async:


      
      Return the size, in bytes, of all values under a prefix.


      :Parameters:

          **prefix** : str
              The prefix of the directory to measure.



      :Returns:

          **nbytes** : int
              The sum of the sizes of the values in the directory (in bytes).







      .. seealso::

          
          :obj:`zarr.Array.nbytes_stored`
              ..
          :obj:`Store.getsize`
              ..
          
      .. rubric:: Notes

      ``getsize_prefix`` is just provided as a potentially faster alternative to
      listing all the keys under a prefix calling :meth:`Store.getsize` on each.

      In general, ``prefix`` should be the path of an Array or Group in the Store.
      Implementations may differ on the behavior when some other ``prefix``
      is provided.



      ..
          !! processed by numpydoc !!


   .. py:method:: is_empty(prefix: str) -> bool
      :async:


      
      Check if the directory is empty.


      :Parameters:

          **prefix** : str
              Prefix of keys to check.



      :Returns:

          bool
              True if the store is empty, False otherwise.











      ..
          !! processed by numpydoc !!


   .. py:method:: list() -> collections.abc.AsyncIterator[str]
      :async:


      
      Retrieve all keys in the store.





      :Returns:

          AsyncIterator[str]
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: list_dir(prefix: str) -> collections.abc.AsyncIterator[str]
      :async:


      
      Retrieve all keys and prefixes with a given prefix and which do not contain the character
      “/” after the given prefix.


      :Parameters:

          **prefix** : str
              ..



      :Returns:

          AsyncIterator[str]
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]
      :async:


      
      Retrieve all keys in the store that begin with a given prefix. Keys are returned relative
      to the root of the store.


      :Parameters:

          **prefix** : str
              ..



      :Returns:

          AsyncIterator[str]
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: open(*args: Any, **kwargs: Any) -> Self
      :classmethod:

      :async:


      
      Create and open the store.


      :Parameters:

          **\*args** : Any
              Positional arguments to pass to the store constructor.

          **\*\*kwargs** : Any
              Keyword arguments to pass to the store constructor.



      :Returns:

          Store
              The opened store instance.











      ..
          !! processed by numpydoc !!


   .. py:method:: set(key: str, value: zarr.core.buffer.Buffer, byte_range: tuple[int, int] | None = None) -> None
      :async:


      
      Store a (key, value) pair.


      :Parameters:

          **key** : str
              ..

          **value** : Buffer
              ..














      ..
          !! processed by numpydoc !!


   .. py:method:: set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None
      :async:


      
      Store a key to ``value`` if the key is not already present.


      :Parameters:

          **key** : str
              ..

          **value** : Buffer
              ..














      ..
          !! processed by numpydoc !!


   .. py:method:: with_read_only(read_only: bool = False) -> MemoryStore

      
      Return a new store with a new read_only setting.

      The new store points to the same location with the specified new read_only state.
      The returned Store is not automatically opened, and this store is
      not automatically closed.

      :Parameters:

          **read_only**
              If True, the store will be created in read-only mode. Defaults to False.



      :Returns:

          A new store of the same type with the new read only attribute.
              ..











      ..
          !! processed by numpydoc !!


   .. py:property:: read_only
      :type: bool


      
      Is the store read-only?
















      ..
          !! processed by numpydoc !!


   .. py:property:: supports_consolidated_metadata
      :type: bool


      
      Does the store support consolidated metadata?.

      If it doesn't an error will be raised on requests to consolidate the metadata.
      Returning `False` can be useful for stores which implement their own
      consolidation mechanism outside of the zarr-python implementation.















      ..
          !! processed by numpydoc !!


   .. py:attribute:: supports_deletes
      :type:  bool
      :value: True


      
      Does the store support deletes?
















      ..
          !! processed by numpydoc !!


   .. py:attribute:: supports_listing
      :type:  bool
      :value: True


      
      Does the store support listing?
















      ..
          !! processed by numpydoc !!


   .. py:property:: supports_partial_writes
      :type: Literal[False]


      
      Does the store support partial writes?

      Partial writes are no longer used by Zarr, so this is always false.















      ..
          !! processed by numpydoc !!


   .. py:attribute:: supports_writes
      :type:  bool
      :value: True


      
      Does the store support writes?
















      ..
          !! processed by numpydoc !!


.. py:class:: TestBuffer(array_like: zarr.core.buffer.core.ArrayLike)

   Bases: :py:obj:`zarr.core.buffer.cpu.Buffer`


   
   Example of a custom Buffer that handles ArrayLike
















   ..
       !! processed by numpydoc !!

   .. py:method:: as_array_like() -> ArrayLike

      
      Returns the underlying array (host or device memory) of this buffer

      This will never copy data.




      :Returns:

          The underlying 1d array such as a NumPy or CuPy array.
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: as_buffer_like() -> zarr.core.common.BytesLike

      
      Returns the buffer as an object that implements the Python buffer protocol.





      :Returns:

          An object that implements the Python buffer protocol
              ..








      .. rubric:: Notes

      Might have to copy data, since the implementation uses `.as_numpy_array()`.



      ..
          !! processed by numpydoc !!


   .. py:method:: as_numpy_array() -> numpy.typing.NDArray[Any]

      
      Returns the buffer as a NumPy array (host memory).





      :Returns:

          NumPy array of this buffer (might be a data copy)
              ..








      .. rubric:: Notes

      Might have to copy data, consider using `.as_array_like()` instead.



      ..
          !! processed by numpydoc !!


   .. py:method:: create_zero_length() -> Self
      :classmethod:


      
      Create an empty buffer with length zero





      :Returns:

          New empty 0-length buffer
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: from_array_like(array_like: ArrayLike) -> Self
      :classmethod:


      
      Create a new buffer of an array-like object


      :Parameters:

          **array_like**
              array-like object that must be 1-dim, contiguous, and byte dtype.



      :Returns:

          New buffer representing `array_like`
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: from_buffer(buffer: zarr.core.buffer.core.Buffer) -> Self
      :classmethod:


      
      Create a new buffer of an existing Buffer

      This is useful if you want to ensure that an existing buffer is
      of the correct subclass of Buffer. E.g., MemoryStore uses this
      to return a buffer instance of the subclass specified by its
      BufferPrototype argument.

      Typically, this only copies data if the data has to be moved between
      memory types, such as from host to device memory.

      :Parameters:

          **buffer**
              buffer object.



      :Returns:

          A new buffer representing the content of the input buffer
              ..








      .. rubric:: Notes

      Subclasses of `Buffer` must override this method to implement
      more optimal conversions that avoid copies where possible



      ..
          !! processed by numpydoc !!


   .. py:method:: from_bytes(bytes_like: zarr.core.common.BytesLike) -> Self
      :classmethod:


      
      Create a new buffer of a bytes-like object (host memory)


      :Parameters:

          **bytes_like**
              bytes-like object



      :Returns:

          New buffer representing `bytes_like`
              ..











      ..
          !! processed by numpydoc !!


   .. py:method:: to_bytes() -> bytes

      
      Returns the buffer as `bytes` (host memory).





      :Returns:

          `bytes` of this buffer (data copy)
              ..






      .. warning::

          Will always copy data, only use this method for small buffers such as metadata
          buffers. If possible, use `.as_numpy_array()` or `.as_array_like()` instead.





      ..
          !! processed by numpydoc !!


