Lazy loading

The ZarrVectorStore class provides a lazy, level-of-detail-aware interface for reading ZVF stores without loading all data into memory. It is the recommended interface for remote stores, large datasets, and interactive applications.

Lazy zarr vectors store access.

Opens a store without reading data. All vertex, attribute, and object access is deferred until .compute() is called, optionally using Dask for parallel I/O.

Usage:

from zarr_vectors.lazy import open_zv

zv = open_zv("scan.zarrvectors")
zv[0].vertices.compute()          # materialise all level-0 vertices
zv[0].vertices[0, 0, 0].compute() # single chunk
zv[0].attributes["intensity"].compute()
class zarr_vectors.lazy.ZVPolylineCollection(group, ndim=3)[source]

Bases: object

Lazy collection of polylines accessible by object ID.

Each polyline is reconstructed by following its object_index manifest and concatenating fragments from the relevant chunks.

Parameters:

  • group (FsGroup) – Resolution level FsGroup.

  • ndim (int) – Number of spatial dimensions.

__init__(group, ndim=3)[source]
Parameters:
Return type:

None

compute()[source]

Materialise all polylines.

Returns:

List of (N_k, D) arrays, one per polyline.

Return type:

list[ndarray[tuple[Any, …], dtype[floating]]]

property count: int

Number of polylines.

filter(*, object_ids=None, length_range=None)[source]

Filter polylines by ID or length range.

Returns a new filtered collection (lazy).

Parameters:

  • object_ids (list[int] | None) – Keep only these IDs.

  • length_range (tuple[float, float] | None) – (min_length, max_length) — keep polylines whose Euclidean path length falls in this range. Requires computing lengths on first access.

Return type:

FilteredPolylineCollection

items()[source]

Iterate over (object_id, delayed_polyline) pairs.

property object_ids: list[int]

List of polyline object IDs.

class zarr_vectors.lazy.ZVStore(root, meta)[source]

Bases: object

Lazy handle to a zarr vectors store.

Attributes are read from .zattrs on first access and cached. Resolution levels are accessed by integer index via __getitem__.

Parameters:

  • root (Group) – An open FsGroup for the store root.

  • meta (RootMetadata) – Parsed RootMetadata.

__init__(root, meta)[source]
Parameters:

  • root (Group)

  • meta (RootMetadata)

Return type:

None

property base_bin_shape: tuple[float, ...] | None
property bin_shape: tuple[float, ...]

Effective bin shape (base_bin_shape or chunk_shape).

property bins_per_chunk: tuple[int, ...]
property bounds: tuple[list[float], list[float]]
property chunk_shape: tuple[float, ...]
property geometry_types: list[str]
property headers: dict[str, dict[str, Any]]

Dict of stored format headers, keyed by format name.

Returns:

{format_name: header_dict} for each stored header. Empty dict if no headers are stored. Typed deserialisation of header dicts is handled by the format package.

property levels: list[int]

Sorted list of available resolution level indices.

property ndim: int
object_levels(oid)[source]

Monotonically-increasing list of levels at which oid is present.

For an ID-preserving pyramid, this is the set of LODs a viewer can pick for the object — the object’s OID is stable across levels and the object is present at level $L$ iff its manifest at level $L$ is non-empty.

Returns [] if the object is absent from every level.

Parameters:

oid (int)

Return type:

list[int]

property path: Path

Filesystem path of the store (LocalBackend only).

set_backend(backend, **backend_kwargs)[source]

Swap the underlying storage backend in place (no data movement).

Useful for switching driver (e.g. "fsspec" → "obstore") or credentials on a store you already have open. Any cached level / array handles are invalidated and will be rebuilt on the next access using the new backend.

Parameters:

  • backend (str | StorageBackend) – Backend name string or a pre-built StorageBackend already pointed at the same URL.

  • **backend_kwargs (Any) – Forwarded to the backend constructor when backend is a string.

Return type:

None

property url: str

Canonical URL of the store root. Portable across backends.

property zv_version: str
class zarr_vectors.lazy.ZVView(group, root_meta, level_meta, all_chunk_keys, spec)[source]

Bases: object

A filtered lazy view of a resolution level.

Created by calling .filter() on a ZVLevel or another ZVView. Each filter narrows the read plan; data is only loaded on .compute().

Parameters:

  • group (FsGroup) – Resolution level FsGroup.

  • root_meta (RootMetadata) – Root metadata.

  • level_meta (LevelMetadata | None) – Level metadata (or None).

  • all_chunk_keys (list[ChunkCoords]) – Full list of chunk keys at this level.

  • spec (FilterSpec) – The accumulated filter constraints.

__init__(group, root_meta, level_meta, all_chunk_keys, spec)[source]
Parameters:

  • group (FsGroup)

  • root_meta (RootMetadata)

  • level_meta (LevelMetadata | None)

  • all_chunk_keys (list[tuple[int, ...]])

  • spec (FilterSpec)

Return type:

None

compute()[source]

Materialise the filtered data.

Returns:

Dict with positions, vertex_count, and optionally object_ids.

Return type:

dict[str, Any]

filter(*, bbox=None, object_ids=None, group_ids=None)[source]

Apply additional filter constraints, returning a new view.

Parameters:

  • bbox (tuple[ndarray[tuple[Any, ...], dtype[_ScalarT]], ndarray[tuple[Any, ...], dtype[_ScalarT]]] | None) – Bounding box (min_corner, max_corner).

  • object_ids (list[int] | None) – Keep only these object IDs.

  • group_ids (list[int] | None) – Keep only objects in these groups (resolved to object IDs via groupings).

Returns:

A new ZVView with the intersection of all constraints.

Return type:

ZVView

property vertices: _FilteredVertices

Lazy filtered vertex accessor.

zarr_vectors.lazy.open_zv(path, *, backend=None, **backend_kwargs)[source]

Open a zarr vectors store lazily.

Reads only root metadata (a few KB). No vertex data is loaded.

Parameters:

  • path (str | Path) – URL or filesystem path to the ZV store.

  • backend (str | None) – Force a backend ("local" / "obstore" / "fsspec"). Auto-detected from the URL scheme by default.

  • **backend_kwargs (Any) – Forwarded to the backend constructor.

Returns:

A ZVStore handle for lazy access.

Return type:

ZVStore

ZarrVectorStore

LazyArray