##### Query arrays in storage [image: .md][image]

This guide covers streaming array-like datasets — "AnnData",
"SpatialData", and generic "HDF5" — directly from disk or cloud
storage. For tabular datasets, see Query tables in storage . For an
in-depth example of distributed data loaders for the "AnnData" format,
see these blog posts:

* blog.lamin.ai/mapped-collection (2024)

* blog.lamin.ai/annbatch (2026)

 # replace with your username and S3 bucket
 lamin login testuser1
 lamin init --storage s3://lamindb-ci/test-arrays

Import lamindb and track this notebook.

 import lamindb as ln
 import numpy as np

#### AnnData

We'll need some test data:

 ln.Artifact("s3://lamindb-ci/test-arrays/pbmc68k.h5ad").save()
 ln.Artifact("s3://lamindb-ci/test-arrays/testfile.hdf5").save()

An "h5ad" artifact stored on s3:

 artifact = ln.Artifact.get(key="pbmc68k.h5ad")

 artifact.path

 adata = artifact.open()

This is an "AnnDataAccessor" — an "AnnData" backed by cloud storage:

 adata

Without subsetting, it references the underlying lazy "h5" or "zarr"
arrays:

 adata.X

You can subset it like a normal "AnnData" object:

 obs_idx = adata.obs.cell_type.isin(["Dendritic cells", "CD14+ Monocytes"]) & (
 adata.obs.percent_mito <= 0.05
 )
 adata_subset = adata[obs_idx]
 adata_subset

Subsets load arrays into memory upon direct access:

 adata_subset.X

To load the entire subset into memory as an actual "AnnData" object,
use "to_memory()":

 adata_subset.to_memory()

You can also add columns to ".obs" and ".var" of a cloud "AnnData"
without downloading it. First, create a new "AnnData" "zarr" artifact:

 adata_subset.to_memory().write_zarr("adata_subset.zarr")
 artifact = ln.Artifact(
 "adata_subset.zarr", description="test add column to adata"
 ).save()

This is how you add a column:

 with artifact.open(mode="r+") as adata_accessor:
 adata_accessor.add_column(where="obs", col_name="ones", col=np.ones(adata_accessor.shape[0]))
 display(adata_accessor)

The version of the artifact is updated after the modification.

 artifact

 artifact.delete(permanent=True)

#### SpatialData

You can also access "AnnData" objects inside "SpatialData" tables:

 db = ln.DB("laminlabs/lamindata")

 artifact = db.Artifact.get(key="visium_aligned_guide_min.zarr")

 access = artifact.open()

 access

 access.tables

This gives you the same "AnnDataAccessor" object as for a normal
"AnnData".

 table = access.tables["table"]

 table

You can subset it and read into memory as an actual "AnnData":

 table_subset = table[table.obs["clone"] == "diploid"]

 table_subset

 adata = table_subset.to_memory()

#### Generic HDF5

Let us query a generic HDF5 artifact:

 artifact = ln.Artifact.get(key="testfile.hdf5")

And get a backed accessor:

 backed = artifact.open()

The returned object contains the ".connection" and "h5py.File" or
"zarr.Group" in ".storage"

 backed

 backed.storage

 lamin delete --force test-arrays