Query arrays in storage .md .md

This guide covers streaming array-like datasets — AnnData, SpatialData, and generic HDF5 — directly from disk or cloud storage. For tabular datasets, see Query tables in storage .

# replace with your username and S3 bucket
lamin login testuser1
lamin init --storage s3://lamindb-ci/test-arrays
Hide code cell output 
→ logged in testuser1

! updating cloud SQLite 's3://lamindb-ci/test-arrays/.lamindb/lamin.db' of instance 'testus

er1/test-arrays'

! locked instance (to unlock and push changes to the cloud SQLite file, call: lamin disconn

ect)
→ initialized lamindb: testuser1/test-arrays

Import lamindb and track this notebook.

import lamindb as ln
import numpy as np

db = ln.DB("laminlabs/lamindata")  # we'll pull the SpatialData example from there
Hide code cell output 
→ connected lamindb: testuser1/test-arrays

! database has modules bionty,pertdb, configure it: lamin settings modules set bionty,pertdb

AnnData

We’ll need some test data:

ln.Artifact("s3://lamindb-ci/test-arrays/pbmc68k.h5ad").save()
ln.Artifact("s3://lamindb-ci/test-arrays/testfile.hdf5").save()
Hide code cell output 
! no run & transform got linked, call `ln.track()` & re-run

! no run & transform got linked, call `ln.track()` & re-run

Artifact(uid='ulAQTAxc4aBHyFW80000', key='testfile.hdf5', description=None, suffix='.hdf5', kind=None, otype=None, size=1400, hash='UCWPjJkhzBjO97rtuo_8Yg', n_files=None, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-07-17 13:46:59 UTC, is_locked=False, version_tag=None, is_latest=True)

An h5ad artifact stored on s3:

artifact = ln.Artifact.get(key="pbmc68k.h5ad")

artifact.path
Hide code cell output 
S3QueryPath('lamindb-ci/test-arrays/pbmc68k.h5ad', protocol='s3')

adata = artifact.open()
Hide code cell output 
! run input wasn't tracked, call `ln.track()` and re-run

This is an AnnDataAccessor — an AnnData backed by cloud storage:

adata
Hide code cell output 
AnnDataAccessor object with n_obs × n_vars = 70 × 765
  constructed for the AnnData object pbmc68k.h5ad
    obs: ['cell_type', 'index', 'louvain', 'n_genes', 'percent_mito']
    obsm: ['X_pca', 'X_umap']
    obsp: ['connectivities', 'distances']
    uns: ['louvain', 'louvain_colors', 'neighbors', 'pca']
    var: ['highly_variable', 'index', 'n_counts']
    varm: ['PCs']

Without subsetting, it references the underlying lazy h5 or zarr arrays:

adata.X
Hide code cell output 
<HDF5 dataset "X": shape (70, 765), type "<f4">

You can subset it like a normal AnnData object:

obs_idx = adata.obs.cell_type.isin(["Dendritic cells", "CD14+ Monocytes"]) & (
    adata.obs.percent_mito <= 0.05
)
adata_subset = adata[obs_idx]
adata_subset
Hide code cell output 
AnnDataAccessorSubset object with n_obs × n_vars = 35 × 765
  obs: ['cell_type', 'index', 'louvain', 'n_genes', 'percent_mito']
  obsm: ['X_pca', 'X_umap']
  obsp: ['connectivities', 'distances']
  uns: ['louvain', 'louvain_colors', 'neighbors', 'pca']
  var: ['highly_variable', 'index', 'n_counts']
  varm: ['PCs']

Subsets load arrays into memory upon direct access:

adata_subset.X
Hide code cell output 
array([[-0.326, -0.191,  0.499, ..., -0.21 , -0.636, -0.49 ],
       [ 0.811, -0.191, -0.728, ..., -0.21 ,  0.604, -0.49 ],
       [-0.326, -0.191,  0.643, ..., -0.21 ,  2.303, -0.49 ],
       ...,
       [-0.326, -0.191, -0.728, ..., -0.21 ,  0.626, -0.49 ],
       [-0.326, -0.191, -0.728, ..., -0.21 , -0.636, -0.49 ],
       [-0.326, -0.191, -0.728, ..., -0.21 , -0.636, -0.49 ]],
      shape=(35, 765), dtype=float32)

To load the entire subset into memory as an actual AnnData object, use to_memory():

adata_subset.to_memory()
Hide code cell output 
AnnData object with n_obs × n_vars = 35 × 765
    obs: 'cell_type', 'n_genes', 'percent_mito', 'louvain'
    var: 'n_counts', 'highly_variable'
    uns: 'louvain', 'louvain_colors', 'neighbors', 'pca'
    obsm: 'X_pca', 'X_umap'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'

You can also add columns to .obs and .var of a cloud AnnData without downloading it. First, create a new AnnData zarr artifact:

adata_subset.to_memory().write_zarr("adata_subset.zarr")
artifact = ln.Artifact(
    "adata_subset.zarr", description="test add column to adata"
).save()
Hide code cell output 
! no run & transform got linked, call `ln.track()` & re-run

This is how you add a column:

with artifact.open(mode="r+") as adata_accessor:
    adata_accessor.add_column(where="obs", col_name="ones", col=np.ones(adata_accessor.shape[0]))
    display(adata_accessor)
Hide code cell output 
! run input wasn't tracked, call `ln.track()` and re-run

AnnDataAccessor object with n_obs × n_vars = 35 × 765
  constructed for the AnnData object 9EtBNxRhyxNeS0sj.zarr
    obs: ['cell_type', 'index', 'louvain', 'n_genes', 'percent_mito', 'ones']
    obsm: ['X_pca', 'X_umap']
    obsp: ['connectivities', 'distances']
    uns: ['louvain', 'louvain_colors', 'neighbors', 'pca']
    var: ['highly_variable', 'index', 'n_counts']
    varm: ['PCs']

! no run & transform got linked, call `ln.track()` & re-run

The version of the artifact is updated after the modification.

artifact
Hide code cell output 
Artifact(uid='9EtBNxRhyxNeS0sj0001', key=None, description='test add column to adata', suffix='.zarr', kind=None, otype=None, size=215962, hash='r4R-x3hgQVsA_0LUE2p1wQ', n_files=123, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-07-17 13:47:07 UTC, is_locked=False, version_tag=None, is_latest=True)

artifact.delete(permanent=True)
Hide code cell output 
→ deleting all versions of this artifact because they all share the same store

SpatialData

You can also access AnnData objects inside SpatialData tables:

artifact = db.Artifact.get(key="visium_aligned_guide_min.zarr")

access = artifact.open()
Hide code cell output 
! run input wasn't tracked, call `ln.track()` and re-run

access
Hide code cell output 
SpatialDataAccessor object
  constructed for the SpatialData object bjH534dxVi1drmLZ.zarr
    with tables: ['table']

access.tables
Hide code cell output 
Accessor for the SpatialData attribute tables
  with keys: ['table']

This gives you the same AnnDataAccessor object as for a normal AnnData.

table = access.tables["table"]

table
Hide code cell output 
AnnDataAccessor object with n_obs × n_vars = 37 × 18085
  constructed for the AnnData object table
    obs: ['_index', 'array_col', 'array_row', 'clone', 'dataset', 'in_tissue', 'region', 'spot_id']
    obsm: ['spatial']
    uns: ['spatial', 'spatialdata_attrs']
    var: ['feature_types', 'gene_ids', 'genome', 'symbols']

You can subset it and read into memory as an actual AnnData:

table_subset = table[table.obs["clone"] == "diploid"]

table_subset
Hide code cell output 
AnnDataAccessorSubset object with n_obs × n_vars = 31 × 18085
  obs: ['_index', 'array_col', 'array_row', 'clone', 'dataset', 'in_tissue', 'region', 'spot_id']
  obsm: ['spatial']
  uns: ['spatial', 'spatialdata_attrs']
  var: ['feature_types', 'gene_ids', 'genome', 'symbols']

adata = table_subset.to_memory()

Generic HDF5

Let us query a generic HDF5 artifact:

artifact = ln.Artifact.get(key="testfile.hdf5")

And get a backed accessor:

backed = artifact.open()
Hide code cell output 
! run input wasn't tracked, call `ln.track()` and re-run

The returned object contains the .connection and h5py.File or zarr.Group in .storage

backed
Hide code cell output 
BackedAccessor(connection=<File-like object S3FileSystem, lamindb-ci/test-arrays/testfile.hdf5>, storage=<HDF5 file "testfile.hdf5>" (mode r)>)

backed.storage
Hide code cell output 
<HDF5 file "testfile.hdf5>" (mode r)>
Hide code cell content 
lamin delete --force test-arrays

→ deleted storage record on hub 76e5f3b018085f52bcd5ca9b4d7e0ce5 | s3://lamindb-ci/test-a

rrays

→ deleted instance record on hub 587a82023ecb5ea28b3a448cb8240f7f