scRNA-seq¶

Here, you’ll learn how to manage a growing number of scRNA-seq datasets as a single queryable collection:

create a dataset (an Artifact) and seed a Collection ()
append a new dataset to the collection ()
query & analyze individual datasets ()
load the collection into memory ()
iterate over the collection to train an ML model ()
concatenate the collection to a single tiledbsoma array store ()

If you’re only interested in using a large curated scRNA-seq collection, see the CELLxGENE guide.

# pip install 'lamindb[jupyter,bionty]'
!lamin init --storage ./test-scrna --modules bionty

import lamindb as ln
import bionty as bt

ln.track()

Populate metadata registries based on an artifact¶

Let us look at the standardized data of Conde et al., Science (2022), available from CELLxGENE. anndata_human_immune_cells() loads a subsampled version:

adata = ln.core.datasets.anndata_human_immune_cells()
adata

Let’s curate this artifact:

curator = ln.Curator.from_anndata(
    adata,
    var_index=bt.Gene.ensembl_gene_id,
    categoricals={
        adata.obs.donor.name: ln.ULabel.name,
        adata.obs.tissue.name: bt.Tissue.name,
        adata.obs.cell_type.name: bt.CellType.name,
        adata.obs.assay.name: bt.ExperimentalFactor.name,
    },
    organism="human",
)

# this runs a while, because this instance is still empty
curator.validate()

curator.add_new_from_var_index()

curator.add_new_from("donor")
curator.add_new_from("cell_type")

curator.validate()

When we create a Artifact object from an AnnData, we automatically curate it with validated features and labels:

artifact = curator.save_artifact(key="datasets/conde22.h5ad")

It is annotated with rich metadata:

artifact.describe()

Show code cell output Hide code cell output

Artifact .h5ad/AnnData
├── General
│   ├── .uid = 'yoNTdfO3SvoxJy110000'
│   ├── .key = 'datasets/conde22.h5ad'
│   ├── .size = 57612943
│   ├── .hash = 't_YJQpYrAyAGhs7Ir68zKj'
│   ├── .n_observations = 1648
│   ├── .path = /home/runner/work/lamin-usecases/lamin-usecases/docs/test-scrna/.lamindb/yoNTdfO3SvoxJy110000.h5ad
│   ├── .created_by = testuser1 (Test User1)
│   ├── .created_at = 2025-04-15 16:34:12
│   └── .transform = 'scRNA-seq'
├── Dataset features
│   ├── var • 36503                 [bionty.Gene]                                                       
│   │   MIR1302-2HG                 float                                                               
│   │   FAM138A                     float                                                               
│   │   OR4F5                       float                                                               
│   │   OR4F29                      float                                                               
│   │   OR4F16                      float                                                               
│   │   LINC01409                   float                                                               
│   │   FAM87B                      float                                                               
│   │   LINC01128                   float                                                               
│   │   LINC00115                   float                                                               
│   │   FAM41C                      float                                                               
│   └── obs • 4                     [Feature]                                                           
│       assay                       cat[bionty.ExperimentalF…  10x 3' v3, 10x 5' v1, 10x 5' v2          
│       cell_type                   cat[bionty.CellType]       CD16-negative, CD56-bright natural kille…
│       donor                       cat[ULabel]                582C, 621B, 637C, 640C, A29, A31, A35, A…
│       tissue                      cat[bionty.Tissue]         blood, bone marrow, caecum, duodenum, il…
└── Labels
    └── .tissues                    bionty.Tissue              blood, thoracic lymph node, spleen, lung…
        .cell_types                 bionty.CellType            classical monocyte, T follicular helper …
        .experimental_factors       bionty.ExperimentalFactor  10x 3' v3, 10x 5' v2, 10x 5' v1          
        .ulabels                    ULabel                     D496, 621B, A29, A36, A35, 637C, A52, A3…

Seed a collection¶

Let’s create a first version of a collection that will encompass many h5ad files when more data is ingested.

Note

To see the result of the incremental growth, take a look at the CELLxGENE Census guide for an instance with ~1k h5ads and ~50 million cells.

collection = ln.Collection(artifact, key="My versioned scRNA-seq collection").save()

For this version 1 of the collection, collection and artifact match each other. But they’re independently tracked and queryable through their registries:

collection.describe()

Access the underlying artifacts like so:

collection.artifacts.df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	_branch_code
id
1	yoNTdfO3SvoxJy110000	datasets/conde22.h5ad	None	.h5ad	dataset	AnnData	57612943	t_YJQpYrAyAGhs7Ir68zKj	None	1648	sha1-fl	True	False	1	1	None	None	True	1	2025-04-15 16:34:12.076000+00:00	1	None	1

See data lineage:

collection.view_lineage()

_images/999c53277e9c48fcc0695ef88073566a5ab98f9c54a57492db352aeb69d239e2.svg