Curate AnnData based on the CELLxGENE schema

This guide shows how to curate an AnnData object with the help of laminlabs/cellxgene against the CELLxGENE schema v5.1.0.

Load your instance where you want to register the curated AnnData object:

# pip install 'lamindb[bionty,jupyter]' cellxgene-lamin
# cellxgene-schema has pinned dependencies. Therefore we recommend installing it into a separate environment using `uv` or `pipx`
# uv tool install cellxgene-schema==5.1.0
!lamin init --storage ./test-cellxgene-curate --modules bionty
Hide code cell output
 initialized lamindb: testuser1/test-cellxgene-curate
import lamindb as ln
import bionty as bt


def get_semi_curated_dataset():
    adata = ln.core.datasets.anndata_human_immune_cells()
    adata.obs["sex_ontology_term_id"] = "PATO:0000384"
    adata.obs["organism"] = "human"
    adata.obs["sex"] = "unknown"
    # create some typos in the metadata
    adata.obs["tissue"] = adata.obs["tissue"].cat.rename_categories({"lung": "lungg"})
    # new donor ids
    adata.obs["donor"] = adata.obs["donor"].astype(str) + "-1"
    # drop animal cell
    adata = adata[adata.obs["cell_type"] != "animal cell", :]
    # remove columns that are reserved in the cellxgene schema
    adata.var.drop(columns=["feature_reference", "feature_biotype"], inplace=True)
    adata.raw.var.drop(
        columns=["feature_name", "feature_reference", "feature_biotype"], inplace=True
    )
    return adata
 connected lamindb: testuser1/test-cellxgene-curate

Let’s start with an AnnData object that we’d like to inspect and curate. We are writing it to disk to run CZI’s cellxgene-schema CLI tool which verifies whether an on-disk h5ad dataset adheres to the cellxgene schema.

adata = get_semi_curated_dataset()
adata.write_h5ad("anndata_human_immune_cells.h5ad")
adata
Hide code cell output
AnnData object with n_obs × n_vars = 1626 × 36503
    obs: 'donor', 'tissue', 'cell_type', 'assay', 'sex_ontology_term_id', 'organism', 'sex'
    var: 'feature_is_filtered'
    uns: 'default_embedding'
    obsm: 'X_umap'

Initially, the cellxgene-schema validator of CZI does not pass and we need to curate the dataset.

!MPLBACKEND=agg uvx cellxgene-schema validate anndata_human_immune_cells.h5ad || exit 1
Hide code cell output
Loading dependencies
Loading validator modules
Starting validation...
Unable to open 'anndata_human_immune_cells.h5ad' with AnnData

Validate and curate metadata

We create a Curate object that references the AnnData object. During instantiation, any :class:~lamindb.Feature records are saved.

curator = ln.curators.CellxGeneAnnDataCatManager(adata, schema_version="5.1.0")
Hide code cell output
✗ Could not find source: ExperimentalFactor
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: CellType
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Phenotype
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Tissue
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Gene
    → consider running `bionty.core.sync_public_sources()`
 Creating control labels in the CellxGene schema.
! you are trying to create a record with name='tissue' but a record with similar name exists: 'TissueType'. Did you mean to load it?
! you are trying to create a record with name='cell' but a record with similar name exists: 'cell culture'. Did you mean to load it?
! 5 terms not validated in feature 'columns': 'assay', 'cell_type', 'sex_ontology_term_id', 'tissue', 'organism'
    → fix typos, remove non-existent values, or save terms via: curator.cat.add_new_from('columns')
validated = curator.validate()
✗ missing required obs columns 'development_stage', 'disease', 'donor_id', 'self_reported_ethnicity', 'suspension_type', 'tissue_type'
    → consider initializing a Curate object with `defaults=cxg.CellxGeneAnnDataCatManager.cxg_categoricals_defaults` to automatically add these columns with default values

Let’s fix the “donor_id” column name:

adata.obs.rename(columns={"donor": "donor_id"}, inplace=True)

For the missing columns, we can pass default values suggested from CELLxGENE which will automatically add them to the AnnData object:

ln.curators.CellxGeneAnnDataCatManager.cxg_categoricals_defaults
Hide code cell output
{'cell_type': 'unknown',
 'development_stage': 'unknown',
 'disease': 'normal',
 'donor_id': 'unknown',
 'self_reported_ethnicity': 'unknown',
 'sex': 'unknown',
 'suspension_type': 'cell',
 'tissue_type': 'tissue'}

Note

CELLxGENE requires columns tissue, organism, and assay to have existing values from the ontologies. Therefore, these columns need to be added and populated manually.

curator = ln.curators.CellxGeneAnnDataCatManager(
    adata,
    defaults=ln.curators.CellxGeneAnnDataCatManager.cxg_categoricals_defaults,
    schema_version="5.1.0",
)
Hide code cell output
 added default value 'unknown' to the adata.obs['development_stage']
 added default value 'normal' to the adata.obs['disease']
 added default value 'unknown' to the adata.obs['self_reported_ethnicity']
 added default value 'cell' to the adata.obs['suspension_type']
 added default value 'tissue' to the adata.obs['tissue_type']
✗ Could not find source: ExperimentalFactor
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: CellType
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: DevelopmentalStage
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Disease
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Phenotype
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Tissue
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Gene
    → consider running `bionty.core.sync_public_sources()`
! 5 terms not validated in feature 'columns': 'development_stage', 'disease', 'self_reported_ethnicity', 'suspension_type', 'tissue_type'
    → fix typos, remove non-existent values, or save terms via: curator.cat.add_new_from('columns')
validated = curator.validate()
validated
Hide code cell output
! 1 term not validated in feature 'tissue': 'lungg'
    → fix typos, remove non-existent values, or save terms via: curator.cat.add_new_from('tissue')
! Starting bulk_create for 36283 Gene records in batches of 10000
! 220 terms not validated in feature 'var_index': 'ENSG00000230699', 'ENSG00000241180', 'ENSG00000226849', 'ENSG00000272482', 'ENSG00000264443', 'ENSG00000242396', 'ENSG00000237352', 'ENSG00000269933', 'ENSG00000286863', 'ENSG00000285808', 'ENSG00000261737', 'ENSG00000230427', 'ENSG00000226822', 'ENSG00000273373', 'ENSG00000259834', 'ENSG00000224167', 'ENSG00000256374', 'ENSG00000234283', 'ENSG00000263464', 'ENSG00000203812', ...
    → fix typos, remove non-existent values, or save terms via: curator.cat.add_new_from('var_index')
False

Remove unvalidated values

We remove all unvalidated genes. These genes may exist in a different release of ensembl but are not valid for the ensembl version of cellxgene schema 5.0.0 (ensembl release 110).

curator.non_validated
Hide code cell output
{'tissue': ['lungg'],
 'var_index': ['ENSG00000230699',
  'ENSG00000241180',
  'ENSG00000226849',
  'ENSG00000272482',
  'ENSG00000264443',
  'ENSG00000242396',
  'ENSG00000237352',
  'ENSG00000269933',
  'ENSG00000286863',
  'ENSG00000285808',
  'ENSG00000261737',
  'ENSG00000230427',
  'ENSG00000226822',
  'ENSG00000273373',
  'ENSG00000259834',
  'ENSG00000224167',
  'ENSG00000256374',
  'ENSG00000234283',
  'ENSG00000263464',
  'ENSG00000203812',
  'ENSG00000272196',
  'ENSG00000237975',
  'ENSG00000235736',
  'ENSG00000272880',
  'ENSG00000227925',
  'ENSG00000238042',
  'ENSG00000237845',
  'ENSG00000270188',
  'ENSG00000287116',
  'ENSG00000236856',
  'ENSG00000226277',
  'ENSG00000237133',
  'ENSG00000224739',
  'ENSG00000230525',
  'ENSG00000227902',
  'ENSG00000237327',
  'ENSG00000285155',
  'ENSG00000232411',
  'ENSG00000239467',
  'ENSG00000225205',
  'ENSG00000272551',
  'ENSG00000280374',
  'ENSG00000226747',
  'ENSG00000272519',
  'ENSG00000236886',
  'ENSG00000229352',
  'ENSG00000286601',
  'ENSG00000227021',
  'ENSG00000259855',
  'ENSG00000233143',
  'ENSG00000228135',
  'ENSG00000273301',
  'ENSG00000237940',
  'ENSG00000271870',
  'ENSG00000237838',
  'ENSG00000286996',
  'ENSG00000223797',
  'ENSG00000233509',
  'ENSG00000269028',
  'ENSG00000239462',
  'ENSG00000286699',
  'ENSG00000273370',
  'ENSG00000261490',
  'ENSG00000251679',
  'ENSG00000249988',
  'ENSG00000272567',
  'ENSG00000270394',
  'ENSG00000249381',
  'ENSG00000272370',
  'ENSG00000272354',
  'ENSG00000251044',
  'ENSG00000248371',
  'ENSG00000251613',
  'ENSG00000272040',
  'ENSG00000182230',
  'ENSG00000249684',
  'ENSG00000233937',
  'ENSG00000248103',
  'ENSG00000204092',
  'ENSG00000261068',
  'ENSG00000236740',
  'ENSG00000236996',
  'ENSG00000232295',
  'ENSG00000271734',
  'ENSG00000236673',
  'ENSG00000227220',
  'ENSG00000236166',
  'ENSG00000112096',
  'ENSG00000285162',
  'ENSG00000228434',
  'ENSG00000229881',
  'ENSG00000286228',
  'ENSG00000237513',
  'ENSG00000285106',
  'ENSG00000226380',
  'ENSG00000270672',
  'ENSG00000225932',
  'ENSG00000244693',
  'ENSG00000283504',
  'ENSG00000283648',
  'ENSG00000268955',
  'ENSG00000272267',
  'ENSG00000255495',
  'ENSG00000253381',
  'ENSG00000254143',
  'ENSG00000253878',
  'ENSG00000259820',
  'ENSG00000226403',
  'ENSG00000229611',
  'ENSG00000233776',
  'ENSG00000269900',
  'ENSG00000283886',
  'ENSG00000261534',
  'ENSG00000237548',
  'ENSG00000239665',
  'ENSG00000256892',
  'ENSG00000249860',
  'ENSG00000271409',
  'ENSG00000224745',
  'ENSG00000261438',
  'ENSG00000231575',
  'ENSG00000260461',
  'ENSG00000234134',
  'ENSG00000255823',
  'ENSG00000248671',
  'ENSG00000254740',
  'ENSG00000254561',
  'ENSG00000282080',
  'ENSG00000256427',
  'ENSG00000286911',
  'ENSG00000287577',
  'ENSG00000246331',
  'ENSG00000287388',
  'ENSG00000276814',
  'ENSG00000271259',
  'ENSG00000287622',
  'ENSG00000255945',
  'ENSG00000261650',
  'ENSG00000256542',
  'ENSG00000230641',
  'ENSG00000275294',
  'ENSG00000236094',
  'ENSG00000237585',
  'ENSG00000223458',
  'ENSG00000261666',
  'ENSG00000280710',
  'ENSG00000203441',
  'ENSG00000230156',
  'ENSG00000275216',
  'ENSG00000215271',
  'ENSG00000286931',
  'ENSG00000258414',
  'ENSG00000258808',
  'ENSG00000277050',
  'ENSG00000273888',
  'ENSG00000258777',
  'ENSG00000258301',
  'ENSG00000258861',
  'ENSG00000259444',
  'ENSG00000260780',
  'ENSG00000244952',
  'ENSG00000259730',
  'ENSG00000258631',
  'ENSG00000258831',
  'ENSG00000273923',
  'ENSG00000259664',
  'ENSG00000259582',
  'ENSG00000261720',
  'ENSG00000277010',
  'ENSG00000260182',
  'ENSG00000262668',
  'ENSG00000232196',
  'ENSG00000260060',
  'ENSG00000260141',
  'ENSG00000261439',
  'ENSG00000260923',
  'ENSG00000215067',
  'ENSG00000263316',
  'ENSG00000262089',
  'ENSG00000273388',
  'ENSG00000264067',
  'ENSG00000272736',
  'ENSG00000214970',
  'ENSG00000263388',
  'ENSG00000262292',
  'ENSG00000256618',
  'ENSG00000221995',
  'ENSG00000226377',
  'ENSG00000273576',
  'ENSG00000267637',
  'ENSG00000283517',
  'ENSG00000282965',
  'ENSG00000286603',
  'ENSG00000265717',
  'ENSG00000278107',
  'ENSG00000273733',
  'ENSG00000273837',
  'ENSG00000286949',
  'ENSG00000256222',
  'ENSG00000280095',
  'ENSG00000278927',
  'ENSG00000278955',
  'ENSG00000224247',
  'ENSG00000272948',
  'ENSG00000233213',
  'ENSG00000277352',
  'ENSG00000239446',
  'ENSG00000231566',
  'ENSG00000256045',
  'ENSG00000228906',
  'ENSG00000228139',
  'ENSG00000261773',
  'ENSG00000237563',
  'ENSG00000228890',
  'ENSG00000226362',
  'ENSG00000278198',
  'ENSG00000273496',
  'ENSG00000277666',
  'ENSG00000278782',
  'ENSG00000277761']}
adata = adata[:, ~adata.var.index.isin(curator.non_validated["var_index"])].copy()
if adata.raw is not None:
    raw_data = adata.raw.to_adata()
    raw_data = raw_data[
        :, ~raw_data.var_names.isin(curator.non_validated["var_index"])
    ].copy()
    adata.raw = raw_data
curator = ln.curators.CellxGeneAnnDataCatManager(adata, schema_version="5.1.0")
✗ Could not find source: ExperimentalFactor
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: CellType
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: DevelopmentalStage
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Disease
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Phenotype
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Tissue
    → consider running `bionty.core.sync_public_sources()`
✗ Could not find source: Gene
    → consider running `bionty.core.sync_public_sources()`

Register new metadata labels

Following the suggestions above to register genes and labels that aren’t present in the current instance:

(Note that our instance is rather empty. Once you filled up the registries, registering new labels won’t be frequently needed)

An error is shown for the tissue label “lungg”, which is a typo, should be “lung”. Let’s fix it:

tissues = curator.lookup(public=True).tissue
tissues.lung
Hide code cell output
Tissue(ontology_id='UBERON:0002048', name='lung', definition='Respiration Organ That Develops As An Outpocketing Of The Esophagus.', synonyms='pulmo', parents=array(['UBERON:0015212', 'UBERON:0005178', 'UBERON:0000171',
       'UBERON:0004119'], dtype=object))
adata.obs["tissue"] = adata.obs["tissue"].cat.rename_categories(
    {"lungg": tissues.lung.name}
)

Let’s validate the object again:

validated = curator.validate()
validated
Hide code cell output
True
adata.obs.head()
Hide code cell output
donor_id tissue cell_type assay sex_ontology_term_id organism sex development_stage disease self_reported_ethnicity suspension_type tissue_type
CZINY-0109_CTGGTCTAGTCTGTAC D496-1 blood classical monocyte 10x 3' v3 PATO:0000384 human unknown unknown normal unknown cell tissue
CZI-IA10244332+CZI-IA10244434_CCTTCGACATACTCTT 621B-1 thoracic lymph node T follicular helper cell 10x 5' v2 PATO:0000384 human unknown unknown normal unknown cell tissue
Pan_T7935491_CTGGTCTGTACATGTC A29-1 spleen memory B cell 10x 5' v1 PATO:0000384 human unknown unknown normal unknown cell tissue
Pan_T7980367_GGGCATCCAGGTGGAT A36-1 lung alveolar macrophage 10x 5' v1 PATO:0000384 human unknown unknown normal unknown cell tissue
Pan_T7935494_ATCATGGTCTACCTGC A29-1 mesenteric lymph node naive thymus-derived CD4-positive, alpha-beta ... 10x 5' v1 PATO:0000384 human unknown unknown normal unknown cell tissue

Save artifact

artifact = curator.save_artifact(
    key=f"my_datasets/dataset-curated-against-cxg-{curator.schema_version}.h5ad"
)
Hide code cell output
! no run & transform got linked, call `ln.track()` & re-run
! 2 unique terms (16.70%) are not validated for name: 'donor_id', 'sex'
artifact.describe()
Hide code cell output
Artifact .h5ad · AnnData · dataset
├── General
│   ├── uid: y2HWwVDgwOfaHc4M0000          hash: t1Fw4r_WZzCxCdPtca_XXh
│   ├── size: 52.1 MB                      n_observations: 1626
│   ├── space: all                         branch: main
│   ├── created_at: 2025-07-07 11:59:54    created_by: testuser1 (Test User1)
│   ├── key: my_datasets/dataset-curated-against-cxg-5.1.0.h5ad
│   └── storage location / path: 
/home/runner/work/cellxgene-lamin/cellxgene-lamin/docs/test-cellxgene-curate/.lamindb/y2HWwVDgwOfaHc4M0000.
h5ad
├── Dataset features
│   ├── var36283                     [bionty.Gene]                                                              
│   │   MIR1302-2HG                     float                                                                      
│   │   FAM138A                         float                                                                      
│   │   OR4F5                           float                                                                      
│   │   OR4F29                          float                                                                      
│   │   OR4F16                          float                                                                      
│   │   LINC01409                       float                                                                      
│   │   FAM87B                          float                                                                      
│   │   LINC01128                       float                                                                      
│   │   LINC00115                       float                                                                      
│   │   FAM41C                          float                                                                      
│   └── obs10                        [Feature]                                                                  
assay                           cat[bionty.ExperimentalFactor]     10x 3' v3, 10x 5' v1, 10x 5' v2         
cell_type                       cat[bionty.CellType]               CD16-negative, CD56-bright natural kill…
development_stage               cat[bionty.DevelopmentalStage]     unknown                                 
disease                         cat[bionty.Disease]                normal                                  
organism                        cat[bionty.Organism]               human                                   
self_reported_ethnicity         cat[bionty.Ethnicity]              unknown                                 
sex_ontology_term_id            cat[bionty.Phenotype]              male                                    
suspension_type                 cat[ULabel]                        cell                                    
tissue                          cat[bionty.Tissue]                 blood, bone marrow, caecum, duodenum, i…
tissue_type                     cat[ULabel]                        tissue                                  
└── Labels
    └── .organisms                      bionty.Organism                    human                                   
        .tissues                        bionty.Tissue                      blood, thoracic lymph node, spleen, mes…
        .cell_types                     bionty.CellType                    classical monocyte, T follicular helper…
        .diseases                       bionty.Disease                     normal                                  
        .phenotypes                     bionty.Phenotype                   male                                    
        .experimental_factors           bionty.ExperimentalFactor          10x 3' v3, 10x 5' v2, 10x 5' v1         
        .developmental_stages           bionty.DevelopmentalStage          unknown                                 
        .ethnicities                    bionty.Ethnicity                   unknown                                 
        .ulabels                        ULabel                             tissue, cell                            

Return an input h5ad file for cellxgene-schema

title = "Cross-tissue immune cell analysis reveals tissue-specific features in humans (for test demo only)"
adata_cxg = curator.to_cellxgene_anndata(is_primary_data=True, title=title)
adata_cxg
Hide code cell output
AnnData object with n_obs × n_vars = 1626 × 36283
    obs: 'donor_id', 'sex_ontology_term_id', 'suspension_type', 'tissue_type', 'tissue_ontology_term_id', 'cell_type_ontology_term_id', 'assay_ontology_term_id', 'organism_ontology_term_id', 'development_stage_ontology_term_id', 'disease_ontology_term_id', 'self_reported_ethnicity_ontology_term_id', 'is_primary_data'
    var: 'feature_is_filtered'
    uns: 'default_embedding', 'title', 'cxg_lamin_schema_reference', 'cxg_lamin_schema_version'
    obsm: 'X_umap'
adata_cxg.write_h5ad("anndata_human_immune_cells_cxg.h5ad")
!MPLBACKEND=agg uvx cellxgene-schema validate anndata_human_immune_cells_cxg.h5ad || exit 1
Hide code cell output
Loading dependencies
Loading validator modules
Starting validation...
Unable to open 'anndata_human_immune_cells_cxg.h5ad' with AnnData

Note

The Curate class is designed to validate all metadata for adherence to ontologies. It does not reimplement all rules of the cellxgene schema and we therefore recommend running the cellxgene-schema if full adherence beyond metadata is a necessity.