Flow cytometry¶

You’ll learn how to manage a growing number of flow cytometry datasets as a single queryable collection.

Specifically, you will

read a single .fcs file as an AnnData and seed a versioned collection with it (, current page)
append a new dataset (a new .fcs file) to create a new version of the collection ()
query individual files and cell markers ()
analyze the collection and store results as plots ()

# !pip install 'lamindb[jupyter,bionty]'
!lamin init --storage ./test-facs --schema bionty

import lamindb as ln
import bionty as bt
import readfcs

bt.settings.organism = "human"  # globally set organism to human

ln.track("OWuTtS4SApon0000")

Ingest a first artifact¶

Access ¶

We start with a flow cytometry file from Alpert et al., Nat. Med. (2019).

Calling the following function downloads the artifact and pre-populates a few relevant registries:

ln.core.datasets.file_fcs_alpert19(populate_registries=True)

PosixPath('Alpert19.fcs')

We use readfcs to read the raw fcs file into memory and create an AnnData object:

adata = readfcs.read("Alpert19.fcs")
adata

It has the following features:

adata.var.head(10)

Show code cell output Hide code cell output

	n	channel	marker	$PnB	$PnR
Time	1	Time		32	2097152
Cell_length	2	Cell_length		32	128
CD57	3	(In113)Dd	CD57	32	8192
Dead	4	(In115)Dd	Dead	32	4096
(Ba138)Dd	5	(Ba138)Dd		32	4096
Bead	6	(Ce140)Dd	Bead	32	16384
CD19	7	(Nd142)Dd	CD19	32	4096
CD4	8	(Nd143)Dd	CD4	32	4096
CD8	9	(Nd144)Dd	CD8	32	4096
IgD	10	(Nd146)Dd	IgD	32	8192

Transform: normalize ¶

In this use case, we’d like to ingest & store curated data, and hence, we split signal and normalize using the pytometry package.

import pytometry as pm

First, we’ll split the signal from heigh and area metadata:

pm.pp.split_signal(adata, var_key="channel", data_type="cytof")

'area' is not in adata.var['signal_type']. Return all.

adata

Normalize the collection:

pm.tl.normalize_arcsinh(adata, cofactor=150)

Note

If the collection was a flow collection, you’ll also have to compensate the data, if possible. The metadata should contain a compensation matrix, which could then be run by the pytometry compensation function. In the case here, its a cyTOF collection, which doesn’t (really) require compensation.

Validate: cell markers ¶

First, we validate features in .var using CellMarker:

validated = bt.CellMarker.validate(adata.var.index)

We see that many features aren’t validated because they’re not standardized.

Hence, let’s standardize feature names & validate again:

adata.var.index = bt.CellMarker.standardize(adata.var.index)
validated = bt.CellMarker.validate(adata.var.index)

The remaining non-validated features don’t appear to be cell markers but rather metadata features.

Let’s move them into adata.obs:

adata.obs = adata[:, ~validated].to_df()
adata = adata[:, validated].copy()

Now we have a clean panel of 35 validated cell markers:

validated = bt.CellMarker.validate(adata.var.index)
assert all(validated)  # all markers are validated

Register: metadata ¶

Next, let’s register the metadata features we moved to .obs.

For this, we create one feature record for each column in the .obs dataframe:

features = ln.Feature.from_df(adata.obs)
ln.save(features)

We use the Experimental Factor Ontology through Bionty to create a “FACS” label:

bt.ExperimentalFactor.public().search("FACS").head(2)  # search the public ontology

	ontology_id	definition	synonyms	parents	molecule	instrument	measurement	__ratio__
name
fluorescence-activated cell sorting	EFO:0009108	A Flow Cytometry Assay That Provides A Method ...	FAC sorting\|FACS	[]	None	None	None	100.0
BALB/c	EFO:0000602	Balb/C Is A Mouse Strain Of Albion Mice.	BALB/cJ\|BALBc\|C	[]	None	None	None	90.0

We found one for “FACS”, let’s save it to our in-house registry:

# import the FACS record from the public ontology and save it to the registry
facs = bt.ExperimentalFactor.from_source(ontology_id="EFO:0009108")
facs.save()

ExperimentalFactor(uid='36GhLFoE', name='fluorescence-activated cell sorting', ontology_id='EFO:0009108', synonyms='FAC sorting|FACS', description='A Flow Cytometry Assay That Provides A Method For Sorting A Heterogeneous Mixture Of Biological Cells Into Two Or More Containers, One Cell At A Time, Based Upon The Specific Light Scattering And Fluorescent Characteristics Of Each Cell.
The Cells Are Suspended In A Stream Of Fluid And Forced Individually Through A Vibrating Nozzle, Then Exposed To A Laser Beam And The Resulting Fluorescence And Scattered Light Is Detected. Finally The Cells Are Sorted By Applying An Electrical Charge To Droplets Of The Fluid And Deflecting It To The Left Or Right Using Charged Electrodes.', created_by_id=1, run_id=1, source_id=65, created_at=2024-11-21 06:55:24 UTC)

We don’t find one for “CyToF”, however, so, let’s create it without importing from a public ontology but label it as a child of “is_cytometry_assay”:

cytof = bt.ExperimentalFactor(name="CyTOF")
cytof.save()
is_cytometry_assay = bt.ExperimentalFactor(name="is_cytometry_assay")
is_cytometry_assay.save()
cytof.parents.add(is_cytometry_assay)
facs.parents.add(is_cytometry_assay)

is_cytometry_assay.view_parents(with_children=True)

Let us look at the content of the registry:

bt.ExperimentalFactor.df()

Show code cell output Hide code cell output

	uid	name	ontology_id	abbr	synonyms	description	molecule	instrument	measurement	source_id	run_id	created_at	created_by_id
id
3	21Qymj4Q	is_cytometry_assay	None	None	None	None	None	None	None	NaN	1	2024-11-21 06:55:24.602458+00:00	1
2	ogoPdeOk	CyTOF	None	None	None	None	None	None	None	NaN	1	2024-11-21 06:55:24.597654+00:00	1
1	36GhLFoE	fluorescence-activated cell sorting	EFO:0009108	None	FAC sorting\|FACS	A Flow Cytometry Assay That Provides A Method ...	None	None	None	65.0	1	2024-11-21 06:55:24.583182+00:00	1

Register: save & annotate with metadata ¶

curate = ln.Curator.from_anndata(adata, var_index=bt.CellMarker.name, categoricals={})
curate.validate()

artifact = curate.save_artifact(description="Alpert19")

Add more labels:

experimental_factors = bt.ExperimentalFactor.lookup()
organisms = bt.Organism.lookup()

artifact.labels.add(experimental_factors.cytof)
artifact.labels.add(organisms.human)

Inspect the saved artifact¶

Inspect features on a high level:

artifact.features

Inspect low-level features in .var:

artifact.features["var"].df().head()

Show code cell output Hide code cell output

	uid	name	description	gene_symbol	ncbi_gene_id	uniprotkb_id	source_id	organism_id	run_id	created_at	created_by_id
id
1	5R8E1YHbOROI	CD57	None	B3GAT1	27087	Q9P2W7	28	1	1	2024-11-21 06:55:22.086146+00:00	1
2	19Sxm5VN87z8	Cd19	None	CD19	930	P15391	28	1	1	2024-11-21 06:55:22.086214+00:00	1
3	5CbKd6B4ILaq	Cd4	None	CD4	920	B4DT49	28	1	1	2024-11-21 06:55:22.086246+00:00	1
4	1xRpnOHIkdyE	CD8	None	CD8A	925	P01732	28	1	1	2024-11-21 06:55:22.086277+00:00	1
5	7fdKraUfUF8w	Igd	None	None	None	None	28	1	1	2024-11-21 06:55:22.086305+00:00	1

Use auto-complete for marker names in the var featureset:

markers = artifact.features["var"].lookup()
markers.cd14

In a plot, we can now easily also show gene symbol and Uniprot ID:

import scanpy as sc

sc.pp.pca(adata)
sc.pl.pca(
    adata,
    color=markers.cd14.name,
    title=(
        f"{markers.cd14.name} / {markers.cd14.gene_symbol} /"
        f" {markers.cd14.uniprotkb_id}"
    ),
)

_images/6e85500e641202e3f445fcd99b9435cd8107886c39a53d3e62bc87e21a83db29.png

Create a collection from the artifact¶

collection = ln.Collection(
    artifact, name="My versioned cytometry collection", version="1"
)
collection

Let’s inspect the features measured in this collection which were inherited from the artifact:

collection.features

This looks all good, hence, let’s save it:

collection.save()