PertCurator¶

Here we use PertCurator to curate perturbation related columns in a subsetted AnnData object of McFarland et al. 2020.

# pip install 'lamindb[jupyter,wetlab]' cellxgene-lamin
!lamin init --storage ./test-pert-curator --modules bionty,wetlab

→ initialized lamindb: anonymous/test-pert-curator

import lamindb as ln
import wetlab as wl
import bionty as bt
import pandas as pd

ln.track()

adata = ln.Artifact.using("laminlabs/lamindata").get(uid="Xk7Qaik9vBLV4PKf0001").load()
adata.obs.head()

→ completing transfer to track Artifact('Xk7Qaik9') as input

→ mapped records:

→ transferred records: Artifact(uid='Xk7Qaik9vBLV4PKf0001'), Storage(uid='D9BilDV2')

/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/lamindb/_record.py:642: FutureWarning: `name` will be removed soon, please pass 'Transfer from `laminlabs/lamindata`' to `description` instead
  transform = Transform(

	depmap_id	cancer	cell_det_rate	cell_line	cell_quality	channel	disease	dose_unit	dose_value	doublet_CL1	doublet_CL2	doublet_GMM_prob	doublet_dev_imp	doublet_z_margin	hash_assignment	hash_tag	num_SNPs	organism	perturbation	perturbation_type	sex	singlet_ID	singlet_dev	singlet_dev_z	singlet_margin	singlet_z_margin	time	tissue_type	tot_reads	nperts	ngenes	ncounts	percent_mito	percent_ribo	chembl-ID
AACTGGTGTCTCTCTG	ACH-000390	True	0.093159	LUDLU-1	normal	nan	lung cancer	µM	0.1	LUDLU1_LUNG	TE14_OESOPHAGUS	2.269468e-10	0.009426	0.403316	nan	nan	481	human	trametinib	drug	Male	LUDLU1_LUNG	0.655877	14.860933	0.462273	12.351139	24	cell_line	787	1	3045	12895.0	3.202792	24.955409	CHEMBL2103875
ATAGGCTCAGATTTCG	ACH-000444	True	0.145728	LU99	normal	2	lung cancer	µM	0.5	LU99_LUNG	MCAS_OVARY	8.562908e-04	0.010173	0.188284	nan	nan	1003	human	afatinib	drug	Male	LU99_LUNG	0.762847	10.648094	0.474590	8.164565	24	cell_line	1597	1	4763	23161.0	7.473771	18.051898	CHEMBL1173655
GCCAAATCAAGCCGTC	ACH-000396	True	0.117330	J82	normal	nan	urinary bladder carcinoma	µM	0.1	J82_URINARY_TRACT	IGR1_SKIN	6.490367e-08	0.009686	1.185862	nan	nan	647	human	dabrafenib	drug	Male	J82_URINARY_TRACT	0.651059	14.740111	0.404508	11.188513	24	cell_line	1159	1	3834	18062.0	2.762706	22.085040	CHEMBL2028663
CGGAGAAGTCGCGTCA	ACH-000997	True	0.005422	HCT-15	low_quality	7	colorectal cancer	µM	0.1	HCT15_LARGE_INTESTINE	NCIH322_LUNG	NaN	0.029753	0.000794	nan	nan	30	human	gemcitabine	drug	Male	HCT15_LARGE_INTESTINE	0.970247	2.852338	0.168971	0.833455	24	cell_line	76	1	178	726.0	70.247934	5.785124	CHEMBL888
TAGTTGGAGATCGATA	ACH-000723	True	0.132708	YD-10B	low_quality	nan	head and neck cancer	nan	NaN	YD10B_UPPER_AERODIGESTIVE_TRACT	647V_URINARY_TRACT	NaN	0.156492	1.556214	nan	nan	874	human	sggpx4-2	CRISPR	Male	YD10B_UPPER_AERODIGESTIVE_TRACT	0.292802	3.272682	0.016459	0.330120	72, 96	cell_line	2105	1	4341	20693.0	0.695887	16.242208	NaN

Curate and register perturbations¶

Required columns:

Either “pert_target” or “pert_name” and “pert_type” (“pert_type” allows: “genetic”, “drug”, “biologic”, “physical”)
If pert_dose = True (default), requires “pert_dose” in form of number+unit. E.g. 10.0nM
If pert_time = True (default), requires “pert_time” in form of number+unit. E.g. 10.0h

# rename the columns to match the expected format
adata.obs["pert_time"] = adata.obs["time"].apply(
    lambda x: str(x).split(", ")[-1] + "h" if pd.notna(x) else x
)  # we only take the last timepoint
adata.obs["pert_dose"] = adata.obs["dose_value"].map(
    lambda x: f"{x}{adata.obs['dose_unit'].iloc[0]}" if pd.notna(x) else None
)
adata.obs.rename(
    columns={"perturbation": "pert_name", "perturbation_type": "pert_type"},
    inplace=True,
)
# fix the perturbation type as suggested by the curator
adata.obs["pert_type"] = adata.obs["pert_type"].cat.rename_categories(
    {"CRISPR": "genetic", "drug": "compound"}
)

adata.obs["tissue_type"] = "cell culture"

curator = wl.PertCurator(adata)

curator.validate()

Show code cell output Hide code cell output

• saving validated records of 'var_index'

✓ added 1869 records from public with Gene.ensembl_gene_id for "var_index": 'ENSG00000102316', 'ENSG00000109472', 'ENSG00000080007', 'ENSG00000203926', 'ENSG00000232301', 'ENSG00000127419', 'ENSG00000266869', 'ENSG00000108960', 'ENSG00000261316', 'ENSG00000126870', 'ENSG00000121797', 'ENSG00000243927', 'ENSG00000143473', 'ENSG00000261488', 'ENSG00000115665', 'ENSG00000180613', 'ENSG00000167283', 'ENSG00000160472', 'ENSG00000273387', 'ENSG00000110768', ...

• saving validated records of 'disease'

✓ added 21 records from public with Disease.name for "disease": 'malignant pancreatic neoplasm', 'sarcoma', 'thyroid cancer', 'breast cancer', 'liver cancer', 'kidney cancer', 'rhabdoid tumor', 'gastric cancer', 'ovarian cancer', 'lung cancer', 'head and neck cancer', 'esophageal cancer', 'bile duct cancer', 'bone cancer', 'neuroblastoma', 'uterine corpus cancer', 'urinary bladder carcinoma', 'skin cancer', 'brain cancer', 'colorectal cancer', ...

• saving validated records of 'sex'

✓ added 2 records from public with Phenotype.name for "sex": 'female', 'male'

• saving validated records of 'cell_line'

✓ added 153 records from public with CellLine.name for "cell_line": 'LN-229 cell', 'NIH:OVCAR-3 cell', 'OAW42 cell', 'MDA-MB-468 cell', 'PANC-1 cell', 'AGS cell', 'NCI-H2009 cell', 'LS1034 cell', 'G-401 cell', 'GB-1 cell', 'SW 480 cell', 'SK-N-SH cell', 'SBC-5 cell', 'RCB1903 cell', 'MIA PaCa-2 cell', 'NCI-H1915 cell', 'RCB0735 cell', 'Ishikawa (Heraklio) 02 ER- cell', 'Daoy cell', 'DK-MG cell', ...

• saving validated records of 'pert_compound'

✓ added 8 records from public with Compound.name for "pert_compound": 'dabrafenib', 'gemcitabine', 'afatinib', 'bortezomib', 'JQ1', 'everolimus', 'trametinib', 'navitoclax'

• mapping "var_index" on Gene.ensembl_gene_id

!   2 terms are not validated: 'ENSG00000255823', 'ENSG00000272370'
    → fix typos, remove non-existent values, or save terms via .add_new_from_var_index()

• mapping "cell_type" on CellType.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("cell_type")

• mapping "development_stage" on DevelopmentalStage.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("development_stage")

• mapping "disease" on Disease.name

!   1 term is not validated: 'pancreatic cancer'
    1 synonym found: "pancreatic cancer" → "malignant pancreatic neoplasm"
    → curate synonyms via .standardize("disease")

• mapping "donor_id" on ULabel.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("donor_id")

• mapping "self_reported_ethnicity" on Ethnicity.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("self_reported_ethnicity")

• mapping "sex" on Phenotype.name

!   3 terms are not validated: 'Male', 'Female', 'Unknown'
    2 synonyms found: "Male" → "male", "Female" → "female"
    → curate synonyms via .standardize("sex")    for remaining terms:
    → fix typos, remove non-existent values, or save terms via .add_new_from("sex")

• mapping "suspension_type" on ULabel.name

!   1 term is not validated: 'cell'
    → fix typos, remove non-existent values, or save terms via .add_new_from("suspension_type")

• mapping "tissue_type" on ULabel.name

!   1 term is not validated: 'cell culture'
    → fix typos, remove non-existent values, or save terms via .add_new_from("tissue_type")

✓ "organism" is validated against Organism.name

• mapping "cell_line" on CellLine.name

!   183 terms are not validated: 'LUDLU-1', 'LU99', 'J82', 'HCT-15', 'YD-10B', 'SNB75', 'SK-MEL-2', 'UM-UC-1', 'AGS', 'COLO-680N', 'KYSE-510', 'Caov-3', 'A-375', 'YD-38', 'PA-TU-8988S', 'LOX IMVI', 'BEN', 'BICR 31', 'SK-MEL-3', 'Sq-1', ...
    153 synonyms found: "LUDLU-1" → "LUDLU-1 cell", "LU99" → "RCB1900 cell", "J82" → "J82 cell", "HCT-15" → "HCT-15 cell", "SNB75" → "SNB75 cell", "SK-MEL-2" → "SK-MEL-2 cell", "AGS" → "AGS cell", "COLO-680N" → "COLO-680N cell", "KYSE-510" → "KYSE-510 cell", "Caov-3" → "Caov-3 cell", "A-375" → "A-375 cell", "PA-TU-8988S" → "PA-TU-8988S cell", "BEN" → "BEN cell", "BICR 31" → "BICR 31 cell", "SK-MEL-3" → "SK-MEL-3 cell", "Sq-1" → "RCB1905 cell", "NIH:OVCAR-3" → "NIH:OVCAR-3 cell", "HPAC" → "HPAC cell", "HCC1143" → "HCC1143 cell", "786-O" → "786-O cell", ...
    → curate synonyms via .standardize("cell_line")    for remaining terms:
    → fix typos, remove non-existent values, or save terms via .add_new_from("cell_line")

• mapping "pert_genetic" on GeneticPerturbation.name

!   4 terms are not validated: 'sggpx4-2', 'sglacz', 'sggpx4-1', 'sgor2j2'
    → fix typos, remove non-existent values, or save terms via .add_new_from("pert_genetic")

• mapping "pert_compound" on Compound.name

!   6 terms are not validated: 'brd3379', 'azd5591', 'control', 'prexasertib', 'taselisib', 'idasanutlin'
    → fix typos, remove non-existent values, or save terms via .add_new_from("pert_compound")

False

Genetic perturbations¶

# register genetic perturbations with their target genes
pert_target_map = {
    "sggpx4-1": "GPX4",
    "sggpx4-2": "GPX4",
    "sgor2j2": "OR2J2",  # cutting control
}

ln.settings.creation.search_names = False
for sg_name, gene_symbol in pert_target_map.items():
    pert = wl.GeneticPerturbation.filter(
        system="CRISPR-Cas9", name=sg_name
    ).one_or_none()
    if pert is None:
        pert = wl.GeneticPerturbation(
            system="CRISPR-Cas9",
            name=sg_name,
            description="cutting control" if sg_name == "sgor2j2" else None,
        ).save()
    target = wl.PerturbationTarget.filter(name=gene_symbol).one_or_none()
    if target is None:
        target = wl.PerturbationTarget(name=gene_symbol).save()
    pert.targets.add(target)
    genes = bt.Gene.filter(symbol=gene_symbol).all()
    if len(genes) == 0:
        genes = bt.Gene.from_values(
            [gene_symbol], field=bt.Gene.symbol, organism="human"
        ).save()
    target.genes.add(*genes)
ln.settings.creation.search_names = True

adata.obs["pert_target"] = adata.obs["pert_genetic"].map(pert_target_map)

# register the negative control without targets: Non-cutting control
wl.GeneticPerturbation(
    name="sglacz", system="CRISPR-Cas9", description="non-cutting control"
).save();

Compounds¶

# the remaining compounds are not in CHEBI and we create records for them
curator.add_new_from("pert_compound")

Curate non-pert metadata¶

# manually fix sex and set assay
adata.obs["sex"] = adata.obs["sex"].astype(str).str.lower()
adata.obs["assay"] = "10x 3' v3"

# subset the adata to only include the validated genes
adata = adata[:, ~adata.var_names.isin(curator.non_validated["var_index"])].copy()

# standardize disease and sex as suggested
curator.standardize("disease")

# Recreate Curator object because we are using a new adata
curator = wl.PertCurator(adata)
curator.validate()

Show code cell output Hide code cell output

→ mapped 'pert_name' to 'pert_genetic'

→ mapped 'pert_name' to 'pert_compound'

✓ added 2 records with Feature.name for "columns": 'assay', 'pert_target'

• saving validated records of 'assay'

✓ added 1 record from public with ExperimentalFactor.name for "assay": '10x 3' v3'

✓ "var_index" is validated against Gene.ensembl_gene_id

✓ "assay" is validated against ExperimentalFactor.name

• mapping "cell_type" on CellType.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("cell_type")

• mapping "development_stage" on DevelopmentalStage.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("development_stage")

• mapping "disease" on Disease.name

!   1 term is not validated: 'pancreatic cancer'
    1 synonym found: "pancreatic cancer" → "malignant pancreatic neoplasm"
    → curate synonyms via .standardize("disease")

• mapping "donor_id" on ULabel.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("donor_id")

• mapping "self_reported_ethnicity" on Ethnicity.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("self_reported_ethnicity")

• mapping "sex" on Phenotype.name

!   1 term is not validated: 'unknown'
    → fix typos, remove non-existent values, or save terms via .add_new_from("sex")

• mapping "suspension_type" on ULabel.name

!   1 term is not validated: 'cell'
    → fix typos, remove non-existent values, or save terms via .add_new_from("suspension_type")

• mapping "tissue_type" on ULabel.name

!   1 term is not validated: 'cell culture'
    → fix typos, remove non-existent values, or save terms via .add_new_from("tissue_type")

✓ "organism" is validated against Organism.name

• mapping "cell_line" on CellLine.name

!   183 terms are not validated: 'LUDLU-1', 'LU99', 'J82', 'HCT-15', 'YD-10B', 'SNB75', 'SK-MEL-2', 'UM-UC-1', 'AGS', 'COLO-680N', 'KYSE-510', 'Caov-3', 'A-375', 'YD-38', 'PA-TU-8988S', 'LOX IMVI', 'BEN', 'BICR 31', 'SK-MEL-3', 'Sq-1', ...
    153 synonyms found: "LUDLU-1" → "LUDLU-1 cell", "LU99" → "RCB1900 cell", "J82" → "J82 cell", "HCT-15" → "HCT-15 cell", "SNB75" → "SNB75 cell", "SK-MEL-2" → "SK-MEL-2 cell", "AGS" → "AGS cell", "COLO-680N" → "COLO-680N cell", "KYSE-510" → "KYSE-510 cell", "Caov-3" → "Caov-3 cell", "A-375" → "A-375 cell", "PA-TU-8988S" → "PA-TU-8988S cell", "BEN" → "BEN cell", "BICR 31" → "BICR 31 cell", "SK-MEL-3" → "SK-MEL-3 cell", "Sq-1" → "RCB1905 cell", "NIH:OVCAR-3" → "NIH:OVCAR-3 cell", "HPAC" → "HPAC cell", "HCC1143" → "HCC1143 cell", "786-O" → "786-O cell", ...
    → curate synonyms via .standardize("cell_line")    for remaining terms:
    → fix typos, remove non-existent values, or save terms via .add_new_from("cell_line")

✓ "pert_target" is validated against PerturbationTarget.name

✓ "pert_genetic" is validated against GeneticPerturbation.name

✓ "pert_compound" is validated against Compound.name

False

curator.add_new_from("all")

curator.validate()

References¶

reference = ln.Reference(
    name="Multiplexed single-cell transcriptional response profiling to define cancer vulnerabilities and therapeutic mechanism of action",
    abbr="McFarland 2020",
    url="https://www.nature.com/articles/s41467-020-17440-w",
    doi="10.1038/s41467-020-17440-w",
    text=(
        "Assays to study cancer cell responses to pharmacologic or genetic perturbations are typically "
        "restricted to using simple phenotypic readouts such as proliferation rate. Information-rich assays, "
        "such as gene-expression profiling, have generally not permitted efficient profiling of a given "
        "perturbation across multiple cellular contexts. Here, we develop MIX-Seq, a method for multiplexed "
        "transcriptional profiling of post-perturbation responses across a mixture of samples with single-cell "
        "resolution, using SNP-based computational demultiplexing of single-cell RNA-sequencing data. We show "
        "that MIX-Seq can be used to profile responses to chemical or genetic perturbations across pools of 100 "
        "or more cancer cell lines. We combine it with Cell Hashing to further multiplex additional experimental "
        "conditions, such as post-treatment time points or drug doses. Analyzing the high-content readout of "
        "scRNA-seq reveals both shared and context-specific transcriptional response components that can identify "
        "drug mechanism of action and enable prediction of long-term cell viability from short-term transcriptional "
        "responses to treatment."
    ),
).save()

Register curated artifact¶

artifact = curator.save_artifact(key="mini_mcFarland20.h5ad")

# link the reference to the artifact
artifact.references.add(reference)

artifact.describe()

Artifact .h5ad/AnnData
├── General
│   ├── .uid = 'rjQ9CS50k5arBLwI0000'
│   ├── .key = 'mini_mcFarland20.h5ad'
│   ├── .size = 2548376
│   ├── .hash = '-Up99P_OUWUKMm84dDKNxA'
│   ├── .n_observations = 1000
│   ├── .path = /home/runner/work/wetlab/wetlab/docs/guide/test-pert-curator/.lamindb/rjQ9CS50k5arBLwI0000.h5ad
│   ├── .created_by = anonymous
│   ├── .created_at = 2025-01-17 14:59:09
│   └── .transform = 'PertCurator'
├── Dataset features/._schemas_m2m
│   ├── var • 1869                  [bionty.Gene]                                                       
│   │   MAGED2                      float                                                               
│   │   CPE                         float                                                               
│   │   DDX43                       float                                                               
│   │   SPANXA2                     float                                                               
│   │   LNCPRESS1                   float                                                               
│   │   TMEM175                     float                                                               
│   │   MMD                         float                                                               
│   │   LINC01834                   float                                                               
│   │   DYNC2I1                     float                                                               
│   │   CCRL2                       float                                                               
│   │   MRPS6                       float                                                               
│   │   KCNH1                       float                                                               
│   │   TBILA                       float                                                               
│   │   SLC5A7                      float                                                               
│   │   GSX2                        float                                                               
│   │   ATP5MG                      float                                                               
│   │   TMEM190                     float                                                               
│   │   GTF2H1                      float                                                               
│   └── obs • 16                    [Feature]                                                           
│       assay                       cat[bionty.ExperimentalF…  10x 3' v3                                
│       cell_line                   cat[bionty.CellLine]       22Rv1, 253J-BV, 42-MG-BA, 639-V, 647-V, …
│       cell_type                   cat[bionty.CellType]       unknown                                  
│       development_stage           cat[bionty.Developmental…  unknown                                  
│       disease                     cat[bionty.Disease]        bile duct cancer, bone cancer, brain can…
│       donor_id                    cat[ULabel]                unknown                                  
│       organism                    cat[bionty.Organism]       human                                    
│       pert_compound               cat[wetlab.Compound]       JQ1, afatinib, azd5591, bortezomib, brd3…
│       pert_genetic                cat[wetlab.GeneticPertur…  sggpx4-1, sggpx4-2, sglacz, sgor2j2      
│       pert_target                 cat[wetlab.PerturbationT…  GPX4, OR2J2                              
│       self_reported_ethnicity     cat[bionty.Ethnicity]      unknown                                  
│       sex                         cat[bionty.Phenotype]      female, male, unknown                    
│       suspension_type             cat[ULabel]                cell                                     
│       tissue_type                 cat[ULabel]                cell culture                             
│       pert_dose                   str                                                                 
│       pert_time                   str                                                                 
└── Labels
    └── .references                 Reference                  Multiplexed single-cell transcriptional …
        .organisms                  bionty.Organism            human                                    
        .cell_types                 bionty.CellType            unknown                                  
        .diseases                   bionty.Disease             sarcoma, thyroid cancer, breast cancer, …
        .cell_lines                 bionty.CellLine            RMUG-S, LN-229, NIH:OVCAR-3, OAW42, MDA-…
        .phenotypes                 bionty.Phenotype           female, male, unknown                    
        .experimental_factors       bionty.ExperimentalFactor  10x 3' v3                                
        .developmental_stages       bionty.DevelopmentalStage  unknown                                  
        .ethnicities                bionty.Ethnicity           unknown                                  
        .compounds                  wetlab.Compound            dabrafenib, gemcitabine, afatinib, borte…
        .perturbation_targets       wetlab.PerturbationTarget  GPX4, OR2J2                              
        .genetic_perturbations      wetlab.GeneticPerturbati…  sggpx4-1, sggpx4-2, sgor2j2, sglacz      
        .ulabels                    ULabel                     unknown, cell, cell culture