Ethnicity

lamindb provides access to the following public Ethnicity ontologies through bionty:

  1. Human Ancestry Ontology

Here we show how to access and search Ethnicity ontologies to standardize new data.

import bionty as bt
import pandas as pd

PublicOntology objects

Let us create a public ontology accessor with .public method, which chooses a default public ontology source from Source. It’s a PublicOntology object, which you can think about as a public registry:

ethnicitys = bt.Ethnicity.public(organism="human")
ethnicitys

→ connected lamindb: testuser1/test-public-ontologies

PublicOntology
Entity: Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

As for registries, you can export the ontology as a DataFrame:

df = ethnicitys.df()
df.head()
name definition synonyms parents
ontology_id
HANCESTRO:0002 region Any Geographic Area Greater Than An Individual... geographical area []
HANCESTRO:0003 country A Collective Generic Term That Refers Here To ... None []
HANCESTRO:0004 ancestry category Population Category Defined Using Ancestry Inf... ancestral group []
HANCESTRO:0005 European Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004]
HANCESTRO:0006 South Asian Includes Individuals Who Either Self-Report Or... None [HANCESTRO:0008]

Unlike registries, you can also export it as a Pronto object via public.ontology.

Look up terms

As for registries, terms can be looked up with auto-complete:

lookup = ethnicitys.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.american

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["American"]

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = ethnicitys.lookup(ethnicitys.ontology_id)

lookup.hancestro_0463

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

Search terms

Search behaves in the same way as it does for registries:

ethnicitys.search("American").head(3)
ontology_id definition synonyms parents __ratio__
name
American HANCESTRO:0463 None None [HANCESTRO:0566] 100.0
African American HANCESTRO:0568 None None [HANCESTRO:0016] 90.0
American Samoan HANCESTRO:0464 None None [HANCESTRO:0566] 90.0

By default, search also covers synonyms:

ethnicitys.search("Caucasian").head(3)
ontology_id definition synonyms parents __ratio__
name
European HANCESTRO:0005 Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004] 100.0
Asian HANCESTRO:0008 Includes Individuals That Either Self-Report O... Asian unspecified [HANCESTRO:0004] 90.0
Asia HANCESTRO:0030 None None [] 90.0

You can turn this off synonym by passing synonyms_field=None:

ethnicitys.search("Caucasian", synonyms_field=None).head(3)
ontology_id definition synonyms parents __ratio__
name
Asian HANCESTRO:0008 Includes Individuals That Either Self-Report O... Asian unspecified [HANCESTRO:0004] 90.0
Asia HANCESTRO:0030 None None [] 90.0
Syrian HANCESTRO:0461 None None [HANCESTRO:0015] 72.0

Search another field (default is .name):

ethnicitys.search(
    "General characterisation of Ancestry of a population",
    field=ethnicitys.definition,
).head()
ontology_id name synonyms parents __ratio__
definition
General Characterisation Of The Ancestry Of A Population Or Individual HANCESTRO:0304 ancestry status None [] 85.245902
A Population With Increased Genetic Homogeneity And Reduced Genetic Variation Due To Cultural Or Geographic Isolation HANCESTRO:0290 genetically isolated population founder population|population isolate [HANCESTRO:0004] 42.857143
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Indian. HANCESTRO:0598 Singaporean Indian None [HANCESTRO:0487] 39.735099
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Malay. HANCESTRO:0597 Singaporean Malay None [HANCESTRO:0007] 38.666667
Population Category Defined Using Ancestry Informative Markers (Aims) Based On Genetic/Genomic Data HANCESTRO:0004 ancestry category ancestral group [] 38.410596

Standardize Ethnicity identifiers

Let us generate a DataFrame that stores a number of Ethnicity identifiers, some of which corrupted:

df_orig = pd.DataFrame(
    index=[
        "Mende",
        "European",
        "South Asian",
        "Arab",
        "This ethnicity does not exist",
    ]
)
df_orig
Mende
European
South Asian
Arab
This ethnicity does not exist

We can check whether any of our values are validated against the ontology reference:

validated = ethnicitys.validate(df_orig.index, ethnicitys.name)
df_orig.index[~validated]

! 1 unique term (20.00%) is not validated: 'This ethnicity does not exist'

Index(['This ethnicity does not exist'], dtype='object')

Ontology source versions

For any given entity, we can choose from a number of versions:

bt.Source.filter(entity="bionty.Ethnicity").df()
Hide code cell output
uid entity organism name version in_db currently_used description url md5 source_website dataframe_artifact_id run_id created_at created_by_id
id
100 MJRq bionty.Ethnicity human hancestro 3.0 False True Human Ancestry Ontology https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None None 2024-11-21 06:52:19.980835+00:00 1 
# only lists the sources that are currently used
bt.Source.filter(entity="bionty.Ethnicity", currently_used=True).df()
uid entity organism name version in_db currently_used description url md5 source_website dataframe_artifact_id run_id created_at created_by_id
id
100 MJRq bionty.Ethnicity human hancestro 3.0 False True Human Ancestry Ontology https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None None 2024-11-21 06:52:19.980835+00:00 1

When instantiating a Bionty object, we can choose a source or version:

source = bt.Source.filter(
    name="hancestro", version="3.0", organism="human"
).one()
ethnicitys= bt.Ethnicity.public(source=source)
ethnicitys

PublicOntology
Entity: Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

The currently used ontologies can be displayed using:

bt.Source.filter(currently_used=True).df()
Hide code cell output
uid entity organism name version in_db currently_used description url md5 source_website dataframe_artifact_id run_id created_at created_by_id
id
1 33TU bionty.Organism vertebrates ensembl release-112 False True Ensembl https://ftp.ensembl.org/pub/release-112/specie... 0ec37e77f4bc2d0b0b47c6c62b9f122d https://www.ensembl.org None None 2024-11-21 06:52:19.976422+00:00 1
6 6bbV bionty.Organism bacteria ensembl release-57 False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org None None 2024-11-21 06:52:19.976600+00:00 1
7 6s9n bionty.Organism fungi ensembl release-57 False True Ensembl http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org None None 2024-11-21 06:52:19.976626+00:00 1
8 2PmT bionty.Organism metazoa ensembl release-57 False True Ensembl http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org None None 2024-11-21 06:52:19.976650+00:00 1
9 7GPH bionty.Organism plants ensembl release-57 False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org None None 2024-11-21 06:52:19.976675+00:00 1
10 4tsk bionty.Organism all ncbitaxon 2023-06-20 False True NCBItaxon Ontology s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon None None 2024-11-21 06:52:19.976700+00:00 1
11 4UGN bionty.Gene human ensembl release-112 False True Ensembl s3://bionty-assets/df_human__ensembl__release-... 4ccda4d88720a326737376c534e8446b https://www.ensembl.org None None 2024-11-21 06:52:19.976726+00:00 1
15 4r4f bionty.Gene mouse ensembl release-112 False True Ensembl s3://bionty-assets/df_mouse__ensembl__release-... 519cf7b8acc3c948274f66f3155a3210 https://www.ensembl.org None None 2024-11-21 06:52:19.976826+00:00 1
19 4RPA bionty.Gene saccharomyces cerevisiae ensembl release-112 False True Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... 11775126b101233525a0a9e2dd64edae https://www.ensembl.org None None 2024-11-21 06:52:19.976928+00:00 1
22 3EYy bionty.Protein human uniprot 2024-03 False True Uniprot s3://bionty-assets/df_human__uniprot__2024-03_... b5b9e7645065b4b3187114f07e3f402f https://www.uniprot.org None None 2024-11-21 06:52:19.977003+00:00 1
25 01RW bionty.Protein mouse uniprot 2024-03 False True Uniprot s3://bionty-assets/df_mouse__uniprot__2024-03_... b1b6a196eb853088d36198d8e3749ec4 https://www.uniprot.org None None 2024-11-21 06:52:19.977077+00:00 1
28 3kDh bionty.CellMarker human cellmarker 2.0 False True CellMarker s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 2024-11-21 06:52:19.977150+00:00 1
29 7bV5 bionty.CellMarker mouse cellmarker 2.0 False True CellMarker s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 2024-11-21 06:52:19.977175+00:00 1
30 6LyR bionty.CellLine all clo 2022-03-21 False True Cell Line Ontology https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO None None 2024-11-21 06:52:19.977199+00:00 1
32 1Lhf bionty.CellType all cl 2024-05-15 False True Cell Ontology http://purl.obolibrary.org/obo/cl/releases/202... 8a8638a9e79567935793e5007704c650 https://obophenotype.github.io/cell-ontology None None 2024-11-21 06:52:19.977248+00:00 1
40 MUtA bionty.Tissue all uberon 2024-08-07 False True Uberon multi-species anatomy ontology http://purl.obolibrary.org/obo/uberon/releases... http://obophenotype.github.io/uberon None None 2024-11-21 06:52:19.977442+00:00 1
49 2L2r bionty.Disease all mondo 2024-06-04 False True Mondo Disease Ontology http://purl.obolibrary.org/obo/mondo/releases/... c47e8edb894c01f2511dfe0751fbc428 https://mondo.monarchinitiative.org None None 2024-11-21 06:52:19.977690+00:00 1
57 4ksw bionty.Disease human doid 2024-05-29 False True Human Disease Ontology http://purl.obolibrary.org/obo/doid/releases/2... bbefd72247d638edfcd31ec699947407 https://disease-ontology.org None None 2024-11-21 06:52:19.977884+00:00 1
65 2a1H bionty.ExperimentalFactor all efo 3.70.0 False True The Experimental Factor Ontology http://www.ebi.ac.uk/efo/releases/v3.70.0/efo.owl https://bioportal.bioontology.org/ontologies/EFO None None 2024-11-21 06:52:19.979985+00:00 1
72 48fB bionty.Phenotype human hp 2024-04-26 False True Human Phenotype Ontology https://github.com/obophenotype/human-phenotyp... e0f2e534eb2ad44a4d45573ef27b508f https://hpo.jax.org None None 2024-11-21 06:52:19.980163+00:00 1
77 4t7Q bionty.Phenotype mammalian mp 2024-06-18 False True Mammalian Phenotype Ontology https://github.com/mgijax/mammalian-phenotype-... 795d8378fe48ec13b41d01a86dd1c86c https://github.com/mgijax/mammalian-phenotype-... None None 2024-11-21 06:52:19.980284+00:00 1
80 sqPX bionty.Phenotype zebrafish zp 2024-04-18 False True Zebrafish Phenotype Ontology https://github.com/obophenotype/zebrafish-phen... 2231ebaa95becf8ff34a33c95a8d4350 https://github.com/obophenotype/zebrafish-phen... None None 2024-11-21 06:52:19.980360+00:00 1
84 6S4q bionty.Phenotype all pato 2024-03-28 False True Phenotype And Trait Ontology http://purl.obolibrary.org/obo/pato/releases/2... 6b1eaacd3d453b34375ce2e31c16328a https://github.com/pato-ontology/pato None None 2024-11-21 06:52:19.980455+00:00 1
86 7Ent bionty.Pathway all go 2024-06-17 False True Gene Ontology https://data.bioontology.org/ontologies/GO/sub... 7fa7ade5e3e26eab3959a7e4bc89ad4f http://geneontology.org None None 2024-11-21 06:52:19.980504+00:00 1
91 3rm9 BFXPipeline all lamin 1.0.0 False True Bioinformatics Pipeline s3://bionty-assets/df_all__lamin__1.0.0__BFXpi... https://lamin.ai None None 2024-11-21 06:52:19.980623+00:00 1
92 ugaI Drug all dron 2024-08-05 False True Drug Ontology https://data.bioontology.org/ontologies/DRON/s... https://bioportal.bioontology.org/ontologies/DRON None None 2024-11-21 06:52:19.980646+00:00 1
96 1GbF bionty.DevelopmentalStage human hsapdv 2024-05-28 False True Human Developmental Stages https://github.com/obophenotype/developmental-... https://github.com/obophenotype/developmental-... None None 2024-11-21 06:52:19.980741+00:00 1
98 10va bionty.DevelopmentalStage mouse mmusdv 2024-05-28 False True Mouse Developmental Stages https://github.com/obophenotype/developmental-... https://github.com/obophenotype/developmental-... None None 2024-11-21 06:52:19.980787+00:00 1
100 MJRq bionty.Ethnicity human hancestro 3.0 False True Human Ancestry Ontology https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None None 2024-11-21 06:52:19.980835+00:00 1
101 5JnV BioSample all ncbi 2023-09 False True NCBI BioSample attributes s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... None None 2024-11-21 06:52:19.980859+00:00 1