Ethnicity

lamindb provides access to the following public Ethnicity ontologies through bionty:

  1. Human Ancestry Ontology

Here we show how to access and search Ethnicity ontologies to standardize new data.

import bionty as bt
import pandas as pd

💡 connected lamindb: testuser1/test-public-ontologies

PublicOntology objects

Let us create a public ontology accessor with .public method, which chooses a default public ontology source from PublicSource. It’s a PublicOntology object, which you can think about as a public registry:

ethnicitys = bt.Ethnicity.public(organism="human")
ethnicitys

PublicOntology
Entity: Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

As for registries, you can export the ontology as a DataFrame:

df = ethnicitys.df()
df.head()
name definition synonyms parents
ontology_id
HANCESTRO:0002 region Any Geographic Area Greater Than An Individual... geographical area []
HANCESTRO:0003 country A Collective Generic Term That Refers Here To ... None []
HANCESTRO:0004 ancestry category Population Category Defined Using Ancestry Inf... ancestral group []
HANCESTRO:0005 European Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004]
HANCESTRO:0006 South Asian Includes Individuals Who Either Self-Report Or... None [HANCESTRO:0008]

Unlike registries, you can also export it as a Pronto object via public.ontology.

Look up terms

As for registries, terms can be looked up with auto-complete:

lookup = ethnicitys.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.american

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["American"]

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = ethnicitys.lookup(ethnicitys.ontology_id)

lookup.hancestro_0463

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

Search terms

Search behaves in the same way as it does for registries:

ethnicitys.search("American").head(3)
ontology_id definition synonyms parents __ratio__
name
American HANCESTRO:0463 None None [HANCESTRO:0566] 100.0
African American HANCESTRO:0568 None None [HANCESTRO:0016] 90.0
African American or Afro-Caribbean HANCESTRO:0016 Includes Individuals Who Either Self-Report Or... None [HANCESTRO:0004] 90.0

By default, search also covers synonyms:

ethnicitys.search("Caucasian").head(3)
ontology_id definition synonyms parents __ratio__
name
European HANCESTRO:0005 Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004] 100.0
Asian HANCESTRO:0008 Includes Individuals That Either Self-Report O... Asian unspecified [HANCESTRO:0004] 90.0
Asia HANCESTRO:0030 None None [] 90.0

You can turn this off synonym by passing synonyms_field=None:

ethnicitys.search("Caucasian", synonyms_field=None).head(3)
ontology_id definition synonyms parents __ratio__
name
Asian HANCESTRO:0008 Includes Individuals That Either Self-Report O... Asian unspecified [HANCESTRO:0004] 90.0
Asia HANCESTRO:0030 None None [] 90.0
Syrian HANCESTRO:0461 None None [HANCESTRO:0015] 72.0

Search another field (default is .name):

ethnicitys.search(
    "General characterisation of Ancestry of a population",
    field=ethnicitys.definition,
).head()
ontology_id name synonyms parents __ratio__
definition
General Characterisation Of The Ancestry Of A Population Or Individual HANCESTRO:0304 ancestry status None [] 85.245902
A Population With Increased Genetic Homogeneity And Reduced Genetic Variation Due To Cultural Or Geographic Isolation HANCESTRO:0290 genetically isolated population founder population|population isolate [HANCESTRO:0004] 42.857143
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Indian. HANCESTRO:0598 Singaporean Indian None [HANCESTRO:0487] 39.735099
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Malay. HANCESTRO:0597 Singaporean Malay None [HANCESTRO:0007] 38.666667
Population Category Defined Using Ancestry Informative Markers (Aims) Based On Genetic/Genomic Data HANCESTRO:0004 ancestry category ancestral group [] 38.410596

Standardize Ethnicity identifiers

Let us generate a DataFrame that stores a number of Ethnicity identifiers, some of which corrupted:

df_orig = pd.DataFrame(
    index=[
        "Mende",
        "European",
        "South Asian",
        "Arab",
        "This ethnicity does not exist",
    ]
)
df_orig
Mende
European
South Asian
Arab
This ethnicity does not exist

We can check whether any of our values are validated against the ontology reference:

validated = ethnicitys.validate(df_orig.index, ethnicitys.name)
df_orig.index[~validated]

❗ 1 term (20.00%) is not validated: This ethnicity does not exist

Index(['This ethnicity does not exist'], dtype='object')

Ontology source versions

For any given entity, we can choose from a number of versions:

bt.PublicSource.filter(entity="Ethnicity").df()
uid entity organism currently_used source source_name version url md5 source_website run_id created_by_id updated_at
id
72 3Tlc Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None 1 2024-07-26 12:19:48.331560+00:00

When instantiating a Bionty object, we can choose a source or version:

public_source = bt.PublicSource.filter(
    source="hancestro", version="3.0", organism="human"
).one()
ethnicitys= bt.Ethnicity.public(public_source=public_source)
ethnicitys

PublicOntology
Entity: Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

The currently used ontologies can be displayed using:

bt.PublicSource.filter(currently_used=True).df()
Hide code cell output
uid entity organism currently_used source source_name version url md5 source_website run_id created_by_id updated_at
id
1 5Dlc Organism vertebrates True ensembl Ensembl release-112 https://ftp.ensembl.org/pub/release-112/specie... 0ec37e77f4bc2d0b0b47c6c62b9f122d https://www.ensembl.org None 1 2024-07-26 12:19:48.322631+00:00
6 2Jzh Organism bacteria True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org None 1 2024-07-26 12:19:48.323129+00:00
7 1kdI Organism fungi True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org None 1 2024-07-26 12:19:48.323223+00:00
8 2mIM Organism metazoa True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org None 1 2024-07-26 12:19:48.323318+00:00
9 2XQ6 Organism plants True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org None 1 2024-07-26 12:19:48.323411+00:00
10 1Vzs Organism all True ncbitaxon NCBItaxon Ontology 2023-06-20 s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon None 1 2024-07-26 12:19:48.323506+00:00
11 1hx4 Gene human True ensembl Ensembl release-112 s3://bionty-assets/df_human__ensembl__release-... 4ccda4d88720a326737376c534e8446b https://www.ensembl.org None 1 2024-07-26 12:19:48.323601+00:00
15 76FX Gene mouse True ensembl Ensembl release-112 s3://bionty-assets/df_mouse__ensembl__release-... 519cf7b8acc3c948274f66f3155a3210 https://www.ensembl.org None 1 2024-07-26 12:19:48.323976+00:00
19 7LW6 Gene saccharomyces cerevisiae True ensembl Ensembl release-112 s3://bionty-assets/df_saccharomyces cerevisiae... 11775126b101233525a0a9e2dd64edae https://www.ensembl.org None 1 2024-07-26 12:19:48.324354+00:00
22 7llW Protein human True uniprot Uniprot 2023-03 s3://bionty-assets/df_human__uniprot__2023-03_... 1c46e85c6faf5eff3de5b4e1e4edc4d3 https://www.uniprot.org None 1 2024-07-26 12:19:48.324642+00:00
24 5U7J Protein mouse True uniprot Uniprot 2023-03 s3://bionty-assets/df_mouse__uniprot__2023-03_... 9d5e9a8225011d3218e10f9bbb96a46c https://www.uniprot.org None 1 2024-07-26 12:19:48.324829+00:00
26 5nkB CellMarker human True cellmarker CellMarker 2.0 s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker None 1 2024-07-26 12:19:48.325016+00:00
27 6AFz CellMarker mouse True cellmarker CellMarker 2.0 s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker None 1 2024-07-26 12:19:48.325111+00:00
28 6cbC CellLine all True clo Cell Line Ontology 2022-03-21 https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO None 1 2024-07-26 12:19:48.325204+00:00
29 3DeN CellType all True cl Cell Ontology 2024-02-13 http://purl.obolibrary.org/obo/cl/releases/202... https://obophenotype.github.io/cell-ontology None 1 2024-07-26 12:19:48.325298+00:00
34 1AyH Tissue all True uberon Uberon multi-species anatomy ontology 2024-02-20 http://purl.obolibrary.org/obo/uberon/releases... 2048667b5fdf93192384bdf53cafba18 http://obophenotype.github.io/uberon None 1 2024-07-26 12:19:48.325766+00:00
39 LoCG Disease all True mondo Mondo Disease Ontology 2024-02-06 http://purl.obolibrary.org/obo/mondo/releases/... 78914fa236773c5ea6605f7570df6245 https://mondo.monarchinitiative.org None 1 2024-07-26 12:19:48.326243+00:00
44 2mou Disease human True doid Human Disease Ontology 2024-01-31 http://purl.obolibrary.org/obo/doid/releases/2... b36c15a4610757094f8db64b78ae2693 https://disease-ontology.org None 1 2024-07-26 12:19:48.326743+00:00
51 4usY ExperimentalFactor all True efo The Experimental Factor Ontology 3.63.0 http://www.ebi.ac.uk/efo/releases/v3.63.0/efo.owl 603e6f6981d53d501c5921aa3940b095 https://bioportal.bioontology.org/ontologies/EFO None 1 2024-07-26 12:19:48.327398+00:00
54 2WLc Phenotype human True hp Human Phenotype Ontology 2024-03-06 https://github.com/obophenotype/human-phenotyp... 36b0d00c24a68edb9131707bc146a4c7 https://hpo.jax.org None 1 2024-07-26 12:19:48.327687+00:00
58 6zE1 Phenotype mammalian True mp Mammalian Phenotype Ontology 2024-02-07 https://github.com/mgijax/mammalian-phenotype-... 31c27ed2c7d5774f8b20a77e4e1fd278 https://github.com/mgijax/mammalian-phenotype-... None 1 2024-07-26 12:19:48.328058+00:00
60 7EnA Phenotype zebrafish True zp Zebrafish Phenotype Ontology 2024-01-22 https://github.com/obophenotype/zebrafish-phen... 01600a5d392419b27fc567362d4cfff8 https://github.com/obophenotype/zebrafish-phen... None 1 2024-07-26 12:19:48.328243+00:00
63 55lY Phenotype all True pato Phenotype And Trait Ontology 2023-05-18 http://purl.obolibrary.org/obo/pato/releases/2... bd472f4971492109493d4ad8a779a8dd https://github.com/pato-ontology/pato None 1 2024-07-26 12:19:48.328519+00:00
64 48aa Pathway all True go Gene Ontology 2023-05-10 https://data.bioontology.org/ontologies/GO/sub... e9845499eadaef2418f464cd7e9ac92e http://geneontology.org None 1 2024-07-26 12:19:48.328612+00:00
67 3rm9 BFXPipeline all True lamin Bioinformatics Pipeline 1.0.0 s3://bionty-assets/bfxpipelines.json a7eff57a256994692fba46e0199ffc94 https://lamin.ai None 1 2024-07-26 12:19:48.328895+00:00
68 5alK Drug all True dron Drug Ontology 2024-03-02 https://data.bioontology.org/ontologies/DRON/s... 84138459de4f65034e979f4e46783747 https://bioportal.bioontology.org/ontologies/DRON None 1 2024-07-26 12:19:48.328989+00:00
70 7CRn DevelopmentalStage human True hsapdv Human Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/HSAPDV/11... 52181d59df84578ed69214a5cb614036 https://github.com/obophenotype/developmental-... None 1 2024-07-26 12:19:48.329178+00:00
71 16tR DevelopmentalStage mouse True mmusdv Mouse Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/MMUSDV/9/... 5bef72395d853c7f65450e6c2a1fc653 https://github.com/obophenotype/developmental-... None 1 2024-07-26 12:19:48.329271+00:00
72 3Tlc Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None 1 2024-07-26 12:19:48.331560+00:00
73 5JnV BioSample all True ncbi NCBI BioSample attributes 2023-09 s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... None 1 2024-07-26 12:19:48.331669+00:00