Jupyter Notebook

Manage biological registries

Registries manage the formalized knowledge & experimental design that anchor dry & wetlab work.

In LaminDB, registries are standard SQL tables, equipped with mechanisms that avoid typos & duplicated data.

In addition, LaminDB makes it easy to import records from public biological ontologies, based on plug-in bionty.

In this notebook, you’ll see how to manage an in-house ontology anchored in public knowledge.

(If you also manage experimental design through registries, you can access all metadata through one API and store it in one simple SQL database.)

Setup

Let us create an instance that has bionty mounted, which extends to include biological registries:

!lamin init --storage ./test-registries --schema bionty
Hide code cell output
💡 connected lamindb: testuser1/test-registries
import lamindb as ln
import bionty as bt

ln.connect("test-registries")
ln.settings.verbosity = "info"
💡 connected lamindb: testuser1/test-registries

Let’s pre-populate the cell type registry with a few records:

bt.Organism.from_public(name="human").save()
bt.CellType.from_public(name="T cell").save()
bt.CellType(name="my T cell subtype").save()
Hide code cell output
✅ created 1 Organism record from Bionty matching name: 'human'
✅ created 1 CellType record from Bionty matching name: 'T cell'
💡 also saving parents of CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:45 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000542'
❗ now recursing through parents: this only happens once, but is much slower than bulk saving
💡 also saving parents of CellType(uid='X6c7osZ5', name='lymphocyte', ontology_id='CL:0000542', description='A Lymphocyte Is A Leukocyte Commonly Found In The Blood And Lymph That Has The Characteristics Of A Large Nucleus, A Neutral Staining Cytoplasm, And Prominent Heterochromatin.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:45 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000842'
💡 also saving parents of CellType(uid='2Jgr5Xx4', name='mononuclear cell', ontology_id='CL:0000842', synonyms='mononuclear leukocyte', description='A Leukocyte With A Single Non-Segmented Nucleus In The Mature Form.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:46 UTC')
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000738', 'CL:0000226'
💡 also saving parents of CellType(uid='3VEAlFdi', name='leukocyte', ontology_id='CL:0000738', synonyms='white blood cell|leucocyte', description='An Achromatic Cell Of The Myeloid Or Lymphoid Lineages Capable Of Ameboid Movement, Found In Blood Or Other Tissue.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
✅ created 4 CellType records from Bionty matching ontology_id: 'CL:0000255', 'CL:0000219', 'CL:0002242', 'CL:0000988'
💡 also saving parents of CellType(uid='4WnpvUTH', name='eukaryotic cell', ontology_id='CL:0000255', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000000'
💡 also saving parents of CellType(uid='2K93w3xO', name='motile cell', ontology_id='CL:0000219', description='A Cell That Moves By Its Own Activities.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
💡 also saving parents of CellType(uid='u3sr1Gdf', name='nucleate cell', ontology_id='CL:0002242', description='A Cell Containing At Least One Nucleus.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
💡 also saving parents of CellType(uid='4Ilrnj9U', name='hematopoietic cell', ontology_id='CL:0000988', synonyms='haematopoietic cell|hemopoietic cell|haemopoietic cell', description='A Cell Of A Hematopoietic Lineage.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
💡 also saving parents of CellType(uid='2cXC7cgF', name='single nucleate cell', ontology_id='CL:0000226', description='A Cell With A Single Nucleus.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:47 UTC')
❗ records with similar names exist! did you mean to load one of them?
uid name ontology_id abbr synonyms description public_source_id run_id created_by_id updated_at
id
1 22LvKd01 T cell CL:0000084 None T-lymphocyte|T lymphocyte|T-cell A Type Of Lymphocyte Whose Defining Characteri... 29 None 1 2024-06-19 23:12:45.218209+00:00
2 X6c7osZ5 lymphocyte CL:0000542 None None A Lymphocyte Is A Leukocyte Commonly Found In ... 29 None 1 2024-06-19 23:12:45.885134+00:00
3 2Jgr5Xx4 mononuclear cell CL:0000842 None mononuclear leukocyte A Leukocyte With A Single Non-Segmented Nucleu... 29 None 1 2024-06-19 23:12:46.586857+00:00
4 3VEAlFdi leukocyte CL:0000738 None white blood cell|leucocyte An Achromatic Cell Of The Myeloid Or Lymphoid ... 29 None 1 2024-06-19 23:12:47.253204+00:00
5 2cXC7cgF single nucleate cell CL:0000226 None None A Cell With A Single Nucleus. 29 None 1 2024-06-19 23:12:47.253352+00:00
6 4WnpvUTH eukaryotic cell CL:0000255 None None None 29 None 1 2024-06-19 23:12:47.933882+00:00
7 2K93w3xO motile cell CL:0000219 None None A Cell That Moves By Its Own Activities. 29 None 1 2024-06-19 23:12:47.934030+00:00
8 u3sr1Gdf nucleate cell CL:0002242 None None A Cell Containing At Least One Nucleus. 29 None 1 2024-06-19 23:12:47.934170+00:00
9 4Ilrnj9U hematopoietic cell CL:0000988 None haematopoietic cell|hemopoietic cell|haemopoie... A Cell Of A Hematopoietic Lineage. 29 None 1 2024-06-19 23:12:47.934307+00:00
10 4bKGljt0 cell CL:0000000 None None A Material Entity Of Anatomical Origin (Part O... 29 None 1 2024-06-19 23:12:48.529727+00:00

Access records in public ontologies

We start with a public ontology for cell types.

.public() gives you access to the ReferenceTable, a class for accessing public ontologies.

ReferenceTable provides simple access to standardized public ontologies that Lamin hosts for reliable and performant access. For most ReferenceTable objects, you can access the underlying ontology through Pronto.

Let’s create a ReferenceTable object:

public = bt.CellType.public()
public
PublicOntology
Entity: CellType
Organism: all
Source: cl, 2024-02-13
#terms: 2918

We can use it to search the public ontology against cell types:

public.search("gamma delta T cell").head(3)
Hide code cell output
ontology_id definition synonyms parents __ratio__
name
gamma-delta T cell CL:0000798 A T Cell That Expresses A Gamma-Delta T Cell R... gammadelta T cell|gamma-delta T-cell|gamma-del... [CL:0000084] 100.000000
CD27-negative gamma-delta T cell CL:0002125 A Circulating Gamma-Delta T Cell That Expresse... gammadelta-17 cells [CL:0000800] 86.486486
mature gamma-delta T cell CL:0000800 A Gamma-Delta T Cell That Has A Mature Phenoty... mature gamma-delta T-lymphocyte|mature gamma-d... [CL:0002419, CL:0000798] 83.720930

And we can also use it to look up cell types with auto-complete:

lookup = public.lookup()
lookup.gamma_delta_t_cell
Hide code cell output
CellType(ontology_id='CL:0000798', name='gamma-delta T cell', definition='A T Cell That Expresses A Gamma-Delta T Cell Receptor Complex.', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', parents=array(['CL:0000084'], dtype=object))

Create records in in-house ontologies

We can now create a record for our in-house SQL registry by passing the result of the lookup in the public ontology to the CellType constructor:

gdt_cell = bt.CellType(lookup.gamma_delta_t_cell)
gdt_cell
Hide code cell output
❗ records with similar names exist! did you mean to load one of them?
uid name ontology_id abbr synonyms description public_source_id run_id created_by_id updated_at
id
1 22LvKd01 T cell CL:0000084 None T-lymphocyte|T lymphocyte|T-cell A Type Of Lymphocyte Whose Defining Characteri... 29.0 None 1 2024-06-19 23:12:45.218209+00:00
3 2Jgr5Xx4 mononuclear cell CL:0000842 None mononuclear leukocyte A Leukocyte With A Single Non-Segmented Nucleu... 29.0 None 1 2024-06-19 23:12:46.586857+00:00
4 3VEAlFdi leukocyte CL:0000738 None white blood cell|leucocyte An Achromatic Cell Of The Myeloid Or Lymphoid ... 29.0 None 1 2024-06-19 23:12:47.253204+00:00
5 2cXC7cgF single nucleate cell CL:0000226 None None A Cell With A Single Nucleus. 29.0 None 1 2024-06-19 23:12:47.253352+00:00
6 4WnpvUTH eukaryotic cell CL:0000255 None None None 29.0 None 1 2024-06-19 23:12:47.933882+00:00
7 2K93w3xO motile cell CL:0000219 None None A Cell That Moves By Its Own Activities. 29.0 None 1 2024-06-19 23:12:47.934030+00:00
8 u3sr1Gdf nucleate cell CL:0002242 None None A Cell Containing At Least One Nucleus. 29.0 None 1 2024-06-19 23:12:47.934170+00:00
9 4Ilrnj9U hematopoietic cell CL:0000988 None haematopoietic cell|hemopoietic cell|haemopoie... A Cell Of A Hematopoietic Lineage. 29.0 None 1 2024-06-19 23:12:47.934307+00:00
10 4bKGljt0 cell CL:0000000 None None A Material Entity Of Anatomical Origin (Part O... 29.0 None 1 2024-06-19 23:12:48.529727+00:00
11 2eNGzkOc my T cell subtype None None None None NaN None 1 2024-06-19 23:12:48.631825+00:00
CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', created_by_id=1, public_source_id=29)

(Alternatively, we could construct the gamma delta T cell via from_public(), which is synonyms-aware.)

bt.CellType.from_public(ontology_id="CL:0000798")
Hide code cell output
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000798'
CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', description='A T Cell That Expresses A Gamma-Delta T Cell Receptor Complex.', created_by_id=1, public_source_id=29)

When we save this record to the registry, logging informs us that we’re also saving parent ontological terms:

gdt_cell.save()
Hide code cell output
💡 also saving parents of CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:50 UTC')
Will I always see parents being saved?

No, this only happens a single time.

  • If we accidentally save the same record again, it will be recognized that the record and all parents are already in the registry.

  • If we save another record that has overlapping parents, only new parents will be saved.

View the ontological hierarchy:

gdt_cell.view_parents()
_images/abfd2e62cbde35910b09380a9851547aff2cf029c2a35a8e0286f57741ad7ea4.svg

Or access the parents directly:

gdt_cell.parents.df()
Hide code cell output
uid name ontology_id abbr synonyms description public_source_id run_id created_by_id updated_at
id
1 22LvKd01 T cell CL:0000084 None T-lymphocyte|T lymphocyte|T-cell A Type Of Lymphocyte Whose Defining Characteri... 29 None 1 2024-06-19 23:12:45.218209+00:00

You can construct custom hierarchies of terms by specifying parents:

my_celltype = bt.CellType.filter(name="my T cell subtype").one()
my_celltype.parents.add(gdt_cell)
gdt_cell.view_parents(distance=2, with_children=True)
_images/d489b6fee84c2d1771a61acdca4a5302fdb608708cf47967f9f25b7e2d739a40.svg

This cell type and all its parents can now be queried & searched in the registry using bionty.CellType.filter and bionty.CellType.search.

Load records for values in data sources

When accessing data sources, one often encounters bulk references to entities that might be corrupted or standardized using different standardization schemes.

Let’s consider an example based on an AnnData object, in the cell_type annotations of this AnnData object, we find 4 references to cell types:

adata = ln.core.datasets.anndata_with_obs()
adata.obs.cell_type.value_counts()
Hide code cell output
T cell                     10
hematopoietic stem cell    10
hepatocyte                 10
my new cell type           10
Name: cell_type, dtype: int64

We’d like to load the corresponding records in our in-house ontology to annotate a dataset.

To this end, you’ll typically use from_values, which will both validate & load records that match the values.

cell_types = bt.CellType.from_values(adata.obs.cell_type)
cell_types
Hide code cell output
✅ loaded 1 CellType record matching name: 'T cell'
✅ created 2 CellType records from Bionty matching name: 'hematopoietic stem cell', 'hepatocyte'
did not create CellType record for 1 non-validated name: 'my new cell type'
[CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:45 UTC'),
 CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29),
 CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29)]

Logging informed us that 3 cell types were validated. Since we loaded these records at the same time, we could readily use them to annotate a dataset.

What happened under-the-hood?

.from_values() performs the following look ups:

  1. If registry records match the values, load these records

  2. If values match synonyms of registry records, load these records

  3. (bionty-only) If no record in the registry matches, attempt to load records from a public reference

  4. (bionty-only) Same as 3. but based on synonyms

No records will be returned if input field values aren’t mappable.

Example:

celltype_names = [
    "gamma-delta T cell",  # existing record with the same name
    "T lymphocyte",  # existing record with synonym
    "hepatocyte",  # public record with the same name
    "HSC",  # public record with synonym
    "my new cell type",  # Not exist in in-house registry, not exist in public reference
]
bionty.CellType.from_values(celltype_names)

This returns records for all names except from “my new cell type”.

If you’d like to add this new value to the registry, do it like so:

my_celltype = bionty.CellType(name="my new cell type")
my_celltype.save()

Alternatively, we can create entries based on ontology ids:

adata.obs.cell_type_id.unique().tolist()
Hide code cell output
['CL:0000084', 'CL:0000037', 'CL:0000182', '']
bt.CellType.from_values(adata.obs.cell_type_id, field=bt.CellType.ontology_id)
Hide code cell output
✅ loaded 1 CellType record matching ontology_id: 'CL:0000084'
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000037', 'CL:0000182'
[CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:45 UTC'),
 CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29),
 CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29)]

If we’re happy with cell_types records, we save them to the registry:

ln.save(cell_types)
Hide code cell output
❗ now recursing through parents: this only happens once, but is much slower than bulk saving
💡 also saving parents of CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:52 UTC')
✅ created 3 CellType records from Bionty matching ontology_id: 'CL:0008001', 'CL:0000723', 'CL:0011026'
💡 also saving parents of CellType(uid='5M0BT5FC', name='hematopoietic precursor cell', ontology_id='CL:0008001', description='Any Hematopoietic Cell That Is A Precursor Of Some Other Hematopoietic Cell Type.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:53 UTC')
💡 also saving parents of CellType(uid='2Dvf9ly5', name='somatic stem cell', ontology_id='CL:0000723', description='A Stem Cell That Can Give Rise To Cell Types Of The Body Other Than Those Of The Germ-Line.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:53 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000034'
💡 also saving parents of CellType(uid='jxDD8ajD', name='stem cell', ontology_id='CL:0000034', synonyms='animal stem cell', description='A Relatively Undifferentiated Cell That Retains The Ability To Divide And Proliferate Throughout Life To Provide Progenitor Cells That Can Differentiate Into Specialized Cells.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:53 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0011115'
💡 also saving parents of CellType(uid='5J0ndawv', name='precursor cell', ontology_id='CL:0011115', description='A Cell That, By Division Or Terminal Differentiation, Can Give Rise To Other Cell Types.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:54 UTC')
💡 also saving parents of CellType(uid='5fX4hLCd', name='progenitor cell', ontology_id='CL:0011026', description='A Precursor Cell That Has A Tendency To Differentiate Into A Specific Type Of Cell. They Are Descendants Of Stem Cells, Only They Are More Constrained In Their Differentiation Potential Or Capacity For Self-Renewal, And Are Often More Limited In Both Senses.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:53 UTC')
💡 also saving parents of CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:52 UTC')
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000066', 'CL:0000417'
💡 also saving parents of CellType(uid='68LNvDH7', name='epithelial cell', ontology_id='CL:0000066', synonyms='epitheliocyte', description='A Cell That Is Usually Found In A Two-Dimensional Sheet With A Free Surface. The Cell Has A Cytoskeleton That Allows For Tight Cell To Cell Contact And For Cell Polarity Where Apical Part Is Directed Towards The Lumen And The Basal Part To The Basal Lamina.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:55 UTC')
💡 also saving parents of CellType(uid='M3aRHlL9', name='endopolyploid cell', ontology_id='CL:0000417', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:55 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000412'
💡 also saving parents of CellType(uid='1035XQsH', name='polyploid cell', ontology_id='CL:0000412', description='A Cell That Contains More Than Two Haploid Sets Of Chromosomes.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:55 UTC')

Now, let’s inspect our in-house registry:

bt.CellType.df()
Hide code cell output
uid name ontology_id abbr synonyms description public_source_id run_id created_by_id updated_at
id
22 1035XQsH polyploid cell CL:0000412 None None A Cell That Contains More Than Two Haploid Set... 29.0 None 1 2024-06-19 23:12:55.916845+00:00
21 M3aRHlL9 endopolyploid cell CL:0000417 None None None 29.0 None 1 2024-06-19 23:12:55.320042+00:00
20 68LNvDH7 epithelial cell CL:0000066 None epitheliocyte A Cell That Is Usually Found In A Two-Dimensio... 29.0 None 1 2024-06-19 23:12:55.319895+00:00
19 5J0ndawv precursor cell CL:0011115 None None A Cell That, By Division Or Terminal Different... 29.0 None 1 2024-06-19 23:12:54.596358+00:00
18 jxDD8ajD stem cell CL:0000034 None animal stem cell A Relatively Undifferentiated Cell That Retain... 29.0 None 1 2024-06-19 23:12:53.814260+00:00
17 5fX4hLCd progenitor cell CL:0011026 None None A Precursor Cell That Has A Tendency To Differ... 29.0 None 1 2024-06-19 23:12:53.223219+00:00
16 2Dvf9ly5 somatic stem cell CL:0000723 None None A Stem Cell That Can Give Rise To Cell Types O... 29.0 None 1 2024-06-19 23:12:53.223080+00:00
15 5M0BT5FC hematopoietic precursor cell CL:0008001 None None Any Hematopoietic Cell That Is A Precursor Of ... 29.0 None 1 2024-06-19 23:12:53.222929+00:00
14 7hggmgo1 hepatocyte CL:0000182 None None The Main Structural Component Of The Liver. Th... 29.0 None 1 2024-06-19 23:12:52.495115+00:00
13 2U8xapxu hematopoietic stem cell CL:0000037 None hemopoietic stem cell|blood forming stem cell A Stem Cell From Which All Cells Of The Lympho... 29.0 None 1 2024-06-19 23:12:52.494964+00:00
12 1HuNn2EP gamma-delta T cell CL:0000798 None gammadelta T cell|gamma-delta T-cell|gamma-del... None 29.0 None 1 2024-06-19 23:12:50.209303+00:00
11 2eNGzkOc my T cell subtype None None None None NaN None 1 2024-06-19 23:12:48.631825+00:00
10 4bKGljt0 cell CL:0000000 None None A Material Entity Of Anatomical Origin (Part O... 29.0 None 1 2024-06-19 23:12:48.529727+00:00
9 4Ilrnj9U hematopoietic cell CL:0000988 None haematopoietic cell|hemopoietic cell|haemopoie... A Cell Of A Hematopoietic Lineage. 29.0 None 1 2024-06-19 23:12:47.934307+00:00
8 u3sr1Gdf nucleate cell CL:0002242 None None A Cell Containing At Least One Nucleus. 29.0 None 1 2024-06-19 23:12:47.934170+00:00
7 2K93w3xO motile cell CL:0000219 None None A Cell That Moves By Its Own Activities. 29.0 None 1 2024-06-19 23:12:47.934030+00:00
6 4WnpvUTH eukaryotic cell CL:0000255 None None None 29.0 None 1 2024-06-19 23:12:47.933882+00:00
5 2cXC7cgF single nucleate cell CL:0000226 None None A Cell With A Single Nucleus. 29.0 None 1 2024-06-19 23:12:47.253352+00:00
4 3VEAlFdi leukocyte CL:0000738 None white blood cell|leucocyte An Achromatic Cell Of The Myeloid Or Lymphoid ... 29.0 None 1 2024-06-19 23:12:47.253204+00:00
3 2Jgr5Xx4 mononuclear cell CL:0000842 None mononuclear leukocyte A Leukocyte With A Single Non-Segmented Nucleu... 29.0 None 1 2024-06-19 23:12:46.586857+00:00
2 X6c7osZ5 lymphocyte CL:0000542 None None A Lymphocyte Is A Leukocyte Commonly Found In ... 29.0 None 1 2024-06-19 23:12:45.885134+00:00
1 22LvKd01 T cell CL:0000084 None T-lymphocyte|T lymphocyte|T-cell A Type Of Lymphocyte Whose Defining Characteri... 29.0 None 1 2024-06-19 23:12:45.218209+00:00

Access records in in-house ontologies

Search:

bt.CellType.search("gamma delta T cell").df().head(2)
Hide code cell output
uid name ontology_id abbr synonyms description public_source_id run_id created_by_id updated_at
id
1 22LvKd01 T cell CL:0000084 None T-lymphocyte|T lymphocyte|T-cell A Type Of Lymphocyte Whose Defining Characteri... 29.0 None 1 2024-06-19 23:12:45.218209+00:00
3 2Jgr5Xx4 mononuclear cell CL:0000842 None mononuclear leukocyte A Leukocyte With A Single Non-Segmented Nucleu... 29.0 None 1 2024-06-19 23:12:46.586857+00:00

Or look up with auto-complete:

cell_types = bt.CellType.lookup()
hsc_record = cell_types.hematopoietic_stem_cell

hsc_record
Hide code cell output
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:52 UTC')

Validate & standardize

Simple validation of an iterable of values works like so:

bt.CellType.validate(["HSC", "blood forming stem cell"])
Hide code cell output
2 terms (100.00%) are not validated for name: HSC, blood forming stem cell
array([False, False])

Because these values don’t comply with the registry, they’re not validated!

You can easily convert these values to validated standardized names based on synonyms like so:

bt.CellType.standardize(["HSC", "blood forming stem cell"])
Hide code cell output
💡 standardized 1/2 terms
['HSC', 'hematopoietic stem cell']

Alternatively, you can use .from_values(), which will only ever create validated records and automatically standardize under-the-hood:

bt.CellType.from_values(["HSC", "blood forming stem cell"])
Hide code cell output
✅ loaded 1 CellType record matching synonyms: 'blood forming stem cell'
did not create CellType record for 1 non-validated name: 'HSC'
[CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:52 UTC')]

We can also add new synonyms to a record like so:

hsc_record.add_synonym("HSCs")

And when we encounter this synonym as a value, it will now be standardized using synonyms-lookup, and mapped on the correct registry record:

bt.CellType.standardize(["HSCs"])
Hide code cell output
💡 standardized 1/1 terms
['hematopoietic stem cell']

A special synonym is .abbr (short for abbreviation), which has its own field and can be assigned via:

hsc_record.set_abbr("HSC")

You can create a lookup object from the .abbr field:

cell_types = bt.CellType.lookup("abbr")
hsc = cell_types.hsc
hsc
Hide code cell output
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', abbr='HSC', synonyms='HSC|hemopoietic stem cell|blood forming stem cell|HSCs', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-06-19 23:12:57 UTC')

The same workflow works for all of bionty’s registries.

Manage registries across organism

Most registries are organism-aware, for instance, Gene:

bt.Gene.from_public(symbol="TCF7", organism="human")
✅ created 1 Gene record from Bionty matching symbol: 'TCF7'
Gene(uid='7IkHKPl0ScQR', symbol='TCF7', ensembl_gene_id='ENSG00000081059', ncbi_gene_ids='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1', created_by_id=1, organism_id=1, public_source_id=11)

Similarly, API calls that interact with multi-organism registries accept a organism argument, e.g.:

bt.Gene.validate(["TCF7", "ABC1"], organism="human")
Hide code cell output
2 terms (100.00%) are not validated for symbol: TCF7, ABC1
array([False, False])

And when working with the same organism throughout your analysis/workflow, you can omit the organism argument by configuring it globally:

bt.settings.organism = "mouse"
bt.Gene.from_public(symbol="Ap5b1")
✅ created 1 Gene record from Bionty matching symbol: 'Ap5b1'
Gene(uid='3b8mHb0MRal4', symbol='Ap5b1', ensembl_gene_id='ENSMUSG00000049562', ncbi_gene_ids='381201', biotype='protein_coding', description='adaptor-related protein complex 5, beta 1 subunit ', synonyms='Gm962', created_by_id=1, organism_id=2, public_source_id=15)

Track underlying ontology versions

Under-the-hood, source ontology versions are automatically tracked:

bt.PublicSource.filter(currently_used=True).df()
Hide code cell output
uid entity organism currently_used source source_name version url md5 source_website run_id created_by_id updated_at
id
1 5Dlc Organism vertebrates True ensembl Ensembl release-112 https://ftp.ensembl.org/pub/release-112/specie... 0ec37e77f4bc2d0b0b47c6c62b9f122d https://www.ensembl.org None 1 2024-06-19 23:12:41.468601+00:00
6 2Jzh Organism bacteria True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org None 1 2024-06-19 23:12:41.469423+00:00
7 1kdI Organism fungi True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org None 1 2024-06-19 23:12:41.469576+00:00
8 2mIM Organism metazoa True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org None 1 2024-06-19 23:12:41.469726+00:00
9 2XQ6 Organism plants True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org None 1 2024-06-19 23:12:41.469877+00:00
10 1Vzs Organism all True ncbitaxon NCBItaxon Ontology 2023-06-20 s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon None 1 2024-06-19 23:12:41.470029+00:00
11 1hx4 Gene human True ensembl Ensembl release-112 s3://bionty-assets/df_human__ensembl__release-... 4ccda4d88720a326737376c534e8446b https://www.ensembl.org None 1 2024-06-19 23:12:41.470179+00:00
15 76FX Gene mouse True ensembl Ensembl release-112 s3://bionty-assets/df_mouse__ensembl__release-... 519cf7b8acc3c948274f66f3155a3210 https://www.ensembl.org None 1 2024-06-19 23:12:41.470789+00:00
19 7LW6 Gene saccharomyces cerevisiae True ensembl Ensembl release-112 s3://bionty-assets/df_saccharomyces cerevisiae... 11775126b101233525a0a9e2dd64edae https://www.ensembl.org None 1 2024-06-19 23:12:41.471392+00:00
22 7llW Protein human True uniprot Uniprot 2023-03 s3://bionty-assets/df_human__uniprot__2023-03_... 1c46e85c6faf5eff3de5b4e1e4edc4d3 https://www.uniprot.org None 1 2024-06-19 23:12:41.471841+00:00
24 5U7J Protein mouse True uniprot Uniprot 2023-03 s3://bionty-assets/df_mouse__uniprot__2023-03_... 9d5e9a8225011d3218e10f9bbb96a46c https://www.uniprot.org None 1 2024-06-19 23:12:41.472140+00:00
26 5nkB CellMarker human True cellmarker CellMarker 2.0 s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker None 1 2024-06-19 23:12:41.472439+00:00
27 6AFz CellMarker mouse True cellmarker CellMarker 2.0 s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker None 1 2024-06-19 23:12:41.472589+00:00
28 6cbC CellLine all True clo Cell Line Ontology 2022-03-21 https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO None 1 2024-06-19 23:12:41.472737+00:00
29 3DeN CellType all True cl Cell Ontology 2024-02-13 http://purl.obolibrary.org/obo/cl/releases/202... https://obophenotype.github.io/cell-ontology None 1 2024-06-19 23:12:41.472887+00:00
34 1AyH Tissue all True uberon Uberon multi-species anatomy ontology 2024-02-20 http://purl.obolibrary.org/obo/uberon/releases... 2048667b5fdf93192384bdf53cafba18 http://obophenotype.github.io/uberon None 1 2024-06-19 23:12:41.473667+00:00
39 LoCG Disease all True mondo Mondo Disease Ontology 2024-02-06 http://purl.obolibrary.org/obo/mondo/releases/... 78914fa236773c5ea6605f7570df6245 https://mondo.monarchinitiative.org None 1 2024-06-19 23:12:41.474417+00:00
44 2mou Disease human True doid Human Disease Ontology 2024-01-31 http://purl.obolibrary.org/obo/doid/releases/2... b36c15a4610757094f8db64b78ae2693 https://disease-ontology.org None 1 2024-06-19 23:12:41.475168+00:00
51 4usY ExperimentalFactor all True efo The Experimental Factor Ontology 3.63.0 http://www.ebi.ac.uk/efo/releases/v3.63.0/efo.owl 603e6f6981d53d501c5921aa3940b095 https://bioportal.bioontology.org/ontologies/EFO None 1 2024-06-19 23:12:41.476224+00:00
54 2WLc Phenotype human True hp Human Phenotype Ontology 2024-03-06 https://github.com/obophenotype/human-phenotyp... 36b0d00c24a68edb9131707bc146a4c7 https://hpo.jax.org None 1 2024-06-19 23:12:41.476675+00:00
58 6zE1 Phenotype mammalian True mp Mammalian Phenotype Ontology 2024-02-07 https://github.com/mgijax/mammalian-phenotype-... 31c27ed2c7d5774f8b20a77e4e1fd278 https://github.com/mgijax/mammalian-phenotype-... None 1 2024-06-19 23:12:41.477300+00:00
60 7EnA Phenotype zebrafish True zp Zebrafish Phenotype Ontology 2024-01-22 https://github.com/obophenotype/zebrafish-phen... 01600a5d392419b27fc567362d4cfff8 https://github.com/obophenotype/zebrafish-phen... None 1 2024-06-19 23:12:41.477600+00:00
63 55lY Phenotype all True pato Phenotype And Trait Ontology 2023-05-18 http://purl.obolibrary.org/obo/pato/releases/2... bd472f4971492109493d4ad8a779a8dd https://github.com/pato-ontology/pato None 1 2024-06-19 23:12:41.478050+00:00
64 48aa Pathway all True go Gene Ontology 2023-05-10 https://data.bioontology.org/ontologies/GO/sub... e9845499eadaef2418f464cd7e9ac92e http://geneontology.org None 1 2024-06-19 23:12:41.478199+00:00
67 3rm9 BFXPipeline all True lamin Bioinformatics Pipeline 1.0.0 s3://bionty-assets/bfxpipelines.json a7eff57a256994692fba46e0199ffc94 https://lamin.ai None 1 2024-06-19 23:12:41.478649+00:00
68 5alK Drug all True dron Drug Ontology 2024-03-02 https://data.bioontology.org/ontologies/DRON/s... 84138459de4f65034e979f4e46783747 https://bioportal.bioontology.org/ontologies/DRON None 1 2024-06-19 23:12:41.478799+00:00
70 7CRn DevelopmentalStage human True hsapdv Human Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/HSAPDV/11... 52181d59df84578ed69214a5cb614036 https://github.com/obophenotype/developmental-... None 1 2024-06-19 23:12:41.479097+00:00
71 16tR DevelopmentalStage mouse True mmusdv Mouse Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/MMUSDV/9/... 5bef72395d853c7f65450e6c2a1fc653 https://github.com/obophenotype/developmental-... None 1 2024-06-19 23:12:41.479245+00:00
72 3Tlc Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None 1 2024-06-19 23:12:41.482881+00:00
73 5JnV BioSample all True ncbi NCBI BioSample attributes 2023-09 s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... None 1 2024-06-19 23:12:41.483041+00:00

Each record is linked to a versioned public source (if it was created from public):

hepatocyte = bt.CellType.filter(name="hepatocyte").one()
hepatocyte.public_source
Hide code cell output
PublicSource(uid='3DeN', entity='CellType', organism='all', currently_used=True, source='cl', source_name='Cell Ontology', version='2024-02-13', url='http://purl.obolibrary.org/obo/cl/releases/2024-02-13/cl.owl', md5='', source_website='https://obophenotype.github.io/cell-ontology', created_by_id=1, updated_at='2024-06-19 23:12:41 UTC')

Create records from specific public ontologies

By default, records are created from the "currently_used" public sources which are configured during the instance initialization, e.g.:

bt.Phenotype.public()
Hide code cell output
PublicOntology
Entity: Phenotype
Organism: human
Source: hp, 2024-03-06
#terms: 18697
bt.Phenotype.sources(currently_used=True).df()
Hide code cell output
uid entity organism currently_used source source_name version url md5 source_website run_id created_by_id updated_at
id
54 2WLc Phenotype human True hp Human Phenotype Ontology 2024-03-06 https://github.com/obophenotype/human-phenotyp... 36b0d00c24a68edb9131707bc146a4c7 https://hpo.jax.org None 1 2024-06-19 23:12:41.476675+00:00
58 6zE1 Phenotype mammalian True mp Mammalian Phenotype Ontology 2024-02-07 https://github.com/mgijax/mammalian-phenotype-... 31c27ed2c7d5774f8b20a77e4e1fd278 https://github.com/mgijax/mammalian-phenotype-... None 1 2024-06-19 23:12:41.477300+00:00
60 7EnA Phenotype zebrafish True zp Zebrafish Phenotype Ontology 2024-01-22 https://github.com/obophenotype/zebrafish-phen... 01600a5d392419b27fc567362d4cfff8 https://github.com/obophenotype/zebrafish-phen... None 1 2024-06-19 23:12:41.477600+00:00
63 55lY Phenotype all True pato Phenotype And Trait Ontology 2023-05-18 http://purl.obolibrary.org/obo/pato/releases/2... bd472f4971492109493d4ad8a779a8dd https://github.com/pato-ontology/pato None 1 2024-06-19 23:12:41.478050+00:00

Sometimes, the default source doesn’t contain the ontology term you are looking for.

You can then specify to create a record from a non-default source. For instance, instead of using untyped labels for iris organisms as Tutorial: Features & labels, we can use the ncbitaxon ontology:


public_source = bt.PublicSource.filter(entity="Organism", source="ncbitaxon").one()
iris_setosa = bt.Organism.from_public(name="iris setosa", public_source=public_source)
iris_setosa.save()

Analogously, you can pass public_source to bulk-create records from a non-default source:


records = bt.Organism.from_values(
    ["iris setosa", "iris versicolor", "iris virginica"], public_source=public_source
)
ln.save(records)
iris_setosa.parents.get(name="iris").view_parents(with_children=True)
Hide code cell content
# clean up test instance
!lamin delete --force test-registries
!rm -r test-registries
💡 deleting instance testuser1/test-registries
rm: cannot remove 'test-registries': No such file or directory