Changelog 2024

Note

💡 LaminDB implements “migration-based versioning”. When upgrading your LaminDB installation to a new minor version in major.minor.patch, you also migrate your database via lamin migrate deploy.

Get notified by watching releases for the lamindb, laminhub-public, and bionty GitHub repositories.

🪜 For older changes, see: 2023 · 2022

2024-11-21 db 0.77.0 | bionty 0.53.0

✨ Enable validation of Literal and other field types PR @sunnyosun

  • A Literal-typed CharField is validated for any Record:

    from lnschema_core import Record, fields

CRISPRType = Literal[
    "CRISPRi",
    "CRISPRa",
]

class Treatment(Record):
    system: CRISPRType = fields.CharField()

Treatment(system="crispr")
#> FieldValidationError:
#>   system: crispr is not a valid value
#>     → valid values are: CRISPRa, CRISPRi

  • You can leverage Django’s rich field validation by inheriting from ValidateFields and using dedicated field types or custom regex expressions:

    from lnschema_core import Record, ValidateFields
from django.core.validators import RegexValidator

class Reference(Record, ValidateFields):
    url: str | None = fields.URLField()
    """URL linking to the reference."""
    doi: str = fields.CharField(
        validators=[
            RegexValidator(
                regex=r"^(?:https?://(?:dx\.)?doi\.org/|doi:|DOI:)?10\.\d+/.*$",
                message="Must be a DOI (10.1000/xyz123 or https://doi.org/10.1000/xyz123)",
            )
        ],
    )
    """A Digital Object Identifier (DOI)."""

Reference(doi="abc.ef", url="myurl.com")
#> FieldValidationError:
#>   url: myurl.com is not valid
#>     → Enter a valid URL.
#>   doi: abc.ef is not valid
#>     → Must be a DOI (10.1000/xyz123 or https://doi.org/10.1000/xyz123)

These possibilities are now leveraged in all schema modules.

Bionty changes.

  • ✨ Add support for local source files in sources.yaml PR @mossjacob

  • ✨ Support taxa="plants" in EnsemblGene PR @mossjacob

2024-11-15 db 0.76.16

New features.

  • ✨ Support saving R code including .qmd and .Rmd PR @falexwolf

  • ✨ Support registering artifacts on Hugging Face PR @Koncopd

Other enhancements.

  • 📝 Add gene symbol FAQ PR @Zethson

  • 🚸 Improve speed and relevance of search PR @Koncopd

  • 🚸 Refactor ln.track() to improve logging and method signature PR @falexwolf

  • 🚸 Enable to query with records from a different database instance PR @falexwolf

  • 🚸 Enable autocompletion for inherited methods in Jupyter PR @Koncopd

  • 🎨 Make EHRCurator immutable PR @Zethson

  • 🚸 Warn if curating against gene symbols PR @Zethson

  • 🚸 Better logging during curation PR PR @Zethson

  • 🚸 Add pip install extras for all schema modules PR @Zethson

Fixes.

  • 🐛 Fix transfering artifacts from a source instance with fewer schema modules PR @sunnyosun

  • 🐛 Fix hard-coded “name” field during search PR @sunnyosun

  • 🐛 Fix registering Gene columns in DataFrameCurator PR @sunnyosun

Deprecations.

  • 🚚 Deprecate import_from_source in favor of import_source PR @Zethson

2024-11-13 hub 0.30

✨ A link that omits the last 4 version-coding characters now links to the latest version of a record, e.g., 13VINnFk89PE → 13VINnFk89PE0006. PR @chaichontat

image

🚸 Filter selectors are more intuitive. PR @chaichontat

image

✨ Plots can now visualize time series. PR @chaichontat @Golodhros

image

Further enhancements.

Fixes.

2024-10-29 db 0.76.15

✨ Stream pyarrow.dataset via Artifact.open() PR @Koncopd

df = pd.DataFrame({"col1": [0, 0, 1, 1], "col2": [6, 7, 8, 9]})
df.to_parquet("df.parquet", engine="pyarrow", partition_cols=["feat1"])
artifact = ln.Artifact("df.parquet", description="A partitioned parquet").save()
with artifact.open() as dataset:    # get pyarrow.dataset.Dataset
    batches = dataset.to_batches()  # get a streaming iterator over batches
    dataset.to_table().to_pandas()  # read into memory and convert to pandas Dataframe

✨ Add .query_parents() and .query_children() to hierachical registries PR @Koncopd

import lamindb as ln

label1 = ln.ULabel(name="label1").save()
label2 = ln.ULabel(name="label2").save()
label3 = ln.ULabel(name="label3").save()
label1.children.add(label2)
label2.children.add(label3)
label3.query_parents()  # returns a QuerySet with label1 and label2
label1.query_children() # returns a QuerySet with label2 and label3

More changes:

2024-10-18 db 0.76.14 | bionty 0.52.0

Features.

  • ✨ Safe handling of renaming internal features and labels PR @sunnyosun

  • ✨ Enable curating multiple categorical features per artifact against the same label registry PR @falexwolf

  • ✨ Add Collection.append() PR @Koncopd

User experience.

  • 🚸 Document how to subclass Curator PR @Zethson

  • 🚸 Automate Curator.add_validated_from and remove it from the API Breaking PR @sunnyosun

  • 🚸 Remove unnecessary BioRecord.list_source() Breaking PR @sunnyosun

Bionty changes.

  • 🎨 Add unique constraints and fix gene_ref_is_symbol to label_ref_is_name PR @falexwolf

  • 🎨 Remove update in add_ontology_from_df() PR @sunnyosun

Fixes.

  • 🐛 Fix tracking of notebooks on PyCharm PR @Koncopd

  • 🐛 Fix loading artifact by key and give clear errors if no artifacts or transforms found PR @Koncopd

Deprecations.

2024-10-14 hub 0.29

✨ Advanced filters. @chaichontat @adamdev21

image

✨ Filter on click. @chaichontat

image

✨ Links to filtered artifacts in record tables. @chaichontat

image

✨ Links to artifacts in Features dashboard. @chaichontat

💄 Prettify plotting panels on Overview page PR @chaichontat

image

Enhancements.

Fixes.

  • 💄 Do not display N/A fields in hover card

  • 🐛 Fix duplicated alias error with filters

  • 🐛 Fix collection copy button

  • 🐛 Deduplicate transform references in “Outputs” section

  • 🐛 Fix hang when loading transform

  • 🐛 Fix transform pane collapsing behavior

  • 🐛 Fix flicker

Deprecations.

  • 🚸 Beta API key is now stable; deprecate legacy API key

2024-10-11 db 0.76.13

  • 🚸 Do not error if a schema module of an instance isn’t installed PR @falexwolf

  • 🚸 More decent failing upon invalid lamin init calls PR @Koncopd

  • 🚸 Clearer feedback when somebody tries to switch the default instance PR @falexwolf

  • 🐛 Fix .get() with .using() PR @sunnyosun

  • 🐛 For .view_lineage() for circular input/outputs PR @sunnyosun

  • 🐛 Fix transferring to target instances that have more schema modules than the source instance PR @sunnyosun

  • 🐛 Fix monkey-patching __getitem__ on QuerySet PR @sunnyosun

  • 🐛 Do not double track runs PR @falexwolf

2024-10-08 db 0.76.12 | bionty 0.51.2

  • ✨ Overhaul save_vitessce_config() to support multiple artifacts and non-.zarr PR @keller-mark

  • 🚸 Query with typed labels through .features PR @falexwolf

  • 📝 Document how to query by dictionary-like run parameters PR @falexwolf

  • 🍱 New ExperimentalFactor version: efo-3.70.0 PR @Zethson

  • 🐛 Fix lamin load ... --with-env across servers PR @Koncopd

2024-10-02 hub 0.28

Major improvements.

Minor changes.

  • 🐛 Fix ulabel filtering on Artifacts page @adamdev21

  • 🐛 Fix collection page @sunnyosun

  • 🐛 Do not re-encode vitessce url if it’s non-lamin @sunnyosun

  • 🚸 A friendly message when a user still drafts a transform and hasn’t provided source code or run report @sunnyosun

  • 🚸 Use key as a download name for a transform @sunnyosun

  • 🚸 Only show linked labels in filters @adamdev21

  • 💄 Order transforms & runs by created_at @sunnyosun

  • 💄 On Overview page, show data formats, feature sets, and artifact size and counts @chaichontat

  • ✨ Support refSpecUrl in VitessceConfig @sunnyosun

2024-10-01 db 0.76.11

  • ✨ Add a reference manager schema module: findrefs PR @falexwolf

  • 🐛 Fix label name display in .describe() PR @falexwolf

  • 🐛 Fix permission error when saving a new artifact on python 3.12 PR @Koncopd

  • 💄 Strip NotebookNotSaved error from report after ln.finish() PR @Koncopd

2024-09-30 db 0.76.10

  • 🚸 Re-worked the CLI: lamin load → lamin connect & lamin get → lamin load PR @Koncopd @ap–

  • ⚡ Improve performance of ln.connect(), lamin connect, and lamin load for a notebook PR @falexwolf

  • ⚡️ Speed up curation workflows through from_values PR @sunnyosun

  • 🚸 Improve lamin load UX for notebooks & scripts PR @falexwolf

  • 🚸 Transfer: Warn about inconsistencies between source & target instances PR @falexwolf

  • 🎨 Move .from_values() from Record to CanValidate PR @falexwolf

  • 📝 Document how to work with run parameters PR @falexwolf

  • ✨ Track transfers as transforms PR @falexwolf

  • ⚡️ Speed up describe PR @sunnyosun

  • 🚸 Minimal ln.track() PR @falexwolf

  • ✨ Enable to lamin load from on-prem domains PR @falexwolf

2024-09-26 db 0.76.9

Curating perturbations

Genetic, compound, and environmental perturbations and their targets can be curated with the wetlab schema. This can be achieved at varying levels of detail. For example, with a high degree of detail:


import wetlab as wl

EGFR_kd = wl.GeneticTreatment(
    system="CRISPR Cas9",
    name="EGFR knockdown",
    sequence="AGCTGACCGTGA",
    on_target_score=85,
    off_target_score=15
).save()

EGFR_gene = bt.Gene.from_source(symbol="EGFR").save()
EGFR_kd_target = wl.TreatmentTarget(name="cell growth").save()
EGFR_kd_target.genes.add(EGFR_gene)

artifact.genetic_treatmends.add(EGFR_kd)

2024-09-23 db 0.76.8

  • 🐛 Ensure is_latest is set to False in previous version if matching on artifact.key PR @falexwolf

  • ✨ Store artifacts under their virtual keys in cache PR @Koncopd

2024-09-18 db 0.76.7 | bionty 0.50.2

  • ✨ Enable getting the latest run environment for a transform PR @falexwolf

  • ✨ Enable displaying images via artifact.load(), add documentation for artifact loaders PR @falexwolf

  • 🚸 Do not throw an error but prompt upon ln.context.track() in a notebook PR @falexwolf

  • 🚸 Allow to use Collection.mapped() without saving the collection PR @Koncopd

  • 🚸 Simplify CLI commands PR @falexwolf

  • 🚸 Add parameter validation in bionty PR @Zethson

Technical changes

  • 🔊 More logging for import_from_source PR @sunnyosun

  • 🔊 Warn instead of hint about missed input tracking PR @Koncopd

  • ✨ Add n_observations to tiledbsoma-like artifacts PR @Koncopd

  • 🎨 Remove stream argument from artifact.load() PR @Koncopd

Ontology versions

  • 🍱 New Tissue version: uberon-2024-03-22 PR @Zethson

  • 🍱 New Disease version: mondo-2024-05-08 PR @Zethson

  • 🍱 New ExperimentalFactor version: efo-3.65.0 PR @Zethson

  • 🍱 New CellType version: cl-2024-04-05 PR @Zethson

Use case changes

  • ✨ Add support for cellxgene-schema 5.1.0 PR @Zethson

  • 📝 Integrate cellxgene guides and add data loader examples PR @Koncopd

  • 📝 Track all AnnData inputs in scrna-tiledbsoma PR @Koncopd

2024-09-09 db 0.76.6 | bionty 0.50

  • ✨ Enable negations in filter() PR @falexwolf

  • ✨ lamin get via key or uid PR @falexwolf

  • 🎨 Replace direct relation of Collection to FeatureSet with indirect relation through Artifact PR @falexwolf

  • 🎨 Remove backward relationships for Run, User & Source foreign keys PR PR @falexwolf @sunnyosun

  • 🐛 Reload in Transform() upon passing existing uid PR @falexwolf

2024-09-05 db 0.76.5

2024-09-04 db 0.76.4

2024-08-30 db 0.76.3 | bionty 0.49

✨ tiledbsoma integration. Guide PR @Koncopd

Example

Create a tiledbsoma.Experiment array store or append AnnData objects to an existing store.

# create new versioned tiledbsoma.Experiment
artifact = ln.integrations.save_tiledbsoma_experiment(
    adatas,
    measurement_name="RNA"
)

# append to existing tiledbsoma.Experiment
revised_artifact = ln.integrations.save_tiledbsoma_experiment(
    adatas,
    measurement_name="RNA",
    revises=artifact
)

Bionty updates.

  • 🍱 Add chebi & chembl PR @Zethson

  • 🍱 Add additional relationship types & update DevelopmentalStage and Tissue PR @Zethson

  • 🍱 New CellLine version: depmap-2024-Q2 PR @Zethson

More changes.

  • 🚚 Deprecate Curate in favor of Curator PR @sunnyosun

  • 🔥 Remove lamin register and password argument of lamin login PR PR @falexwolf

2024-08-26 hub 0.27

✨ Instance overview page. @chaichontat

Screenshot
image

💄 Show Vitessce button next to dataset instead of VitessceConfig file. @sunnyosun

🏗️ Much improved on-prem deployment. @fredericenard

2024-08-23 db 0.76.2

🚸 Simplify versioning. PR @falexwolf

Semantic version strings in .version are now optional as in git.
image
For Artifact & Transform, you can now also create new versions by passing the key argument.
artifact_v1 = ln.Artifact.from_df(df, key="my_datasets/my_study1.parquet").save()
# below automatically creates a new version of artifact_v1 because the `key` matches
artifact_v2 = ln.Artifact.from_df(df_updated, key="my_datasets/my_study1.parquet").save()

  • 🚚 Deprecate is_new_version_of argument in favor of revises

  • 🚚 Deprecate passing version to constructors; rather set .version after creating records

More changes.

2024-08-16 db 0.76.1

🚸 Overhauled context tracking experience with ln.context.track() Details & PR @falexwolf @chaichontat

ln.context.uid = "FPnfDtJz8qbE0000"  # <-- auto-generated by ln.context.track()

# track the execution of your notebook or script with inputs & outputs
ln.context.track()
What was the previous experience?

Now:

image

Previously:

image
How does it look on the hub?

If you don’t label with a semantic version tag, you’ll get an auto-generated revision id.

image

⚠️ Breaking change: ln.track() now returns None instead of a Run. Access the run via ln.context.run instead.

More changes:

  • 🚸 Update .get() to accept expressions so that it can replace .filter(...).one() PR @falexwolf

  • ✨ MappedCollection compatible with latest scdataloader PR @Koncopd PR @jkobject

2024-08-14 db 0.76

2024-08-10 hub 0.26

2024-08-08 db 0.75.1

🚸 Improved the cellxgene_lamin curation guide.

2024-08-08 bionty 0.48

New ontology versions.

  • 🍱 New Tissue version: uberon-2024-05-13 PR @Zethson

  • 🍱 New Tissue version: uberon-2024-01-18 PR @Zethson

  • 🍱 New Phenotype version: zp-2024-04-18 PR @Zethson

  • 🍱 New Phenotype version: pato-2024-03-28 PR @Zethson

  • 🍱 New Phenotype version: mp-2024-06-18 PR @Zethson

  • 🍱 New Phenotype version: hp-2024-04-26 PR @Zethson

  • 🍱 New Pathway version: pw-7.84 PR @Zethson

  • 🍱 New Pathway version: go-2024-06-17 PR @Zethson

  • 🍱 New Disease version: mondo-2024-06-04 PR @Zethson

  • 🍱 New Disease version: doid-2024-05-29 PR @Zethson

  • 🍱 New Disease version: mondo-2024-01-03 PR @Zethson

  • 🍱 New ExperimentalFactor version: efo-3.66.0 PR @Zethson

  • 🍱 New ExperimentalFactor version: efo-3.62.0 PR @Zethson

  • 🍱 New Drug version: dron-2024-08-05 PR @Zethson

  • 🍱 New CellType version: cl-2024-05-15 PR @Zethson

  • 🍱 New CellType version: cl-2024-01-04 PR @Zethson

2024-08-03 db 0.75

✨ Track mutations of array stores. Guide PR @Koncopd

  • Artifacts that store mutable arrays can lead to non-reproducible queries.

  • To monitor reproduciblity and data lineage, mutations are now tracked when a context manager and Artifact.open(mode="w") for tiledbsoma array stores is used:

    with artifact.open(mode="w") as array:
    # mutate `artifact`

# `artifact` now points to a new version of the artifact with an updated hash

🚸 A better structured API. PR @falexwolf

  • 🚸 Easier typing & maintenance of categorical fields via typing.Literal instead of Django’s migration-dependent CharField.choices

  • 🚸 Less clutter in auto-complete

    • 🚚 All fields pointing to link records start with links_

    • 🚚 Several fields for Artifact are now private via _ prefix: accessor, key_is_virtual, feature_values, param_values, hash_type, previous_runs

  • 🎨 More consistency

    • 🚚 Rename Transform.parents to Transform.predecessors to disambiguate procedural/temporal from ontological/conceptual hierachies

    • 🎨 Feature names are now guaranteed to be unique in a lamindb instance Feature.name

    • 🎨 Consistent length of hash fields: HASH_LENGTH=22

    • 🚚 Rename input_of to input_of_runs

    • 🎨 Transform.latest_report is now a property point to Transform.latest_run.report to simplify the schema

    • 🎨 Artifact.type now defaults to None when passing a path so that auxiliary files and folders aren’t labeled as dataset

  • 🚸 Better definition of Collection

    • 🚚 Rename fields .artifact to .meta_artifact and .unordered_artifacts to .artifacts

    • Iteration over an ordered QuerySet of artifacts is now possible via .ordered_artifacts

    • For collections that have a single data artifact, access it via .data_artifact

  • 🏗️ Towards searchable source code

    • 🚚 Rename Transform.source_code to Transform._source_code_artifact

    • Re-introduce Transform.source_code as a text field together with a field hash

Better storage management.

  • 🚸 Enable deleting artifacts in all managed storage locations of the current instance PR @falexwolf

  • ♻️ Do not write storage records to hub for local test instances PR @falexwolf

  • 🐛 Fix populating storage.instance_uid during init_instance PR @falexwolf

Various updates.

2024-08-03 bionty 0.47

🏗️ Bionty is now a single Python package. PR PR PR PR PR

⚠️ Migration: Once you load an instance, you’ll be asked to uninstall lnschema_bionty and lamin migrate deploy

  • On the SQL level, tables are now prefixed with bionty_ instead of lnschema_bionty_

  • On the Django level, you can mount the bionty instead of the lnschema_bionty apps

🚸 You can now import from in-house ontology sources. PR @sunnyosun

  • 🚚 Rename PublicSource to Source & from_public to from_source

  • Import from any parquet file into your registry, akin to how Bionty imports public ontology sources

User experience.

  • ⚡ Performantly import bulk records via .import_from_source()

  • 🚸 More reliable ontology_id field recognition

  • ✨ Better error message for synonym duplications PR @Zethson

  • 🚚 All link model fields start with links_ PR falexwolf

  • 🎨 CellMarker.name is now unique together with organism PR sunnyosun

New ontologies.

  • ✨ Add ICD ontology for Disease PR PR Zethson

  • 🍱 New Protein version: uniprot-2024-03 PR sunnyosun

  • 🍱 New Gene version: ensembl-111/112 PR Zethson

  • 🍱 New ExperimentalFactor version: efo-3.63 PR Zethson

  • 🍱 New CellType version: cl-2024-02-13 PR Zethson

  • 🍱 New Tissue version: uberon-2024-02-20 PR Zethson

  • 🍱 New Organism version: ensembl-release-111 & ensembl-release-112 PR sunnyosun

  • 🍱 New Disease version: mondo-2024-02-06 PR Zethson

  • 🍱 New Disease version: DOID-2024-01-31 PR Zethson

  • 🍱 New Phenotype version: hp-2024-03-06 PR Zethson

  • 🍱 New Phenotype version: mp-2024-02-07 PR Zethson

  • 🍱 New Phenotype version: zp-2024-01-22 PR Zethson

  • 🍱 New Pathway version: pw-7.82 PR Zethson

  • 🍱 New Drug version: DRON-2024-03-02 PR Zethson

2024-07-26 hub 0.25

Overhauled the REST API: better performance and architecture.

UI improvements.

2024-07-26 db 0.74.3

⚡ Speed up populating parent records by an order of magnitude, remove the parents keyword (PR @sunnyosun).

Features.

  • ✨ Allow for multiple local storage locations with the same root path PR @falexwolf

  • ✨ Add add_from_df method to BioRecord PR @sunnyosun

Chores.

2024-07-22 db 0.74.2

The API is now cleaner and fields are typed.

Details

All users who don’t use Django outside of lamindb can set Django’s internal API that clutters the Record name spaces by running: lamin set private-django-api on the command line.

tiledbsoma is now better supported.

  • ✨ Artifact.open() for tiledbsoma stores PR @Koncopd

Better names.

  • 🚚 Deprecate Artifact.backed() in favor of Artifact.open() PR @Koncopd

  • 🚚 Deprecate Annotate in favor of Curate PR @falexwolf

  • 🚚 Deprecate Registry in favor of Record PR @falexwolf

Better documentation.

Security updates & bug fixes.

  • 🔒 Enable Ruff security rules (bandit) & CodeQL PR @Zethson

  • 🐛 Fix return values of .save() for a few classes PR @falexwolf

2024-07-01 hub 0.24

2024-06-26 db 0.74.1

♻️ Refactor ln.settings PR @falexwolf.

  • ✨ Pass custom names for scripts via ln.settings.transform.name = "My script"

  • ⚠️ ln.settings.storage returns a StorageSettings object (root via ln.settings.storage.root)

Features.

  • ✨ Support different join types in QuerySet.df() PR @insavchuk

Use cases.

Docs.

2024-06-20 db 0.74

✨ You can now distinguish model-like and dataset-like artifacts via a type field in the Artifact registry.

  • 🚸 Leverage artifact.params.add_values() to annotate model-like artifacts like you leverage artifact.features.add_values() to annotate dataset-like artifacts

  • 🏗️ Add type field to Artifact, allow linking model-like artifacts against params, validate params akin to validating features, enable features-based annotation with non-ulabels PR @falexwolf

  • 🚸 Support dict in add_values PR @Zethson

♻️ Refactor after upath upgrade. PR PR @Koncopd

2024-06-13 db 0.73.2

  • 🐛 Fix clashing reverse accessors for .previous_runs and .run PR @falexwolf

  • 🐛 Import IPython inside view PR @Koncopd

2024-06-05 db 0.73.1

  • 🏗️ Instantly synchronize instance schema with the hub PR @fredericenard

  • ⬆️ Upgrade universal_pathlib to 0.2.2 PR @Koncopd

  • 🐛 Fix generation of uid for manual Transform constructor PR @falexwolf

  • 🔥 Deleting artifact.stage() in favor of artifact.cache() (was deprecated in 0.70.0)

2024-05-29 db 0.73.0

Annotating & querying by features improved:

  • ✨ Support non-categorical feature values PR @falexwolf

  • ✨ Annotate dict-style with features & values PR @falexwolf

  • ✨ Query by features via .features.filter(key=value) PR @falexwolf

  • 🏗️ Feature values decoupled from feature sets PR @falexwolf

Example:

# annotate dict-style (feature & category names get validated)
artifact.features.add_values({
    "species": "setosa",
    "scientist": ["Barbara McClintock", "Edgar Anderson"],
    "instrument": "Leica IIIc Camera",
    "temperature": 27.6,
    "study": "Study 0: initial plant gathering",
    "is_awesome": True
})

# get the dict back
artifact.features.get_values()

# query by feature
ln.Artifact.features.filter(is_awesome=True)

Various improvements:

  • 🚚 Additional non-breaking constraints in the core schema PR @falexwolf

  • 🚸 Make .upload_from(), .download_to(), and .view_tree() more user friendly PR @falexwolf PR @Koncopd

  • 🚸 More intuitive version updating dialogue PR @falexwolf

  • 🐛 Actually add tracking run for entities beyond Artifact & Collection PR @falexwolf

  • 🚸 ln.track() returns run PR @falexwolf

  • 🚸 Better duplicate detection and search PR @falexwolf

  • 🚸 Prettier .describe() PR @falexwolf

  • 🚸 More interactivity in lamin save PR @falexwolf

  • 🚸 create flag in .from_values() PR @falexwolf

  • 🚸 Better ordering of fields in dataframe & record representations PR @falexwolf

  • 📝 Improved API reference: docs now show relationship attributes PR @falexwolf

2024-05-19 db 0.72.1

2024-05-19 db 0.72.0

  • ✨ Extend managed access for AWS S3 to arbitrary paths PR @Koncopd @fredericenard

  • ✨ Extended data lineage tracking PR @falexwolf

    • Now store all creating runs and all updating runs for any entity, not just for Artifact & Collection, e.g., runs can now have CellType record outputs

    • Code is simpler through inheritance from two new base classes: TracksRun and TracksUpdates

  • ♻️ Briefer and richer syntax for denoting feature types, renamed Feature.type to Feature.dtype, e.g., for categorical features, a valid type can be: cat[ULabel|bionty.Drug] PR @falexwolf

  • ✨ Support non-categorical metadata PR @falexwolf

    • Track non-categorical features: int, float, bool, datetime, lists & dictionaries stored in a FeatureValue registry

    • Track arbitrary typed parameters for runs through a Param registry analogous to the Feature registry: this replaces the hard-to-validate, hard-to-migrate, and hard-to-query json field of Run

  • 🏗️ Refactor link models PR PR @falexwolf

    • All annotation-related links are now stratified by Feature: what held for ULabel now also holds CellType and all other Bionty registries

    • Indicate whether semantic keys were used during validation to enable warnings upon renames

    • Protect artifact annotations rather than cascade delete them

    • More consistent naming of link models, e.g., ulabels.artifact_links instead of ulabels.artifactulabel_set

    • Dropped linking Bionty entities directly against Collection

    • Pruned & squashed migrations for faster instance creation

2024-05-14 db 0.71.3

2024-05-07 db 0.71.2

2024-05-07 db 0.71.1

  • 🚸 Upload source code of scripts upon ln.finish() and no longer upon ln.track() PR @falexwolf

  • 🎨 Make features.add_feature_set public PR @sunnyosun

  • 🎨 Use the same uid for the same feature set in transfer PR @sunnyosun

  • 🎨 Upon upload switch to virtual key PR @falexwolf

  • ⚡️ Zarr and cache improvements PR @Koncopd

  • ♻️ Extend valid suffixes to composite suffixes PR @falexwolf

  • 🔥 Remove little-used artifact.view_tree() PR @falexwolf

2024-05-01 db 0.71.0

2024-04-24 db 0.70.4

  • ✨ Allow passing path to .from_anndata PR @sunnyosun

  • 🚸 In .setup.delete(), check for data deletion & delete from hub PR @falexwolf

  • ⚡️ Speed up latest_version PR @falexwolf

  • 🚸 Better user feedback on folder-like artifacts PR @falexwolf

2024-04-22 db 0.70.3

  • 🚸 Update metadata like description upon re-running PR @falexwolf

  • 🐛 Fix detection of AnnData in zarr and h5ad, refactor directory upload PR @Koncopd

  • 🚸 Raise error if transforms of type notebook or script are passed manually PR @falexwolf

2024-04-19 db 0.70.2

  • ♻️ In Vitessce integration, separate VitessceConfig from its referenced artifacts PR @falexwolf

  • 🚸 In ln.finish(), remove flag i_saved_the_notebook PR @falexwolf

2024-04-18 db 0.70.1

2024-04-17 db 0.70.0

  • 🚸 Update data source in case transform is re-run PR @falexwolf

  • 🚸 Enable to label transforms via transform.ulabels PR @falexwolf

  • 🚚 Deprecate stage() in favor of cache() PR @falexwolf

2024-04-12 db 0.69.10

  • ✨ Add .obsm and .layers to MappedCollection and rename label_keys to obs_keys PR @Koncopd

  • 🚸 Eliminate kwargs PR @sunnyosun

  • ✨ Introduce Annotate.from_mudata PR @sunnyosun

2024-04-08 db 0.69.9

2024-04-04 db 0.69.8

2024-04-03 db 0.69.7

  • ✨ Add ability to upload arbitrary files or folders from CLI PR @falexwolf

  • 🐛 Fix anndata backed mode incompatibility with scipy 1.13.0 f

2024-04-02 db 0.69.6

  • 🚑️ Temp fix region for non-hosted buckets PR @sunnyosun

2024-03-30 db 0.69.5

2024-03-30 db 0.69.4

2024-03-28 db 0.69.3

  • ✨ Introduce annotation flow via Annotate.from_df and Annotate.from_anndata PR 1 2 3 @sunnyosun

2024-03-26 db 0.69.2

2024-03-18 db 0.69.1

✨ To try out, add lamindb.validation with the Validator class PR @sunnyosun

2024-03-17 db 0.69.0

Main new features:

  • ✨ Integrate lamindb with git PR PR @falexwolf

  • ✨ Introduce ln.finish(), track run finish times as run.finished_at, rename run.run_at to run.started_at, upload notebooks during ln.finish() PR @falexwolf

  • 🚸 Upload script source code and environment during ln.track() PR @falexwolf

Other changes:

2024-03-11 db 0.68.2

  • 🚸 Move transform & run artifacts into cache before uploading PR @falexwolf

  • 🚸 More sensible transform types PR @falexwolf

  • 🚚 Rename lnschema_lamin1 to wetlab PR @falexwolf

2024-03-08 db 0.68.1

  • 🚸 You can now use ln.connect() to connect to a LaminDB instance PR @falexwolf

  • 🚸 You can no longer delete data from non-default storage locations, as these might be tracked in other instances PR @sunnyosun

  • 🚸 Enable transferring data from local instances to remote instances PR @sunnyosun

2024-03-01 db 0.68.0

🚸 Decouple features linking from Artifact construction PR 1 2 3 @sunnyosun.

# default constructor for PathLike
artifact = ln.Artifact("mysc.h5ad", description="raw data")
# from_ constructors for other types
artifact = ln.Artifact.from_anndata(mysc_adata, description="raw data")  # no longer links features
artifact = artifact.save()

# high-level feature linking
artifact.features.add_from_anndata(var_field=bt.Gene.ensembl_gene_id)
artifact.features.add_from_df()

# low-level feature linking
meta = ln.Feature.from_values(mysc_adata.obs.columns, field="name")
genes = bt.Gene.from_values(mysc_adata.var.ensembl_gene_id, field="ensembl_gene_id")
artifact.features.add(genes, slot="obs")
artifact.features.add(genes, slot="var")

# labels linking (no change)
labels = ln.ULabel.from_values(adata.obs.donor, field=...)
ln.save(labels)
artifact.labels.add(labels)

2024-02-02 db 0.67.3

2024-01-14 db 0.67.2

  • ✨ Enable staging notebooks & code using the CLI PR @falexwolf

2024-01-12 db 0.67.1

2024-01-11 db 0.67.0

  • 🚚 Rename .bionty to .public, .from_bionty to .from_public PR @sunnyosun

2024-01-09 db 0.66.1

  • 🐛 Fix id matching in view_lineage PR @sunnyosun

  • ♻️ Fix connection time outs PR @Koncopd

  • ♻️ Incorporate edge cases in inner and outer join in Collection.mapped PR @Koncopd

  • 🎨 Not create organism records when calling .bionty() PR @sunnyosun

2024-01-07 db 0.66.0

2024-01-05 db 0.65.1

2024-01-02 db 0.65.0

bionty

Name

PR

Developer

Date

Version

🚚 Rename Bionty to PublicOntology class

536

sunnyosun

2024-01-12

0.36.0

🚚 Rename bionty to bionty-base

539

sunnyosun

🚚 Rename PublicSource to Source

263

sunnyosun

2024-07-26

0.44.0

🎨 Do not add obsolete terms due to ontology_id duplication

261

sunnyosun

2024-07-25

⚡️ Speed up parents

259

sunnyosun

2024-07-22

🚚 Rename registry to record

256

falexwolf

2024-07-17

♻️ Consciously use class method

255

falexwolf

2024-07-10

🐛 Fix clashing reverse accessors between .previous_runs and .run

249

falexwolf

2024-06-13

0.43.0

♻️ Reformulate data lineage, remove json field from run

247

falexwolf

2024-05-19

0.42.0

♻️ Protect gene, protein, cell_marker & pathway in their FeatureSet relationships

246

falexwolf

2024-05-18

🏗️ Naming conventions for link tables, protecting deletion in link tables, maintaining integrity upon label & feature renames

245

falexwolf

2024-05-18

♻️ Account for migrations in lnschema_core

244

falexwolf

2024-05-17

🔥 Prune migrations

243

falexwolf

2024-05-16

🏗️ Spell out link tables with Artifact and link features

239

falexwolf

2024-05-16

🔥 Remove linking Collection to all Bionty entities

238

falexwolf

2024-05-15

✨ Add sources

237

sunnyosun

2024-05-14

🎨 Fix passing arguments to from_public

236

sunnyosun

2024-05-13

🐛 Fix organism

235

sunnyosun

2024-05-08

🚑️ Fix public_source in inspect

232

sunnyosun

2024-04-18

🐛 Fix syncing public sources

230

sunnyosun

2024-04-11

✨ Add PublicSource.set_as_currently_used

223

sunnyosun

2024-03-14

0.41.4

✏️ Fix encoding

213

sunnyosun

2024-01-12

0.38.4

🚑️ Re-encode PublicSource

212

sunnyosun

2024-01-10

0.38.3

🚚 Rename .bionty to .public

208

sunnyosun

2024-01-09

🚸 Do not create organism when calling bionty

206

sunnyosun

2024-01-08

🚚 Rename Dataset to Collection

205

falexwolf

2023-12-23

0.37.0

🚚 Rename File to Artifact

201

falexwolf

2023-12-11

0.36.0

⬆️ Enabled ID conversion via return_field

200

sunnyosun

2023-12-02

🐛 Fix save parents with bionty_source

198

sunnyosun

2023-11-23

✨ Added functions to sync bionty source

197

sunnyosun

2023-11-16

0.35.1

🚚 Rename FK organism to organisms in BiontySource

196

sunnyosun

2023-11-13

0.35.0

✨ Add parents to Organism

193

sunnyosun

2023-10-23

0.34.0

🚚 Rename Species to Organism

192

sunnyosun

2023-10-19

0.33.0

✏️ Fix uid encoder

190

sunnyosun

2023-10-18

0.32.2

🚚 Migrate to integer primary keys

188

falexwolf

2023-10-13

0.32.0

🔇 Mute repetitive warning for saving parents

187

sunnyosun

2023-09-20

0.31.2

⬆️ Silence download bar for <5MB files

186

sunnyosun

2023-09-08

0.31.1

🎨 Return None if no records are created for from_bionty

185

sunnyosun

2023-09-07

0.31.0

✨ Allow non-default bionty sources in LaminDB

184

sunnyosun

2023-09-07

♻️ Add DB-facing constructor

180

falexwolf

2023-08-28

0.30.1

🎨 Make species not nullable in Gene, Protein, CellType

179

sunnyosun

2023-08-24

♻️ Use CanValidate & HasParents

176

falexwolf

2023-08-23

0.30.0

🚚 Rename map_synonyms to standardize

173

sunnyosun

2023-08-16

0.29.5

🚚 Rename ORM to Registry

167

sunnyosun

2023-08-05

🎇 Added stable_id to Gene

166

sunnyosun

2023-07-31

0.29.0

🚚 Rename select to filter

164

falexwolf

2023-07-31

🎨 Switch biontysource id to hash id

163

sunnyosun

2023-07-25

0.28.2

🚸 New settings auto_save_parents

162

falexwolf

2023-07-25

0.28.1

🎨 Link all entities to files and datasets

160

sunnyosun

2023-07-24

0.28.0

🚚 Replace lamin_logger with lamin_utils

161

falexwolf

2023-07-24

✏️ Fix docstrings

159

sunnyosun

2023-07-21

0.27.4

🎨 Make species name unique

158

sunnyosun

2023-07-21

🎨 Gene table unique on ensembl id

152

sunnyosun

2023-07-18

0.27.0

🎨 Fix migration

150

sunnyosun

2023-07-18

🚚 Rename Readout to ExperimentalFactor

149

sunnyosun

2023-07-17

🚸 Trigger instance loading for users

147

falexwolf

2023-07-06

0.26.2

⚡️ Speed up parents ingestion

146

falexwolf

2023-07-04

0.26.1

🚸 Add settings

142

falexwolf

2023-06-30

0.25.4

🚑️ Allow bulk saving parents without causing conflicts

141

sunnyosun

2023-06-30

0.25.2

✨ Added self-referential parents, renamed abbr and description

139

sunnyosun

2023-06-29

0.25.0

♻️ Replaced ln.parse with orm.from_values

138

sunnyosun

2023-06-27

0.24.0

🚚 Move add_synonym to BaseORM

137

sunnyosun

2023-06-22

0.23.5

⬆️ Fix synonyms duplication check

136

sunnyosun

2023-06-22

🚑️ Do not check existing synonyms when db table is empty

134

sunnyosun

2023-06-20

🎨 Create multiple records with .from_bionty

133

sunnyosun

2023-06-20

0.23.3

📈 Save to DB if record already exists for add_synonyms

132

sunnyosun

2023-06-20

0.23.2

🚚 Fixed all related_names

131

sunnyosun

2023-06-20

0.23.1

🏷️ Change name to a required field for ontology tables

130

sunnyosun

2023-06-19

0.23.0

🎨 Consolidate schema, remove non-crucial fields

127

sunnyosun

2023-06-16

0.22.0

🚚 Rename source_key to source in BiontySource table

125

sunnyosun

2023-06-10

0.21.3

🚚 Rename Featureset to FeatureSet

124

sunnyosun

2023-06-10

0.21.2

⏪ Change species uid to 4 char

122

sunnyosun

2023-06-09

0.21.1

🎨 Change on_delete to PROTECT

121

sunnyosun

2023-06-09

0.21.0

🎨 Remove unnecessary null=True, blank=True

118

sunnyosun

2023-06-08

0.19a6

♻️ Simplified species code

116

sunnyosun

2023-06-08

0.19a4

🚚 Rename BiontyVersions to BiontySources

115

sunnyosun

2023-06-07

0.19a3

🚚 Migrate lnschema-bionty to Django

112

sunnyosun

2023-06-05

0.19a1

✨ Added Features.pathways

111

sunnyosun

2023-06-02

0.18.3

✨ Add add_synonym method

109

sunnyosun

2023-06-01

🚑 Must pass a species when creating record from lookup

108

sunnyosun

2023-05-30

0.18.2

✨ Added Pathway.genes

107

sunnyosun

2023-05-30

0.18.1

🚑 Fix created_by_id in Readout

106

sunnyosun

2023-05-28

0.18.0

🎨 Change .bionty to a method

105

sunnyosun

2023-05-28

🚑 Default species to None and not pass to Bionty

104

sunnyosun

2023-05-27

0.18a6

🚑 Fix 0.18.0 sqlite migration script

103

sunnyosun

2023-05-27

✨ Add synonyms, definition, short_name columns

102

sunnyosun

2023-05-27

0.18a5

🚑 Fix table name for id_encoder

101

sunnyosun

2023-05-27

⬆️ Update lamindb and bionty versions

100

sunnyosun

2023-05-27

🏷️ Restored Optional type annotation

99

sunnyosun

2023-05-26

🏗️ Remove SQL-level schema modules

96

falexwolf

2023-05-25

0.18a2

🚑 Fix last migration

98

falexwolf

2023-05-25

0.17.1

🍱 Add CreatedBy and CreatedAt to all entities

95

sunnyosun

2023-05-25

0.17.0

🚚 Rename bt.Entity to bt.Bionty

94

sunnyosun

2023-05-25

⬆️ Upgrade to bionty 0.13

92

sunnyosun

2023-05-09

0.16.4

✨ Added synonyms field to CellMarker

91

sunnyosun

2023-05-09

0.16.3

💚 Fix dependencies

90

sunnyosun

2023-04-27

0.16.2

🎨 Move ._entity to private

89

sunnyosun

2023-04-27

0.16.1

💥 Enable .bionty for accessing bionty functionalities

87

sunnyosun

2023-04-24

0.16.0

⚡ Fix adding species

84

sunnyosun

2023-04-19

0.15.2

✨ Add ._entity to knowledge class for features parsing

83

sunnyosun

2023-04-18

0.15.1

🎨 New design of the knowledge API

80

sunnyosun

2023-04-17

0.15.0

♻️ Refactor knowledge

77

sunnyosun

2023-03-28

0.14.0

✨ Added Readout

75

sunnyosun

2023-03-27

0.13.0

📝 Move knowledge guide from lamindb here

71

falexwolf

2023-03-27

⬆️ Updated bionty and lamindb deps

74

sunnyosun

2023-03-27

0.12.2

👷 Upgrade to latest cookiecutter

73

falexwolf

2023-03-27

🚑 Fix get_term

72

sunnyosun

2023-03-25

0.12.1

♻️ Switch to class property

70

sunnyosun

2023-03-24

0.12.0

✨ Expose bionty functions with knowledge decorator

69

sunnyosun

2023-03-24

0.11.0

🐛 Fix Readout population

68

sunnyosun

2023-03-23

0.10.1

⬆️ Update CellMarker columns for bionty 0.10

64

sunnyosun

2023-03-21

0.10.0

✨ Add CellLine, Pathway, Phenotype

63

sunnyosun

2023-03-21

0.9.0

💚 Fix CI

62

sunnyosun

2023-03-21

⬆️ Upgrade to bionty 0.9.0

61

sunnyosun

2023-03-20

0.8.1

🎨 Allow hashing external IDs

60

falexwolf

2023-03-10

⬆️ Stable release of bionty 0.8.0

59

sunnyosun

2023-03-09

0.8.0

⬆️ Bionty>=0.8rc3

58

sunnyosun

2023-03-09

0.8rc2

✨ Use ontology_id as the id for Tissue, CellType, Disease

57

sunnyosun

2023-03-07

0.8.0rc1

⬆️ Upgrade bionty

56

falexwolf

2023-03-06

🔥 Remove migration files

51

sunnyosun

2023-02-14

0.7.0

⬆️ Upgrade and rename lndb_setup to lndb (v0.32.0)

49

bpenteado

2023-02-14

0.6.9

🐛 Fixed taxon_id type

48

sunnyosun

2023-01-17

0.6.8

⬆️ Upgrade dependencies

47

sunnyosun

2023-01-17

0.6.7

⏪ Revert the last PR

46

sunnyosun

2023-01-17

🐛 Fixed typo for postgres

45

sunnyosun

2023-01-16

0.6.6

🎨 Change Species.taxon_id type to int

43

sunnyosun

2023-01-13

👷 Extend CI to py3.8-3.10

42

sunnyosun

2023-01-12

🚚 Rename Species.common_name to Species.name

41

falexwolf

2023-01-05

🐛 Fix migration for sqlite

40

falexwolf

2023-01-05

0.6.5

♻️ Treat link tables as in all other schema modules

39

falexwolf

2023-01-05

0.6.4

🚚 Move knowledge version tables into module

38

falexwolf

2023-01-05

0.6.3

🧪 Added test for ontology autopopulate

37

sunnyosun

2022-12-09

✅ Add migrations test

36

falexwolf

2022-12-08

✨ Added bionty_version table

35

sunnyosun

2022-12-07

0.6.2

🎨 Updated bionty_versions

34

sunnyosun

2022-12-05

✨ Added bionty_versions table

33

sunnyosun

2022-11-18

0.6.1

🏗️ Introduce relationships, move Features table to core

32

falexwolf

2022-11-17

0.6.0

⚡ Implemented knowledge as a decorator

31

sunnyosun

2022-11-11

0.5.4

✨ Added populator for ontologies

30

sunnyosun

2022-11-11

♻️ Refactor knowledge population

29

falexwolf

2022-11-06

🐛 Fixed postgres migration script

28

sunnyosun

2022-11-06

0.5.3

✨ Use bionty to populate species

27

sunnyosun

2022-11-06

0.5.2

🐛 Fix prefix

26

falexwolf

2022-11-03

0.5.1

🎨 Modularize and capitalize

24

falexwolf

2022-11-03

0.5.0

🎨 Updated cell marker id gen

23

sunnyosun

2022-10-25

0.4.5

✏️ Updated species id to be 3 chars

22

sunnyosun

2022-10-24

0.4.4

🧑‍💻 Added naming convention

20

sunnyosun

2022-10-20

🩹 Fix type annotations of primary keys

19

falexwolf

2022-09-29

0.4.3

✏️ Changed ids to str

18

sunnyosun

2022-09-29

0.4.2

🍱 Added description and version columns to the gene table

17

sunnyosun

2022-09-27

0.4.1

🚚 Update primary keys to universal ids

16

falexwolf

2022-09-22

0.4.0

🚚 Rename column in version table, add testdb, and migration script

15

falexwolf

2022-09-22

0.3.1

✨ Add migrations infra

14

falexwolf

2022-09-22

0.3.0

🩹 Added unique constraints

13

sunnyosun

2022-09-22

🐛 Fixed API ref

12

sunnyosun

2022-09-06

0.2.4

🎨 Import additional modules, species -> species_id

11

sunnyosun

2022-08-28

0.2.3

🚚 Rename gene_symbol to symbol, protein_name to name

10

sunnyosun

2022-08-28

0.2.2

⬆️ Update schema to match bionty 0.2.0

9

sunnyosun

2022-08-28

0.2.1

🚚 Rename lndb-schema-bionty to lnschema-bionty

8

falexwolf

2022-08-19

0.2.0

🔧 Add schema module id

7

falexwolf

2022-08-18

🍱 Updated the gene table to match new ensembl table

6

sunnyosun

2022-08-18

0.1.4

🎨 Changed type of species and entrez_gene_id in gene table to int

5

sunnyosun

2022-08-15

0.1.3

🚚 Move biolab to lnschema-wetlab

4

falexwolf

2022-07-31

✨ Introduced featureset to replace geneset and proteinset

2

sunnyosun

2022-07-31

0.1.2

🎨 Changed resolution to platform in readout_type

commit

sunnyosun

2022-07-23

0.1.1

🚚 Move code from lamindb-schema

1

falexwolf

2022-07-23

0.1.0

wetlab

Name

PR

Developer

Date

Version

♻️ Model categoricals via simple Literal

60

falexwolf

2024-08-02

✨ Improved support for perturbations

56

Zethson

2024-07-29

🚚 Rename lnschema-lamin1 to wetlab

47

sunnyosun

2024-03-08

0.27.0

nbproject

Name

PR

Developer

Date

Version

⬆️ Upgrade to pydantic v2

284

falexwolf

2024-07-23

⚡️ Warn instead of raising the exception when ipylab is not installed

283

Koncopd

2024-05-08

0.10.3

♻️ Make ipylab an optional dependency

282

falexwolf

2024-05-06

🔇 Silence erroneous logging

279

falexwolf

2024-02-27

0.10.1

🚸 Init version at 1

277

falexwolf

2024-01-11

0.10.0