Changelog 2025
ยถ
Note
Get notified by watching releases for git repositories: lamindb ยท laminhub ยท lamin-cli ยท laminr ยท nf-lamin ยท bionty
2025-12-18 hub 1.6.0ยถ
โจ Creating sheets from scratch + creating schemas & features + updating features of a sheet PR @falexwolf
๐ Fix error on instance
Settingspage if no schema modules are defined PR @chaichontat๐ Update style for
New instancebutton PR @chaichontat
2025-12-16 hub 1.5.2ยถ
๐ Fix ancestry resolution for nested
Record[...]dtypes PR @chaichontat๐ธ Always show delete confirmation prompt in sheets PR @chaichontat
2025-12-15 R 1.2.1ยถ
2025-12-15 hub 1.5.1ยถ
๐ Enforce branch code filter also when
showAll=truePR @chaichontat๐ Access tables now show a loading state PR @chaichontat
โก๏ธ Faster load for instance collaborators PR @chaichontat
2025-12-14 db 1.17.0ยถ
โ ๏ธ Run lamin migrate deploy
All instances connected to LaminHub have been migrated and there is no need to act.
If you are an admin of a self-managed instance, please migrate your database with lamin migrate deploy.
For details see the Compatibility Matrix and the source code.
LaminDB instances can now easily be snapshot as an SQLite file. We leverage this to provide https access to clones of public instances that are hosted on S3, avoiding firewall-related time outs if the Postgres-protocol or ports are blacklisted and generally improving latency of queries.
โจ Add
ln.DBto conveniently query all entities in any instance PR @Zethsonโจ Add
lamin ioto manage snapshots, exports & imports PR @Zethsonโจ Read-only queries of public instances through shadow SQLite clones PR @Zethson
More convenient access to feature values:
โจ Implement
FeatureManager.__getitem__for getting values records PR @sunnyosun
General UX improvements:
๐ธ Enable queries to other instances also in case no default instance is configured PR @Koncopd
๐ธ Introduce
.describe()on allBaseSQLRecordobjects and classes + onQuerySetPR @falexwolf๐ธ Warn if a user passes a string to the
dtypeargument ofFeatureand add validation for existence of record types PR @falexwolf๐ธ No more warning about not being connected given
ln.DB()now exists PR @falexwolf
General data integrity:
๐ธ Propagate re-organization of record types to dtypes PR @falexwolf
๐ธ Maintain a history of feature renaming events in the feature record, simplify handling label rename events PR @falexwolf
๐ธ When deleting a record type, delete its descendants, too PR @falexwolf
๐ธ Be more strict about constraining types in
HasTypePR @falexwolf
Validation & annotation:
๐ธ Make adding optional features convenient through
schema.add_optional_features()PR @falexwolf๐ธ Better handle
organismconstraints where they are needed, default tobt.settings.organism = NonePR @sunnyosun๐ธ Improve error behavior if a
typeisnโt found during validation: raise anIntegrityErrorinstead ofInvalidArgumentPR @falexwolf๐ธ Leverage schema feature association when annotating artifacts through curators PR @falexwolf
๐ธ If matching the same term multiple times at different levels of a type hierarchy during validation, annotate with top-level match PR @falexwolf
๐ Resolve feature name clashes during usage of
artifact.features.add_values()PR @falexwolf๐ Fix a bug where a changed set of valid features led to violating the per-
slotunique constraint inArtifactSchemarather than updating the existing inferred schema link PR @falexwolf
Sheets:
๐ธ Better name for
record.type_to_dataframe():record.to_dataframe()PR @falexwolf๐๏ธ Add an
extra_dataJSON field toRecordPR @falexwolf๐ Gracefully handle corrupted
dateanddatetimevalues in sheets PR @falexwolf๐ Fix type casting to pandas in sheet exports with
dtype="num"PR @falexwolf๐ Consistent filtering behavior for
kind__startswith='__lamindb_'artifacts acrossto_dataframe()in different contexts PR @falexwolf๐ Respect soft delete in sheet export PR @falexwolf
๐ Fix sheet export with mixed single- and multi-item lists PR @falexwolf
๐ Fix sheet export when multiple features with the same name in an instance PR @falexwolf
๐ Fix sheet export of columns that contain only nulls and add a broad test suite PR @falexwolf
๐ Honor
listdtypeover the defaultsetin sheet export PR @falexwolf๐ Fix sheet export of features with list dtype and additional field qualifiers PR @falexwolf
๐ Fix a bug where when a list dtype categorical feature was present in a sheet without the corresponding non-list one, itโd be exlcuded from the sheet export PR @falexwolf
๐ In sheet export, do not limit to just 100 records PR @falexwolf
โป๏ธ Allow
__lamindb_...type columns in sheet export also ifschema.maximal_set = TruePR @falexwolf
General UX:
โก๏ธ Faster
lamin initfrom compressed SQLite file PR @apโ @falexwolf๐ธ Suggest using
.from_dataframe()when passing aDataFrame-like path toArtifact()PR @falexwolf๐ธ Give a hint on re-login while connected to an instance PR @Koncopd
๐ธ Allow omitting
entityinlamin loadPR @falexwolf๐ธ Enable
skip_hash_lookupin transform PR @falexwolf๐ธ Show objects when storage is non-empty on deletion PR @Koncopd
๐ธ Clarify DB vendor vs. dialect PR @falexwolf
๐ธ Better error message if notebook filename is
Untitled.ipynbPR @falexwolf๐ธ Allow permanent deletion from CLI PR @falexwolf
General bug fixes:
๐ Fix
logger.mute()before importing lamindb and refactorRootLoggerPR @falexwolf๐
Artifact()incorrectly returned latest version instead of the matching one whenhashwas found in version history andkeymatched both versions PR @falexwolf๐ Honor
dev-dirinlamin loadPR @falexwolf๐ Honor
dev-dirinlamin savePR @falexwolf๐ Fix
view_parents()andquery_relatives()for non-default instances PR @Zethson
Migrations:
๐๏ธ Add multiple fields to
TransformPR @falexwolf๐๏ธ Add
ArtifactArtifactandArtifactRunto enabledtype=Artifactanddtype=Runin artifact relationships PR @falexwolf๐๏ธ Add
RecordCollectionandCollectionRecordPR @falexwolf
General docs:
๐ More extensive documentation for
Artifact.otypePR @falexwolf
General refactoring:
โป๏ธ Allow skipping to mark the storage root when registering a storage PR @Koncopd
โป๏ธ Add
query_ulabels(),query_features(),query_schemas(), etc. to allHasTyperegistries PR @falexwolfโป๏ธ Add
LAMIN_S3_ANONenv variable to forceanonmode for all non-managed s3 buckets PR @Koncopdโป๏ธ Centralize
Curatorunit tests in thecoreworker PR @falexwolfโป๏ธ Adapt code to rate limiting initialization of instances PR @Koncopd
โป๏ธ Remove logic that checks admin permissions client side before migrate step PR @falexwolf
โป๏ธ Corrections and clarifications for
get_parents()PR @Koncopdโป๏ธ Harmonize lineage graph display PR @falexwolf
Dependencies:
โ Add
pgtriggerPR @falexwolf
Independent tests:
โ Also run core unit tests against an SQLite database, not just against Postgres PR @falexwolf
2025-12-13 hub 1.5.0ยถ
โจ Add a
Create instancebutton to the UI PR @fredericenardโจ Introduce guests in an organization PR @fredericenard
โจ Add per-account instance quota enforcement PR @fredericenard
โจ Support date filling PR @chaichontat
๐ธ A specialized dict cell in sheets PR @chaichontat
๐ธ Improve the design of the account storage & infrastructure pages PR @fredericenard
๐ธ Improve robustness of sheet-to-artifact export PR @fredericenard
๐ Propagate org membership loss to team members and instance & space collaborators PR @fredericenard
๐ Enforce absolute
typehierarchy within relationaldtypesPR @chaichontat๐ Default for
Feature.nullableshould beTruePR @sunnyosun๐ Reject empty non-nullable features with dtype
list[...]PR @chaichontat๐ Fix edge cases in the S3 storage metadata endpoint PR @chaichontat
๐ Skip existing transform name validation PR @chaichontat
๐ Bar chart renders as horizontal stacked bar with long category labels PR @sunnyosun
๐ Fix pagination display in team members management dialog PR @sunnyosun
๐ Refine landing page PR @falexwolf
2025-12-04 hub 1.4.2ยถ
๐ Fix a breadth of edge cases for sheet export PR @falexwolf
2025-12-04 hub 1.4.1ยถ
๐ Fix populating sheets from transforms in presence of list dtypes PR @chaichontat
๐ Fix sheet export for list dtypes by upgrading lamindb PR @fredericenard
2025-12-03 hub 1.4.0ยถ
โจ In
Launchview: addlistanddictdtypesupport for run metadata PR @chaichontat๐ธ In
Launchview: better design forselectandcreatebuttons and behaviors PR @chaichontat๐ Fix list feature parsing from function response PR @sunnyosun
๐ In CSV validation view, do not mark missing values as required for nullable and optional features PR @falexwolf
2025-12-02 hub 1.3.1ยถ
๐ Fix data propagation in
dtypelistfields PR @chaichontat
2025-12-02 hub 1.3.0ยถ
โจ In
Sheeteditor andcsvimport, support storing extra data e.g. from non-validated columns PR @chaichontat
2025-12-01 nf 0.2.2ยถ
2025-12-01 hub 1.2.0ยถ
โจ Support display of list
dtypein Records PR @chaichontat๐ธ On
Launchpage, consolidate buttons and allow dialogs to fill page PR @chaichontat๐ธ On
Launchpage, renameImport to SheettoEdit in browserand addSelectbutton PR @sunnyosun๐ธ On
Launchpage, set initial artifact filter to schema name PR @chaichontat๐ In lineage viewer, show deeper lineage for collections PR @falexwolf
2025-11-27 hub 1.1.1ยถ
๐ Preserve column name
__lamindb_record_name__in csv upload PR @chaichontat
2025-11-26 hub 1.1.0ยถ
โจ Directly load artifacts and files into sheets PR @chaichontat
โจ Support executable transform UIDs PR @chaichontat
โจ Recognize column name
__lamindb_record_name__in csvs to indicate record names for sheet imports PR @chaichontat๐ธ Auto-filename generation for uploads PR @chaichontat
๐ธ Improve visual distinction between optional and required features for a schema PR @chaichontat
๐ Render images in markdown PR @sunnyosun
๐ Render markdown bullets PR @sunnyosun
2025-11-23 hub 1.0.0ยถ
LaminHub is a data collaboration hub built on LaminDB, similar to how GitHub is built on git.
It provides a GUI for LaminDB and makes it easy to manage many LaminDB instances including hosting, access management, and soon, audit logs.
Through the open-source LaminDB, developers retain full access to their data with zero lock-in danger. Instead of depending on rate-limited REST APIs, developers build directly with Postgres & diverse open storage formats on object stores.
You can give it a try by exploring public omics data collections at lamin.ai/explore. Itโs free and no account is required.
LaminHub is a SaaS product. For private data & commercial usage, see: lamin.ai/pricing.
Incremental changes since 0.52.0
๐ธ Make simple records clickable and add markdown page to record detailed view PR @falexwolf
๐ Do not include
cat[andcat[Record[indtypedisplay PR @falexwolf๐ Fix help message text color and alignment PR @sunnyosun
2025-11-20 hub 0.52.0ยถ
๐ธ Hide entrypoint selection if there is only a single one, rename โfunctionโ to โentrypointโ in transform launch PR @falexwolf
๐ธ Update type requirement to use an artifact type for execution PR @chaichontat
๐ Exclude deleted record types when determining dtypes PR @falexwolf
2025-11-19 hub 0.51.0ยถ
๐ธ Implement collapsible sections for
Launchpage PR @chaichontat๐ Fix editing context in quick mode on
Launchpage PR @chaichontat
2025-11-18 hub 0.50.0ยถ
๐ธ Select artifacts in transform execute PR @chaichontat
๐ธ Better annotation definition in the
Schemapage PR @chaichontat
2025-11-17 hub 0.49.0ยถ
๐ธ Consolidate create new/select existing sheet logic PR @chaichontat
๐ธ Inline form for run metadata PR @chaichontat
๐ธ Auto-append run
uidto Nextflow launch output dir PR @chaichontat๐ Add autoscroll to fill handler PR @chaichontat
๐ Ensure a new run is visible right after launching it PR @chaichontat
๐ Fix faceted branch filtering PR @chaichontat
๐ Prevent a selector from creating new entities through
PR @chaichontat ๐ Fix direct links to individual ulabels PR @chaichontat
2025-11-16 hub 0.48.0ยถ
๐ธ Create a tooltip to help orient the user in the โCreate recordโ dialogue PR @chaichontat
๐ธ Auto-select unique transforms during sheet generation PR @chaichontat
๐ In lineage viewer, fix missing display of downstream lineage for collections PR @falexwolf
๐ Fix record labels not showing on artifact detail page PR @falexwolf
๐ Enhance color for
Recordicon PR @falexwolf๐ Make it easy to create a single record PR @falexwolf
๐ Consistently display records under root vs. sub types & exclude types from sheets PR @falexwolf
๐ Add display of
dateanddatetimevalues in sheets PR @falexwolf๐ Show Records at the top of the filtering sidebar for Artifact PR @falexwolf
2025-11-14 db 1.16.2ยถ
๐ Fix re-validating an existing artifact with an external schema and external features PR @falexwolf
2025-11-13 db 1.16.1ยถ
Bug fixes:
๐ Do not log CLI args in notebooks PR @falexwolf
๐ Fix ignoring proxy settings from env variables PR @Koncopd
๐ Fix
ArtifactfromhttpURL if leveraging virtual keys PR @Koncopd๐ Fix no mimetype for URL in
get_stat_file_cloudPR @Koncopd
UX:
๐ธ Log if switching to read-only storage location of another instance also when using
ln.settings.storagePR @falexwolf
2025-11-12 db 1.16.0ยถ
Features:
โจ Introduce
artifact.features.set_values()PR @falexwolfโจ Make it easy to re-run scripts by tracking
cli_argsPR @falexwolf
UX:
๐ธ Pass
stringtofrom_source()for look up across all semantic fields, no longer do that upon passingnameor any other specific field PR @sunnyosun๐ธ Enable passing a list of record objects to
record.features.add_values()PR @falexwolf
Bug fixes:
Deprecations:
๐ Deprecate
Registry.using()in favor ofRegistry.connect()PR @falexwolf๐ Deprecate the
dataargument ofArtifactin favor ofpathand polish class docstrings PR @falexwolf๐ Deprecate
itype='Composite'PR @falexwolf
Dependencies:
Refactoring:
โป๏ธ Refactor
artifacttests PR @falexwolfโป๏ธ Re-organize
.features.add_values()and.features.remove_values()tests and make consistent acrossArtifact&RecordPR @falexwolfโป๏ธ Refactor and simplify
FeatureManager.remove_values()PR @falexwolf
2025-11-05 db 1.15.2ยถ
Note: v1.15.1 had an unnecessary print statement, producing unexpected logs in to_dataframe(include="features").
Bug fixes.
๐ Correctly deal with
None,date, anddatetimeinrecord.features.add_values()PR @falexwolf๐ Cast
intandfloatwhen loading from database where appropriate PR @falexwolf๐ Reset federated AWS credentials cache after logout or re-login PR @Koncopd
๐ Deal with corrupted state in which no
is_latestrecord is found PR @falexwolf๐ Do not restrict modules during
django_reset()PR @falexwolf
UX.
2025-10-30 db 1.15.0ยถ
Features.
โจ Add
Record.featuresto conveniently populate features of records PR1 PR2 @falexwolf
UX:
๐ธ Improve the experience of working in a local development directory:
dev-dir(calledwork-dirin beta) PR @falexwolf๐ธ Support external features in
DataFrameCuratorPR @falexwolf๐ธ Enable passing
parquet_kwargstoArtifact.from_dataframe()for configuringpyarrow.ParquetWriterPR1 PR2 @falexwolf๐ธ Implement database access debugging function PR @Koncopd
Database:
๐๏ธ Remove
db_indexonReference.textfield PR @falexwolf
Bug fixes:
๐ Handle
run=Nonecorrectly when populating subsequent creating runs of collections PR @falexwolf๐ Allow
features.remove_values()to work with features ofdtype=list[cat]PR @sunnyosun
Performance:
โก๏ธ More optimal recursive childrens query PR @falexwolf
Docs:
๐ Simplify API reference PR @falexwolf
๐ Refactor
Recorddocs PR @falexwolf
2025-10-17 nf 0.2.1ยถ
2025-10-27 db 1.14.0ยถ
Features:
UX:
๐ธ More consistent
.to_dataframe()acrossSQLRecord.to_dataframe()andQuerySet.to_datafame(), all now set a defaultlimitof 100 and order by latest creation - also, one can now indicate including features viainclude="features"PR @falexwolf๐ธ Enable users to annotate & validate without having to learn about the
Schemaconcept PR @falexwolf๐ธ Allow features to have reserved Django field names PR @sunnyosun
๐ธ Log writing in-memory objects into cache on artifact creation PR @Koncopd
๐ธ Simplify lineage & ontology graph displays PR @falexwolf
๐ธ Ask for extra storage-level delete confirmation in case
_real_keyis notNonePR @falexwolf๐ธ Better message upon start of a script in
ln.track()PR @falexwolf๐ธ Better dialogue for user delete within existing storage location PR @falexwolf
๐ธ Add fallback behavior if artifact was moved in storage PR @falexwolf
๐ธ Clearer error and logging messages for
sync_git_repoandln.finish()PR @falexwolf๐ธ Prettier instance settings PR @falexwolf
๐ธ Enable passing
sync_git_repovia env variable PR @falexwolf๐ธ Return a
SQLRecordListinArtifact.from_dir()PR @falexwolf๐ธ Populate
referenceandreference_typefor sliding transforms PR @sunnyosun
Improvements to .describe():
๐ธ Re-design
.describe()and refactor the related code base PR @falexwolf๐ธ Add
transform.describe()PR @falexwolf๐ธ Improve
schema.describe(), harmonize withdescribe()ofArtifact,Collection, andRunPR @falexwolf๐ธ Enable
run.describe()PR @falexwolf๐ธ Better way of displaying the schema slot identifier type in
.describe()PR @falexwolf๐ธ Strip
catfrom featuredtypedisplay PR @falexwolf
Bug fixes:
๐ Attach schema when validating with external features PR @falexwolf
๐ Serialize
IterableofSQLRecordin params tracking PR @falexwolf๐ Correctly handle empty lists in curation PR @sunnyosun
๐ Fix s3 upload and download problems arising from local aws configs PR @Koncopd
๐ Fix exporting sheets with features of dtype
UserPR @sunnyosun๐ Also include
key&storagelocation in error handling uponskip_hash_lookup = TruePR @falexwolf๐ Capture outputs of root logger in
LogStreamTrackerPR @falexwolf๐ Exclude records in trash from unique constraint on
nameunder atypeand addis_valid_typeconstraints to all typed entities PR @falexwolf๐ Fix a bug that prevented re-curating existing artifacts in some cases PR @falexwolf
๐ Set db tokens for connections in all contexts (threads, tasks), disable task-specific database wrappers in jupyter for ipykernel>=7.0.0 PR @Koncopd
๐ Raise error in curators when dtype is list but values are not PR @sunnyosun
๐ Respect
feature.dtypeinFeatureManager.get_values()PR @sunnyosun
Dependencies:
โฌ๏ธ Upgrade Django to 5.2 PR @falexwolf
Architecture:
๐๏ธ For instances connected to the hub, they can now be migrated through the API PR @fredericenard
Deprecations:
๐ Deprecate
dtype='cat'in favor ofdtype=ln.RecordPR @falexwolf
Configuration:
Refactors:
โป๏ธ Allow collections with duplicated artifacts PR @falexwolf
โป๏ธ Move
define_valid_features.pyand similar scripts into the package PR @falexwolfโป๏ธ Refactor removing features PR @falexwolf
โป๏ธ Add a
RecordRunlink model PR @falexwolfโป๏ธ Fix type annotation and double-loading of dataframe in
DataFrameCuratorPR @falexwolfโป๏ธ Enable bulk-re-creating records that would violate a unique constraint PR @sunnyosun
โป๏ธ Adapt curator tests to new
describe()format PR @falexwolfโป๏ธ Improve dealing with ad hoc annotations of lists of categorical features PR @falexwolf
โป๏ธ Add
init_local_sqliteto create empty copies of existing Postgres instances PR @Zethson
Docs:
๐ Improve
manage-ontologiesguide PR @sunnyosun๐ README v4 PR @falexwolf
2025-10-16 db 1.13.1ยถ
Features:
โจ Add
Transform.from_git()PR @falexwolf
Bug fixes:
๐ Fix serializing & curating against dtypes in deeply nested type hierarchies PR @falexwolf @sunnyosun
๐ Never return existing transform with same
keyfrom trash PR @falexwolf
UX:
๐ธ Add
restore()to all registries with soft-delete PR @falexwolf๐ธ Format on-prem UI URLs PR @falexwolf
๐ธ Better error message for trying to write on a public connection. PR @Koncopd
Refactor:
โป๏ธ Map features based on name during transfer PR @sunnyosun
Configurability:
๐ง Introduce
skip_hash_lookupforCollectionPR @falexwolf๐ง Do not request db token when
use_root_db_userisTrueonconnectPR @Koncopd
Dependencies:
2025-10-13 db 1.13.0ยถ
โ ๏ธ Run lamin migrate deploy
All instances connected to LaminHub have been migrated and there is no need to act.
If you are an admin of a self-managed instance, please migrate your database with lamin migrate deploy.
Necessary database migrations in this release consist in a new field.
For details see the Compatibility Matrix and the source code.
Features:
โจ Enable tracking run parameters without validation through a new
Run.paramsfield PR @falexwolf
UX:
๐ธ Allow
init()after another instance was connected PR @falexwolf๐ธ Limit name search upon record creation to records of same type PR @falexwolf
๐ธ Better UX for
lamin annotateCLI command PR @falexwolf๐ธ Ignore cycles arising between input artifacts and re-created artifacts PR @falexwolf
๐ธ Speed up
Curator.add_new_from()PR @sunnyosun๐ธ Add clear errors for locked records, test RLS for locked records and
Blockmodels PR @Koncopd๐ธ Polish logging during
ln.track()PR @falexwolf
Bug fixes:
๐ Require a restricted storage location when writing an
Artifactto a restricted space PR @falexwolf๐ Fix hash calculation for hybrid composite schemas, refactor external feature annotation in
Artifact.from_dataframe()PR @falexwolf๐ Sanitize dataframe proxy for
dictvalidation to enable schema-based validation of categoricals in dicts PR @falexwolf๐ Fix annotation with features of
dtype = list[str]PR @falexwolf๐ Fix schema mutation when removing features PR @falexwolf
๐ Do not retrieve records from trash based on hash lookup PR @falexwolf @Koncopd
Configurability:
๐ง Introduce a
skip_hash_lookupargument forArtifactPR @falexwolf๐ง Add an environment variable to set
instance.api_urlPR @Koncopd
Dependencies:
2025-10-09 R 1.2.0ยถ
๐ธ Simplified setup. PR @lazappi
Install the laminr package.
install.packages("laminr", dependencies = TRUE)
Create a LaminDB instance:
lc <- import_module("lamin_cli")
lc$init(storage = "./mydata", modules = "bionty")
Or if you have write access to an instance, login and connect to it:
lc$login()
lc$connect("<account>/<instance>")
More UX:
Configurability:
๐ง Add a
laminr_status()function that returns alaminr_statusobject with information useful for debugging PR @lazappi๐ง Add a
get_current_lamin_settings()function that returns the current LaminDB settings as an R list PR @lazappi๐ง Add
LAMINR_LAMINDB_VERSIONandLAMINR_LAMINDB_OPTIONSenvironment variables PR @lazappi
Bug fixes:
๐ Handle missing setting values in
lamin_settings()PR @lazappi๐ Handle list columns when creating artifacts from data frames PR @lazappi
๐ Handle missing document context ID when detecting the current path for tracking PR @lazappi
๐ Make sure additional arguments are correctly passed to
require_module()PR @lazappi๐ Improve suffix matching when loading artifacts PR @lazappi
Refactoring:
2025-10-07 db 1.12.1ยถ
2025-10-07 db 1.12.0ยถ
โ ๏ธ Run lamin migrate deploy
All instances connected to LaminHub have been migrated and there is no need to act.
If you are an admin of a self-managed instance, please migrate your database with lamin migrate deploy.
Necessary database migrations in this release consist in new fields and auxiliary tables.
For details see the Compatibility Matrix and the source code.
Features:
โจ Add PyTorch Lightning integration with a
Callbackclass PR PR Guide @Zethsonโจ Allow virtual keys with custom real storage keys PR @Koncopd
Beta features:
โจ Introduce a
work-diras a local development directory to organize scripts & notebooks PR @falexwolf
Improvements to the Record class:
๐ธ Expand
Recordso that it can act as an ontology akin toULabelPR @falexwolf๐ธ Add conditional unique constraint for
Record.namegiventypeandspacePR @falexwolf
Database schema:
๐๏ธ Enable
dtype=UserPR @falexwolf๐๏ธ Enable attaching markdown blocks to all entities PR2 @falexwolf
๐๏ธ Add an
is_lockedfield toSQLRecordPR1 PR2 @falexwolf๐๏ธ Make
descriptionfields unlimited length for all registries and increase the max length forArtifact.keyandTransform.keyto 1024 chars PR @falexwolf๐๏ธ Add a
descriptionfield toProject, replacePersonregistry withRecordregistry PR @falexwolf๐๏ธ Make feature
dtypenon-nullable on the database level ifis_type=FalsePR @falexwolf
UX:
๐ธ Add the current branch and the main branch by default for filtering PR @Koncopd
๐ธ Enable creating projects from CLI PR @falexwolf
๐ธ Consistent delete
permanentargument acrossQuerySet,BaseSQLRecordandSQLRecordPR @Koncopd๐ธ Raise an error in
AnnDataCuratorwhenschema.otype != 'AnnData', no exceptions forotype is NonePR @sunnyosun๐ธ Replace default organism warning with targeted message during validation error PR @falexwolf
๐ธ Change logging for django module variable reset to debug PR @falexwolf
๐ธ Improve logging when violating versioning constraints PR @falexwolf
๐ธ De-duplicate data lineage in
Collection.cache()PR @falexwolf
Bugs:
๐ Fix dealing with uniqueness constraints of soft-deleted ontology records PR @sunnyosun
๐ Extend
query_relatives()to respect branches PR @falexwolf๐ Fix small chunksize for very large files when uploading to s3 PR @Koncopd
๐ Set run status code to completion upon
ln.tracked()PR @falexwolf
Code organization:
๐จ Expose main CLI commands as functions on the
lamin_cliroot API for LaminR PR @falexwolf๐จ Deprecate
str_as_catinFeature.from_dict()PR @Zethson๐จ Do no longer allow
uploadas a transform type PR @falexwolf
Dependencies:
2025-09-30 nf 0.2.0ยถ
โจ Allow an advanced user to specify the transform and/or run UID in the config PR @rcannood
๐ Remove dangling files from previous plugin mechanism PR @rcannood
๐ Throw more understandable error when an instance canโt be found PR @rcannood
๐ Make logging consistent with other Lamin packages PR @falexwolf
2025-09-23 db 1.11.3ยถ
Note: v1.11.2 didnโt update internal dependencies correctly and was yanked on PyPI.
๐ Fix inconsistency between
.get()and.filter()due tobranch_idPR @Koncopd๐ Add
copy()to iteration oversys.modulesPR @falexwolf๐ธ Throw a clearer error when creating a schema without features & components and
is_type=TruePR @sunnyosun๐ธ Fix the cache warning on
ln.track()in scripts on Windows PR @Koncopd
2025-09-18 db 1.11.1ยถ
2025-09-14 db 1.11.0 | bionty 1.7.0 | wetlab 1.5.0ยถ
๐๏ธ Always import the entire public API & enable re-connecting in same Python session PR1 PR2 PR3 PR4 @falexwolf @Koncopd
Additional package re-organization
Because users tend to forget commonly used extras in installation commands, a few optional dependencies are now part of the default dependencies.
๐๏ธ Include
wetlab&biontyin lamindb dependencies PR @falexwolf๐๏ธ Integrate the 3 small Jupyter-related dependencies PR @falexwolf
๐๏ธ Pause maintenance of
clinicorePR @falexwolf
Data validation:
โจ Support validating
attrsof Pandas DataFrames PR @Zethsonimport lamindb as ln study_metadata_schema = ln.Schema([ ln.Feature(name="temperature", dtype=float).save(), ln.Feature(name="experiment", dtype=str).save(), ]).save() schema = ln.Schema( slots={"attrs": study_metadata_schema}, # validates keys in df.attrs otype="DataFrame", ).save()
โจ Support validating linked features: replace
str_as_ulabelargument withschemaargument in.features.add_values()and enable passing feature annotations toln.ArtifactPR PR @Zethsonimport lamindb as ln schema = ln.Schema([ ln.Feature(name="species", dtype=str).save(), ln.Feature(name="split", dtype=str).save() ]).save() artifact = ln.Artifact( "./test_file.txt", key="test.txt", features={"species": "bird", "split": "train"}, schema=schema, ).save()
โจ Unstructured slot validation of scverse datastructures PR @Zethson
Other features:
โจ Fine-grained storage access management PR @falexwolf
โจ Allow getting artifacts by path via
Artifact.get(path="...")PR @Koncopdโจ Soft delete for all entities PR @falexwolf @Zethson
โจ Implement lazy artifact saving for streaming data PR @Koncopd
CELLxGENE-compliant curation:
โจ Enable validating & annotating datasets with a CELLxGENE schema. Corresponding new guide. PR @Zethson
Bug fixes:
๐ Include a project feature in a sheet and test it PR @falexwolf
๐ Add missing
RecordUserlink model PR @falexwolf๐ Fix
queryset.to_dataframe(include=...)withusingand annotations PR @Koncopd๐ Fix adding a column to a cloud
AnnDataobject with consolidated metadata PR @Koncopd๐ Do not reset latest version when deleting latest versions of folder artifacts PR @Koncopd
Refactoring:
โป๏ธ Rename
.list()to.to_list(),.from_df()to.from_dataframe()PR @Koncopdโป๏ธ Refactor croissant file mapping PR @falexwolf
โป๏ธ Refactor and prettify
login()andlamin loginPR @falexwolfโป๏ธ Add
typerecord constraint PR @chaichontatโป๏ธ Transfer feature based on uid instead of name PR @sunnyosun
โป๏ธ Simplify
init_storage()PR @falexwolfโป๏ธ Adapt to new convention for tracking R environments PR @falexwolf
UX:
๐ธ Better
lamin deleteand getting versioned entities from trash PR @falexwolf๐ธ Migrate easily in presence of fine-grained access connection strings PR @falexwolf
๐ธ Print warning when creating spaces on SQLite or local instances PR @falexwolf
๐ธ Expose available spaces for a user on fine-grained access instances PR @Koncopd
๐ธ Parse and populate title in R notebooks on
lamin savePR @Koncopd๐ธ Allow updating reports of notebooks on
lamin savePR @Koncopd
Docs:
Upgrades:
Bionty:
โจ Add a script to register new standard ontology in
bionty-assetsPR @namsaraevaโป๏ธ Add indexes PR @falexwolf
Wetlab:
Assets:
Use cases:
๐ Refactor bulk RNA curation guide to being schema-based PR @Zethson
โฌ๏ธ Adapt
sc-imaging2andfacs2to anndata 0.12 PR @namsaraeva PR @sunnyosun
2025-08-12 db 1.10.2ยถ
โจ Add
ProjectRecordfor annotating sheets with projects PR @falexwolf๐ธ Ask for additional confirmation when creating storage locations through switching storage settings PR @falexwolf
๐ Fix
AnnDataAccessorforAnnDataobjects with indices stored as integers PR @Koncopd๐ Annotate artifacts passed to Curator with Schema PR @Zethson
2025-08-06 db 1.10.1ยถ
2025-07-29 db 1.10.0ยถ
Features.
โจ Add
curate_from_croissant()to curate from MLCommonsCroissantfiles PR @falexwolfโจ Allow to receive and add extra parameters for managed buckets PR @Koncopd
โจ Enable reverting database migrations PR @falexwolf
โจ Allow getting settings via the CLI PR @falexwolf
UX.
๐ธ Stricter hash uniqueness on artifact and more indexes PR @falexwolf
๐ธ Hide
VitessceConfigartifacts PR @falexwolf๐ธ Move
.datasetsfrom.coreto.examplesPR @falexwolf๐ธ Cache
branchandspacesettings PR @falexwolf
Refactors and bug fixes.
โป๏ธ Adapt huggingface sync to the changes in their API PR @Koncopd
๐ Fix auto-search for corrupted local storage location PR @falexwolf
2025-07-23 db 1.9.1ยถ
๐ Fix keep-artifacts-local mode when no local storage location is found PR @falexwolf
๐ Enable anonymous users to access public folders on AWS S3 PR @Koncopd
๐ธ Create records under subtype via
Curator.add_new_fromPR @sunnyosun๐ธ On
transformdelete, deleteTransformProjectlinks because they might be protected through a run of the same transform PR @falexwolf
2025-07-21 db 1.9.0 | bionty 1.6.1ยถ
Features.
โจ Enable validating & annotating datasets with a CELLxGENE schema. Corresponding new guide. PR @Zethson
โจ Add
lamin annotate, enable string-based annotation with non-ulabels, overhaul CLI docs PR @falexwolf
UX.
๐ธ Enable
ln.track()andln.finish()for notebooks running on remote servers PR @falexwolf๐ธ Improve UX of working in
keep-artifacts-localmode PR @falexwolf๐ธ Expose
storage,branch, andspacein theArtifactconstructor PR @falexwolf
Bug fixes.
๐ Fix
search_local_rootinkeep-artifacts-localfor storage roots without access PR @Koncopd๐ Fix sync for cache synchronization for timestamps with a fractional part PR PR @Koncopd
๐ Rework
get_storage_region()to make it reliable and use it inupath.to_url()PR @Koncopd๐ Fix creation of a
Schemawithis_type=TruePR @sunnyosun๐ Fix
indexhash calculation forSchemaPR @sunnyosun
Performance.
โก๏ธ Speed up
describe()by 6x PR @falexwolfโก๏ธ Speed up
lamin connectby 2x PR @falexwolfโก๏ธ Implement performant synchronization for directories PR @Koncopd
Docs.
๐ Overhaul the
Curate datasetsguide PR @sunnyosun๐ Add an actual
README.mdPR @falexwolf
Bionty.
โป๏ธ New error message when url of ontology doesnโt exist PR @namsaraeva
โป๏ธ Remove Disease constructor overloads PR @namsaraeva
โป๏ธ Update visibility of
source.dataframe_artifactPR @sunnyosunโป๏ธ Make sure ensembl organism has a
synonymscolumn PR @sunnyosunโป๏ธ Adapt to renaming of
UPath.synchronize()toUPath.synchronize_to()PR PR @Koncopdโป๏ธ Remove pronto warning filters PR @namsaraeva
โป๏ธ Added logging to
Ontology.to_df()PR @namsaraeva
2025-07-14 db 1.8.0ยถ
๐ธ Improve the experience of working in
keep-artifacts-localmode PR @falexwolfโก๏ธ Use native
polarsObject Store by default PR @Koncopd๐ธ Improve error message when attempting to curate against unsaved schema PR @Zethson
๐ธ Improve UX for labeling unsaved records from other instances PR @Zethson
๐ Properly ignore
ln.track()and tracking warnings on read-only connections PR @Koncopd
2025-07-07 db 1.7.1ยถ
2025-07-06 db 1.7.0 | bionty 1.6.0ยถ
โ ๏ธ Consider lamin migrate deploy
All instances connected to LaminHub have been migrated and there is no need to act.
If you are an admin of a self-managed instance, please migrate your database with lamin migrate deploy.
Necessary database migrations in this release consist in new link tables that can interfere with certain deletion calls.
Features.
โจ Enable switching branches and spaces via the CLI PR @falexwolf
โจ Upload the R environment tracked in LaminR PR @falexwolf
โจ Create export functionality for records, enable annotating artifacts by records, enable tracking lineage of records, treat sheets as a record type PR PR PR @falexwolf
โจ Support
cat_filtersto enable specifying Source versions PR @Zethsonโจ Add
dtypepathPR @sunnyosun
UX improvements.
๐ธ Improved storage location management: conveniently create & delete storage locations PR PR PR PR PR @falexwolf @Koncopd
๐ธ Introduce run status codes, save source code & run environment of scripts upon
ln.track()instead ofln.finish()PR @falexwolf๐ธ Simplify default transfer mode: no longer transfer annotations PR @falexwolf
๐ธ Add
settings.is_connectedto check if the current instance is properly connected for use PR @Koncopd๐ธ No longer persist instance settings on
ln.connect()PR @Koncopd๐ธ Case-insensitive uniqueness and lower-case pre-defined branch and space names PR @falexwolf
๐ธ Warn when no categorical values are validated rather than throwing an error, e.g. when validating column names of a dataframe or labels of a label vector PR @sunnyosun
๐ธ Introduce two-column layout in
Artifact.describe()and add information likespace,branch, andkindPR @falexwolf๐ธ Support nested sub types in curation PR PR @sunnyosun @falexwolf
๐ธ Enable inferring all features of a queryset of records or artifacts by passing
features="queryset"to.df()PR @falexwolf๐ธ Mark data transfers via transform key
__lamindb_transfer__PR @falexwolf๐ธ More aggressive checks for
anon_publicrequests to S3 for public instances PR @Koncopd๐ธ Mark upload failures of artifacts via a private boolean indicating a successful save event:
artifact._is_saved_to_storage_locationPR @Koncopd๐ธ Correct treatment of carriage returns in logs PR @Koncopd
๐ธ Add
keepwith default"first"toLookupclass PR @sunnyosun๐ธ Batched bulk saving for persisting large numbers of records efficiently PR @sunnyosun
๐ธ Define all exceptions in
lamindb_setup.errorsPR @falexwolf๐ธ Leverage panderas lazy validation PR @falexwolf
๐ธ Enable setting
overwrite_versionsinArtifactcreator PR @falexwolf๐ธ Default to hiding artifacts with
kind = "__lamindb_run__"in regular queries PR @falexwolf๐ธ Mark as
py.typedto allow mypy inspection and fix return type ofBaseSQLRecord.savePR PR @apโ๐ธ Enable passing
branchandspaceeverywhere PR @falexwolf
Bug fixes.
๐ Auto-version scripts in absence of
ln.track()PR @falexwolf๐ Fix updates of re-loaded
Schemaauxiliary fields PR @Zethson๐ Fix a bug preventing backwards compatibility of instance settings files PR @Koncopd
๐ Have transfer comply with
settings.annotation.n_max_recordsand fix a bug related to repeated schema transfer PR @falexwolf๐ Fix transfer from instance with superset of module to instance with subset of modules in presence of schema-annotation in the additional modules PR @falexwolf
Bionty changes.
๐ธ Allow using public references without prefix filters PR @sunnyosun
โฌ๏ธ Add new ontology versions & fix registration bugs PR PR @Zethson
โป๏ธ Downgrade sources versions in sources.yaml PR @namsaraeva
โป๏ธ Do not upload dataframe if instance is bionty-assets PR @sunnyosun
โป๏ธ Make treatment of
Recordlink tables consistent with lamindb PR @falexwolf๐ Fix
import_sourcewith non-existingparentsin the reference PR @namsaraeva
Deprecated code.
๐ฅ Remove long-deprecated
ln.setup.load()and LaminDB v1 migration logic PR @falexwolf
2025-06-23 nf 0.1.0ยถ
2025-06-10 db 1.6.2ยถ
2025-06-03 db 1.6.1 | bionty 1.5.0ยถ
Bionty.
โจ Flexible ontology sources PR @sunnyosun
LaminDB.
๐ธ Enable passing
--branchand--spacetolamin savePR @falexwolf๐ Fix query of feature-associated labels from non-ULabel registries PR @sunnyosun
2025-06-01 db 1.6.0 | bionty 1.4.0ยถ
โ ๏ธ Consider lamin migrate deploy
All instances connected to LaminHub have been migrated and there is no need to act.
If you are an admin of a self-managed instance, please migrate your database with lamin migrate deploy.
The migrations in this release do not break old LaminDB clients with the exception of writing to the Param registry: the data in the corresponding SQL table got moved into the Feature registry.
The bulk of database-level changes was made in this PR @falexwolf.
remove unique constraint from
Feature.namereplace hard unique constraint on
Transform.hashandArtifact.hash, with conditional unique constraint: hash can be duplicated for different keysnew names for how instances are referred to for
type, these donโt clash with the newrecordconcept:Ulabel.ulabels,Feature.features,Schema.schemas,Project.projectsdefault space uid is now a
"A"for"All"Feature._expect_manynow defaults toNoneso that the auto-display of single values as opposed to a set makes sense, and a user can enforce one (single) or the other (many) in the futurehash is populated for all
FeatureValuerecords so that there is an easy way to universally identify a unique feature value
Changes to registries.
๐๏ธ Integrate the
Paraminto theFeatureregistry PR @falexwolf โ the change is backward compatible on the Python/R level โ on the SQL level, records are transferred from thelamindb_paramtable to thelamindb_featuretable during migrationsโจ Introduce a
Branchregistry PR @falexwolfโป๏ธ Rename
RecordtoSQLRecordPR PR @falexwolfโจ Introduce a flexible
Recordregistry to manage any kind of entity without database migrations PR @falexwolf
Data curation.
โจ Add schema-based
TiledbsomaExperimentCuratorPR PR @Zethsonโจ Support curating lists as values in
DataFrameCuratorPR @sunnyosun
Bug fixes.
๐ Fix transfer for cases in which genes are insufficiently populated PR @falexwolf
Dependency changes.
โฌ๏ธ No longer install
contenttypesPR @falexwolf
UX improvements.
๐ธ Do no longer duplicate tracking of predecessors through the corresponding link table on
TransformPR @falexwolf๐ธ Add
is_run_inputtoArtifact.get()andCollection.get()PR @Koncopd๐ธ Clearer error in
parse_cat_dtypeif cat dtype contains a module name and the module is not found PR @Koncopd๐ธ Better error message when user passes manual
uidtotrack()+ anticipate that the user might want to create new transforms in some cases also if hash matches PR @falexwolf๐ธ Improve setting relationships of unsaved records UX PR @Zethson
๐ธ Improve
DoesNotExisterror message uponDBRecord.get()PR @Zethsonโป๏ธ Set current space when transferring records PR @Koncopd
โป๏ธ Mark internal lamindb-produced artifacts with
kind="__lamindb__"instead of_branch_code=0PR @falexwolf
2025-05-13 db 1.5.3ยถ
2025-05-13 db 1.5.2ยถ
๐ Reset
SpatialDatapath when access in-memory representation PR @Zethson๐ธ Do not validate twice within
Artifact.from_X(...)when passing schema PR @falexwolf
2025-05-08 db 1.5.1ยถ
๐ Fix a too strict unique constraint in composite schemas PR @falexwolf
๐ Fix display of parents & children in
view_parents(with_children)PR @falexwolfโฌ๏ธ Adapt
save_tiledbsoma_experimenttotiledbsoma==1.16.2PR @Koncopd
2025-05-07 db 1.5.0 | bionty 1.3.2ยถ
Data lineage.
๐ธ Make notebook & script tracking via
ln.track()robust to renames PR @falexwolfโจ Enable executing notebooks via
jupyter nbconvert --executePR @falexwolf
CLI updates.
โจ Enable cloud paths for
lamin savePR @Koncopdlamin save s3://my-bucket/my-file.txt
โจ Enable labeling with project during
lamin savePR @falexwolflamin save ./my-folder --project my-project
Streaming artifacts.
โจ Enable
polarsinArtifact.open()andCollection.open()PR @Koncopdโจ Enable
.load(),.open(), and.mapped()on query sets of artifacts PR @Koncopd
Curation & schemas.
โจ Enable curating the index of a dataframe PR @falexwolf
schema = ln.Schema( features=[ ln.Feature(name="required_feature", dtype=str).save(), ], index=ln.Feature(name="sample", dtype=ln.ULabel).save(), ).save()
๐ธ Enable passing a
ULabeltype todtypePR @falexwolfperturbation_type = ln.ULabel.get(name="Perturbation") # perturbation_type.is_type is True ln.Feature(name="perturbation", dtype=perturbation_type)
๐ธ Handle schema updates decently PR @falexwolf
๐ธ Do not annotate with more than
n_max_records = 1000PR @falexwolf๐ธ Introduce a submodule
lamindb.exampleswith schemas PR @falexwolf๐ธ Enable validating against nested dicts in
spatialdataPR @falexwolf๐ธ Better handle validation of ensembl gene IDs and add curator representation PR @Zethson
๐ธ Prettier
Schema.describe()PR @sunnyosun๐ธ
AnnData: enable explicit transposition invarschema definition PR @falexwolf๐ Rename the
componentsargument ofSchema()toslotsPR @falexwolf๐ Fix respecting
schema.ordered_setinDataFramevalidation PR @sunnyosun
Bulk annotation with features & queries via features.
โจ Support feature dtype
dictPR @falexwolfln.Feature(name="metadata_details", dtype=dict).save()
๐ธ For artifacts, improve (1) bulk annotation with features + (2) queries by features PR @falexwolf
General UX improvements.
๐ธ Do not raise exceptions on problems with
copy_or_move_to_cachewithinArtifact.savePR @Koncopd๐ธ Allow passing
keytosave_vitessce_config()PR @namsaraeva
Docs.
๐ Document
uidgeneration, prettify API reference docs PR @falexwolf
Refactoring.
General refactoring.
โป๏ธ Eliminate monkey patching of
django.db.models.QuerySetanddjango.db.models.ManagerPR @Koncopdโป๏ธ Avoid non-lazy loads of settings on import of
lamindb.modelsPR @Koncopd
Refactoring for curation & schemas.
โป๏ธ Restore validation error messages & add their fine-grained testing PR @falexwolf
โป๏ธ Can save csv artifacts in
DataFrameCuratorPR @sunnyosunโป๏ธ Clearer naming conventions in the internal curator codebase PR @falexwolf
โป๏ธ Separate
CatManagerusage for.catattribute and as legacy interface PR @falexwolfโป๏ธ Separate legacy curators from new curators PR @falexwolf
โป๏ธ Execute curator examples and also show them in the curation guide PR @falexwolf
โป๏ธ Refactor annotating with inferred feature sets PR @falexwolf
Fine-grained access management (in beta).
๐ธ Better access management errors on
Record.save()PR @Koncopd๐ Fix
.usingwith fine-grained access instances and permissions test PR @Koncopdโ Temp table based authentication (adapt tests) PR @Koncopd
๐ธ Delete version family if user wants to retain store by passing
storage=Falsetoartifact.delete(), but retain warning PR @falexwolf
2025-04-25 bionty 1.3.1ยถ
๐ Fixed downloading old Ensembl versions. PR @sunnyosun
If you upgraded to bionty 1.3.0 and used Ensembl versions below 108, please clear the cached ontology source files.
import bionty as bt
import shutil
shutil.rmtree(bt.base.settings.dynamicdir)
2025-04-24 R 1.1.0ยถ
LaminR is now documented on docs.lamin.ai.
The previous docs site laminr.lamin.ai continues to host developer docs.
๐ Update documentation site to match the main docs website PR PR @lazappi
๐ท Separate Seurat analysis from rest of the introduction notebook PR @falexwolf
โป๏ธ Make R and Python quickstarts parallel PR @falexwolf
โป๏ธ Move
setup.Rmdtolamin-docsPR @falexwolf
New features.
โจ Improved Python dependency management with
reticulate, deprecatedinstall_lamindb()PR @lazappiโจ Add tracking of the R environment using
paklockfiles PR @lazappi
Bug fixes.
๐ Enable setting wrapped object slots like
artifact$description,artifact$key, etc. PR @lazappi๐ Fix an issue that was preventing
lamin_connect()from being run multiple times with the same instance PR @lazappi๐ Properly clear and delete temporary instances created using
lamin_init_temp()PR @lazappi
Other changes.
2025-04-15 db 1.4.0 | bionty 1.3.0ยถ
โจ Add schema as an argument to Artifact.from_X(). PR @falexwolf
artifact = ln.Artifact.from_df(df, key="my_dataset.parquet", schema=schema).save()
โจ Enable defining simple schemas that merely enforce a feature identifier type. PR @falexwolf
schema = ln.Schema(itype=ln.Feature).save() # <-- enforce valid feature identifiers, no need to define specific required features
โจ Enable defining optional features on a per-schema level & improve schema hash calculation. PR @sunnyosun
schema = ln.Schema(
features=[
ln.Feature(name="sample_id", dtype=str).save() # required
ln.Feature(name="sample_name", dtype=str).with_config(optional=True) # optional
],
).save()
โจ Introduce lamin run with a Modal backend. PR @ragyhaddad
lamin run my_script.py --project my_project # <-- will run the script on Modal
โจ Support auto-download of Ensembl genes of all organisms. PR @sunnyosun
gene_ontology = bt.base.Gene(source="ensembl", organism="rabbit", version='release-103')
gene_ontology.register_source_in_lamindb() # register the new ontology source in lamindb
source = bt.Source.get(entity="bionty.Gene", name="ensembl", organism="rabbit", version='release-103')
bt.Gene.import_source(source=source) # import all genes from that source
๐ธ Enable querying by features & params through Artifact.filter() and Run.filter(). PR @falexwolf
ln.Artifact.filter(scientist="Barbara McClintock")
User experience.
๐ธ
from_sourceno longer returnsNonebut throws aNoResultFoundexception if the look up in the public ontology fails PR @sunnyosun๐ธ Allow renaming artifacts & transforms within the same version family PR @falexwolf
๐ธ Better support
minimal_set,maximal_set,ordered_setin curators PR @sunnyosun๐ธ Enable passing the stem uid to
lamin savePR @falexwolf๐ธ No longer throw an error but merely print a warning when attempting to update a schema PR @falexwolf
๐ธ Enable plain notebook uploads by making a default run for notebook in case no run is found PR @falexwolf
๐ธ Enable to authenticate and set the current instance through environment variables PR @falexwolf
๐ธ Show link to hub in
view_lineage()and render lineage through graphviz also in scripts PR @falexwolf๐ธ Order
IsVersioned.versionsquery set PR @falexwolf๐ธ Do not print warning about missing schema modules PR @falexwolf
Refactors.
โป๏ธ Eliminate duplicated parsing & record creation during curation PR @falexwolf
โป๏ธ Remove
verbosityandorganismarguments onCatManagerlevel PR PR @falexwolfโป๏ธ Organize categorical curation code with
CatColumnPR @sunnyosunโป๏ธ Add
return_graphargument toview_lineage()PR @lazappi
Docs.
๐ Compare lamindb with pydantic and pandera in an FAQ doc PR @falexwolf
๐ Document access any Ensembl genes PR @sunnyosun
Bugs.
๐ Fix validation of
var_indexPR @sunnyosun๐ Fix
numcodecs==0.16.0incompatibility withzarr v2PR @Koncopd๐ Fix organism passing to
from_sourcePR @sunnyosun๐ Return an empty set not a dict for modules in instance settings PR @falexwolf
Bionty.
๐ธ Make the default organism
"human"instead ofNonePR @falexwolfโฌ๏ธ Support Python 3.13 & remove support for Python 3.9 PR @Zethson
โป๏ธ Improve Ensembl prefix detection PR @sunnyosun
โป๏ธ Use
UPath.synchronizeins3_bionty_assetsPR @Koncopd
2025-03-27 db 1.3.2 | bionty 1.2.1ยถ
๐ Fix bionty ontology sources sync through
reticulatePR @falexwolf๐ Fix data transfer through when target instances has no schema modules PR @falexwolf
2025-03-26 db 1.3.1 | bionty 1.2.0ยถ
In Bionty, you can now add custom ontology sources through the Source registry.
df = pd.read_csv("./our_inhouse_genes.csv") # a csv describing gene metadata e.g. from parsing a GTF file
custom_source = bt.Source(entity="bionty.Gene", organism="human", name="Our genes", version="2025-04-01").save()
bt.Gene.add_source(custom_source, df=df) # couple the custom source to the Gene registry
Detailed changes
Bionty now relies on a single file source.yaml to reference public sources.
โจ Enable update existing records to a new ontology PRPR @sunnyosun
โจ Robust support of custom sources PR @sunnyosun
โป๏ธ Refactor
sync_public_sourcesPR @sunnyosunโป๏ธ Refactor default source configuration PR @sunnyosun
โป๏ธ Make EFO parsing the same as other ontologies PR @sunnyosun
โป๏ธ No longer use local source yaml files PR @sunnyosun
โป๏ธ Move source tests from lamindb to bionty PR @sunnyosun
โป๏ธ Standardize organism scientific names from ensembl source PR @sunnyosun
โป๏ธ Increase uid length for
Sourceto 8 chars PR @falexwolf
LaminDB changes.
๐ Enable transferring features pointing to multiple labels PR @sunnyosun
๐ More extensive validation for updates to
artifact.keyandartifact.suffixPR @falexwolf๐ธ Refactor conventions for files written during init: the SQLite file is now
.lamindb/lamin.dband the storage marker is.lamindb/storage_uid.txtPR @falexwolf๐ธ Make upload of large directories more robust by reducing batch size PR @Koncopd
๐ธ Avoid requiring
coerce_dtypefor"int"and"float"in case an integer or floatpd.Series.dtypeonly deviates by numerical precision/range PR @falexwolf๐ธ In
AnnDataCurator, make'obs'schema optional and allow'uns'schema PR @falexwolf
2025-03-16 db 1.3.0ยถ
New features.
โจ Add schema-based
SpatialDataCuratorPR1 PR2 PR3 @Zethsonโจ Add schema-based
MuDataCuratorPR @sunnyosunโจ Add
lamin getfor artifacts andlamin loadfor collections PR @Zethson @falexwolf
Other changes.
โฌ๏ธ Support CELLxGENE schema 5.2.0 PR1 PR2 @sunnyosun
๐ธ Skip
ln.track()when connected in read-only mode PR @falexwolf๐ธ Error if trying to register an instance without a storage in the hub PR @Koncopd
๐ธ Refactor
organismconstraints during validation PR @sunnyosun๐ธ Add more constructor signatures and specific inherited types PR @falexwolf
๐ธ No logging message if database is behind by minor version PR @falexwolf
๐ Re-structure curation guides PR1 PR2 @falexwolf
๐ Integrate tutorials into introduction guide PR @falexwolf
2025-03-10 R 1.0.0ยถ
โจ laminr now has feature parity with lamindb. PR @lazappi
Run
install_lamindb(), which will ensurelamindb >= 1.2in the Python environment used byreticulate.Replace
db <- connect()withln <- import_module("lamindb")and see the โDetailed changesโ dropdown.
The ln object is largely similar to the db object in laminr < v1 and matches lamindbโs Python API (. โ $).
Detailed changes
What |
Before |
After |
|---|---|---|
Connect to the default LaminDB instance |
|
|
Start tracking |
|
|
Get an artifact from another instance |
|
|
Create an artifact from a path |
|
|
Finish tracking |
|
|
See the updated โGet startedโ vignette for more information.
User-facing changes:
Add an
import_module()function to import Python modules with additional functionality, e.g.,import_module("lamindb")for lamindbAdd functions for accessing more
laminCLI commandsAdd a new โIntroductionโ vignette that replicates the code from the Python lamindb introduction guide
Internal changes:
Add an internal
wrap_python()function to wrap Python objects while replacing Python methods with R methods as needed, leaving most work to {reticulate}Update the internal
check_requires()function to handle Python packagesAdd custom
cache()/load()methods to theArtifactclassAdd custom
track()/finish()methods to the lamindb module
2025-03-09 db 1.2.0ยถ
โจ Enable to auto-link entities to projects. PR @falexwolf
ln.track(project="My project")
๐ธ Better support for spatialdata with Artifact.from_spatialdata() and artifact.load(). PR1 PR2 @Zethson
๐ธ Introduce .slots in Schema, Curator, and artifact.features to access schemas and curators by dataset slot. PR @sunnyosun
schema.slots["obs"] # -> schema for .obs slot of AnnData
curator.slots["obs"] # -> curator for .obs slot of AnnData
artifact.features["obs"] # -> feature set for .obs slot of AnnData
๐๏ธ Re-structured the internal API away from monkey-patching Django models. PR @falexwolf
โ ๏ธ Use of internal API
If you used the internal API, you might experience a breaking change. The most drastic change is that all internal registry-related functionality is now re-exported under lamindb.models.
๐ธ When re-creating an Artifact, link subsequent runs instead of updating .run and linking previous runs. PR @falexwolf
On the hub.
More details here. @chaichontat
Before |
After |
|---|---|
An artifact is only shown as an output for the latest run that created the artifact. Previous runs donโt show it. |
All runs that (re-)create an artifact show it as an output. |
More changes:
โจ Enable
Artifact.open()andArtifact.load()for.gzfiles PR @Koncopd๐ Fix passing a path to
ln.track()when no path found bynbprojectPR @Koncopd๐ Do not overwrite
._state_dbof records when the current instance is passed to.usingPR @Koncopd๐ธ Do not show track warning for read-only connections PR @Koncopd
๐ธ Raise
NotImplementedErrorinArtifact.load()if there is no loader PR @Koncopd
2025-02-27 db 1.1.1ยถ
๐ธ Make the
obsandvarDataFrameCuratorobjects accessible viaAnnDataCurator.slotsPR @sunnyosun๐ธ Better error message upon re-creation of schema with same name and different hash PR @falexwolf
๐ธ Raise consistency error if a source path suffix doesnโt match the artifact
keysuffix PR @falexwolf๐ธ Automatically add missing columns upon
DataFrameCurator.standardize()ifnullableisTruePR @falexwolf๐ธ Allow specifying
fsspecupload options inArtifact.savePR @Koncopd๐ธ Populate
Artifact.n_observationsinArtifact.from_df()PR @Koncopd๐ Run
pip freezewith current python interpreter PR @apโ๐ Fix notebook re-run with same hash PR @falexwolf
2025-02-18 db 1.1.0ยถ
โ ๏ธ The FeatureSet registry got renamed to Schema.
All your code is backward compatible. The Schema registry encompasses feature sets as a special case.
โจ Conveniently track functions including inputs, outputs, and parameters with a decorator: ln.tracked(). PR1 PR2 @falexwolf
@ln.tracked()
def subset_dataframe(
input_artifact_key: str, # all arguments tracked as parameters of the function run
output_artifact_key: str,
subset_rows: int = 2,
subset_cols: int = 2,
) -> None:
artifact = ln.Artifact.get(key=input_artifact_key)
df = artifact.load() # auto-tracked as input
new_df = df.iloc[:subset_rows, :subset_cols]
ln.Artifact.from_df(new_df, key=output_artifact_key).save() # auto-tracked as output
โจ Make sub-types of ULabel, Feature, Schema, Project, Param, and Reference. PR @falexwolf
On the hub.
More details here. @awgaan @chaichontat
Before |
After |
|---|---|
perturbation = ln.ULabel(name="Perturbation", is_type=True).save()
ln.ULabel(name="DMSO", type=perturbation).save()
ln.ULabel(name="IFNG", type=perturbation).save()
โจ Use an overhauled dataset curation flow. @falexwolf @Zethson @sunnyosun
support persisting validation constraints as a
pandera-compatible schemasupport validating any feature type, no longer just categoricals
make the relationship between features, dataset schema, and curator evident
Detailed changes for the overhauled curation flow.
โ ๏ธ The API gained the lamindb.curators module as the new way to access Curator classes for different data structures.
This release introduces the schema-based
DataFrameCuratorandAnnDataCuratorThe old-style curation flow for categoricals based on
lamindb.Curator.from_objecttype()continues to work
Before |
After |
|---|---|
Key PRs.
โจ Overhaul curation guides + enable default values and filters on valid categories for features PR @falexwolf
โจ Schema-based curators:
AnnDataCuratorPR @falexwolfโจ Schema-based curators:
DataFrameCuratorPR @falexwolf
Enabling PRs.
โจ Allow passing
artifacttoCuratorPR @sunnyosun๐จ A
ManyToManybetweenSchema.componentsand.compositesPR @falexwolfโป๏ธ Mark
Schemafields as non-editable PR @falexwolfโจ Add auxiliary field
nullabletoFeaturePR @falexwolfโป๏ธ Prettify
AnnDataCuratorimplementation PR @falexwolf๐ธ Better error for malformed categorical dtype PR @falexwolf
๐จ A
ManyToManybetweenSchema.componentsand.compositesPR @falexwolf๐ Restore
.feature_setsas aManyToManyFieldPR @falexwolf๐ Rename
CatCuratortoCatManagerPR @falexwolf๐จ Let
Curator.validate()throw an error PR @falexwolfโป๏ธ Re-purpose
BaseCuratorasCurator, introduceCatCuratorand consolidate shared logic underCatCuratorPR @falexwolfโป๏ธ Refactor
organismhandling in curators PR @falexwolf๐ฅ Eliminate all logic related to
using_keyin curators PR @falexwolf๐ Bulk-rename old-style curators to
CatCuratorPR @falexwolf๐จ Self-contained definition of
CellxGeneschema / validation constraints PR @falexwolf๐ Move
PertCuratorfromwetlabhere and addCellxGeneCuratortest PR @falexwolf๐ Move CellXGene
Curatorfromcellxgene-laminhere PR @falexwolf
schema = ln.Schema(
name="small_dataset1_obs_level_metadata",
features=[
ln.Feature(name="CD8A", dtype=int).save(), # integer counts for CD8A marker
ln.Feature(name="perturbation", dtype=ln.ULabel).save(), # a categorical feature that validates against the ULabel registry
ln.Feature(name="sample_note", dtype=str).save(), # a note for the sample
],
).save()
df = pd.DataFrame({
"CD8A": [1, 4, 0],
"perturbation": ["DMSO", ],
"sample_note": ["value_1", "value_2", "value_3"],
"temperature": [22.2, 25.7, 27.3],
})
curator = ln.curators.DataFrameCurator(df, schema)
artifact = curator.save_artifact(key="example_datasets/dataset1.parquet") # validates compliance with schema, annotates with metadata
assert artifact.schema == schema # the validating schema
โจ Easily filter on a validating schema. @falexwolf @Zethson @sunnyosun
On the hub.
With the Schema filter button, find all datasets that satisfy a given schema (โ explore).
schema = ln.Schema.get(name="small_dataset1_obs_level_metadata") # get a schema
ln.Artifact.filter(schema=schema).df() # filter all datasets that were validated by the schema
โจ Collection.open() returns a pyarrow dataset. PR @Koncopd
df = pd.DataFrame({"feat1": [0, 0, 1, 1], "feat2": [6, 7, 8, 9]})
df[:2].to_parquet("df1.parquet", engine="pyarrow")
df[2:].to_parquet("df2.parquet", engine="pyarrow")
artifact1 = ln.Artifact(shard1, key="df1.parquet").save()
artifact2 = ln.Artifact(shard2, key="df2.parquet").save()
collection = ln.Collection([artifact1, artifact2], key="parquet_col")
dataset = collection.open() # backed by files in the cloud storage
dataset.to_table().to_pandas().head()
โจ Support s3-compatible endpoint urls, say your on-prem MinIO deployment. PR @Koncopd
Speed up instance creation through squashed migrations.
โก Squash migrations PR1 PR2 @falexwolf
Tiledbsoma.
โจ Support
endpoint_urlin operations with tiledbsoma PR1 PR2 @Koncopdโจ Add
Artifact.from_tiledbsomato populaten_observationsPR @Koncopd
MappedCollection.
๐ Allow filtering on
np.naninobs_filterofMappedCollectionPR @Koncopd๐ Fix labels for
NaNin categorical columns forMappedCollectionPR @Koncopd
SpatialDataCurator.
๐ Fix
var_indexstandardization ofSpatialDataCuratorPR1 PR2 @Zethson๐ Fix sample level metadata optional in
SpatialDataCatManagerPR @Zethson
Core functionality.
โจ Allow checking the need for syncing without actually syncing PR @Koncopd
โจ Check for corrupted cache in
Artifact.load()&Artifact.open()PR PR @Koncopdโจ Infer
n_observationsinArtifact.from_anndataPR @Koncopd๐ Account for VSCode appending languageid to markdown cell in notebook tracking PR @falexwolf
๐ Normalize module names for robust checking in
_check_instance_setup()PR @Koncopd๐ Fix idempotency of
Featurecreation whendescriptionis passed and improve filter and get error behavior PR @Zethson๐ธ Make new version upon passing existing
keytoCollectionPR @falexwolf๐ธ Throw better error upon checking
instance.moduleswhen loading a lamindb schema module PR @Koncopd๐ธ Validate existing records in the DB irrespective of whether an ontology
sourceis passed or not PR @sunnyosun๐ธ Full guarantee of avoiding duplicating
Transform,Artifact&Collectionin concurrent runs PR @falexwolf๐ธ Better user feedback during keyword validation in
Recordconstructor PR @Zethson๐ธ Improve local storage not found warning message PR @Zethson
๐ธ Better error message when attempting to save a file while not being connected to an instance PR @Zethson
๐ธ Error for non-keyword parameters for
Artifact.from_xmethods PR @Zethson
Housekeeping.
2025-01-23 db 1.0.5ยถ
2025-01-21 db 1.0.4ยถ
๐ Revert Collection.description back to unlimited length TextField. PR @falexwolf
2025-01-21 db 1.0.3ยถ
๐ธ In track(), improve logging in RStudio sessions. PR @falexwolf
2025-01-20 R 0.4.0ยถ
๐ Migrate to lamindb v1 PR @falexwolf
๐ธ Improve the user experience for setting up Python & reticulate PR @lazappi
2025-01-20 db 1.0.2ยถ
๐ Improvments for lamindb v1 migrations. PR @falexwolf
add a
.descriptionfield toSchemaenable labeling
RunwithULabeladd a
.predecessorsand.successorsfield toProjectakin to whatโs present onTransformmake
.uidfields not editable
2025-01-18 db 1.0.1ยถ
๐ Block non-admin users from confirming the dialogue for integrating lnschema-core. PR @falexwolf
2025-01-17 db 1.0.0ยถ
This release makes the API consistent, integrates lnschema_core & ourprojects into the lamindb package, and introduces a breadth of database migrations to enable future features without disruption. Youโll now need at least Python 3.10.
Your code will continue to run as is, but you will receive warnings about a few renamed API components.
What |
Before |
After |
|---|---|---|
Dataset vs. model |
|
|
Python object for |
|
|
Number of files |
|
|
|
|
|
|
|
|
Consecutiveness field |
|
|
Run initiator |
|
|
|
|
|
Migration guide:
Upon
lamin connect account/instanceyou will be prompted to confirm migrating away fromlnschema_coreAfter that, you will be prompted to call
lamin migrate deployto apply database migrations
New features:
โจ Allow filtering by multiple
obscolumns inMappedCollectionPR @Koncopdโจ In git sync, also search git blob hash in non-default branches PR @Zethson
โจ Add relationship with
Projectto everything exceptRun,Storage&Userso that you can easily filter for the entities relevant to your project PR @falexwolfโจ Capture logs of scripts during
ln.track()PR1 PR2 @falexwolf @Koncopdโจ Support
"|"-seperated multi-values inCuratorPR @sunnyosun๐ธ Accept
Noneinconnect()and improve migration dialogue PR @falexwolf
UX improvements:
๐ธ Simplify the
ln.track()experience PR @falexwolfyou can omit the
uidargumentyou can organize transforms in folders
versioning is fully automated (requirement for 1.)
you can save scripts and notebooks without running them (corollary of 1.)
you avoid the interactive prompt in a notebook and the throwing of an error in a script (corollary of 1.)
you are no longer required to add a title in a notebook
๐ธ Raise error when modifying
Artifact.keyin problematic ways PR1 PR2 @sunnyosun @Koncopd๐ธ Better error message on running
ln.track()within Python terminal PR @Koncopd๐ธ Hide traceback for
InstanceNotEmptyusing Click Exception PR @Zethson๐ธ Only auto-search
._name_fieldin sub-classes ofCanCuratePR @falexwolf๐ธ Simplify installation & API overview PR @falexwolf
๐ธ Make
lamin_run_uidcategorical in tiledbsoma stores PR @Koncopd๐ธ Raise
ValueErrorwhen trying to search aNonevalue PR @Zethson
Bug fixes:
๐ Skip deleting storage when deleting outdated versions of folder-like artifacts PR @Koncopd
๐ Let
SOMACurator()validate and annotate all.obscolumns PR @falexwolf๐ Fix renaming of feature sets PR @sunnyosun
๐ Do not raise an exception when default AWS credentials fail PR @Koncopd
๐ Only map synonyms when field is name PR @sunnyosun
๐ Fix
sourcein.from_valuesPR @sunnyosun๐ Fix creating instances with storage in the current local working directory PR @Koncopd
๐ Fix NA values in
Curator.add_new_from()PR @sunnyosun
Refactors, renames & maintenance:
๐๏ธ Integrate
lnschema-coreintolamindbPR1 PR2 @falexwolf @Koncopd๐๏ธ Integrate
ourprojectsinto lamindb PR @falexwolfโป๏ธ Manage
created_at,updated_aton the database-level, makecreated_bynot editable PR @falexwolf๐ Rename transform type โglueโ to โlinkerโ PR @falexwolf
๐ Deprecate the
--schemaargument oflamin initin favor of--modulesPR @falexwolf
DevOps:
Detailed list of database migrations
Those not yet announced above will be announced with the functionality they enable.
โป๏ธ Add
contenttypesDjango plugin PR @falexwolf๐ Prepare introduction of persistable
Curatorobjects by renamingFeatureSettoSchemaon the database-level PR @falexwolf๐ Add a
.typeforeign key toULabel,Feature,FeatureSet,Reference,ParamPR @falexwolf๐ Introduce
RunData,TidyTable, andTidyTableDatain the database PR @falexwolf
All remaining database schema changes were made in this PR @falexwolf. Data migrations happen automatically.
remove
_source_code_artifactfrom Transform, itโs been deprecated since 0.75data migration: for all transforms that have
_source_code_artifactpopulated, populatesource_code
rename
Transform.nametoTransform.descriptionbecause itโs analogous toArtifact.descriptionbackward compat:
in the
Transformconstructor usenameto populatekeyin all cases in which onlynameis passedreturn the same transform based on
keyin casesource_code is Nonevia._name_field = "key"
data migrations:
there already was a legacy
descriptionfield that was never exposed on the constructor; to be safe, we concatenated potential data in it on the new description fieldfor all transforms that have
key=Noneandname!=None, usenameto pre-populatekey
rename
Collection.nametoCollection.keyfor consistency withArtifact&Transformand the high likelihood of you wanting to organize them hierarchicallya
_branch_codeinteger on every record to model pull requestsinclude
visibilitywithin that coderepurpose
visibility=0as_branch_code=0as โarchiveโput an index on it
code a โdraftโ as _branch_code = 2, and โdraft prsโ as negative branch codes
rename values
"number"to"num"in dtypean
._auxjson field onRecorda SmallInteger
run._status_codethat allows to writefinished_atin clean up operations so that there is a run time also for aborted runsrename
Run.is_consecutivetoRun._is_consecutivea
_template_idFK to store the information of the generating template (whether a record is a template is coded via _branch_code)rename
_accessortootypeto publicly declare the data format assuffix, accessorrename
Artifact.typetoArtifact.kinda FK to artifact
run._logfilewhich holds logsa
hashfield onParamValueandFeatureValueto enforce uniqueness without running the danger of failure for large dictionariesadd a boolean field
._expect_manytoFeature/Paramthat defaults toTrue/Falseand indicates whether values for this feature/param are expected to occur a single or multiple times for every single artifact/runfor feature
if itโs
True(default), the values come from an observation-level aggregation and a dtype ofdatetimeon the observation-level meanset[datetime]on the artifact-levelif itโs
Falseitโs an artifact-level value anddatetimemeansdatetime; this is an edge case because an arbitrary artifact would always be a set of arbitrary measurements that would need to be aggregated (โone just happens to measure a single cell line in that artifactโ)
for param
if itโs
False(default), the values mean artifact/run-level values anddatetimemeansdatetimeif itโs
True, the values would be from an aggregation, this seems like an edge case but say when characterizing a model ensemble trained with different parameters it could be relevant
remove the
.transformforeign key from artifact and collection for consistency with all other records; introduce a property and a simple filter statement instead that maintains the same UXstore provenance metadata for
TransformULabel,RunParamValue,ArtifactParamValueenable linking projects & references to transforms & collections
rename
Run.parenttoRun.initiated_by_runintroduce a boolean flag on artifact thatโs called
_overwrite_versions, which indicates whether versions are overwritten or stored separately; it defaults toFalsefor file-like artifacts and toTruefor folder-like artifactsRename
n_objectston_filesfor more clarityAdd a
Spaceregistry to lamindb with an FK on everyBasicRecordadd a name column to
Runso that a specific run can be used as a named specific analysisremove
_previous_runsfield on everything exceptArtifact&Collection