Transfer data¶

This guide shows how to transfer data from a source database into the currently connected database.

# pip install lamindb
!lamin init --storage ./test-transfer --modules bionty

import lamindb as ln

ln.track("ITeOtm7bhtdq")

Query all artifacts in the laminlabs/lamindata instance and filter them to their latest versions.

# query all latest artifact versions
artifacts = ln.Artifact.connect("laminlabs/lamindata").filter(is_latest=True)

# convert the QuerySet to a DataFrame and show the latest 5 versions
artifacts.to_dataframe().head()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	version	is_latest	is_locked	created_at	branch_id	space_id	storage_id	run_id	schema_id	created_by_id
id
12355	Jco0MsJRch3qr8va0000	None	Output artifact for run 1457	.json	None	None	80	TI5jQI2bFHG6Ap2Tl-X4lw	NaN	NaN	None	True	False	2025-11-13 13:09:29.217619+00:00	1	1	2	1457.0	NaN	18
12354	HGXyzt2O6JAmE7yU0000	None	Output artifact for run 1456	.json	None	None	80	-cWEGjZgdNJvOMkZ42jbVg	NaN	NaN	None	True	False	2025-11-13 13:09:20.672169+00:00	1	1	2	1456.0	NaN	18
12353	8jSRE4Uy9o1rE6u60000	None	Output artifact for run 1453	.json	None	None	80	i4jMxVFaY8QFSi4n1XTL3w	NaN	NaN	None	True	False	2025-11-12 20:10:22.983611+00:00	1	1	2	1453.0	NaN	18
12352	hufaiPqM4YjYrWUH0000	None	Output artifact for run 1452	.json	None	None	80	qJ_le6dX_oJ30O9a4UsGTw	NaN	NaN	None	True	False	2025-11-12 20:10:15.053733+00:00	1	1	2	1452.0	NaN	18
12351	IoLEXQcBcMx0FELM0000	None	Output artifact for run 1449	.json	None	None	80	H65Kn7LVW9r8ak_4UEESng	NaN	NaN	None	True	False	2025-11-12 09:58:19.999322+00:00	1	1	2	1449.0	NaN	18

You can now further subset or search the QuerySet. Here we query by whether the description contains “tabula sapiens”.

artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.describe()

By saving the artifact record that’s currently attached to the source database instance, you transfer it to the default database instance.

artifact.save()

The artifact record has been transferred to the current database without feature & label annotations, but with updated data lineage.

artifact.describe()

You see that the data itself remained in the original storage location, which has been added to the current instance’s storage location as a read-only location (indicated by the fact that the instance_uid doesn’t match the current instance).

ln.Storage.to_dataframe()

Show code cell output Hide code cell output

	uid	root	description	type	region	instance_uid	is_locked	created_at	branch_id	space_id	created_by_id	run_id
id
2	D9BilDV2	s3://lamindata	None	s3	us-east-1	4XIuR0tvaiXM	False	2023-04-22 05:50:06.537267+00:00	1	1	1	2.0
1	jRsfEBkyHO0c	/home/runner/work/lamindb/lamindb/docs/test-tr...	None	local	None	1FHu5eE0uxm4	False	2025-11-14 00:09:59.555000+00:00	1	1	1	NaN

See the state of the database.

ln.view()

Show code cell output Hide code cell output

****************

* module: core *

****************

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	version	is_latest	is_locked	created_at	branch_id	space_id	storage_id	run_id	schema_id	created_by_id
id
1	dPraor9rU1EofcFb6Wph	tabula_sapiens_lung.h5ad	Part of Tabula Sapiens, a benchmark, first-dra...	.h5ad	None	None	3899435772	8mB1KK2wd51F6HQdvqipcQ	None	None	None	True	False	2023-07-14 19:00:30.621330+00:00	1	1	2	2	None	1

Run

	uid	name	started_at	finished_at	params	reference	reference_type	is_locked	created_at	branch_id	space_id	transform_id	report_id	_logfile_id	environment_id	created_by_id	initiated_by_run_id
id
2	TRb8WqzlgOBruvTT	None	2025-11-14 00:10:06.251000+00:00	None	None	None	None	False	2025-11-14 00:10:06.251000+00:00	1	1	2	None	None	None	1	1.0
1	QETAsSpjGWzDGcfF	None	2025-11-14 00:10:03.077680+00:00	None	None	None	None	False	2025-11-14 00:10:03.078000+00:00	1	1	1	None	None	None	1	NaN

Storage

	uid	root	description	type	region	instance_uid	is_locked	created_at	branch_id	space_id	created_by_id	run_id
id
2	D9BilDV2	s3://lamindata	None	s3	us-east-1	4XIuR0tvaiXM	False	2023-04-22 05:50:06.537267+00:00	1	1	1	2.0
1	jRsfEBkyHO0c	/home/runner/work/lamindb/lamindb/docs/test-tr...	None	local	None	1FHu5eE0uxm4	False	2025-11-14 00:09:59.555000+00:00	1	1	1	NaN

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	version	is_latest	is_locked	created_at	branch_id	space_id	created_by_id	_template_id
id
2	4XIuR0tvaiXM0000	__lamindb_transfer__/4XIuR0tvaiXM	Transfer from `laminlabs/lamindata`	function	None	None	None	None	None	True	False	2025-11-14 00:10:06.247000+00:00	1	1	1	None
1	ITeOtm7bhtdq0000	transfer.ipynb	Transfer data	notebook	None	None	None	None	None	True	False	2025-11-14 00:10:03.072000+00:00	1	1	1	None

******************

* module: bionty *

******************

Source

	uid	entity	organism	name	in_db	currently_used	description	url	md5	source_website	version	is_locked	created_at	branch_id	space_id	created_by_id	run_id	dataframe_artifact_id
id
33	5JnVODh4	BioSample	all	ncbi	False	True	NCBI BioSample attributes	s3://bionty-assets/df_all__ncbi__2023-09__BioS...	None	https://www.ncbi.nlm.nih.gov/biosample/docs/at...	2023-09	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
32	MJRqduf9	bionty.Ethnicity	human	hancestro	False	True	Human Ancestry Ontology	http://purl.obolibrary.org/obo/hancestro/relea...	None	https://github.com/EBISPOT/hancestro	3.0	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
31	10va5JSt	bionty.DevelopmentalStage	mouse	mmusdv	False	True	Mouse Developmental Stages	https://github.com/obophenotype/developmental-...	None	https://github.com/obophenotype/developmental-...	2024-05-28	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
30	1GbFkOdz	bionty.DevelopmentalStage	human	hsapdv	False	True	Human Developmental Stages	https://github.com/obophenotype/developmental-...	None	https://github.com/obophenotype/developmental-...	2024-05-28	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
29	1atB0WnU	Drug	all	chebi	False	False	Chemical Entities of Biological Interest	s3://bionty-assets/df_all__chebi__2024-07-27__...	None	https://www.ebi.ac.uk/chebi/	2024-07-27	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
28	ugaIoIlj	Drug	all	dron	False	True	Drug Ontology	http://purl.obolibrary.org/obo/dron/releases/2...	None	https://bioportal.bioontology.org/ontologies/DRON	2024-08-05	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None
27	3rm9aOzL	BFXPipeline	all	lamin	False	True	Bioinformatics Pipeline	s3://bionty-assets/df_all__lamin__1.0.0__BFXpi...	None	https://lamin.ai	1.0.0	False	2025-11-14 00:09:59.660000+00:00	1	1	1	None	None

View lineage:

artifact.view_lineage()

The transferred dataset is linked to a special type of transform that stores the slug and uid of the source instance:

artifact.transform.description

The transform key has the form f"__lamindb_transfer__/{source_instance.uid}":

artifact.transform.key

The current notebook run is linked as the initiated_by_run of the “transfer run”:

artifact.run.initiated_by_run.transform

Upon re-transferring a record, it will identify that the record already exists in the target database and simply map the record.

artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.save()

If you also want to transfer annotations of the artifact, you can pass transfer="annotations" to save(). Just note that this might populate your target database with metadata that doesn’t match the conventions you want to enforce.

artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.save(transfer="annotations")

The artifact is now annotated.

artifact.describe()