Track notebooks, scripts & functions

For tracking pipelines, see: Pipelines – workflow managers.

# pip install 'lamindb[jupyter]'
!lamin init --storage ./test-track

Show code cell output Hide code cell output

→ initialized lamindb: testuser1/test-track

Track a notebook or script

Call track() to register your notebook or script as a transform and start capturing inputs & outputs of a run.

import lamindb as ln

ln.track()  # initiate a tracked notebook/script run

# your code automatically tracks inputs & outputs

ln.finish()  # mark run as finished, save execution report, source code & environment

Here is how a notebook with run report looks on the hub.

Explore it here.

You find your notebooks and scripts in the Transform registry (along with pipelines & functions). Run stores executions. You can use all usual ways of querying to obtain one or multiple transform records, e.g.:

transform = ln.Transform.get(key="my_analyses/my_notebook.ipynb")
transform.source_code  # source code
transform.runs  # all runs
transform.latest_run.report  # report of latest run
transform.latest_run.environment  # environment of latest run

To load a notebook or script from the hub, search or filter the transform page and use the CLI.

lamin load https://lamin.ai/laminlabs/lamindata/transform/13VINnFk89PE

Use projects

You can link the entities created during a run to a project.

import lamindb as ln

my_project = ln.Project(name="My project").save()  # create a project

ln.track(project="My project")  # auto-link entities to "My project"

ln.Artifact(ln.core.datasets.file_fcs(), key="my_file.fcs").save()  # save an artifact

Show code cell output Hide code cell output

→ connected lamindb: testuser1/test-track

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

→ created Transform('89Ql1W9ey0hA0000'), started new Run('p5PYLMIv...') at 2025-09-07 19:24:19 UTC

→ notebook imports: lamindb==1.11a1

• recommendation: to identify the notebook across renames, pass the uid: ln.track("89Ql1W9ey0hA", project="My project")

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

Artifact(uid='yKQ2kEfT2DmwghyU0000', is_latest=True, key='my_file.fcs', suffix='.fcs', size=19330507, hash='rCPvmZB19xs4zHZ7p_-Wrg', branch_id=1, space_id=1, storage_id=1, run_id=1, created_by_id=1, created_at=2025-09-07 19:24:21 UTC)

Filter entities by project, e.g., artifacts:

ln.Artifact.filter(projects=my_project).to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
1	yKQ2kEfT2DmwghyU0000	my_file.fcs	None	.fcs	None	None	19330507	rCPvmZB19xs4zHZ7p_-Wrg	None	None	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:21.915000+00:00	1	{'af': {'0': True}}	1

Access entities linked to a project.

display(my_project.artifacts.to_dataframe())
display(my_project.transforms.to_dataframe())
display(my_project.runs.to_dataframe())

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
1	yKQ2kEfT2DmwghyU0000	my_file.fcs	None	.fcs	None	None	19330507	rCPvmZB19xs4zHZ7p_-Wrg	None	None	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:21.915000+00:00	1	{'af': {'0': True}}	1

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	_aux	branch_id
id
1	89Ql1W9ey0hA0000	track.ipynb	Track notebooks, scripts & functions	notebook	None	None	None	None	1	None	None	True	2025-09-07 19:24:18.993000+00:00	1	None	1

	uid	name	started_at	finished_at	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	created_at	created_by_id	_aux	branch_id
id
1	p5PYLMIvW5OgQVPV	None	2025-09-07 19:24:19.001517+00:00	None	None	None	None	-1	1	1	None	None	None	None	2025-09-07 19:24:19.002000+00:00	1	None	1

Use spaces

You can write the entities created during a run into a space that you configure on LaminHub. This is particularly useful if you want to restrict access to a space. Note that this doesn’t affect bionty entities who should typically be commonly accessible.

ln.track(space="Our team space")

Track parameters

In addition to tracking source code, run reports & environments, you can track run parameters.

Track run parameters

First, define valid parameters, e.g.:

ln.Feature(name="input_dir", dtype=str).save()
ln.Feature(name="learning_rate", dtype=float).save()
ln.Feature(name="preprocess_params", dtype="dict").save()

Show code cell output Hide code cell output

Feature(uid='dJumOlHB46Fj', name='preprocess_params', dtype='dict', array_rank=0, array_size=0, branch_id=1, space_id=1, created_by_id=1, run_id=1, created_at=2025-09-07 19:24:22 UTC)

If you hadn’t defined these parameters, you’d get a ValidationError in the following script.

run_track_with_params.py

import argparse
import lamindb as ln

if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--input-dir", type=str)
    p.add_argument("--downsample", action="store_true")
    p.add_argument("--learning-rate", type=float)
    args = p.parse_args()
    params = {
        "input_dir": args.input_dir,
        "learning_rate": args.learning_rate,
        "preprocess_params": {
            "downsample": args.downsample,  # nested parameter names & values in dictionaries are not validated
            "normalization": "the_good_one",
        },
    }
    ln.track(params=params)

    # your code

    ln.finish()

Run the script.

!python scripts/run_track_with_params.py  --input-dir ./mydataset --learning-rate 0.01 --downsample

Show code cell output Hide code cell output

→ connected lamindb: testuser1/test-track

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

→ created Transform('K9Q2juractQM0000'), started new Run('iZ1uBXdo...') at 2025-09-07 19:24:24 UTC
→ params: input_dir=./mydataset, learning_rate=0.01, preprocess_params={'downsample': True, 'normalization': 'the_good_one'}
• recommendation: to identify the script across renames, pass the uid: ln.track("K9Q2juractQM", params={...})
! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions
! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

Query by run parameters

Query for all runs that match a certain parameters:

ln.Run.filter(
    learning_rate=0.01, input_dir="./mydataset", preprocess_params__downsample=True
).to_dataframe()

Show code cell output Hide code cell output

	uid	name	started_at	finished_at	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	created_at	created_by_id	_aux	branch_id
id
2	iZ1uBXdoFoI5rsOd	None	2025-09-07 19:24:24.646189+00:00	2025-09-07 19:24:26.008623+00:00	None	None	True	0	1	2	3	None	2	None	2025-09-07 19:24:24.646000+00:00	1	None	1

Note that:

• preprocess_params__downsample=True traverses the dictionary preprocess_params to find the key "downsample" and match it to True

• nested keys like "downsample" in a dictionary do not appear in Feature and hence, do not get validated

Access parameters of a run

Below is how you get the parameter values that were used for a given run.

run = ln.Run.filter(learning_rate=0.01).order_by("-started_at").first()
run.features.get_values()

Show code cell output Hide code cell output

{'input_dir': './mydataset',
 'learning_rate': 0.01,
 'preprocess_params': {'downsample': True, 'normalization': 'the_good_one'}}

Here is how it looks on the hub.

Explore parameter values

If you want to query all parameter values together with other feature values, use FeatureValue.

ln.models.FeatureValue.to_dataframe(include=["feature__name", "created_by__handle"])

Show code cell output Hide code cell output

	value	hash	feature__name	created_by__handle
id
1	./mydataset	71I4KdtOlqWZYoR9KaVTvw	input_dir	testuser1
2	0.01	BIF-_RHBU2Sm7COXgAOIYg	learning_rate	testuser1
3	{'downsample': True, 'normalization': 'the_goo...	4ehQH8UO25aNM181K_gloQ	preprocess_params	testuser1

Track functions

If you want more-fined-grained data lineage tracking, use the tracked() decorator.

In a notebook

ln.Feature(name="subset_rows", dtype="int").save()  # define parameters
ln.Feature(name="subset_cols", dtype="int").save()
ln.Feature(name="input_artifact_key", dtype="str").save()
ln.Feature(name="output_artifact_key", dtype="str").save()

Feature(uid='RctBFOICVDwq', name='output_artifact_key', dtype='str', array_rank=0, array_size=0, branch_id=1, space_id=1, created_by_id=1, run_id=1, created_at=2025-09-07 19:24:26 UTC)

Define a function and decorate it with tracked():

@ln.tracked()
def subset_dataframe(
    input_artifact_key: str,
    output_artifact_key: str,
    subset_rows: int = 2,
    subset_cols: int = 2,
) -> None:
    artifact = ln.Artifact.get(key=input_artifact_key)
    dataset = artifact.load()
    new_data = dataset.iloc[:subset_rows, :subset_cols]
    ln.Artifact.from_dataframe(new_data, key=output_artifact_key).save()

Prepare a test dataset:

df = ln.core.datasets.small_dataset1(otype="DataFrame")
input_artifact_key = "my_analysis/dataset.parquet"
artifact = ln.Artifact.from_dataframe(df, key=input_artifact_key).save()

Run the function with default params:

ouput_artifact_key = input_artifact_key.replace(".parquet", "_subsetted.parquet")
subset_dataframe(input_artifact_key, ouput_artifact_key)

Query for the output:

subsetted_artifact = ln.Artifact.get(key=ouput_artifact_key)
subsetted_artifact.view_lineage()

This is the run that created the subsetted_artifact:

subsetted_artifact.run

Run(uid='4KKLhN4x5C60aELq', started_at=2025-09-07 19:24:26 UTC, finished_at=2025-09-07 19:24:26 UTC, branch_id=1, space_id=1, transform_id=3, created_by_id=1, initiated_by_run_id=1, created_at=2025-09-07 19:24:26 UTC)

This is the function that created it:

subsetted_artifact.run.transform

Transform(uid='6iRACK34VaHL0000', is_latest=True, key='track.ipynb/subset_dataframe.py', type='function', hash='CUqkJpolJY1Q1tqyCoWIWg', branch_id=1, space_id=1, created_by_id=1, created_at=2025-09-07 19:24:26 UTC)

This is the source code of this function:

subsetted_artifact.run.transform.source_code

'@ln.tracked()\ndef subset_dataframe(\n    input_artifact_key: str,\n    output_artifact_key: str,\n    subset_rows: int = 2,\n    subset_cols: int = 2,\n) -> None:\n    artifact = ln.Artifact.get(key=input_artifact_key)\n    dataset = artifact.load()\n    new_data = dataset.iloc[:subset_rows, :subset_cols]\n    ln.Artifact.from_dataframe(new_data, key=output_artifact_key).save()\n'

These are all versions of this function:

subsetted_artifact.run.transform.versions.to_dataframe()

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	_aux	branch_id
id
3	6iRACK34VaHL0000	track.ipynb/subset_dataframe.py	None	function	@ln.tracked()\ndef subset_dataframe(\n inpu...	CUqkJpolJY1Q1tqyCoWIWg	None	None	1	None	None	True	2025-09-07 19:24:26.618000+00:00	1	None	1

This is the initating run that triggered the function call:

subsetted_artifact.run.initiated_by_run

Run(uid='p5PYLMIvW5OgQVPV', started_at=2025-09-07 19:24:19 UTC, branch_id=1, space_id=1, transform_id=1, created_by_id=1, created_at=2025-09-07 19:24:19 UTC)

This is the transform of the initiating run:

subsetted_artifact.run.initiated_by_run.transform

Transform(uid='89Ql1W9ey0hA0000', is_latest=True, key='track.ipynb', description='Track notebooks, scripts & functions', type='notebook', branch_id=1, space_id=1, created_by_id=1, created_at=2025-09-07 19:24:18 UTC)

These are the parameters of the run:

subsetted_artifact.run.features.get_values()

{'input_artifact_key': 'my_analysis/dataset.parquet',
 'output_artifact_key': 'my_analysis/dataset_subsetted.parquet',
 'subset_cols': 2,
 'subset_rows': 2}

These input artifacts:

subsetted_artifact.run.input_artifacts.to_dataframe()

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
4	IIXqsd36tSlceLEH0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	9868	aUPgpQYtBZ_zvZvl637r9g	None	3	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:26.596000+00:00	1	{'af': {'0': True}}	1

These are output artifacts:

subsetted_artifact.run.output_artifacts.to_dataframe()

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
5	rgG52ngwPuHbxPhq0000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3238	Jgf9d64lHjcra4Su3v-jSg	None	2	md5	True	False	1	1	None	None	True	3	2025-09-07 19:24:26.665000+00:00	1	{'af': {'0': True}}	1

Re-run the function with a different parameter:

subsetted_artifact = subset_dataframe(
    input_artifact_key, ouput_artifact_key, subset_cols=3
)
subsetted_artifact = ln.Artifact.get(key=ouput_artifact_key)
subsetted_artifact.view_lineage()

Show code cell output Hide code cell output

→ creating new artifact version for key='my_analysis/dataset_subsetted.parquet' (storage: '/home/runner/work/lamindb/lamindb/docs/test-track')

We created a new run:

subsetted_artifact.run

Run(uid='AFDnqu7NIyDX5Ksz', started_at=2025-09-07 19:24:27 UTC, finished_at=2025-09-07 19:24:27 UTC, branch_id=1, space_id=1, transform_id=3, created_by_id=1, initiated_by_run_id=1, created_at=2025-09-07 19:24:27 UTC)

With new parameters:

subsetted_artifact.run.features.get_values()

{'input_artifact_key': 'my_analysis/dataset.parquet',
 'output_artifact_key': 'my_analysis/dataset_subsetted.parquet',
 'subset_cols': 3,
 'subset_rows': 2}

And a new version of the output artifact:

subsetted_artifact.run.output_artifacts.to_dataframe()

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
6	rgG52ngwPuHbxPhq0001	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3852	0Hn71VkZunxiCiSd6qhMIQ	None	2	md5	True	False	1	1	None	None	True	4	2025-09-07 19:24:27.191000+00:00	1	{'af': {'0': True}}	1

See the state of the database:

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
6	rgG52ngwPuHbxPhq0001	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3852	0Hn71VkZunxiCiSd6qhMIQ	None	2.0	md5	True	False	1	1	None	None	True	4	2025-09-07 19:24:27.191000+00:00	1	{'af': {'0': True}}	1
5	rgG52ngwPuHbxPhq0000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3238	Jgf9d64lHjcra4Su3v-jSg	None	2.0	md5	True	False	1	1	None	None	False	3	2025-09-07 19:24:26.665000+00:00	1	{'af': {'0': True}}	1
4	IIXqsd36tSlceLEH0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	9868	aUPgpQYtBZ_zvZvl637r9g	None	3.0	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:26.596000+00:00	1	{'af': {'0': True}}	1
1	yKQ2kEfT2DmwghyU0000	my_file.fcs	None	.fcs	None	None	19330507	rCPvmZB19xs4zHZ7p_-Wrg	None	NaN	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:21.915000+00:00	1	{'af': {'0': True}}	1

Feature

	uid	name	dtype	is_type	unit	description	array_rank	array_size	array_shape	proxy_dtype	synonyms	_expect_many	_curation	space_id	type_id	run_id	created_at	created_by_id	_aux	branch_id
id
7	RctBFOICVDwq	output_artifact_key	str	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:26.546000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
6	om1HaELEO6Px	input_artifact_key	str	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:26.537000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
5	CsGgRby5i0bD	subset_cols	int	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:26.529000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
4	4nmyf9L7UIag	subset_rows	int	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:26.520000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
3	dJumOlHB46Fj	preprocess_params	dict	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:22.067000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
2	rqEiQWjAuK41	learning_rate	float	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:22.059000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
1	K1hYTdf2gJ1Q	input_dir	str	None	None	None	0	0	None	None	None	None	None	1	None	1	2025-09-07 19:24:22.050000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1

FeatureValue

	value	hash	space_id	feature_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	./mydataset	71I4KdtOlqWZYoR9KaVTvw	1	1	NaN	2025-09-07 19:24:24.664000+00:00	1	None	1
2	0.01	BIF-_RHBU2Sm7COXgAOIYg	1	2	NaN	2025-09-07 19:24:24.666000+00:00	1	None	1
3	{'downsample': True, 'normalization': 'the_goo...	4ehQH8UO25aNM181K_gloQ	1	3	NaN	2025-09-07 19:24:24.668000+00:00	1	None	1
4	2	yB5yjZ1ML2NvBn-JzBSGLA	1	4	1.0	2025-09-07 19:24:26.637000+00:00	1	None	1
5	2	yB5yjZ1ML2NvBn-JzBSGLA	1	5	1.0	2025-09-07 19:24:26.639000+00:00	1	None	1
6	my_analysis/dataset.parquet	1ImgyYl4KlCl3XCd-aQE9Q	1	6	1.0	2025-09-07 19:24:26.641000+00:00	1	None	1
7	my_analysis/dataset_subsetted.parquet	G9luXJ51Hi4-Csrifos0Lw	1	7	1.0	2025-09-07 19:24:26.643000+00:00	1	None	1

Project

	uid	name	is_type	abbr	url	start_date	end_date	_status_code	space_id	type_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	HCkRp4HAbI45	My project	False	None	None	None	None	0	1	None	None	2025-09-07 19:24:18.186000+00:00	1	None	1

Run

	uid	name	started_at	finished_at	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	created_at	created_by_id	_aux	branch_id
id
1	p5PYLMIvW5OgQVPV	None	2025-09-07 19:24:19.001517+00:00	NaT	None	None	None	-1.0	1	1	NaN	None	NaN	NaN	2025-09-07 19:24:19.002000+00:00	1	None	1
2	iZ1uBXdoFoI5rsOd	None	2025-09-07 19:24:24.646189+00:00	2025-09-07 19:24:26.008623+00:00	None	None	True	0.0	1	2	3.0	None	2.0	NaN	2025-09-07 19:24:24.646000+00:00	1	None	1
3	4KKLhN4x5C60aELq	None	2025-09-07 19:24:26.622637+00:00	2025-09-07 19:24:26.673727+00:00	None	None	None	NaN	1	3	NaN	None	NaN	1.0	2025-09-07 19:24:26.623000+00:00	1	None	1
4	AFDnqu7NIyDX5Ksz	None	2025-09-07 19:24:27.147498+00:00	2025-09-07 19:24:27.198990+00:00	None	None	None	NaN	1	3	NaN	None	NaN	1.0	2025-09-07 19:24:27.147000+00:00	1	None	1

Storage

	uid	root	description	type	region	instance_uid	space_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	eqdAUVwnA0xZ	/home/runner/work/lamindb/lamindb/docs/test-track	None	local	None	73KPGC58ahU9	1	None	2025-09-07 19:24:15.092000+00:00	1	None	1

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	_aux	branch_id
id
3	6iRACK34VaHL0000	track.ipynb/subset_dataframe.py	None	function	@ln.tracked()\ndef subset_dataframe(\n inpu...	CUqkJpolJY1Q1tqyCoWIWg	None	None	1	None	None	True	2025-09-07 19:24:26.618000+00:00	1	None	1
2	K9Q2juractQM0000	run_track_with_params.py	run_track_with_params.py	script	import argparse\nimport lamindb as ln\n\nif __...	nRUs3ZjuVTbKtBmSXpVQ5A	None	None	1	None	None	True	2025-09-07 19:24:24.644000+00:00	1	None	1
1	89Ql1W9ey0hA0000	track.ipynb	Track notebooks, scripts & functions	notebook	None	None	None	None	1	None	None	True	2025-09-07 19:24:18.993000+00:00	1	None	1

In a script

run_workflow.py

import argparse
import lamindb as ln

ln.Param(name="run_workflow_subset", dtype=bool).save()


@ln.tracked()
def subset_dataframe(
    artifact: ln.Artifact,
    subset_rows: int = 2,
    subset_cols: int = 2,
    run: ln.Run | None = None,
) -> ln.Artifact:
    dataset = artifact.load(is_run_input=run)
    new_data = dataset.iloc[:subset_rows, :subset_cols]
    new_key = artifact.key.replace(".parquet", "_subsetted.parquet")
    return ln.Artifact.from_dataframe(new_data, key=new_key, run=run).save()


if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--subset", action="store_true")
    args = p.parse_args()

    params = {"run_workflow_subset": args.subset}

    ln.track(params=params)

    if args.subset:
        df = ln.core.datasets.small_dataset1(otype="DataFrame")
        artifact = ln.Artifact.from_dataframe(
            df, key="my_analysis/dataset.parquet"
        ).save()
        subsetted_artifact = subset_dataframe(artifact)

    ln.finish()

!python scripts/run_workflow.py --subset

Show code cell output Hide code cell output

→ connected lamindb: testuser1/test-track

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions

→ created Transform('MguuFdGODFWK0000'), started new Run('bhJpcrzm...') at 2025-09-07 19:24:30 UTC
→ params: run_workflow_subset=True
• recommendation: to identify the script across renames, pass the uid: ln.track("MguuFdGODFWK", params={...})

! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions
→ returning existing artifact with same hash: Artifact(uid='IIXqsd36tSlceLEH0000', is_latest=True, key='my_analysis/dataset.parquet', suffix='.parquet', kind='dataset', otype='DataFrame', size=9868, hash='aUPgpQYtBZ_zvZvl637r9g', n_observations=3, branch_id=1, space_id=1, storage_id=1, run_id=1, created_by_id=1, created_at=2025-09-07 19:24:26 UTC); to track this artifact as an input, use: ln.Artifact.get()

→ returning existing artifact with same hash: Artifact(uid='rgG52ngwPuHbxPhq0001', is_latest=True, key='my_analysis/dataset_subsetted.parquet', suffix='.parquet', kind='dataset', otype='DataFrame', size=3852, hash='0Hn71VkZunxiCiSd6qhMIQ', n_observations=2, branch_id=1, space_id=1, storage_id=1, run_id=4, created_by_id=1, created_at=2025-09-07 19:24:27 UTC); to track this artifact as an input, use: ln.Artifact.get()
! creating spaces manually on sqlite databases is possible for demo purposes, but does *not* affect access permissions
→ returning existing artifact with same hash: Artifact(uid='CCySSNFxwftS8FUi0000', is_latest=True, description='log streams of run iZ1uBXdoFoI5rsOd', suffix='.txt', kind='__lamindb_run__', size=0, hash='1B2M2Y8AsgTpgAmY7PhCfg', branch_id=1, space_id=1, storage_id=1, created_by_id=1, created_at=2025-09-07 19:24:26 UTC); to track this artifact as an input, use: ln.Artifact.get()
! updated description from log streams of run iZ1uBXdoFoI5rsOd to log streams of run bhJpcrzmWZVvfCMS

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	run_id	created_at	created_by_id	_aux	branch_id
id
6	rgG52ngwPuHbxPhq0001	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3852	0Hn71VkZunxiCiSd6qhMIQ	None	2.0	md5	True	False	1	1	None	None	True	4	2025-09-07 19:24:27.191000+00:00	1	{'af': {'0': True}}	1
5	rgG52ngwPuHbxPhq0000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3238	Jgf9d64lHjcra4Su3v-jSg	None	2.0	md5	True	False	1	1	None	None	False	3	2025-09-07 19:24:26.665000+00:00	1	{'af': {'0': True}}	1
4	IIXqsd36tSlceLEH0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	9868	aUPgpQYtBZ_zvZvl637r9g	None	3.0	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:26.596000+00:00	1	{'af': {'0': True}}	1
1	yKQ2kEfT2DmwghyU0000	my_file.fcs	None	.fcs	None	None	19330507	rCPvmZB19xs4zHZ7p_-Wrg	None	NaN	md5	True	False	1	1	None	None	True	1	2025-09-07 19:24:21.915000+00:00	1	{'af': {'0': True}}	1

Feature

	uid	name	dtype	is_type	unit	description	array_rank	array_size	array_shape	proxy_dtype	synonyms	_expect_many	_curation	space_id	type_id	run_id	created_at	created_by_id	_aux	branch_id
id
8	bi4idMuzZ56P	run_workflow_subset	bool	None	None	None	0	0	None	None	None	None	None	1	None	NaN	2025-09-07 19:24:29.989000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
7	RctBFOICVDwq	output_artifact_key	str	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:26.546000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
6	om1HaELEO6Px	input_artifact_key	str	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:26.537000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
5	CsGgRby5i0bD	subset_cols	int	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:26.529000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
4	4nmyf9L7UIag	subset_rows	int	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:26.520000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
3	dJumOlHB46Fj	preprocess_params	dict	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:22.067000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1
2	rqEiQWjAuK41	learning_rate	float	None	None	None	0	0	None	None	None	None	None	1	None	1.0	2025-09-07 19:24:22.059000+00:00	1	{'af': {'0': None, '1': True, '2': False}}	1

FeatureValue

	value	hash	space_id	feature_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	./mydataset	71I4KdtOlqWZYoR9KaVTvw	1	1	NaN	2025-09-07 19:24:24.664000+00:00	1	None	1
2	0.01	BIF-_RHBU2Sm7COXgAOIYg	1	2	NaN	2025-09-07 19:24:24.666000+00:00	1	None	1
3	{'downsample': True, 'normalization': 'the_goo...	4ehQH8UO25aNM181K_gloQ	1	3	NaN	2025-09-07 19:24:24.668000+00:00	1	None	1
4	2	yB5yjZ1ML2NvBn-JzBSGLA	1	4	1.0	2025-09-07 19:24:26.637000+00:00	1	None	1
5	2	yB5yjZ1ML2NvBn-JzBSGLA	1	5	1.0	2025-09-07 19:24:26.639000+00:00	1	None	1
6	my_analysis/dataset.parquet	1ImgyYl4KlCl3XCd-aQE9Q	1	6	1.0	2025-09-07 19:24:26.641000+00:00	1	None	1
7	my_analysis/dataset_subsetted.parquet	G9luXJ51Hi4-Csrifos0Lw	1	7	1.0	2025-09-07 19:24:26.643000+00:00	1	None	1

Project

	uid	name	is_type	abbr	url	start_date	end_date	_status_code	space_id	type_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	HCkRp4HAbI45	My project	False	None	None	None	None	0	1	None	None	2025-09-07 19:24:18.186000+00:00	1	None	1

Run

	uid	name	started_at	finished_at	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	created_at	created_by_id	_aux	branch_id
id
1	p5PYLMIvW5OgQVPV	None	2025-09-07 19:24:19.001517+00:00	NaT	None	None	None	-1.0	1	1	NaN	None	NaN	NaN	2025-09-07 19:24:19.002000+00:00	1	None	1
2	iZ1uBXdoFoI5rsOd	None	2025-09-07 19:24:24.646189+00:00	2025-09-07 19:24:26.008623+00:00	None	None	True	0.0	1	2	3.0	None	2.0	NaN	2025-09-07 19:24:24.646000+00:00	1	None	1
3	4KKLhN4x5C60aELq	None	2025-09-07 19:24:26.622637+00:00	2025-09-07 19:24:26.673727+00:00	None	None	None	NaN	1	3	NaN	None	NaN	1.0	2025-09-07 19:24:26.623000+00:00	1	None	1
4	AFDnqu7NIyDX5Ksz	None	2025-09-07 19:24:27.147498+00:00	2025-09-07 19:24:27.198990+00:00	None	None	None	NaN	1	3	NaN	None	NaN	1.0	2025-09-07 19:24:27.147000+00:00	1	None	1
5	bhJpcrzmWZVvfCMS	None	2025-09-07 19:24:30.000958+00:00	2025-09-07 19:24:31.365054+00:00	None	None	True	0.0	1	4	3.0	None	2.0	NaN	2025-09-07 19:24:30.001000+00:00	1	None	1
6	sjo1oGZxZmcoU1do	None	2025-09-07 19:24:31.325987+00:00	2025-09-07 19:24:31.363370+00:00	None	None	None	NaN	1	5	NaN	None	NaN	5.0	2025-09-07 19:24:31.326000+00:00	1	None	1

Storage

	uid	root	description	type	region	instance_uid	space_id	run_id	created_at	created_by_id	_aux	branch_id
id
1	eqdAUVwnA0xZ	/home/runner/work/lamindb/lamindb/docs/test-track	None	local	None	73KPGC58ahU9	1	None	2025-09-07 19:24:15.092000+00:00	1	None	1

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	_aux	branch_id
id
5	t1JTXlN7TNj20000	run_workflow.py/subset_dataframe.py	None	function	@ln.tracked()\ndef subset_dataframe(\n arti...	9NYMDP5l5Iuu9F8VrO3vWQ	None	None	1	None	None	True	2025-09-07 19:24:31.324000+00:00	1	None	1
4	MguuFdGODFWK0000	run_workflow.py	run_workflow.py	script	import argparse\nimport lamindb as ln\n\nln.Pa...	9Lj_NWDi9zlVsBt72GhOTg	None	None	1	None	None	True	2025-09-07 19:24:29.999000+00:00	1	None	1
3	6iRACK34VaHL0000	track.ipynb/subset_dataframe.py	None	function	@ln.tracked()\ndef subset_dataframe(\n inpu...	CUqkJpolJY1Q1tqyCoWIWg	None	None	1	None	None	True	2025-09-07 19:24:26.618000+00:00	1	None	1
2	K9Q2juractQM0000	run_track_with_params.py	run_track_with_params.py	script	import argparse\nimport lamindb as ln\n\nif __...	nRUs3ZjuVTbKtBmSXpVQ5A	None	None	1	None	None	True	2025-09-07 19:24:24.644000+00:00	1	None	1
1	89Ql1W9ey0hA0000	track.ipynb	Track notebooks, scripts & functions	notebook	None	None	None	None	1	None	None	True	2025-09-07 19:24:18.993000+00:00	1	None	1

Sync scripts with git

To sync with your git commit, add the following line to your script:

ln.settings.sync_git_repo = <YOUR-GIT-REPO-URL>

synced_with_git.py

import lamindb as ln

ln.settings.sync_git_repo = "https://github.com/..."
ln.track()
# your code
ln.finish()

You’ll now see the GitHub emoji clickable on the hub.

Manage notebook templates

A notebook acts like a template upon using lamin load to load it. Consider you run:

lamin load https://lamin.ai/account/instance/transform/Akd7gx7Y9oVO0000

Upon running the returned notebook, you’ll automatically create a new version and be able to browse it via the version dropdown on the UI.

Additionally, you can:

• label using ULabel, e.g., transform.ulabels.add(template_label)

• tag with an indicative version string, e.g., transform.version = "T1"; transform.save()

Saving a notebook as an artifact

Sometimes you might want to save a notebook as an artifact. This is how you can do it:

lamin save template1.ipynb --key templates/template1.ipynb --description "Template for analysis type 1" --registry artifact

Show code cell content Hide code cell content

assert run.features.get_values() == {
    "input_dir": "./mydataset",
    "learning_rate": 0.01,
    "preprocess_params": {"downsample": True, "normalization": "the_good_one"},
}

assert my_project.artifacts.exists()
assert my_project.transforms.exists()
assert my_project.runs.exists()

# clean up test instance
!rm -r ./test-track
!lamin delete --force test-track

• deleting instance testuser1/test-track