Track notebooks & scripts

In addition to tracking Python scripts, LaminDB tracks interactive analyses performed in notebooks.

By calling track() in a notebook or script, input data, and output data get automatically registered associated with the run.


Provenance tracking of notebooks & scripts is analogous to tracking pipelines, scripts & UI data manipulation, see Project flow.


!lamin init --storage ./test-track
Hide code cell output
💡 connected lamindb: testuser1/test-track
import lamindb as ln

ln.settings.verbosity = "hint"
💡 connected lamindb: testuser1/test-track

Initiate tracking

Call track() to auto-generate IDs to track data lineage. Copy these into your cell above track().

ln.settings.transform.stem_uid = "9priar0hoE5u"
ln.settings.transform.version = "0"
💡 notebook imports: lamindb==0.74.0
💡 saved: Transform(uid='9priar0hoE5u6K79', version='0', name='Track notebooks & scripts', key='track', type='notebook', created_by_id=1, updated_at='2024-06-19 23:12:50 UTC')
💡 saved: Run(uid='6v8nJW7DuRur7Td5I0F7', transform_id=1, created_by_id=1)
💡 tracked pip freeze > /home/runner/.cache/lamindb/run_env_pip_6v8nJW7DuRur7Td5I0F7.txt
Run(uid='6v8nJW7DuRur7Td5I0F7', started_at='2024-06-19 23:12:50 UTC', is_consecutive=True, transform_id=1, created_by_id=1)

LaminDB now automatically tracks all input and output data.

Save run reports and source artifact

If you want to save a notebook including its run report & source artifact, run:


See how transforms with execution reports looks in LaminHub:

Query for a notebook or script

In the API, filter the Transform registry to obtain a notebook record:

import lamindb as ln

transform = ln.Transform.filter(name="Track notebooks").one()
# Your notebook is linked with to its source artifact (stripped of its output cells) and execution report (with the notebook's output cells)

On LaminHub, use the UI filter in the Transforms view.

Hide code cell content
# clean up test instance
!lamin delete --force test-track
!rm -r test-track
💡 deleting instance testuser1/test-track
rm: cannot remove 'test-track': No such file or directory