Redun 
¶
Here, we’ll see how to track redun workflows with lamindb.
Note
This guide is based on github.com/ricomnl/bioinformatics-pipeline-tutorial.
Amend the workflow¶
Here is how to instrument a redun workflow for tracking with lamindb:
Add
ln.track()to themain()task (see on GitHub)Register desired output files or folders by creating artifacts for them (see on GitHub):
ln.Artifact(output_path, key="data/results.tar.gz").save()
Add a
finish()task that callsln.finish()(see on GitHub)Optionally cache/stage input files (see on GitHub)
Why not use @ln.flow() for main()?
Because main() in redun is typically a scheduler/executor task rather than a task that performs the actual computation. ln.flow() would then just track the execution time of scheduling, and not an actual compute run.
If one wanted to use @ln.flow() it’s advisable to wrap the scheduling main() task:
@ln.flow()
def run_pipeline(...):
scheduler = Scheduler()
result = scheduler.run(main(...)) # run the main task
ln.Artifact(result.path, key="data/results.tgz").save()
return result
Run redun¶
Let’s see what the input files are:
ls ./fasta
Show code cell output
KLF4.fasta
MYC.fasta
PO5F1.fasta
SOX2.fasta
Create a lamindb test instance:
# pip install lamindb redun git+http://github.com/laminlabs/redun-lamin-fasta
lamin init --storage ./test-redun-lamin
Show code cell output
→ set dev-dir: /home/runner/work/redun-lamin/redun-lamin/docs
→ initialized lamindb: testuser1/test-redun-lamin
Register each input file individually as an artifact:
import lamindb as ln
import json
ln.Artifact.from_dir("./fasta").save()
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
! folder is outside existing storage location, will copy files from ./fasta to /home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/fasta
! no run & transform got linked, call `ln.track()` & re-run
! no run & transform got linked, call `ln.track()` & re-run
! no run & transform got linked, call `ln.track()` & re-run
! no run & transform got linked, call `ln.track()` & re-run
SQLRecordList([Artifact(uid='FJT1x4bsb4AUmvxG0000', key='fasta/MYC.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=536, hash='WGbEtzPw-3bQEGcngO_pHQ', n_files=None, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-08-25 14:20:43 UTC, is_locked=False, version_tag=None, is_latest=True),
Artifact(uid='hYNRNvbV6G5IVjBv0000', key='fasta/SOX2.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=414, hash='C5q_yaFXGk4SAEpfdqBwnQ', n_files=None, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-08-25 14:20:43 UTC, is_locked=False, version_tag=None, is_latest=True),
Artifact(uid='vUbTVuhZbOpzjdqQ0000', key='fasta/PO5F1.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=477, hash='-7iJgveFO9ia0wE1bqVu6g', n_files=None, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-08-25 14:20:43 UTC, is_locked=False, version_tag=None, is_latest=True),
Artifact(uid='2NvGgv1TszeykQ3g0000', key='fasta/KLF4.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=609, hash='LyuoYkWs4SgYcH7P7JLJtA', n_files=None, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=None, schema_id=None, created_by_id=1, created_at=2026-08-25 14:20:43 UTC, is_locked=False, version_tag=None, is_latest=True)])
Run the redun workflow:
redun run workflow.py main --input-dir ./fasta --tag run=test-run 1> run_logs.txt 2>run_logs.txt
Inspect the logs:
cat run_logs.txt
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
→ script invoked with: run wor
kflow.py main --input-dir ./fasta --tag run=test-run
→ created Transform('bUTFA0d9W1gl000
0', key='workflow.py'), started new Run('kmFWceBP0bfH6Seq') at 2026-08-25 14:20:52 UTC
•
tip: to identify the script across renames, pass the uid: ln.track("bUTFA0d9W1gl")
File(path=/home/r
unner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=f315a483)
cs_pipelin
e_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/d
ocs/test-redun-lamin/.lamindb/FJT1x4bsb4AUmvxG0000.fasta, hash=6fc8bc0b), enzyme_regex='[KR]', misse
d_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 893a27b6: bioinformatics_
pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-
lamin/docs/test-redun-lamin/.lamindb/hYNRNvbV6G5IVjBv0000.fasta, hash=a9f56138), enzyme_regex='[KR]'
, missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job deda680a: bioinfor
matics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin
/redun-lamin/docs/test-redun-lamin/.lamindb/vUbTVuhZbOpzjdqQ0000.fasta, hash=b8ec1b21), enzyme_regex
='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 1cc65cb8: b
ioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redu
n-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/2NvGgv1TszeykQ3g0000.fasta, hash=2f82db8b), enzym
e_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job f4f39
8f4: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner
/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/FJT1x4bsb4AUmvxG0000.fasta, hash=6fc8bc
0b), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/F
JT1x4bsb4AUmvxG0000.peptides.txt, hash=03ae4030), amino_acid='C') on default
[redun] Run Job a6ba
adf4: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runne
r/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/hYNRNvbV6G5IVjBv0000.fasta, hash=a9f56
138), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/
hYNRNvbV6G5IVjBv0000.peptides.txt, hash=3d623979), amino_acid='C') on default
[redun] Run Job 6e6
6fbe0: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runn
er/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/vUbTVuhZbOpzjdqQ0000.fasta, hash=b8ec
1b21), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data
/vUbTVuhZbOpzjdqQ0000.peptides.txt, hash=6c2df573), amino_acid='C') on default
[redun] Run Job a6
5be983: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/run
ner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/2NvGgv1TszeykQ3g0000.fasta, hash=2f8
2db8b), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/dat
a/2NvGgv1TszeykQ3g0000.peptides.txt, hash=33f6e460), amino_acid='C') on default
[redun] Run Job 6
020c95a: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/wo
rk/redun-lamin/redun-lamin/docs/test-redun-lamin/data/FJT1x4bsb4AUmvxG0000.count.tsv, hash=4c7c1aa6)
) on default
[redun] Run Job 45a018d0: bioinformatics_pipeline_tutorial.lib.plot_count_task(inpu
t_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/hYNRNvbV6G5IV
jBv0000.count.tsv, hash=22f6e743)) on default
[redun] Run Job 2137f646: bioinformatics_pipeline_
tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/te
st-redun-lamin/data/vUbTVuhZbOpzjdqQ0000.count.tsv, hash=452ec5d0)) on default
[redun] Run Job ee
fea09c: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/wor
k/redun-lamin/redun-lamin/docs/test-redun-lamin/data/2NvGgv1TszeykQ3g0000.count.tsv, hash=b4110171))
on default
[redun] Run Job 6afd02c1: bioinformatics_pipeline_tutorial.lib.get_report_task(input
_counts=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/FJT1x4bsb4AU
mvxG0000.count.tsv, hash=4c7c1aa6), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-re
dun-l...) on default
[redun] Run Job 969f2c01: bioinformatics_pipeline_tutorial.lib.archive_resu
lts_task(inputs_plots=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/dat
a/FJT1x4bsb4AUmvxG0000.plot.png, hash=1a009b73), File(path=/home/runner/work/redun-lamin/redun-lamin
/docs/test-redun-la..., input_report=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-r
edun-lamin/data/protein_report.tsv, hash=6fd7d07e)) on default
[redun] Run Job 86e406bb: redun_l
amin_fasta.finish(results_archive=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redu
n-lamin/data/results.tgz, hash=f315a483)) on default
[redun]
[redun] | JOB STATUS 2026/08/25 14:21:
00
[redun] | TASK PENDING RUNNING FAILED CA
CHED DONE TOTAL
[redun] |
[redun] | ALL
0 0 0 0 16 16
[redun] | bioinformatics_pipeline_tutorial.lib.arc
hive_results_task 0 0 0 0 1 1
[redun] | bioinformatics_pipelin
e_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioi
nformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4
4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0
0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0
0 0 0 4 4
[redun] | redun_lamin_fasta.finish
0 0 0 0 1 1
[redun] | redun_lamin_fasta.main
0 0 0 0 1 1
[redun]
[redun]
[redun
] Execution duration: 8.40 seconds
View data lineage¶
artifact = ln.Artifact.get(key="data/results.tgz")
artifact.view_lineage()
Show code cell output
artifact.transform.describe()
Show code cell output
Transform: workflow.py (0000) | description: CLI: redun ├── uid: bUTFA0d9W1gl0000 │ hash: zjZMEeofUMp3FV6fxbgjYg type: script │ branch: main space: all │ created_at: 2026-08-25 14:20:52 UTC created_by: testuser1 └── source_code: │ """workflow.py.""" │ │ # This code is based on a copy from https://github.com/ricomnl/bioinformatics-pi … │ # Copyright Rico Meinl 2022 │ from enum import Enum │ │ import lamindb as ln │ from redun import File, task │ │ import redun_lamin_fasta │ from redun_lamin_fasta.lib import ( │ archive_results_task, │ count_amino_acids_task, │ digest_protein_task, │ get_report_task, │ plot_count_task, │ ) │ │ redun_namespace = redun_lamin_fasta.__name__ │ │ │ class Executor(str, Enum): │ default = "default" │ process = "process" │ batch = "batch" │ batch_debug = "batch_debug" │ │ │ @task() │ def finish(results_archive: File) -> File: │ …
artifact.run.describe()
Show code cell output
Run: kmFWceB (workflow.py) ├── uid: kmFWceBP0bfH6Seq transform: workflow.py (0000) │ started_at: 2026-08-25 14:20:52 UTC finished_at: 2026-08-25 14:21:00 UTC │ status: completed │ branch: main space: all │ created_at: 2026-08-25 14:20:52 UTC created_by: testuser1 ├── cli_args: │ │ run workflow.py main --input-dir ./fasta --tag run=test-run ├── report: HWoOA9Z │ │ → connected lamindb: testuser1/test-redun-lamin │ │ → created Transform('bUTFA0d9W1gl0000', key='workflow.py'), started new Run('kmF … │ │ • tip: to identify the script across renames, pass the uid: ln.track("bUTFA0d9W1 … └── environment: LRFnIxY │ aiobotocore==3.9.0 │ aiohappyeyeballs==2.7.1 │ aiohttp==3.14.3 │ aioitertools==0.13.0 │ …
Explore the run on the hub¶
→ lamin.ai/laminlabs/lamindata/transform/taasWKawCiNA
View the database content¶
ln.view()
Show code cell output
Artifact
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | ... | is_latest | is_locked | created_at | branch_id | created_on_id | space_id | storage_id | run_id | schema_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 6 | qssTVyxA8d0rz8IG0000 | data/results.tgz | None | None | None | 94832 | MEatPkxGIbDFTuIJwmvUqg | None | None | ... | True | False | 2026-08-25 14:20:59.933000+00:00 | 1 | 1 | 1 | 1 | 1.0 | None | 1 | |
| 4 | 2NvGgv1TszeykQ3g0000 | fasta/KLF4.fasta | None | .fasta | None | None | 609 | LyuoYkWs4SgYcH7P7JLJtA | None | None | ... | True | False | 2026-08-25 14:20:43.634000+00:00 | 1 | 1 | 1 | 1 | NaN | None | 1 |
| 3 | vUbTVuhZbOpzjdqQ0000 | fasta/PO5F1.fasta | None | .fasta | None | None | 477 | -7iJgveFO9ia0wE1bqVu6g | None | None | ... | True | False | 2026-08-25 14:20:43.633000+00:00 | 1 | 1 | 1 | 1 | NaN | None | 1 |
| 2 | hYNRNvbV6G5IVjBv0000 | fasta/SOX2.fasta | None | .fasta | None | None | 414 | C5q_yaFXGk4SAEpfdqBwnQ | None | None | ... | True | False | 2026-08-25 14:20:43.632000+00:00 | 1 | 1 | 1 | 1 | NaN | None | 1 |
| 1 | FJT1x4bsb4AUmvxG0000 | fasta/MYC.fasta | None | .fasta | None | None | 536 | WGbEtzPw-3bQEGcngO_pHQ | None | None | ... | True | False | 2026-08-25 14:20:43.632000+00:00 | 1 | 1 | 1 | 1 | NaN | None | 1 |
5 rows × 22 columns
Run
| uid | name | description | entrypoint | started_at | finished_at | params | extra_data | reference | reference_type | ... | created_at | branch_id | created_on_id | space_id | transform_id | report_id | environment_id | plan_id | created_by_id | initiated_by_run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 1 | kmFWceBP0bfH6Seq | None | None | None | 2026-08-25 14:20:52.470000+00:00 | 2026-08-25 14:21:00.500270+00:00 | None | None | None | None | ... | 2026-08-25 14:20:52.470000+00:00 | 1 | 1 | 1 | 1 | 7 | 5 | None | 1 | None |
1 rows × 22 columns
Storage
| uid | root | description | type | region | instance_uid | is_locked | created_at | branch_id | created_on_id | space_id | created_by_id | run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||
| 1 | EW7uCsJ3lgQD | /home/runner/work/redun-lamin/redun-lamin/docs... | None | local | None | iQlBPgD8uaqR | False | 2026-08-25 14:20:42.474000+00:00 | 1 | 1 | 1 | 1 | None |
Transform
| uid | key | description | kind | source_code | hash | reference | reference_type | version_tag | is_latest | is_locked | created_at | branch_id | created_on_id | space_id | environment_id | plan_id | run_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||
| 1 | bUTFA0d9W1gl0000 | workflow.py | CLI: redun | script | """workflow.py."""\n\n# This code is based on ... | zjZMEeofUMp3FV6fxbgjYg | None | None | None | True | False | 2026-08-25 14:20:52.467000+00:00 | 1 | 1 | 1 | None | None | None | 1 |
Appendix¶
Map the redun execution id¶
Export the run information from redun:
redun log --exec --exec-tag run=test-run --format json --no-pager > redun_exec.json
Map it on the run reference:
# load the redun execution id from the JSON and store it in the LaminDB run record
with open("redun_exec.json") as file:
redun_exec = json.loads(file.readline())
artifact.run.reference = redun_exec["id"]
artifact.run.reference_type = "redun_id"
artifact.run.save()
Show code cell output
Run(uid='kmFWceBP0bfH6Seq', name=None, description=None, entrypoint=None, started_at=2026-08-25 14:20:52 UTC, finished_at=2026-08-25 14:21:00 UTC, params=None, extra_data=None, reference='cbad5e72-69d7-40b6-b6c9-76d08a905280', reference_type='redun_id', cli_args='run workflow.py main --input-dir ./fasta --tag run=test-run', branch_id=1, created_on_id=1, space_id=1, transform_id=1, report_id=7, environment_id=5, plan_id=None, created_by_id=1, initiated_by_run_id=None, created_at=2026-08-25 14:20:52 UTC, is_locked=False)
Map the redun run report¶
While lamindb auto-tracks the logs of the main python process you might also want to link the dedicated redun logs:
report = ln.Artifact(
"run_logs.txt",
description=f"Redun run report of {redun_exec['id']}",
run=False,
kind="__lamindb_run__", # mark as auxiliary artifact for the run
).save()
artifact.run.report = report
artifact.run.save()
Show code cell output
Run(uid='kmFWceBP0bfH6Seq', name=None, description=None, entrypoint=None, started_at=2026-08-25 14:20:52 UTC, finished_at=2026-08-25 14:21:00 UTC, params=None, extra_data=None, reference='cbad5e72-69d7-40b6-b6c9-76d08a905280', reference_type='redun_id', cli_args='run workflow.py main --input-dir ./fasta --tag run=test-run', branch_id=1, created_on_id=1, space_id=1, transform_id=1, report_id=8, environment_id=5, plan_id=None, created_by_id=1, initiated_by_run_id=None, created_at=2026-08-25 14:20:52 UTC, is_locked=False)