Redun¶
Here, we’ll see how to track redun workflow runs with LaminDB.
Note
This use case is based on github.com/ricomnl/bioinformatics-pipeline-tutorial.
# pip install lamindb redun git+http://github.com/laminlabs/redun-lamin-fasta
!lamin init --storage ./test-redun-lamin
Show code cell output
→ initialized lamindb: testuser1/test-redun-lamin
Amend the workflow¶
import lamindb as ln
import json
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
Let’s amend a redun workflow.py to register input & output artifacts in LaminDB:
To track the workflow run in LaminDB, add (see on GitHub):
ln.track(params=params)
To register the output file via LaminDB, add (see on GitHub):
ln.Artifact(output_path, description="results").save()
Run redun¶
Let’s see what the input files are:
!ls ./fasta
Show code cell output
KLF4.fasta MYC.fasta PO5F1.fasta SOX2.fasta
And call the workflow:
!redun run workflow.py main --input-dir ./fasta --tag run=test-run 1> run_logs.txt 2>run_logs.txt
Inspect the logs:
!cat run_logs.txt
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
→ script invoked with: run workflow.py main --input-dir ./fasta --tag run=test-run
→ created Transform('KIjVpH3l8kjg0000', key='workflow.py'), started new Run('7k6cXUwqJ2g36lpN') at 2025-12-14 22:40:14 UTC
→ params: input_dir='./fasta', amino_acid='C', enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75, executor=<Executor.default: 'default'>
9[redun] Run Job 3249817e: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/h0ziqf8gWchXDEpy0000.fasta, hash=3ae095ae), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job bff9bb87: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/hoFile(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=195f9f11)
6626), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 05c636d6: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/7fxdlN7iGVFnem6p0000.fasta, hash=f02bae01), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 6566c7f3: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/yLiXYVcaCGSfraVH0000.fasta, hash=9f094987), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 4a16c45e: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/h0ziqf8gWchXDEpy0000.fasta, hash=3ae095ae), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/h0ziqf8gWchXDEpy0000.peptides.txt, hash=ec09ec50), amino_acid='C') on default
[redun] Run Job 4e408214: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/aoLRlMdamLfUy0Dd0000.fasta, hash=e4156626), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/aoLRlMdamLfUy0Dd0000.peptides.txt, hash=e3ce0460), amino_acid='C') on default
[redun] Run Job 8ff7c07e: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/7fxdlN7iGVFnem6p0000.fasta, hash=f02bae01), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/7fxdlN7iGVFnem6p0000.peptides.txt, hash=654320fe), amino_acid='C') on default
[redun] Run Job 7823baca: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/yLiXYVcaCGSfraVH0000.fasta, hash=9f094987), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/yLiXYVcaCGSfraVH0000.peptides.txt, hash=9002d4e7), amino_acid='C') on default
[redun] Run Job 6fcdeead: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/h0ziqf8gWchXDEpy0000.count.tsv, hash=4c1115a6)) on default
[redun] Run Job 61acbee5: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/aoLRlMdamLfUy0Dd0000.count.tsv, hash=50d5750f)) on default
[redun] Run Job 1ffd22e6: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/7fxdlN7iGVFnem6p0000.count.tsv, hash=9670e9da)) on default
[redun] Run Job 663a0bca: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/yLiXYVcaCGSfraVH0000.count.tsv, hash=bd26e62b)) on default
[redun] Run Job 750caa36: bioinformatics_pipeline_tutorial.lib.get_report_task(input_counts=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/h0ziqf8gWchXDEpy0000.count.tsv, hash=4c1115a6), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-l...) on default
[redun]
[redun] | JOB STATUS 2025/12/14 22:40:35
[redun] | TASK PENDING RUNNING FAILED CACHED DONE TOTAL
[redun] |
[redun] | ALL 2 5 0 0 9 16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task 1 0 0 0 0 1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0 4 0 0 0 4
[redun] | redun_lamin_fasta.finish 1 0 0 0 0 1
[redun] | redun_lamin_fasta.main 0 1 0 0 0 1
[redun]
[redun]
[redun]
[redun] | JOB STATUS 2025/12/14 22:40:55
[redun] | TASK PENDING RUNNING FAILED CACHED DONE TOTAL
[redun] |
[redun] | ALL 2 5 0 0 9 16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task 1 0 0 0 0 1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0 4 0 0 0 4
[redun] | redun_lamin_fasta.finish 1 0 0 0 0 1
[redun] | redun_lamin_fasta.main 0 1 0 0 0 1
[redun]
[redun]
[redun] Run Job ed311e5a: bioinformatics_pipeline_tutorial.lib.archive_results_task(inputs_plots=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/h0ziqf8gWchXDEpy0000.plot.png, hash=fbb42011), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-la..., input_report=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/protein_report.tsv, hash=198fd809)) on default
[redun] Run Job d8717ca3: redun_lamin_fasta.finish(results_archive=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=195f9f11)) on default
[redun]
[redun] | JOB STATUS 2025/12/14 22:41:00
[redun] | TASK PENDING RUNNING FAILED CACHED DONE TOTAL
[redun] |
[redun] | ALL 0 0 0 0 16 16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0 0 0 0 4 4
[redun] | redun_lamin_fasta.finish 0 0 0 0 1 1
[redun] | redun_lamin_fasta.main 0 0 0 0 1 1
[redun]
[redun]
[redun] Execution duration: 47.02 seconds
View data lineage:
artifact = ln.Artifact.get(key="data/results.tgz")
artifact.view_lineage()
Show code cell output
Track the redun execution id¶
If we want to be able to query LaminDB for redun execution ID, this here is a way to get it:
# export the run information from redun
!redun log --exec --exec-tag run=test-run --format json --no-pager > redun_exec.json
# load the redun execution id from the JSON and store it in the LaminDB run record
with open("redun_exec.json") as file:
redun_exec = json.loads(file.readline())
artifact.run.reference = redun_exec["id"]
artifact.run.reference_type = "redun_id"
artifact.run.save()
Run(uid='7k6cXUwqJ2g36lpN', name=None, started_at=2025-12-14 22:40:14 UTC, finished_at=2025-12-14 22:41:00 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='afc69eb3-1e39-4fa6-99fb-05b0aa236d06', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=7, environment_id=1, created_by_id=2, initiated_by_run_id=None, created_at=2025-12-14 22:40:14 UTC, is_locked=False)
Track the redun run report¶
While lamindb tracks logs of the main python process, the redun logs currently don’t get tracked and artifact.run.report is an empty text file.
This is how you can manually attach a report.
report = ln.Artifact(
"run_logs.txt",
description=f"Redun run report of {redun_exec['id']}",
run=False,
kind="__lamindb_run__", # mark as auxiliary artifact for the run
).save()
artifact.run.report = report
artifact.run.save()
Run(uid='7k6cXUwqJ2g36lpN', name=None, started_at=2025-12-14 22:40:14 UTC, finished_at=2025-12-14 22:41:00 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='afc69eb3-1e39-4fa6-99fb-05b0aa236d06', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=8, environment_id=1, created_by_id=2, initiated_by_run_id=None, created_at=2025-12-14 22:40:14 UTC, is_locked=False)
View transforms and runs in LaminHub¶
View the database content¶
ln.view()
Show code cell output
Artifact
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | version | is_latest | is_locked | created_at | branch_id | space_id | storage_id | run_id | schema_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||||||
| 6 | pRGDPXmocDxRn9i80000 | data/results.tgz | None | .tgz | None | None | 94329 | YeL8oso2PkMtVb7XZavuOA | None | None | None | True | False | 2025-12-14 22:41:00.929000+00:00 | 1 | 1 | 2 | 1.0 | None | 2 |
| 5 | yLiXYVcaCGSfraVH0000 | fasta/KLF4.fasta | None | .fasta | None | None | 609 | LyuoYkWs4SgYcH7P7JLJtA | None | None | None | True | False | 2025-12-14 22:40:14.961000+00:00 | 1 | 1 | 2 | NaN | None | 2 |
| 4 | 7fxdlN7iGVFnem6p0000 | fasta/PO5F1.fasta | None | .fasta | None | None | 477 | -7iJgveFO9ia0wE1bqVu6g | None | None | None | True | False | 2025-12-14 22:40:14.960000+00:00 | 1 | 1 | 2 | NaN | None | 2 |
| 3 | aoLRlMdamLfUy0Dd0000 | fasta/MYC.fasta | None | .fasta | None | None | 536 | WGbEtzPw-3bQEGcngO_pHQ | None | None | None | True | False | 2025-12-14 22:40:14.959000+00:00 | 1 | 1 | 2 | NaN | None | 2 |
| 2 | h0ziqf8gWchXDEpy0000 | fasta/SOX2.fasta | None | .fasta | None | None | 414 | C5q_yaFXGk4SAEpfdqBwnQ | None | None | None | True | False | 2025-12-14 22:40:14.959000+00:00 | 1 | 1 | 2 | NaN | None | 2 |
Run
| uid | name | started_at | finished_at | params | reference | reference_type | is_locked | created_at | branch_id | space_id | transform_id | report_id | _logfile_id | environment_id | created_by_id | initiated_by_run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||
| 1 | 7k6cXUwqJ2g36lpN | None | 2025-12-14 22:40:14.224327+00:00 | 2025-12-14 22:41:00.956925+00:00 | {'input_dir': './fasta', 'amino_acid': 'C', 'e... | afc69eb3-1e39-4fa6-99fb-05b0aa236d06 | redun_id | False | 2025-12-14 22:40:14.225000+00:00 | 1 | 1 | 1 | 8 | None | 1 | 2 | None |
Storage
| uid | root | description | type | region | instance_uid | is_locked | created_at | branch_id | space_id | created_by_id | run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||
| 2 | BxyfPAhGGqCC | /home/runner/work/redun-lamin/redun-lamin/docs... | None | local | None | iQlBPgD8uaqR | False | 2025-12-14 22:39:56.944000+00:00 | 1 | 1 | 2 | None |
Transform
| uid | key | description | type | source_code | hash | reference | reference_type | config | is_flow | version | is_latest | is_locked | created_at | branch_id | space_id | flow_id | environment_id | created_by_id | _template_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||||||
| 1 | KIjVpH3l8kjg0000 | workflow.py | None | script | """workflow.py."""\n\n# This code is a copy fr... | Qn4m7TedTE04S_x3VFxq8w | None | None | None | False | None | True | False | 2025-12-14 22:40:14.222000+00:00 | 1 | 1 | None | None | 2 | None |
Delete the test instance:
Show code cell content
!rm -rf test-redun-lamin
!lamin delete --force test-redun-lamin