Redun¶
Here, we’ll see how to track redun workflow runs with LaminDB.
Note
This use case is based on github.com/ricomnl/bioinformatics-pipeline-tutorial.
# pip install lamindb redun git+http://github.com/laminlabs/redun-lamin-fasta
!lamin init --storage ./test-redun-lamin
Show code cell output
→ initialized lamindb: testuser1/test-redun-lamin
Amend the workflow¶
import lamindb as ln
import json
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
Let’s amend a redun workflow.py to register input & output artifacts in LaminDB:
To track the workflow run in LaminDB, add (see on GitHub):
ln.track(params=params)
To register the output file via LaminDB, add (see on GitHub):
ln.Artifact(output_path, description="results").save()
Run redun¶
Let’s see what the input files are:
!ls ./fasta
Show code cell output
KLF4.fasta MYC.fasta PO5F1.fasta SOX2.fasta
And call the workflow:
!redun run workflow.py main --input-dir ./fasta --tag run=test-run 1> run_logs.txt 2>run_logs.txt
Inspect the logs:
!cat run_logs.txt
Show code cell output
→ connected lamindb: testuser1/test-redun-lamin
→ script invoked with: run workflow.py main --input-dir ./fasta --tag run=test-run
→ created Transform('55NcgaxPLTwH0000', key='workflow.py'), started new Run('nla4VvQyBkuUlV9f') at 2025-11-14 00:10:25 UTC
→ params: input_dir='./fasta', amino_acid='C', enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75, executor=<Executor.default: 'default'>
9[redun] Run Job bfc4e704: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/kDRcbjcdngpI02ed0000.fasta, hash=19f4840e), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 2d5d4f7d: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/hoFile(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=74d00ab1)
8dc3), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job c4bdf322: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/3jqUPHtTNsOJJj2I0000.fasta, hash=f13e9aa0), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 93bb99c3: bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/Dk1nYU13GkySsV1M0000.fasta, hash=866fb0c2), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run Job 7a6ffcca: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/kDRcbjcdngpI02ed0000.fasta, hash=19f4840e), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/kDRcbjcdngpI02ed0000.peptides.txt, hash=71b088dc), amino_acid='C') on default
[redun] Run Job e22937bb: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/mLD8ByZQxgbWfaLQ0000.fasta, hash=92bb8dc3), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/mLD8ByZQxgbWfaLQ0000.peptides.txt, hash=d1147e76), amino_acid='C') on default
[redun] Run Job 0d6dc70d: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/3jqUPHtTNsOJJj2I0000.fasta, hash=f13e9aa0), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/3jqUPHtTNsOJJj2I0000.peptides.txt, hash=f6db06cf), amino_acid='C') on default
[redun] Run Job 8a1f1989: bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/Dk1nYU13GkySsV1M0000.fasta, hash=866fb0c2), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/Dk1nYU13GkySsV1M0000.peptides.txt, hash=e50ced8c), amino_acid='C') on default
[redun] Run Job eb7c321b: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/kDRcbjcdngpI02ed0000.count.tsv, hash=48ced0d7)) on default
[redun] Run Job d3dcc934: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/mLD8ByZQxgbWfaLQ0000.count.tsv, hash=e8662c54)) on default
[redun] Run Job 17ec5339: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/3jqUPHtTNsOJJj2I0000.count.tsv, hash=b7f6bcac)) on default
[redun] Run Job b17c3b9f: bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/Dk1nYU13GkySsV1M0000.count.tsv, hash=a4376ba6)) on default
[redun] Run Job 8f057424: bioinformatics_pipeline_tutorial.lib.get_report_task(input_counts=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/kDRcbjcdngpI02ed0000.count.tsv, hash=48ced0d7), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-l...) on default
[redun]
[redun] | JOB STATUS 2025/11/14 00:10:46
[redun] | TASK PENDING RUNNING FAILED CACHED DONE TOTAL
[redun] |
[redun] | ALL 2 5 0 0 9 16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task 1 0 0 0 0 1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0 4 0 0 0 4
[redun] | redun_lamin_fasta.finish 1 0 0 0 0 1
[redun] | redun_lamin_fasta.main 0 1 0 0 0 1
[redun]
[redun]
[redun] Run Job 55a44b97: bioinformatics_pipeline_tutorial.lib.archive_results_task(inputs_plots=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/kDRcbjcdngpI02ed0000.plot.png, hash=e9c0db68), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-la..., input_report=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/protein_report.tsv, hash=abb375e0)) on default
[redun] Run Job caf80d5d: redun_lamin_fasta.finish(results_archive=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=74d00ab1)) on default
[redun]
[redun] | JOB STATUS 2025/11/14 00:10:46
[redun] | TASK PENDING RUNNING FAILED CACHED DONE TOTAL
[redun] |
[redun] | ALL 0 0 0 0 16 16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task 0 0 0 0 4 4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task 0 0 0 0 1 1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task 0 0 0 0 4 4
[redun] | redun_lamin_fasta.finish 0 0 0 0 1 1
[redun] | redun_lamin_fasta.main 0 0 0 0 1 1
[redun]
[redun]
[redun] Execution duration: 21.94 seconds
View data lineage:
artifact = ln.Artifact.get(key="data/results.tgz")
artifact.view_lineage()
Show code cell output
Track the redun execution id¶
If we want to be able to query LaminDB for redun execution ID, this here is a way to get it:
# export the run information from redun
!redun log --exec --exec-tag run=test-run --format json --no-pager > redun_exec.json
# load the redun execution id from the JSON and store it in the LaminDB run record
with open("redun_exec.json") as file:
redun_exec = json.loads(file.readline())
artifact.run.reference = redun_exec["id"]
artifact.run.reference_type = "redun_id"
artifact.run.save()
Run(uid='nla4VvQyBkuUlV9f', name=None, started_at=2025-11-14 00:10:25 UTC, finished_at=2025-11-14 00:10:46 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='fda8ef61-468f-49a9-9d96-1d64e131af2d', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=7, environment_id=1, created_by_id=1, initiated_by_run_id=None, created_at=2025-11-14 00:10:25 UTC, is_locked=False)
Track the redun run report¶
While lamindb tracks logs of the main python process, the redun logs currently don’t get tracked and artifact.run.report is an empty text file.
This is how you can manually attach a report.
report = ln.Artifact(
"run_logs.txt",
description=f"Redun run report of {redun_exec['id']}",
run=False,
kind="__lamindb_run__", # mark as auxiliary artifact for the run
).save()
artifact.run.report = report
artifact.run.save()
Run(uid='nla4VvQyBkuUlV9f', name=None, started_at=2025-11-14 00:10:25 UTC, finished_at=2025-11-14 00:10:46 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='fda8ef61-468f-49a9-9d96-1d64e131af2d', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=8, environment_id=1, created_by_id=1, initiated_by_run_id=None, created_at=2025-11-14 00:10:25 UTC, is_locked=False)
View transforms and runs in LaminHub¶
View the database content¶
ln.view()
Show code cell output
Artifact
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | version | is_latest | is_locked | created_at | branch_id | space_id | storage_id | run_id | schema_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||||||
| 6 | nbi06jILYouH2ILG0000 | data/results.tgz | None | .tgz | None | None | 94327 | 9-77vtrI4hUDj2PCq6B1iw | None | None | None | True | False | 2025-11-14 00:10:46.820000+00:00 | 1 | 1 | 1 | 1.0 | None | 1 |
| 5 | Dk1nYU13GkySsV1M0000 | fasta/KLF4.fasta | None | .fasta | None | None | 609 | LyuoYkWs4SgYcH7P7JLJtA | None | None | None | True | False | 2025-11-14 00:10:25.884000+00:00 | 1 | 1 | 1 | NaN | None | 1 |
| 4 | 3jqUPHtTNsOJJj2I0000 | fasta/PO5F1.fasta | None | .fasta | None | None | 477 | -7iJgveFO9ia0wE1bqVu6g | None | None | None | True | False | 2025-11-14 00:10:25.884000+00:00 | 1 | 1 | 1 | NaN | None | 1 |
| 3 | mLD8ByZQxgbWfaLQ0000 | fasta/MYC.fasta | None | .fasta | None | None | 536 | WGbEtzPw-3bQEGcngO_pHQ | None | None | None | True | False | 2025-11-14 00:10:25.883000+00:00 | 1 | 1 | 1 | NaN | None | 1 |
| 2 | kDRcbjcdngpI02ed0000 | fasta/SOX2.fasta | None | .fasta | None | None | 414 | C5q_yaFXGk4SAEpfdqBwnQ | None | None | None | True | False | 2025-11-14 00:10:25.883000+00:00 | 1 | 1 | 1 | NaN | None | 1 |
Run
| uid | name | started_at | finished_at | params | reference | reference_type | is_locked | created_at | branch_id | space_id | transform_id | report_id | _logfile_id | environment_id | created_by_id | initiated_by_run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||
| 1 | nla4VvQyBkuUlV9f | None | 2025-11-14 00:10:25.089062+00:00 | 2025-11-14 00:10:46.846144+00:00 | {'input_dir': './fasta', 'amino_acid': 'C', 'e... | fda8ef61-468f-49a9-9d96-1d64e131af2d | redun_id | False | 2025-11-14 00:10:25.089000+00:00 | 1 | 1 | 1 | 8 | None | 1 | 1 | None |
Storage
| uid | root | description | type | region | instance_uid | is_locked | created_at | branch_id | space_id | created_by_id | run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||
| 1 | wZ41vYDNsrei | /home/runner/work/redun-lamin/redun-lamin/docs... | None | local | None | iQlBPgD8uaqR | False | 2025-11-14 00:10:11.673000+00:00 | 1 | 1 | 1 | None |
Transform
| uid | key | description | type | source_code | hash | reference | reference_type | version | is_latest | is_locked | created_at | branch_id | space_id | created_by_id | _template_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||
| 1 | 55NcgaxPLTwH0000 | workflow.py | None | script | """workflow.py."""\n\n# This code is a copy fr... | Qn4m7TedTE04S_x3VFxq8w | None | None | None | True | False | 2025-11-14 00:10:25.087000+00:00 | 1 | 1 | 1 | None |
Delete the test instance:
Show code cell content
!rm -rf test-redun-lamin
!lamin delete --force test-redun-lamin
• deleting instance testuser1/test-redun-lamin