Redun¶

Here, we’ll see how to track redun workflow runs with LaminDB.

Note

This use case is based on github.com/ricomnl/bioinformatics-pipeline-tutorial.

# pip install lamindb redun git+http://github.com/laminlabs/redun-lamin-fasta
!lamin init --storage ./test-redun-lamin

Amend the workflow¶

import lamindb as ln
import json

Let’s amend a redun workflow.py to register input & output artifacts in LaminDB:

To track the workflow run in LaminDB, add (see on GitHub):
```
ln.track(params=params)
```
To register the output file via LaminDB, add (see on GitHub):
```
ln.Artifact(output_path, description="results").save()
```

Run redun¶

Let’s see what the input files are:

!ls ./fasta

And call the workflow:

!redun run workflow.py main --input-dir ./fasta --tag run=test-run  1> run_logs.txt 2>run_logs.txt

Inspect the logs:

!cat run_logs.txt

Show code cell output Hide code cell output

→ connected lamindb: testuser1/test-redun-lamin
→ created Transform('1UuEEqCbvEHH0000', key='workflow.py'), started new Run('KdTqc2nwfW796oac') at 2025-11-05 21:32:41 UTC
→ params: input_dir='./fasta', amino_acid='C', enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75, executor=<Executor.default: 'default'>
• recommendation: to identify the script across renames, pass the uid: ln.track("1UuEE[redun] Run    Job b1b68409:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/OI5N3FH7J2nUeZit0000.fasta, hash=b30ea24e), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    JobFile(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=20b7788c)
k/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/AtoSSEKLA2zBec2B0000.fasta, hash=6bfa5600), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job a26711df:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/aS5mZPJS69mewxG50000.fasta, hash=8a3ca39c), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job 8ff2e578:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/G22A4tGPyqyOKpLO0000.fasta, hash=e962305d), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job 2a1cbc3c:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/OI5N3FH7J2nUeZit0000.fasta, hash=b30ea24e), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/OI5N3FH7J2nUeZit0000.peptides.txt, hash=b6d85c4f), amino_acid='C') on default
[redun] Run    Job 7d84180a:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/AtoSSEKLA2zBec2B0000.fasta, hash=6bfa5600), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/AtoSSEKLA2zBec2B0000.peptides.txt, hash=2539213f), amino_acid='C') on default
[redun] Run    Job dbf8b7a0:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/aS5mZPJS69mewxG50000.fasta, hash=8a3ca39c), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/aS5mZPJS69mewxG50000.peptides.txt, hash=d791aa19), amino_acid='C') on default
[redun] Run    Job 43aafd39:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/G22A4tGPyqyOKpLO0000.fasta, hash=e962305d), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/G22A4tGPyqyOKpLO0000.peptides.txt, hash=1920391c), amino_acid='C') on default
[redun] Run    Job a601488a:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/OI5N3FH7J2nUeZit0000.count.tsv, hash=538524a3)) on default
[redun] Run    Job 063ee9a3:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/AtoSSEKLA2zBec2B0000.count.tsv, hash=a0743bb9)) on default
[redun] Run    Job b1f4c940:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/aS5mZPJS69mewxG50000.count.tsv, hash=50e8da56)) on default
[redun] Run    Job 4452ace3:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/G22A4tGPyqyOKpLO0000.count.tsv, hash=60b4f3ea)) on default
[redun] Run    Job 25dbf5ce:  bioinformatics_pipeline_tutorial.lib.get_report_task(input_counts=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/OI5N3FH7J2nUeZit0000.count.tsv, hash=538524a3), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-l...) on default
[redun] Run    Job bb0df457:  bioinformatics_pipeline_tutorial.lib.archive_results_task(inputs_plots=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/OI5N3FH7J2nUeZit0000.plot.png, hash=2fae6b59), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-la..., input_report=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/protein_report.tsv, hash=ae24675a)) on default
[redun] Run    Job 6a0045cb:  redun_lamin_fasta.finish(results_archive=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=20b7788c)) on default
[redun] 
[redun] | JOB STATUS 2025/11/05 21:32:46
[redun] | TASK                                                        PENDING RUNNING  FAILED  CACHED    DONE   TOTAL
[redun] | 
[redun] | ALL                                                               0       0       0       0      16      16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task         0       0       0       0       1       1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task       0       0       0       0       4       4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task          0       0       0       0       4       4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task              0       0       0       0       1       1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task              0       0       0       0       4       4
[redun] | redun_lamin_fasta.finish                                          0       0       0       0       1       1
[redun] | redun_lamin_fasta.main                                            0       0       0       0       1       1
[redun] 
[redun] 
[redun] Execution duration: 5.37 seconds

View data lineage:

artifact = ln.Artifact.get(key="data/results.tgz")
artifact.view_lineage()

Track the redun execution id¶

If we want to be able to query LaminDB for redun execution ID, this here is a way to get it:

# export the run information from redun
!redun log --exec --exec-tag run=test-run --format json --no-pager > redun_exec.json
# load the redun execution id from the JSON and store it in the LaminDB run record
with open("redun_exec.json") as file:
    redun_exec = json.loads(file.readline())
artifact.run.reference = redun_exec["id"]
artifact.run.reference_type = "redun_id"
artifact.run.save()

Run(uid='KdTqc2nwfW796oac', name=None, started_at=2025-11-05 21:32:41 UTC, finished_at=2025-11-05 21:32:46 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='6331f7e9-ccba-4e57-8616-097f2ba2fe6b', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=7, environment_id=1, created_by_id=1, initiated_by_run_id=None, created_at=2025-11-05 21:32:41 UTC, is_locked=False)

Track the redun run report¶

While lamindb tracks logs of the main python process, the redun logs currently don’t get tracked and artifact.run.report is an empty text file.

This is how you can manually attach a report.

report = ln.Artifact(
    "run_logs.txt",
    description=f"Redun run report of {redun_exec['id']}",
    run=False,
    kind="__lamindb_run__",  # mark as auxiliary artifact for the run
).save()
artifact.run.report = report
artifact.run.save()

Run(uid='KdTqc2nwfW796oac', name=None, started_at=2025-11-05 21:32:41 UTC, finished_at=2025-11-05 21:32:46 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='6331f7e9-ccba-4e57-8616-097f2ba2fe6b', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=8, environment_id=1, created_by_id=1, initiated_by_run_id=None, created_at=2025-11-05 21:32:41 UTC, is_locked=False)

View transforms and runs in LaminHub¶

View the database content¶

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	version	is_latest	is_locked	created_at	branch_id	space_id	storage_id	run_id	schema_id	created_by_id
id
6	1Fz8DHMZqxPixkTo0000	data/results.tgz	None	.tgz	None	None	94331	805vtJ_ibQHwEPZk9EnNeg	None	None	None	True	False	2025-11-05 21:32:46.139000+00:00	1	1	1	1.0	None	1
5	G22A4tGPyqyOKpLO0000	fasta/KLF4.fasta	None	.fasta	None	None	609	LyuoYkWs4SgYcH7P7JLJtA	None	None	None	True	False	2025-11-05 21:32:41.905000+00:00	1	1	1	NaN	None	1
4	aS5mZPJS69mewxG50000	fasta/PO5F1.fasta	None	.fasta	None	None	477	-7iJgveFO9ia0wE1bqVu6g	None	None	None	True	False	2025-11-05 21:32:41.904000+00:00	1	1	1	NaN	None	1
3	AtoSSEKLA2zBec2B0000	fasta/MYC.fasta	None	.fasta	None	None	536	WGbEtzPw-3bQEGcngO_pHQ	None	None	None	True	False	2025-11-05 21:32:41.904000+00:00	1	1	1	NaN	None	1
2	OI5N3FH7J2nUeZit0000	fasta/SOX2.fasta	None	.fasta	None	None	414	C5q_yaFXGk4SAEpfdqBwnQ	None	None	None	True	False	2025-11-05 21:32:41.903000+00:00	1	1	1	NaN	None	1

Run

	uid	name	started_at	finished_at	params	reference	reference_type	is_locked	created_at	branch_id	space_id	transform_id	report_id	_logfile_id	environment_id	created_by_id	initiated_by_run_id
id
1	KdTqc2nwfW796oac	None	2025-11-05 21:32:41.018733+00:00	2025-11-05 21:32:46.164853+00:00	{'input_dir': './fasta', 'amino_acid': 'C', 'e...	6331f7e9-ccba-4e57-8616-097f2ba2fe6b	redun_id	False	2025-11-05 21:32:41.019000+00:00	1	1	1	8	None	1	1	None

Storage

	uid	root	description	type	region	instance_uid	is_locked	created_at	branch_id	space_id	created_by_id	run_id
id
1	ecWFpEwqihb0	/home/runner/work/redun-lamin/redun-lamin/docs...	None	local	None	iQlBPgD8uaqR	False	2025-11-05 21:32:25.685000+00:00	1	1	1	None

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	version	is_latest	is_locked	created_at	branch_id	space_id	created_by_id	_template_id
id
1	1UuEEqCbvEHH0000	workflow.py	None	script	"""workflow.py."""\n\n# This code is a copy fr...	Qn4m7TedTE04S_x3VFxq8w	None	None	None	True	False	2025-11-05 21:32:41.016000+00:00	1	1	1	None

Delete the test instance: