Redun¶

Here, we’ll see how to track redun workflow runs with LaminDB.

Note

This use case is based on github.com/ricomnl/bioinformatics-pipeline-tutorial.

# pip install lamindb redun git+http://github.com/laminlabs/redun-lamin-fasta
!lamin init --storage ./test-redun-lamin

Amend the workflow¶

import lamindb as ln
import json

Let’s amend a redun workflow.py to register input & output artifacts in LaminDB:

To track the workflow run in LaminDB, add (see on GitHub):
```
ln.track(params=params)
```
To register the output file via LaminDB, add (see on GitHub):
```
ln.Artifact(output_path, description="results").save()
```

Run redun¶

Let’s see what the input files are:

!ls ./fasta

And call the workflow:

!redun run workflow.py main --input-dir ./fasta --tag run=test-run  1> run_logs.txt 2>run_logs.txt

Inspect the logs:

!cat run_logs.txt

Show code cell output Hide code cell output

→ connected lamindb: testuser1/test-redun-lamin
→ created Transform('eKH08gq1CMVc0000', key='workflow.py'), started new Run('8VDlshi3n9mmNtvl') at 2025-10-16 11:48:08 UTC
→ params: input_dir=./fasta, amino_acid=C, enzyme_regex=[KR], missed_cleavages=0, min_length=4, max_length=75, executor=Executor.default
• recommendation: to identify the script across renames, pass the uid: ln.track("eKH08gq1CMVc", params={.[redun] Run    Job 9227170c:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/5xx2NvhSiLGx8V4p0000.fasta, hash=68cad206), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on defaultFile(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=2b667f90)
th=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/CTx25qSPmKoSd1rO0000.fasta, hash=b1cb3e4c), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job ccbe1477:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/fbTea3G2QX8s2Kon0000.fasta, hash=8298e5e6), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job cdafa232:  bioinformatics_pipeline_tutorial.lib.digest_protein_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/9eiojKWrZtKITwbq0000.fasta, hash=4d0eafff), enzyme_regex='[KR]', missed_cleavages=0, min_length=4, max_length=75) on default
[redun] Run    Job 2f19b37d:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/5xx2NvhSiLGx8V4p0000.fasta, hash=68cad206), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/5xx2NvhSiLGx8V4p0000.peptides.txt, hash=6532a76d), amino_acid='C') on default
[redun] Run    Job 13b4348a:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/CTx25qSPmKoSd1rO0000.fasta, hash=b1cb3e4c), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/CTx25qSPmKoSd1rO0000.peptides.txt, hash=6d3ab5e2), amino_acid='C') on default
[redun] Run    Job 6094c98c:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/fbTea3G2QX8s2Kon0000.fasta, hash=8298e5e6), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/fbTea3G2QX8s2Kon0000.peptides.txt, hash=85c2d801), amino_acid='C') on default
[redun] Run    Job f2062696:  bioinformatics_pipeline_tutorial.lib.count_amino_acids_task(input_fasta=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/.lamindb/9eiojKWrZtKITwbq0000.fasta, hash=4d0eafff), input_peptides=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/9eiojKWrZtKITwbq0000.peptides.txt, hash=00030270), amino_acid='C') on default
[redun] Run    Job 8c9c60c6:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/5xx2NvhSiLGx8V4p0000.count.tsv, hash=b3a4c594)) on default
[redun] Run    Job 2eca252b:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/CTx25qSPmKoSd1rO0000.count.tsv, hash=ed4702a9)) on default
[redun] Run    Job eb4ab6b9:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/fbTea3G2QX8s2Kon0000.count.tsv, hash=f558c3c2)) on default
[redun] Run    Job 54fd1288:  bioinformatics_pipeline_tutorial.lib.plot_count_task(input_count=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/9eiojKWrZtKITwbq0000.count.tsv, hash=dafdc3c0)) on default
[redun] Run    Job f129fe4c:  bioinformatics_pipeline_tutorial.lib.get_report_task(input_counts=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/5xx2NvhSiLGx8V4p0000.count.tsv, hash=b3a4c594), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-l...) on default
[redun] Run    Job e737da17:  bioinformatics_pipeline_tutorial.lib.archive_results_task(inputs_plots=[File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/5xx2NvhSiLGx8V4p0000.plot.png, hash=782904c2), File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-la..., input_report=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/protein_report.tsv, hash=deb262b4)) on default
[redun] Run    Job e28cb194:  redun_lamin_fasta.finish(results_archive=File(path=/home/runner/work/redun-lamin/redun-lamin/docs/test-redun-lamin/data/results.tgz, hash=2b667f90)) on default
[redun] 
[redun] | JOB STATUS 2025/10/16 11:48:14
[redun] | TASK                                                        PENDING RUNNING  FAILED  CACHED    DONE   TOTAL
[redun] | 
[redun] | ALL                                                               0       0       0       0      16      16
[redun] | bioinformatics_pipeline_tutorial.lib.archive_results_task         0       0       0       0       1       1
[redun] | bioinformatics_pipeline_tutorial.lib.count_amino_acids_task       0       0       0       0       4       4
[redun] | bioinformatics_pipeline_tutorial.lib.digest_protein_task          0       0       0       0       4       4
[redun] | bioinformatics_pipeline_tutorial.lib.get_report_task              0       0       0       0       1       1
[redun] | bioinformatics_pipeline_tutorial.lib.plot_count_task              0       0       0       0       4       4
[redun] | redun_lamin_fasta.finish                                          0       0       0       0       1       1
[redun] | redun_lamin_fasta.main                                            0       0       0       0       1       1
[redun] 
[redun] 
[redun] Execution duration: 5.49 seconds

View data lineage:

artifact = ln.Artifact.get(key="data/results.tgz")
artifact.view_lineage()

Track the redun execution id¶

If we want to be able to query LaminDB for redun execution ID, this here is a way to get it:

# export the run information from redun
!redun log --exec --exec-tag run=test-run --format json --no-pager > redun_exec.json
# load the redun execution id from the JSON and store it in the LaminDB run record
with open("redun_exec.json") as file:
    redun_exec = json.loads(file.readline())
artifact.run.reference = redun_exec["id"]
artifact.run.reference_type = "redun_id"
artifact.run.save()

Run(uid='8VDlshi3n9mmNtvl', started_at=2025-10-16 11:48:08 UTC, finished_at=2025-10-16 11:48:14 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='c29d622a-5292-4138-9eef-af5a7f50f6ad', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=7, environment_id=1, created_by_id=1, created_at=2025-10-16 11:48:08 UTC, is_locked=False)

Track the redun run report¶

While lamindb tracks logs of the main python process, the redun logs currently don’t get tracked and artifact.run.report is an empty text file.

This is how you can manually attach a report.

report = ln.Artifact(
    "run_logs.txt",
    description=f"Redun run report of {redun_exec['id']}",
    run=False,
    kind="__lamindb_run__",  # mark as auxiliary artifact for the run
).save()
artifact.run.report = report
artifact.run.save()

Run(uid='8VDlshi3n9mmNtvl', started_at=2025-10-16 11:48:08 UTC, finished_at=2025-10-16 11:48:14 UTC, params={'input_dir': './fasta', 'amino_acid': 'C', 'enzyme_regex': '[KR]', 'missed_cleavages': 0, 'min_length': 4, 'max_length': 75, 'executor': 'default'}, reference='c29d622a-5292-4138-9eef-af5a7f50f6ad', reference_type='redun_id', branch_id=1, space_id=1, transform_id=1, report_id=8, environment_id=1, created_by_id=1, created_at=2025-10-16 11:48:08 UTC, is_locked=False)

View transforms and runs in LaminHub¶

View the database content¶

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_overwrite_versions	space_id	storage_id	schema_id	version	is_latest	is_locked	run_id	created_at	created_by_id	_aux	_real_key	branch_id
id
6	mKthUm69iWbtk7UX0000	data/results.tgz	None	.tgz	None	None	94324	vrlR5ARsmtj4Ck8Ro-yu5Q	None	None	md5	False	False	1	1	None	None	True	False	1.0	2025-10-16 11:48:14.087000+00:00	1	None	None	1
5	9eiojKWrZtKITwbq0000	fasta/KLF4.fasta	None	.fasta	None	None	609	LyuoYkWs4SgYcH7P7JLJtA	None	None	md5	True	False	1	1	None	None	True	False	NaN	2025-10-16 11:48:09.671000+00:00	1	{'af': {'0': True}}	None	1
3	CTx25qSPmKoSd1rO0000	fasta/MYC.fasta	None	.fasta	None	None	536	WGbEtzPw-3bQEGcngO_pHQ	None	None	md5	True	False	1	1	None	None	True	False	NaN	2025-10-16 11:48:09.670000+00:00	1	{'af': {'0': True}}	None	1
4	fbTea3G2QX8s2Kon0000	fasta/PO5F1.fasta	None	.fasta	None	None	477	-7iJgveFO9ia0wE1bqVu6g	None	None	md5	True	False	1	1	None	None	True	False	NaN	2025-10-16 11:48:09.670000+00:00	1	{'af': {'0': True}}	None	1
2	5xx2NvhSiLGx8V4p0000	fasta/SOX2.fasta	None	.fasta	None	None	414	C5q_yaFXGk4SAEpfdqBwnQ	None	None	md5	True	False	1	1	None	None	True	False	NaN	2025-10-16 11:48:09.669000+00:00	1	{'af': {'0': True}}	None	1

Run

	uid	name	started_at	finished_at	params	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	is_locked	created_at	created_by_id	_aux	branch_id
id
1	8VDlshi3n9mmNtvl	None	2025-10-16 11:48:08.856678+00:00	2025-10-16 11:48:14.115129+00:00	{'input_dir': './fasta', 'amino_acid': 'C', 'e...	c29d622a-5292-4138-9eef-af5a7f50f6ad	redun_id	True	0	1	1	8	None	1	None	False	2025-10-16 11:48:08.857000+00:00	1	None	1

Storage

	uid	root	description	type	region	instance_uid	space_id	is_locked	run_id	created_at	created_by_id	_aux	branch_id
id
1	QevNNRQx3hLn	/home/runner/work/redun-lamin/redun-lamin/docs...	None	local	None	iQlBPgD8uaqR	1	False	None	2025-10-16 11:47:54.191000+00:00	1	None	1

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	is_locked	created_at	created_by_id	_aux	branch_id
id
1	eKH08gq1CMVc0000	workflow.py	None	script	"""workflow.py."""\n\n# This code is a copy fr...	Qn4m7TedTE04S_x3VFxq8w	None	None	1	None	None	True	False	2025-10-16 11:48:08.854000+00:00	1	None	1

Delete the test instance: