Will data & metadata stay in sync?¶

Here, we walk through different errors that can occur while saving artifacts & metadata records, and show that the LaminDB instance does not get corrupted by dangling metadata or artifacts.

Transactions within Python across data & metadata are ACID.

If an upload process is externally killed and Python cannot run clean-up operations anymore, the artifact is internally still flagged with artifact._storage_ongoing = True. This is visible on the UI. You can then re-run lamin save or artifact.save() to attempt uploading the artifact a second time.

!lamin init --storage ./test-acid

→ initialized lamindb: testuser1/test-acid

import pytest
import lamindb as ln
from upath import UPath

ln.settings.verbosity = "debug"

→ connected lamindb: testuser1/test-acid

open("sample.fasta", "w").write(">seq1\nACGT\n")

Save error due to failed upload within Python¶

Let’s try to save an artifact to a storage location without permission.

artifact = ln.Artifact("sample.fasta", key="sample.fasta")

! no run & transform got linked, call `ln.track()` & re-run

• path content will be copied to default storage upon `save()` with key 'sample.fasta'

Because the public API only allows you to set a default storage for which you have permission, we need to hack it:

ln.settings.storage._root = UPath("s3://nf-core-awsmegatests")

This raises an exception but nothing gets saved:

with pytest.raises(PermissionError) as error:
    artifact.save()
print(error.exconly())
assert len(ln.Artifact.filter()) == 0

✓ storing artifact 'Cvgd2kkiNA8Q3EEe0000' at 's3://nf-core-awsmegatests/.lamindb/Cvgd2kkiNA8Q3EEe0000.fasta'

! could not upload artifact: Artifact(uid='Cvgd2kkiNA8Q3EEe0000', version_tag=None, is_latest=True, key='sample.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=11, hash='83rEPcAoBHmYiIuyBYrFKg', n_files=None, n_observations=None, branch_id=1, space_id=1, storage_id=3, run_id=None, schema_id=None, created_by_id=3, created_at=2026-01-21 14:13:41 UTC, is_locked=False)

PermissionError: Access Denied

Save error during bulk creation¶

artifacts = [artifact, "this is not a record"]

This raises an exception but nothing gets saved:

with pytest.raises(Exception) as error:
    ln.save(artifacts)
print(error.exconly())
assert len(ln.Artifact.filter()) == 0  # nothing got saved

AttributeError: 'str' object has no attribute '_state'

If a list of data objects is passed to ln.save() and the upload of one of these data objects fails, the successful uploads are maintained and a RuntimeError is raised, listing the successfully uploaded data objects up until that point.

Save error due to externally aborted upload¶

Back to a proper storage location:

ln.settings.storage._root = UPath("./test-acid").absolute()

The save operation works:

artifact.save()

✓ storing artifact 'Cvgd2kkiNA8Q3EEe0000' at '/home/runner/work/lamindb/lamindb/docs/faq/test-acid/.lamindb/Cvgd2kkiNA8Q3EEe0000.fasta'

Artifact(uid='Cvgd2kkiNA8Q3EEe0000', version_tag=None, is_latest=True, key='sample.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=11, hash='83rEPcAoBHmYiIuyBYrFKg', n_files=None, n_observations=None, branch_id=1, space_id=1, storage_id=3, run_id=None, schema_id=None, created_by_id=3, created_at=2026-01-21 14:13:41 UTC, is_locked=False)

Let’s pretend the upload was killed.

artifact._storage_ongoing = True
artifact.save()
artifact.path.unlink()
assert artifact._aux == {"so": 1}  # storage/upload is ongoing

We can re-run it:

artifact = ln.Artifact("sample.fasta", key="sample.fasta").save()

! no run & transform got linked, call `ln.track()` & re-run

→ returning artifact with same hash: Artifact(uid='Cvgd2kkiNA8Q3EEe0000', version_tag=None, is_latest=True, key='sample.fasta', description=None, suffix='.fasta', kind=None, otype=None, size=11, hash='83rEPcAoBHmYiIuyBYrFKg', n_files=None, n_observations=None, branch_id=1, space_id=1, storage_id=3, run_id=None, schema_id=None, created_by_id=3, created_at=2026-01-21 14:13:41 UTC, is_locked=False); to track this artifact as an input, use: ln.Artifact.get()

✓ storing artifact 'Cvgd2kkiNA8Q3EEe0000' at '/home/runner/work/lamindb/lamindb/docs/faq/test-acid/.lamindb/Cvgd2kkiNA8Q3EEe0000.fasta'

assert not artifact._storage_ongoing
assert artifact._aux is None