##### Will data & metadata stay in sync? [image: .md][image]

Here, we walk through different errors that can occur while saving
artifacts & metadata records, and show that the LaminDB instance does
not get corrupted by dangling metadata or artifacts.

Transactions within Python across data & metadata are ACID.

If an upload process is externally killed and Python cannot run clean-
up operations anymore, the artifact is internally still flagged with
"artifact._storage_ongoing = True". This is visible on the UI. You can
then re-run "lamin save" or "artifact.save()" to attempt uploading the
artifact a second time.

 !lamin init --storage ./test-acid

 import pytest
 import lamindb as ln
 from upath import UPath

 ln.settings.verbosity = "debug"

 open("sample.fasta", "w").write(">seq1\nACGT\n")

#### Save error due to failed upload within Python

Let's try to save an artifact to a storage location without
permission.

 artifact = ln.Artifact("sample.fasta", key="sample.fasta")

Because the public API only allows you to set a default storage for
which you have permission, we need to hack it:

 ln.settings.storage._root = UPath("s3://nf-core-awsmegatests")

This raises an exception but nothing gets saved:

 with pytest.raises(PermissionError) as error:
 artifact.save()
 print(error.exconly())
 assert len(ln.Artifact.filter()) == 0

#### Save error during bulk creation

 artifacts = [artifact, "this is not a record"]

This raises an exception but nothing gets saved:

 with pytest.raises(Exception) as error:
 ln.save(artifacts)
 print(error.exconly())
 assert len(ln.Artifact.filter()) == 0  # nothing got saved

If a list of data objects is passed to "ln.save()" and the upload of
one of these data objects fails, the successful uploads are maintained
and a "RuntimeError" is raised, listing the successfully uploaded data
objects up until that point.

#### Save error due to externally aborted upload

Back to a proper storage location:

 ln.settings.storage._root = UPath("./test-acid").absolute()

The save operation works:

 artifact.save()

Let's pretend the upload was killed.

 artifact._storage_ongoing = True
 artifact.save()
 artifact.path.unlink()
 assert artifact._aux == {"so": 1}  # storage/upload is ongoing

We can re-run it:

 artifact = ln.Artifact("sample.fasta", key="sample.fasta").save()

 assert not artifact._storage_ongoing
 assert artifact._aux is None

 !rm -r ./test-acid
 !lamin delete --force test-acid