Keep artifacts local in a cloud instance¶

If you want to default to keeping artifacts local in a cloud instance, enable keep_artifacts_local.

Let us first create a cloud instance that woul store artifacts exclusively on S3.

!lamin login testuser1
!lamin init --storage s3://lamindb-ci/keep-artifacts-local

Let’s import lamindb and track the current notebook run.

# !pip install 'lamindb[jupyter]'
import lamindb as ln

ln.track("l9lFf83aPwRc0000")

Toggling setting “keep artifacts local”¶

You checkmark the “Keep artifacts local” box on the instance settings tab.

For the current test instance, we toggle it through the following private attribute.

ln.setup.settings.instance._keep_artifacts_local = True

Create a local storage location¶

Call the following for a – potentially pre-existing – root path and a unique host identifier.

ln.Storage(root="./our_local_storage", host="abc-institute-drive1").save()

Now, you have two storage locations: one in the S3 bucket, and the other locally.

ln.Storage.df()

Show code cell output Hide code cell output

	uid	root	description	type	region	instance_uid	space_id	run_id	created_at	created_by_id	_aux	branch_id
id
2	Ua5xIK2qsS15	/home/runner/work/lamindb/lamindb/docs/faq/our...	None	local	abc-institute-drive1	6uGWmLpZlNoJ	1	1.0	2025-07-14 06:42:57.460000+00:00	1	None	1
1	IO79fB989dSX	s3://lamindb-ci/keep-artifacts-local	None	s3	us-west-1	6uGWmLpZlNoJ	1	NaN	2025-07-14 06:42:49.389000+00:00	1	None	1

Use a local storage location¶

If you save an artifact in keep-artifacts-local mode, by default, it’s stored in local storage.

original_filepath = ln.core.datasets.file_fcs()
artifact = ln.Artifact(original_filepath, key="example_datasets/file1.fcs").save()
local_path = artifact.path  # local storage path
local_path

You’ll see the .fcs file named by the uid in your .lamindb/ directory under ./our_local_storage/:

assert artifact.path.exists()
assert artifact.path.as_posix().startswith(ln.settings.local_storage.root.as_posix())
ln.settings.local_storage.root.view_tree()

Pre-existing artifacts¶

Assume you already have a file in your local storage location:

file_in_local_storage = ln.core.datasets.file_bam()
file_in_local_storage.rename("./our_local_storage/output.bam")
ln.UPath("our_local_storage/").view_tree()

When registering an artifact for it, it remains where it is.

my_existing_file = ln.Artifact("./our_local_storage/output.bam").save()
ln.UPath("our_local_storage/").view_tree()

The storage path of the artifact matches the pre-existing file:

my_existing_file.path

Switching between local storage locations¶

You might have several local storage locations. Here is how you can switch between them.

ln.Storage(root="./our_local_storage2", host="abc-institute-drive1").save()
ln.settings.local_storage = "./our_local_storage2"  # switch to the new storage location

Ingest a file into the new local storage location.

filepath = ln.core.datasets.file_fastq()
artifact3 = ln.Artifact(filepath, key="example_datasets/file.fastq.gz").save()

Inspect where all the files are.

ln.Artifact.df(include=["storage__root", "storage__region"])

Show code cell output Hide code cell output

	uid	key	storage__root	storage__region
id
3	dfzOjFBQ8fvA5Hrg0000	example_datasets/file.fastq.gz	/home/runner/work/lamindb/lamindb/docs/faq/our...	abc-institute-drive1
2	Uqqa0uwRt824iynX0000	output.bam	/home/runner/work/lamindb/lamindb/docs/faq/our...	abc-institute-drive1
1	GBKjRwNaekCOmlNL0000	example_datasets/file1.fcs	/home/runner/work/lamindb/lamindb/docs/faq/our...	abc-institute-drive1

Upload a local artifact to the cloud¶

If you’d like to upload an artifact to the cloud storage location to more easily share it or view it through web applications, you pass upload=True to the save() method.

artifact.save(upload=True)

→ moved local artifact to cache: /home/runner/.cache/lamindb/GBKjRwNaekCOmlNL0000.fcs

Artifact(uid='GBKjRwNaekCOmlNL0000', is_latest=True, key='example_datasets/file1.fcs', suffix='.fcs', size=19330507, hash='rCPvmZB19xs4zHZ7p_-Wrg', branch_id=1, space_id=1, storage_id=1, run_id=1, created_by_id=1, created_at=2025-07-14 06:42:58 UTC)

You now see the artifact in the S3 bucket:

ln.settings.storage.root.view_tree()

2 sub-directories & 3 files with suffixes '.db', '.txt', '.fcs'
s3://lamindb-ci/keep-artifacts-local
└── .lamindb/
    ├── GBKjRwNaekCOmlNL0000.fcs
    ├── lamin.db
    ├── storage_uid.txt
    └── _exclusion/

And it’s no longer present in local storage:

assert artifact.path.exists()
assert not local_path.exists()
assert artifact.path.as_posix().startswith(ln.settings.storage.root.as_posix())
ln.settings.local_storage.root.view_tree()

1 sub-directory & 2 files with suffixes '.fastq.gz', '.txt'
/home/runner/work/lamindb/lamindb/docs/faq/our_local_storage2
└── .lamindb/
    ├── dfzOjFBQ8fvA5Hrg0000.fastq.gz
    └── storage_uid.txt

Upload directly to the cloud¶

You can also directly upload via upload=True:

filepath = ln.core.datasets.file_mini_csv()
artifact2 = ln.Artifact(filepath, key="example_datasets/mini.csv").save(upload=True)
artifact2.path

S3QueryPath('s3://lamindb-ci/keep-artifacts-local/.lamindb/2DxW9bAH4AUvFCRE0000.csv')

Now we have two files on S3:

ln.Artifact.df(include="storage__root")

	uid	key	storage__root
id
4	2DxW9bAH4AUvFCRE0000	example_datasets/mini.csv	s3://lamindb-ci/keep-artifacts-local
3	dfzOjFBQ8fvA5Hrg0000	example_datasets/file.fastq.gz	/home/runner/work/lamindb/lamindb/docs/faq/our...
2	Uqqa0uwRt824iynX0000	output.bam	/home/runner/work/lamindb/lamindb/docs/faq/our...
1	GBKjRwNaekCOmlNL0000	example_datasets/file1.fcs	s3://lamindb-ci/keep-artifacts-local

Update storage description¶

You can add a description to the storage location by using the description field.

storage_record = ln.Storage.get(root__endswith="our_local_storage")
storage_record.description = "Our shared directory for project X"
storage_record.save()
ln.Storage.df()

	uid	root	description	type	region	instance_uid	space_id	run_id	created_at	created_by_id	_aux	branch_id
id
3	Bl99S684Ge0l	/home/runner/work/lamindb/lamindb/docs/faq/our...	None	local	abc-institute-drive1	6uGWmLpZlNoJ	1	1.0	2025-07-14 06:42:59.187000+00:00	1	None	1
2	Ua5xIK2qsS15	/home/runner/work/lamindb/lamindb/docs/faq/our...	Our shared directory for project X	local	abc-institute-drive1	6uGWmLpZlNoJ	1	1.0	2025-07-14 06:42:57.460000+00:00	1	None	1
1	IO79fB989dSX	s3://lamindb-ci/keep-artifacts-local	None	s3	us-west-1	6uGWmLpZlNoJ	1	NaN	2025-07-14 06:42:49.389000+00:00	1	None	1

Delete the test instance¶

Delete the artifacts:

artifact.delete(permanent=True)
artifact2.delete(permanent=True)
artifact3.delete(permanent=True)
my_existing_file.delete(permanent=True, storage=False)

→ a file/folder remains here: /home/runner/work/lamindb/lamindb/docs/faq/our_local_storage/output.bam

Delete the instance:

ln.setup.delete("keep-artifacts-local", force=True)