Curate MNIST
¶
# pip install lamindb torch torchvision lightning
lamin init --storage ./lamin-mlops
Show code cell output
! using anonymous user (to identify, call: lamin login)
→ initialized database anonymous/lamin-mlops in /home/runner/work/lamin-mlops/lamin-mlops
/docs
import lamindb as ln
from pathlib import Path
ln.track()
Show code cell output
→ connected lamindb: anonymous/lamin-mlops
→ created Transform('KIk0AmfDKDrQ0000', key='mnist.ipynb'), started new Run('aYa2QqMC4hRT2pN0') at 2026-10-03 08:33:59 UTC
• tip: to identify the notebook across renames, pass the uid: ln.track("KIk0AmfDKDrQ")
Download the MNIST dataset and save it in LaminDB to keep track of the training data that is associated with our model.
from torchvision.datasets import MNIST
from torchvision.transforms import ToTensor
dataset = MNIST(Path.cwd() / "download_mnist", download=True, transform=ToTensor())
Show code cell output
0%| | 0.00/9.91M [00:00<?, ?B/s]
100%|██████████| 9.91M/9.91M [00:00<00:00, 139MB/s]
0%| | 0.00/28.9k [00:00<?, ?B/s]
100%|██████████| 28.9k/28.9k [00:00<00:00, 29.3MB/s]
0%| | 0.00/1.65M [00:00<?, ?B/s]
100%|██████████| 1.65M/1.65M [00:00<00:00, 37.6MB/s]
0%| | 0.00/4.54k [00:00<?, ?B/s]
100%|██████████| 4.54k/4.54k [00:00<00:00, 25.8MB/s]
# no need for the zipped files
rm -r download_mnist/MNIST/raw/*.gz
ls -r download_mnist/MNIST/raw
Show code cell output
train-labels-idx1-ubyte
train-images-idx3-ubyte
t10k-labels-idx1-ubyte
t10k-images-idx3-ubyte
training_data_artifact = ln.Artifact(
"download_mnist/",
key="testdata/mnist",
kind="dataset",
description="Complete MNIST dataset directory containing training and test data",
).save()
training_data_artifact
Show code cell output
! did not find database on hub, but you're not logged in, so will miss private databases
Artifact(uid='WBZ3k6GhS7j7OOiM0000', key='testdata/mnist', description='Complete MNIST dataset directory containing training and test data', suffix='', kind='dataset', otype=None, size=54950048, hash='amFx_vXqnUtJr0kmxxWK2Q', n_files=4, n_observations=None, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=1, schema_id=None, created_by_id=1, created_at=2026-10-03 08:34:09 UTC, is_locked=False, version_tag=None, is_latest=True)
ln.finish()
Show code cell output
→ finished Run('aYa2QqMC4hRT2pN0') after 10s at 2026-10-03 08:34:09 UTC