MLFlow¶
LaminDB can be integrated with MLflow to track model checkpoints as artifacts linked against training runs.
# pip install lamindb torchvision lightning wandb
!lamin init --storage ./lamin-mlops
Show code cell output
→ connected lamindb: anonymous/lamin-mlops
import lightning as pl
import lamindb as ln
from lamindb.integrations import lightning as ll
import mlflow
from pathlib import Path
from torch import utils
from torchvision.datasets import MNIST
from torchvision.transforms import ToTensor
from autoencoder import LitAutoEncoder
Show code cell output
→ connected lamindb: anonymous/lamin-mlops
Tracking models in both LaminDB and MLFlow
It is not always necessary to track all model parameters and metrics in both LaminDB and MLFlow. However, if specific artifacts or runs should be queryable by specific model attributes such as, for example, the learning rate, then these attributes should be tracked. Below, we show exemplary how to do that for the batch size and learning rate but the approach generalizes to more features.
# define model run parameters, features, and labels so that validation passes later on
MODEL_CONFIG = {"batch_size": 32, "lr": 0.001}
hyperparameter = ln.Feature(name="Autoencoder hyperparameter", is_type=True).save()
hyperparams = ln.Feature.from_dict(MODEL_CONFIG, type=hyperparameter)
ln.save(hyperparams)
metrics_to_annotate = ["train_loss", "val_loss", "current_epoch"]
for metric in metrics_to_annotate:
dtype = int if metric == "current_epoch" else float
ln.Feature(name=metric, dtype=dtype).save()
# create all MLflow related features like 'mlflow_run_id'
ln.examples.mlflow.save_mlflow_features()
Show code cell output
→ returning existing Feature record with same name: 'Autoencoder hyperparameter'
→ returning existing Feature record with same name: 'batch_size'
→ returning existing Feature record with same name: 'train_loss'
→ returning existing Feature record with same name: 'val_loss'
→ returning existing Feature record with same name: 'current_epoch'
! you are trying to create a record with name='mlflow_experiment_name' but a record with similar name exists: 'mlflow_experiment_id'. Did you mean to load it?
# track this notebook/script run so that all checkpoint artifacts are associated with the source code
ln.track(params=MODEL_CONFIG, project=ln.Project(name="MLflow tutorial").save())
Show code cell output
→ created Transform('q971UuXXd03J0000', key='mlflow.ipynb'), started new Run('LSyrIuOBOxaUWNPW') at 2025-10-13 20:32:06 UTC
→ params: batch_size=32, lr=0.001
→ notebook imports: autoencoder lamindb==1.13.0 lightning==2.5.5 mlflow-skinny==3.4.0 mlflow-tracing==3.4.0 mlflow==3.4.0 torch==2.8.0 torchvision==0.23.0
• recommendation: to identify the notebook across renames, pass the uid: ln.track("q971UuXXd03J", project="MLflow tutorial", params={...})
Define a model¶
We use a basic PyTorch Lightning autoencoder as an example model.
Code of LitAutoEncoder
Simple autoencoder model¶
import torch
import lightning
from torch import optim, nn
class LitAutoEncoder(lightning.LightningModule):
def __init__(self, hidden_size: int, bottleneck_size: int) -> None:
super().__init__()
self.encoder = nn.Sequential(
nn.Linear(28 * 28, hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, bottleneck_size),
)
self.decoder = nn.Sequential(
nn.Linear(bottleneck_size, hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, 28 * 28),
)
self.save_hyperparameters()
def training_step(
self, batch: tuple[torch.Tensor, torch.Tensor], batch_idx: int
) -> torch.Tensor:
x, y = batch
x = x.view(x.size(0), -1)
z = self.encoder(x)
x_hat = self.decoder(z)
loss = nn.functional.mse_loss(x_hat, x)
self.log("train_loss", loss, on_epoch=True)
return loss
def validation_step(
self, batch: tuple[torch.Tensor, torch.Tensor], batch_idx: int
) -> torch.Tensor:
x, y = batch
x = x.view(x.size(0), -1)
z = self.encoder(x)
x_hat = self.decoder(z)
loss = nn.functional.mse_loss(x_hat, x)
self.log("val_loss", loss, on_epoch=True)
return loss
def configure_optimizers(self) -> optim.Optimizer:
optimizer = optim.Adam(self.parameters(), lr=1e-3)
return optimizer
Query & download the MNIST dataset¶
We saved the MNIST dataset in a curation notebook which now shows up in the Artifact registry:
ln.Artifact.filter(kind="dataset").to_dataframe()
Show code cell output
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | _hash_type | _key_is_virtual | _overwrite_versions | space_id | storage_id | schema_id | version | is_latest | is_locked | run_id | created_at | created_by_id | _aux | _real_key | branch_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||||||
| 1 | OzuOXgshTJQEUq200000 | testdata/mnist | Complete MNIST dataset directory containing tr... | dataset | None | 54950048 | amFx_vXqnUtJr0kmxxWK2Q | 4 | None | md5-d | True | True | 1 | 1 | None | None | True | False | 1 | 2025-10-13 20:31:18.384000+00:00 | 1 | {'af': {'0': True}} | None | 1 |
Let’s get the dataset:
artifact = ln.Artifact.get(key="testdata/mnist")
artifact
Show code cell output
Artifact(uid='OzuOXgshTJQEUq200000', is_latest=True, key='testdata/mnist', description='Complete MNIST dataset directory containing training and test data', suffix='', kind='dataset', size=54950048, hash='amFx_vXqnUtJr0kmxxWK2Q', n_files=4, branch_id=1, space_id=1, storage_id=1, run_id=1, created_by_id=1, created_at=2025-10-13 20:31:18 UTC, is_locked=False)
And download it to a local cache:
path = artifact.cache()
path
Show code cell output
PosixUPath('/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops/.lamindb/OzuOXgshTJQEUq20')
Create a PyTorch-compatible dataset:
dataset = MNIST(path.as_posix(), transform=ToTensor())
dataset
Show code cell output
Dataset MNIST
Number of datapoints: 60000
Root location: /home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops/.lamindb/OzuOXgshTJQEUq20
Split: Train
StandardTransform
Transform: ToTensor()
Monitor training with MLflow¶
Train our example model and track the training progress with MLflow.
# enable MLFlow PyTorch autologging
mlflow.pytorch.autolog()
with mlflow.start_run() as mlflow_run:
train_dataset = MNIST(
root="./data", train=True, download=True, transform=ToTensor()
)
val_dataset = MNIST(root="./data", train=False, download=True, transform=ToTensor())
train_loader = utils.data.DataLoader(train_dataset, batch_size=32)
val_loader = utils.data.DataLoader(val_dataset, batch_size=32)
# create model
autoencoder = LitAutoEncoder(hidden_size=32, bottleneck_size=16)
# Create a LaminDB LightningCallback which also (optionally) annotates checkpoints by desired metrics
lamindb_callback = ll.Callback(
path=Path("model_checkpoints") / f"{mlflow_run.info.run_id}_last_epoch.ckpt",
key=f"testmodels/mlflow/{mlflow_run.info.run_id}.ckpt",
features={
"mlflow_run_id": mlflow_run.info.run_id,
"mlflow_run_name": mlflow_run.info.run_name,
**{
metric: None for metric in metrics_to_annotate
}, # auto-populated through callback
},
)
# Train model
trainer = pl.Trainer(
limit_train_batches=3,
max_epochs=5,
callbacks=[lamindb_callback],
)
trainer.fit(
model=autoencoder, train_dataloaders=train_loader, val_dataloaders=val_loader
)
# Register model_summary.txt
local_model_summary_path = (
f"{mlflow_run.info.artifact_uri.removeprefix('file://')}/model_summary.txt"
)
mlflow_model_summary_af = ln.Artifact(
local_model_summary_path,
key=local_model_summary_path,
kind="model",
).save()
Show code cell output
INFO:pytorch_lightning.utilities.rank_zero:💡 Tip: For seamless cloud uploads and versioning, try installing [litmodels](https://pypi.org/project/litmodels/) to enable LitModelCheckpoint, which syncs automatically with the Lightning model registry.
INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/lightning/pytorch/trainer/connectors/logger_connector/logger_connector.py:76: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `lightning.pytorch` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default
| Name | Type | Params | Mode
-----------------------------------------------
0 | encoder | Sequential | 25.6 K | train
1 | decoder | Sequential | 26.4 K | train
-----------------------------------------------
52.1 K Trainable params
0 Non-trainable params
52.1 K Total params
0.208 Total estimated model params size (MB)
8 Modules in train mode
0 Modules in eval mode
Sanity Checking: | | 0/? [00:00<?, ?it/s]
Sanity Checking: | | 0/? [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 50%|█████ | 1/2 [00:00<00:00, 61.38it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00, 88.84it/s]
/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:433: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance.
/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:433: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance.
/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/lightning/pytorch/loops/fit_loop.py:310: The number of training batches (3) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
Training: | | 0/? [00:00<?, ?it/s]
Training: | | 0/? [00:00<?, ?it/s]
Epoch 0: 0%| | 0/3 [00:00<?, ?it/s]
Epoch 0: 33%|███▎ | 1/3 [00:00<00:00, 106.66it/s]
Epoch 0: 33%|███▎ | 1/3 [00:00<00:00, 99.92it/s, v_num=0]
Epoch 0: 67%|██████▋ | 2/3 [00:00<00:00, 114.15it/s, v_num=0]
Epoch 0: 67%|██████▋ | 2/3 [00:00<00:00, 109.73it/s, v_num=0]
Epoch 0: 100%|██████████| 3/3 [00:00<00:00, 119.20it/s, v_num=0]
Epoch 0: 100%|██████████| 3/3 [00:00<00:00, 115.43it/s, v_num=0]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: | | 0/? [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/313 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 1/313 [00:00<00:01, 194.77it/s]
Validation DataLoader 0: 1%| | 2/313 [00:00<00:01, 179.29it/s]
Validation DataLoader 0: 1%| | 3/313 [00:00<00:01, 172.28it/s]
Validation DataLoader 0: 1%|▏ | 4/313 [00:00<00:01, 168.01it/s]
Validation DataLoader 0: 2%|▏ | 5/313 [00:00<00:01, 165.31it/s]
Validation DataLoader 0: 2%|▏ | 6/313 [00:00<00:01, 164.89it/s]
Validation DataLoader 0: 2%|▏ | 7/313 [00:00<00:01, 163.53it/s]
Validation DataLoader 0: 3%|▎ | 8/313 [00:00<00:01, 162.77it/s]
Validation DataLoader 0: 3%|▎ | 9/313 [00:00<00:01, 162.64it/s]
Validation DataLoader 0: 3%|▎ | 10/313 [00:00<00:01, 162.57it/s]
Validation DataLoader 0: 4%|▎ | 11/313 [00:00<00:01, 162.10it/s]
Validation DataLoader 0: 4%|▍ | 12/313 [00:00<00:01, 162.44it/s]
Validation DataLoader 0: 4%|▍ | 13/313 [00:00<00:01, 162.52it/s]
Validation DataLoader 0: 4%|▍ | 14/313 [00:00<00:01, 162.61it/s]
Validation DataLoader 0: 5%|▍ | 15/313 [00:00<00:01, 162.82it/s]
Validation DataLoader 0: 5%|▌ | 16/313 [00:00<00:01, 162.68it/s]
Validation DataLoader 0: 5%|▌ | 17/313 [00:00<00:01, 162.73it/s]
Validation DataLoader 0: 6%|▌ | 18/313 [00:00<00:01, 162.83it/s]
Validation DataLoader 0: 6%|▌ | 19/313 [00:00<00:01, 162.91it/s]
Validation DataLoader 0: 6%|▋ | 20/313 [00:00<00:01, 162.76it/s]
Validation DataLoader 0: 7%|▋ | 21/313 [00:00<00:01, 162.83it/s]
Validation DataLoader 0: 7%|▋ | 22/313 [00:00<00:01, 162.82it/s]
Validation DataLoader 0: 7%|▋ | 23/313 [00:00<00:01, 162.82it/s]
Validation DataLoader 0: 8%|▊ | 24/313 [00:00<00:01, 162.66it/s]
Validation DataLoader 0: 8%|▊ | 25/313 [00:00<00:01, 162.40it/s]
Validation DataLoader 0: 8%|▊ | 26/313 [00:00<00:01, 161.68it/s]
Validation DataLoader 0: 9%|▊ | 27/313 [00:00<00:01, 161.18it/s]
Validation DataLoader 0: 9%|▉ | 28/313 [00:00<00:01, 160.93it/s]
Validation DataLoader 0: 9%|▉ | 29/313 [00:00<00:01, 160.55it/s]
Validation DataLoader 0: 10%|▉ | 30/313 [00:00<00:01, 160.41it/s]
Validation DataLoader 0: 10%|▉ | 31/313 [00:00<00:01, 160.42it/s]
Validation DataLoader 0: 10%|█ | 32/313 [00:00<00:01, 159.37it/s]
Validation DataLoader 0: 11%|█ | 33/313 [00:00<00:01, 158.91it/s]
Validation DataLoader 0: 11%|█ | 34/313 [00:00<00:01, 158.96it/s]
Validation DataLoader 0: 11%|█ | 35/313 [00:00<00:01, 159.04it/s]
Validation DataLoader 0: 12%|█▏ | 36/313 [00:00<00:01, 159.20it/s]
Validation DataLoader 0: 12%|█▏ | 37/313 [00:00<00:01, 159.29it/s]
Validation DataLoader 0: 12%|█▏ | 38/313 [00:00<00:01, 159.40it/s]
Validation DataLoader 0: 12%|█▏ | 39/313 [00:00<00:01, 159.57it/s]
Validation DataLoader 0: 13%|█▎ | 40/313 [00:00<00:01, 159.66it/s]
Validation DataLoader 0: 13%|█▎ | 41/313 [00:00<00:01, 159.76it/s]
Validation DataLoader 0: 13%|█▎ | 42/313 [00:00<00:01, 159.83it/s]
Validation DataLoader 0: 14%|█▎ | 43/313 [00:00<00:01, 159.88it/s]
Validation DataLoader 0: 14%|█▍ | 44/313 [00:00<00:01, 160.00it/s]
Validation DataLoader 0: 14%|█▍ | 45/313 [00:00<00:01, 160.17it/s]
Validation DataLoader 0: 15%|█▍ | 46/313 [00:00<00:01, 160.34it/s]
Validation DataLoader 0: 15%|█▌ | 47/313 [00:00<00:01, 160.52it/s]
Validation DataLoader 0: 15%|█▌ | 48/313 [00:00<00:01, 160.67it/s]
Validation DataLoader 0: 16%|█▌ | 49/313 [00:00<00:01, 160.78it/s]
Validation DataLoader 0: 16%|█▌ | 50/313 [00:00<00:01, 160.71it/s]
Validation DataLoader 0: 16%|█▋ | 51/313 [00:00<00:01, 160.74it/s]
Validation DataLoader 0: 17%|█▋ | 52/313 [00:00<00:01, 160.76it/s]
Validation DataLoader 0: 17%|█▋ | 53/313 [00:00<00:01, 160.70it/s]
Validation DataLoader 0: 17%|█▋ | 54/313 [00:00<00:01, 160.69it/s]
Validation DataLoader 0: 18%|█▊ | 55/313 [00:00<00:01, 160.65it/s]
Validation DataLoader 0: 18%|█▊ | 56/313 [00:00<00:01, 160.49it/s]
Validation DataLoader 0: 18%|█▊ | 57/313 [00:00<00:01, 160.18it/s]
Validation DataLoader 0: 19%|█▊ | 58/313 [00:00<00:01, 160.08it/s]
Validation DataLoader 0: 19%|█▉ | 59/313 [00:00<00:01, 160.08it/s]
Validation DataLoader 0: 19%|█▉ | 60/313 [00:00<00:01, 159.99it/s]
Validation DataLoader 0: 19%|█▉ | 61/313 [00:00<00:01, 159.96it/s]
Validation DataLoader 0: 20%|█▉ | 62/313 [00:00<00:01, 159.93it/s]
Validation DataLoader 0: 20%|██ | 63/313 [00:00<00:01, 159.96it/s]
Validation DataLoader 0: 20%|██ | 64/313 [00:00<00:01, 159.89it/s]
Validation DataLoader 0: 21%|██ | 65/313 [00:00<00:01, 159.84it/s]
Validation DataLoader 0: 21%|██ | 66/313 [00:00<00:01, 159.86it/s]
Validation DataLoader 0: 21%|██▏ | 67/313 [00:00<00:01, 159.92it/s]
Validation DataLoader 0: 22%|██▏ | 68/313 [00:00<00:01, 159.96it/s]
Validation DataLoader 0: 22%|██▏ | 69/313 [00:00<00:01, 160.01it/s]
Validation DataLoader 0: 22%|██▏ | 70/313 [00:00<00:01, 160.06it/s]
Validation DataLoader 0: 23%|██▎ | 71/313 [00:00<00:01, 160.10it/s]
Validation DataLoader 0: 23%|██▎ | 72/313 [00:00<00:01, 160.17it/s]
Validation DataLoader 0: 23%|██▎ | 73/313 [00:00<00:01, 160.23it/s]
Validation DataLoader 0: 24%|██▎ | 74/313 [00:00<00:01, 160.01it/s]
Validation DataLoader 0: 24%|██▍ | 75/313 [00:00<00:01, 159.53it/s]
Validation DataLoader 0: 24%|██▍ | 76/313 [00:00<00:01, 159.42it/s]
Validation DataLoader 0: 25%|██▍ | 77/313 [00:00<00:01, 159.43it/s]
Validation DataLoader 0: 25%|██▍ | 78/313 [00:00<00:01, 159.39it/s]
Validation DataLoader 0: 25%|██▌ | 79/313 [00:00<00:01, 159.40it/s]
Validation DataLoader 0: 26%|██▌ | 80/313 [00:00<00:01, 159.39it/s]
Validation DataLoader 0: 26%|██▌ | 81/313 [00:00<00:01, 159.38it/s]
Validation DataLoader 0: 26%|██▌ | 82/313 [00:00<00:01, 159.36it/s]
Validation DataLoader 0: 27%|██▋ | 83/313 [00:00<00:01, 159.36it/s]
Validation DataLoader 0: 27%|██▋ | 84/313 [00:00<00:01, 159.37it/s]
Validation DataLoader 0: 27%|██▋ | 85/313 [00:00<00:01, 159.25it/s]
Validation DataLoader 0: 27%|██▋ | 86/313 [00:00<00:01, 159.27it/s]
Validation DataLoader 0: 28%|██▊ | 87/313 [00:00<00:01, 159.34it/s]
Validation DataLoader 0: 28%|██▊ | 88/313 [00:00<00:01, 159.19it/s]
Validation DataLoader 0: 28%|██▊ | 89/313 [00:00<00:01, 159.02it/s]
Validation DataLoader 0: 29%|██▉ | 90/313 [00:00<00:01, 158.80it/s]
Validation DataLoader 0: 29%|██▉ | 91/313 [00:00<00:01, 158.40it/s]
Validation DataLoader 0: 29%|██▉ | 92/313 [00:00<00:01, 158.07it/s]
Validation DataLoader 0: 30%|██▉ | 93/313 [00:00<00:01, 157.80it/s]
Validation DataLoader 0: 30%|███ | 94/313 [00:00<00:01, 157.46it/s]
Validation DataLoader 0: 30%|███ | 95/313 [00:00<00:01, 157.24it/s]
Validation DataLoader 0: 31%|███ | 96/313 [00:00<00:01, 157.03it/s]
Validation DataLoader 0: 31%|███ | 97/313 [00:00<00:01, 156.87it/s]
Validation DataLoader 0: 31%|███▏ | 98/313 [00:00<00:01, 156.62it/s]
Validation DataLoader 0: 32%|███▏ | 99/313 [00:00<00:01, 156.47it/s]
Validation DataLoader 0: 32%|███▏ | 100/313 [00:00<00:01, 156.32it/s]
Validation DataLoader 0: 32%|███▏ | 101/313 [00:00<00:01, 156.11it/s]
Validation DataLoader 0: 33%|███▎ | 102/313 [00:00<00:01, 155.92it/s]
Validation DataLoader 0: 33%|███▎ | 103/313 [00:00<00:01, 155.90it/s]
Validation DataLoader 0: 33%|███▎ | 104/313 [00:00<00:01, 155.88it/s]
Validation DataLoader 0: 34%|███▎ | 105/313 [00:00<00:01, 155.92it/s]
Validation DataLoader 0: 34%|███▍ | 106/313 [00:00<00:01, 155.93it/s]
Validation DataLoader 0: 34%|███▍ | 107/313 [00:00<00:01, 156.00it/s]
Validation DataLoader 0: 35%|███▍ | 108/313 [00:00<00:01, 156.05it/s]
Validation DataLoader 0: 35%|███▍ | 109/313 [00:00<00:01, 156.06it/s]
Validation DataLoader 0: 35%|███▌ | 110/313 [00:00<00:01, 156.07it/s]
Validation DataLoader 0: 35%|███▌ | 111/313 [00:00<00:01, 156.08it/s]
Validation DataLoader 0: 36%|███▌ | 112/313 [00:00<00:01, 156.09it/s]
Validation DataLoader 0: 36%|███▌ | 113/313 [00:00<00:01, 156.08it/s]
Validation DataLoader 0: 36%|███▋ | 114/313 [00:00<00:01, 156.07it/s]
Validation DataLoader 0: 37%|███▋ | 115/313 [00:00<00:01, 156.03it/s]
Validation DataLoader 0: 37%|███▋ | 116/313 [00:00<00:01, 156.03it/s]
Validation DataLoader 0: 37%|███▋ | 117/313 [00:00<00:01, 156.05it/s]
Validation DataLoader 0: 38%|███▊ | 118/313 [00:00<00:01, 156.06it/s]
Validation DataLoader 0: 38%|███▊ | 119/313 [00:00<00:01, 156.08it/s]
Validation DataLoader 0: 38%|███▊ | 120/313 [00:00<00:01, 156.03it/s]
Validation DataLoader 0: 39%|███▊ | 121/313 [00:00<00:01, 155.80it/s]
Validation DataLoader 0: 39%|███▉ | 122/313 [00:00<00:01, 155.55it/s]
Validation DataLoader 0: 39%|███▉ | 123/313 [00:00<00:01, 155.42it/s]
Validation DataLoader 0: 40%|███▉ | 124/313 [00:00<00:01, 155.23it/s]
Validation DataLoader 0: 40%|███▉ | 125/313 [00:00<00:01, 155.26it/s]
Validation DataLoader 0: 40%|████ | 126/313 [00:00<00:01, 155.26it/s]
Validation DataLoader 0: 41%|████ | 127/313 [00:00<00:01, 155.17it/s]
Validation DataLoader 0: 41%|████ | 128/313 [00:00<00:01, 155.02it/s]
Validation DataLoader 0: 41%|████ | 129/313 [00:00<00:01, 154.86it/s]
Validation DataLoader 0: 42%|████▏ | 130/313 [00:00<00:01, 154.86it/s]
Validation DataLoader 0: 42%|████▏ | 131/313 [00:00<00:01, 154.73it/s]
Validation DataLoader 0: 42%|████▏ | 132/313 [00:00<00:01, 154.60it/s]
Validation DataLoader 0: 42%|████▏ | 133/313 [00:00<00:01, 154.46it/s]
Validation DataLoader 0: 43%|████▎ | 134/313 [00:00<00:01, 154.30it/s]
Validation DataLoader 0: 43%|████▎ | 135/313 [00:00<00:01, 154.31it/s]
Validation DataLoader 0: 43%|████▎ | 136/313 [00:00<00:01, 154.11it/s]
Validation DataLoader 0: 44%|████▍ | 137/313 [00:00<00:01, 153.92it/s]
Validation DataLoader 0: 44%|████▍ | 138/313 [00:00<00:01, 153.91it/s]
Validation DataLoader 0: 44%|████▍ | 139/313 [00:00<00:01, 153.79it/s]
Validation DataLoader 0: 45%|████▍ | 140/313 [00:00<00:01, 153.69it/s]
Validation DataLoader 0: 45%|████▌ | 141/313 [00:00<00:01, 153.70it/s]
Validation DataLoader 0: 45%|████▌ | 142/313 [00:00<00:01, 153.79it/s]
Validation DataLoader 0: 46%|████▌ | 143/313 [00:00<00:01, 153.82it/s]
Validation DataLoader 0: 46%|████▌ | 144/313 [00:00<00:01, 153.90it/s]
Validation DataLoader 0: 46%|████▋ | 145/313 [00:00<00:01, 153.90it/s]
Validation DataLoader 0: 47%|████▋ | 146/313 [00:00<00:01, 153.95it/s]
Validation DataLoader 0: 47%|████▋ | 147/313 [00:00<00:01, 153.97it/s]
Validation DataLoader 0: 47%|████▋ | 148/313 [00:00<00:01, 154.06it/s]
Validation DataLoader 0: 48%|████▊ | 149/313 [00:00<00:01, 154.04it/s]
Validation DataLoader 0: 48%|████▊ | 150/313 [00:00<00:01, 153.94it/s]
Validation DataLoader 0: 48%|████▊ | 151/313 [00:00<00:01, 153.88it/s]
Validation DataLoader 0: 49%|████▊ | 152/313 [00:00<00:01, 153.75it/s]
Validation DataLoader 0: 49%|████▉ | 153/313 [00:00<00:01, 153.71it/s]
Validation DataLoader 0: 49%|████▉ | 154/313 [00:01<00:01, 153.75it/s]
Validation DataLoader 0: 50%|████▉ | 155/313 [00:01<00:01, 153.64it/s]
Validation DataLoader 0: 50%|████▉ | 156/313 [00:01<00:01, 153.63it/s]
Validation DataLoader 0: 50%|█████ | 157/313 [00:01<00:01, 153.64it/s]
Validation DataLoader 0: 50%|█████ | 158/313 [00:01<00:01, 153.67it/s]
Validation DataLoader 0: 51%|█████ | 159/313 [00:01<00:01, 153.68it/s]
Validation DataLoader 0: 51%|█████ | 160/313 [00:01<00:00, 153.69it/s]
Validation DataLoader 0: 51%|█████▏ | 161/313 [00:01<00:00, 153.65it/s]
Validation DataLoader 0: 52%|█████▏ | 162/313 [00:01<00:00, 153.58it/s]
Validation DataLoader 0: 52%|█████▏ | 163/313 [00:01<00:00, 153.53it/s]
Validation DataLoader 0: 52%|█████▏ | 164/313 [00:01<00:00, 153.52it/s]
Validation DataLoader 0: 53%|█████▎ | 165/313 [00:01<00:00, 153.55it/s]
Validation DataLoader 0: 53%|█████▎ | 166/313 [00:01<00:00, 153.58it/s]
Validation DataLoader 0: 53%|█████▎ | 167/313 [00:01<00:00, 153.63it/s]
Validation DataLoader 0: 54%|█████▎ | 168/313 [00:01<00:00, 153.67it/s]
Validation DataLoader 0: 54%|█████▍ | 169/313 [00:01<00:00, 153.71it/s]
Validation DataLoader 0: 54%|█████▍ | 170/313 [00:01<00:00, 153.77it/s]
Validation DataLoader 0: 55%|█████▍ | 171/313 [00:01<00:00, 153.83it/s]
Validation DataLoader 0: 55%|█████▍ | 172/313 [00:01<00:00, 153.88it/s]
Validation DataLoader 0: 55%|█████▌ | 173/313 [00:01<00:00, 153.91it/s]
Validation DataLoader 0: 56%|█████▌ | 174/313 [00:01<00:00, 153.96it/s]
Validation DataLoader 0: 56%|█████▌ | 175/313 [00:01<00:00, 154.00it/s]
Validation DataLoader 0: 56%|█████▌ | 176/313 [00:01<00:00, 154.02it/s]
Validation DataLoader 0: 57%|█████▋ | 177/313 [00:01<00:00, 154.06it/s]
Validation DataLoader 0: 57%|█████▋ | 178/313 [00:01<00:00, 154.10it/s]
Validation DataLoader 0: 57%|█████▋ | 179/313 [00:01<00:00, 154.14it/s]
Validation DataLoader 0: 58%|█████▊ | 180/313 [00:01<00:00, 154.21it/s]
Validation DataLoader 0: 58%|█████▊ | 181/313 [00:01<00:00, 154.28it/s]
Validation DataLoader 0: 58%|█████▊ | 182/313 [00:01<00:00, 154.33it/s]
Validation DataLoader 0: 58%|█████▊ | 183/313 [00:01<00:00, 154.35it/s]
Validation DataLoader 0: 59%|█████▉ | 184/313 [00:01<00:00, 154.41it/s]
Validation DataLoader 0: 59%|█████▉ | 185/313 [00:01<00:00, 154.44it/s]
Validation DataLoader 0: 59%|█████▉ | 186/313 [00:01<00:00, 154.47it/s]
Validation DataLoader 0: 60%|█████▉ | 187/313 [00:01<00:00, 154.51it/s]
Validation DataLoader 0: 60%|██████ | 188/313 [00:01<00:00, 154.55it/s]
Validation DataLoader 0: 60%|██████ | 189/313 [00:01<00:00, 154.60it/s]
Validation DataLoader 0: 61%|██████ | 190/313 [00:01<00:00, 154.66it/s]
Validation DataLoader 0: 61%|██████ | 191/313 [00:01<00:00, 154.71it/s]
Validation DataLoader 0: 61%|██████▏ | 192/313 [00:01<00:00, 154.75it/s]
Validation DataLoader 0: 62%|██████▏ | 193/313 [00:01<00:00, 154.75it/s]
Validation DataLoader 0: 62%|██████▏ | 194/313 [00:01<00:00, 154.78it/s]
Validation DataLoader 0: 62%|██████▏ | 195/313 [00:01<00:00, 154.82it/s]
Validation DataLoader 0: 63%|██████▎ | 196/313 [00:01<00:00, 154.85it/s]
Validation DataLoader 0: 63%|██████▎ | 197/313 [00:01<00:00, 154.88it/s]
Validation DataLoader 0: 63%|██████▎ | 198/313 [00:01<00:00, 154.90it/s]
Validation DataLoader 0: 64%|██████▎ | 199/313 [00:01<00:00, 154.92it/s]
Validation DataLoader 0: 64%|██████▍ | 200/313 [00:01<00:00, 154.94it/s]
Validation DataLoader 0: 64%|██████▍ | 201/313 [00:01<00:00, 154.93it/s]
Validation DataLoader 0: 65%|██████▍ | 202/313 [00:01<00:00, 154.95it/s]
Validation DataLoader 0: 65%|██████▍ | 203/313 [00:01<00:00, 154.98it/s]
Validation DataLoader 0: 65%|██████▌ | 204/313 [00:01<00:00, 155.01it/s]
Validation DataLoader 0: 65%|██████▌ | 205/313 [00:01<00:00, 155.04it/s]
Validation DataLoader 0: 66%|██████▌ | 206/313 [00:01<00:00, 155.08it/s]
Validation DataLoader 0: 66%|██████▌ | 207/313 [00:01<00:00, 155.10it/s]
Validation DataLoader 0: 66%|██████▋ | 208/313 [00:01<00:00, 155.12it/s]
Validation DataLoader 0: 67%|██████▋ | 209/313 [00:01<00:00, 155.14it/s]
Validation DataLoader 0: 67%|██████▋ | 210/313 [00:01<00:00, 155.17it/s]
Validation DataLoader 0: 67%|██████▋ | 211/313 [00:01<00:00, 155.19it/s]
Validation DataLoader 0: 68%|██████▊ | 212/313 [00:01<00:00, 155.20it/s]
Validation DataLoader 0: 68%|██████▊ | 213/313 [00:01<00:00, 155.20it/s]
Validation DataLoader 0: 68%|██████▊ | 214/313 [00:01<00:00, 155.21it/s]
Validation DataLoader 0: 69%|██████▊ | 215/313 [00:01<00:00, 155.24it/s]
Validation DataLoader 0: 69%|██████▉ | 216/313 [00:01<00:00, 155.27it/s]
Validation DataLoader 0: 69%|██████▉ | 217/313 [00:01<00:00, 155.29it/s]
Validation DataLoader 0: 70%|██████▉ | 218/313 [00:01<00:00, 155.21it/s]
Validation DataLoader 0: 70%|██████▉ | 219/313 [00:01<00:00, 155.19it/s]
Validation DataLoader 0: 70%|███████ | 220/313 [00:01<00:00, 155.18it/s]
Validation DataLoader 0: 71%|███████ | 221/313 [00:01<00:00, 155.07it/s]
Validation DataLoader 0: 71%|███████ | 222/313 [00:01<00:00, 154.98it/s]
Validation DataLoader 0: 71%|███████ | 223/313 [00:01<00:00, 154.86it/s]
Validation DataLoader 0: 72%|███████▏ | 224/313 [00:01<00:00, 154.76it/s]
Validation DataLoader 0: 72%|███████▏ | 225/313 [00:01<00:00, 154.69it/s]
Validation DataLoader 0: 72%|███████▏ | 226/313 [00:01<00:00, 154.63it/s]
Validation DataLoader 0: 73%|███████▎ | 227/313 [00:01<00:00, 154.61it/s]
Validation DataLoader 0: 73%|███████▎ | 228/313 [00:01<00:00, 154.52it/s]
Validation DataLoader 0: 73%|███████▎ | 229/313 [00:01<00:00, 154.52it/s]
Validation DataLoader 0: 73%|███████▎ | 230/313 [00:01<00:00, 154.53it/s]
Validation DataLoader 0: 74%|███████▍ | 231/313 [00:01<00:00, 154.53it/s]
Validation DataLoader 0: 74%|███████▍ | 232/313 [00:01<00:00, 154.57it/s]
Validation DataLoader 0: 74%|███████▍ | 233/313 [00:01<00:00, 154.59it/s]
Validation DataLoader 0: 75%|███████▍ | 234/313 [00:01<00:00, 154.61it/s]
Validation DataLoader 0: 75%|███████▌ | 235/313 [00:01<00:00, 154.62it/s]
Validation DataLoader 0: 75%|███████▌ | 236/313 [00:01<00:00, 154.61it/s]
Validation DataLoader 0: 76%|███████▌ | 237/313 [00:01<00:00, 154.63it/s]
Validation DataLoader 0: 76%|███████▌ | 238/313 [00:01<00:00, 154.66it/s]
Validation DataLoader 0: 76%|███████▋ | 239/313 [00:01<00:00, 154.70it/s]
Validation DataLoader 0: 77%|███████▋ | 240/313 [00:01<00:00, 154.72it/s]
Validation DataLoader 0: 77%|███████▋ | 241/313 [00:01<00:00, 154.74it/s]
Validation DataLoader 0: 77%|███████▋ | 242/313 [00:01<00:00, 154.77it/s]
Validation DataLoader 0: 78%|███████▊ | 243/313 [00:01<00:00, 154.80it/s]
Validation DataLoader 0: 78%|███████▊ | 244/313 [00:01<00:00, 154.81it/s]
Validation DataLoader 0: 78%|███████▊ | 245/313 [00:01<00:00, 154.76it/s]
Validation DataLoader 0: 79%|███████▊ | 246/313 [00:01<00:00, 154.78it/s]
Validation DataLoader 0: 79%|███████▉ | 247/313 [00:01<00:00, 154.80it/s]
Validation DataLoader 0: 79%|███████▉ | 248/313 [00:01<00:00, 154.74it/s]
Validation DataLoader 0: 80%|███████▉ | 249/313 [00:01<00:00, 154.67it/s]
Validation DataLoader 0: 80%|███████▉ | 250/313 [00:01<00:00, 154.60it/s]
Validation DataLoader 0: 80%|████████ | 251/313 [00:01<00:00, 154.54it/s]
Validation DataLoader 0: 81%|████████ | 252/313 [00:01<00:00, 154.52it/s]
Validation DataLoader 0: 81%|████████ | 253/313 [00:01<00:00, 154.55it/s]
Validation DataLoader 0: 81%|████████ | 254/313 [00:01<00:00, 154.49it/s]
Validation DataLoader 0: 81%|████████▏ | 255/313 [00:01<00:00, 154.51it/s]
Validation DataLoader 0: 82%|████████▏ | 256/313 [00:01<00:00, 154.53it/s]
Validation DataLoader 0: 82%|████████▏ | 257/313 [00:01<00:00, 154.48it/s]
Validation DataLoader 0: 82%|████████▏ | 258/313 [00:01<00:00, 154.44it/s]
Validation DataLoader 0: 83%|████████▎ | 259/313 [00:01<00:00, 154.46it/s]
Validation DataLoader 0: 83%|████████▎ | 260/313 [00:01<00:00, 154.48it/s]
Validation DataLoader 0: 83%|████████▎ | 261/313 [00:01<00:00, 154.50it/s]
Validation DataLoader 0: 84%|████████▎ | 262/313 [00:01<00:00, 154.52it/s]
Validation DataLoader 0: 84%|████████▍ | 263/313 [00:01<00:00, 154.54it/s]
Validation DataLoader 0: 84%|████████▍ | 264/313 [00:01<00:00, 154.57it/s]
Validation DataLoader 0: 85%|████████▍ | 265/313 [00:01<00:00, 154.61it/s]
Validation DataLoader 0: 85%|████████▍ | 266/313 [00:01<00:00, 154.65it/s]
Validation DataLoader 0: 85%|████████▌ | 267/313 [00:01<00:00, 154.68it/s]
Validation DataLoader 0: 86%|████████▌ | 268/313 [00:01<00:00, 154.71it/s]
Validation DataLoader 0: 86%|████████▌ | 269/313 [00:01<00:00, 154.73it/s]
Validation DataLoader 0: 86%|████████▋ | 270/313 [00:01<00:00, 154.75it/s]
Validation DataLoader 0: 87%|████████▋ | 271/313 [00:01<00:00, 154.78it/s]
Validation DataLoader 0: 87%|████████▋ | 272/313 [00:01<00:00, 154.81it/s]
Validation DataLoader 0: 87%|████████▋ | 273/313 [00:01<00:00, 154.84it/s]
Validation DataLoader 0: 88%|████████▊ | 274/313 [00:01<00:00, 154.87it/s]
Validation DataLoader 0: 88%|████████▊ | 275/313 [00:01<00:00, 154.89it/s]
Validation DataLoader 0: 88%|████████▊ | 276/313 [00:01<00:00, 154.92it/s]
Validation DataLoader 0: 88%|████████▊ | 277/313 [00:01<00:00, 154.93it/s]
Validation DataLoader 0: 89%|████████▉ | 278/313 [00:01<00:00, 154.93it/s]
Validation DataLoader 0: 89%|████████▉ | 279/313 [00:01<00:00, 154.94it/s]
Validation DataLoader 0: 89%|████████▉ | 280/313 [00:01<00:00, 154.95it/s]
Validation DataLoader 0: 90%|████████▉ | 281/313 [00:01<00:00, 154.97it/s]
Validation DataLoader 0: 90%|█████████ | 282/313 [00:01<00:00, 154.98it/s]
Validation DataLoader 0: 90%|█████████ | 283/313 [00:01<00:00, 154.99it/s]
Validation DataLoader 0: 91%|█████████ | 284/313 [00:01<00:00, 155.00it/s]
Validation DataLoader 0: 91%|█████████ | 285/313 [00:01<00:00, 155.03it/s]
Validation DataLoader 0: 91%|█████████▏| 286/313 [00:01<00:00, 155.04it/s]
Validation DataLoader 0: 92%|█████████▏| 287/313 [00:01<00:00, 155.07it/s]
Validation DataLoader 0: 92%|█████████▏| 288/313 [00:01<00:00, 155.10it/s]
Validation DataLoader 0: 92%|█████████▏| 289/313 [00:01<00:00, 155.06it/s]
Validation DataLoader 0: 93%|█████████▎| 290/313 [00:01<00:00, 155.08it/s]
Validation DataLoader 0: 93%|█████████▎| 291/313 [00:01<00:00, 155.10it/s]
Validation DataLoader 0: 93%|█████████▎| 292/313 [00:01<00:00, 155.13it/s]
Validation DataLoader 0: 94%|█████████▎| 293/313 [00:01<00:00, 155.17it/s]
Validation DataLoader 0: 94%|█████████▍| 294/313 [00:01<00:00, 155.20it/s]
Validation DataLoader 0: 94%|█████████▍| 295/313 [00:01<00:00, 155.21it/s]
Validation DataLoader 0: 95%|█████████▍| 296/313 [00:01<00:00, 155.24it/s]
Validation DataLoader 0: 95%|█████████▍| 297/313 [00:01<00:00, 155.26it/s]
Validation DataLoader 0: 95%|█████████▌| 298/313 [00:01<00:00, 155.26it/s]
Validation DataLoader 0: 96%|█████████▌| 299/313 [00:01<00:00, 155.26it/s]
Validation DataLoader 0: 96%|█████████▌| 300/313 [00:01<00:00, 155.28it/s]
Validation DataLoader 0: 96%|█████████▌| 301/313 [00:01<00:00, 155.31it/s]
Validation DataLoader 0: 96%|█████████▋| 302/313 [00:01<00:00, 155.33it/s]
Validation DataLoader 0: 97%|█████████▋| 303/313 [00:01<00:00, 155.35it/s]
Validation DataLoader 0: 97%|█████████▋| 304/313 [00:01<00:00, 155.38it/s]
Validation DataLoader 0: 97%|█████████▋| 305/313 [00:01<00:00, 155.42it/s]
Validation DataLoader 0: 98%|█████████▊| 306/313 [00:01<00:00, 155.44it/s]
Validation DataLoader 0: 98%|█████████▊| 307/313 [00:01<00:00, 155.45it/s]
Validation DataLoader 0: 98%|█████████▊| 308/313 [00:01<00:00, 155.45it/s]
Validation DataLoader 0: 99%|█████████▊| 309/313 [00:01<00:00, 155.47it/s]
Validation DataLoader 0: 99%|█████████▉| 310/313 [00:01<00:00, 155.47it/s]
Validation DataLoader 0: 99%|█████████▉| 311/313 [00:02<00:00, 155.49it/s]
Validation DataLoader 0: 100%|█████████▉| 312/313 [00:02<00:00, 155.52it/s]
Validation DataLoader 0: 100%|██████████| 313/313 [00:02<00:00, 155.65it/s]
Epoch 0: 100%|██████████| 3/3 [00:02<00:00, 1.47it/s, v_num=0]
! calling anonymously, will miss private instances
Epoch 0: 100%|██████████| 3/3 [00:02<00:00, 1.22it/s, v_num=0]
Epoch 0: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 1: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 1: 33%|███▎ | 1/3 [00:00<00:00, 121.52it/s, v_num=0]
Epoch 1: 33%|███▎ | 1/3 [00:00<00:00, 113.11it/s, v_num=0]
Epoch 1: 67%|██████▋ | 2/3 [00:00<00:00, 125.24it/s, v_num=0]
Epoch 1: 67%|██████▋ | 2/3 [00:00<00:00, 120.70it/s, v_num=0]
Epoch 1: 100%|██████████| 3/3 [00:00<00:00, 125.40it/s, v_num=0]
Epoch 1: 100%|██████████| 3/3 [00:00<00:00, 122.68it/s, v_num=0]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: | | 0/? [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/313 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 1/313 [00:00<00:01, 187.95it/s]
Validation DataLoader 0: 1%| | 2/313 [00:00<00:01, 168.11it/s]
Validation DataLoader 0: 1%| | 3/313 [00:00<00:01, 167.19it/s]
Validation DataLoader 0: 1%|▏ | 4/313 [00:00<00:01, 164.15it/s]
Validation DataLoader 0: 2%|▏ | 5/313 [00:00<00:01, 163.57it/s]
Validation DataLoader 0: 2%|▏ | 6/313 [00:00<00:01, 164.12it/s]
Validation DataLoader 0: 2%|▏ | 7/313 [00:00<00:01, 164.28it/s]
Validation DataLoader 0: 3%|▎ | 8/313 [00:00<00:01, 163.71it/s]
Validation DataLoader 0: 3%|▎ | 9/313 [00:00<00:01, 163.24it/s]
Validation DataLoader 0: 3%|▎ | 10/313 [00:00<00:01, 163.44it/s]
Validation DataLoader 0: 4%|▎ | 11/313 [00:00<00:01, 162.12it/s]
Validation DataLoader 0: 4%|▍ | 12/313 [00:00<00:01, 161.88it/s]
Validation DataLoader 0: 4%|▍ | 13/313 [00:00<00:01, 162.33it/s]
Validation DataLoader 0: 4%|▍ | 14/313 [00:00<00:01, 162.64it/s]
Validation DataLoader 0: 5%|▍ | 15/313 [00:00<00:01, 161.94it/s]
Validation DataLoader 0: 5%|▌ | 16/313 [00:00<00:01, 162.10it/s]
Validation DataLoader 0: 5%|▌ | 17/313 [00:00<00:01, 161.81it/s]
Validation DataLoader 0: 6%|▌ | 18/313 [00:00<00:01, 161.43it/s]
Validation DataLoader 0: 6%|▌ | 19/313 [00:00<00:01, 160.92it/s]
Validation DataLoader 0: 6%|▋ | 20/313 [00:00<00:01, 161.08it/s]
Validation DataLoader 0: 7%|▋ | 21/313 [00:00<00:01, 161.20it/s]
Validation DataLoader 0: 7%|▋ | 22/313 [00:00<00:01, 160.89it/s]
Validation DataLoader 0: 7%|▋ | 23/313 [00:00<00:01, 161.12it/s]
Validation DataLoader 0: 8%|▊ | 24/313 [00:00<00:01, 160.97it/s]
Validation DataLoader 0: 8%|▊ | 25/313 [00:00<00:01, 161.01it/s]
Validation DataLoader 0: 8%|▊ | 26/313 [00:00<00:01, 161.02it/s]
Validation DataLoader 0: 9%|▊ | 27/313 [00:00<00:01, 160.62it/s]
Validation DataLoader 0: 9%|▉ | 28/313 [00:00<00:01, 160.56it/s]
Validation DataLoader 0: 9%|▉ | 29/313 [00:00<00:01, 160.55it/s]
Validation DataLoader 0: 10%|▉ | 30/313 [00:00<00:01, 160.55it/s]
Validation DataLoader 0: 10%|▉ | 31/313 [00:00<00:01, 160.60it/s]
Validation DataLoader 0: 10%|█ | 32/313 [00:00<00:01, 160.64it/s]
Validation DataLoader 0: 11%|█ | 33/313 [00:00<00:01, 160.59it/s]
Validation DataLoader 0: 11%|█ | 34/313 [00:00<00:01, 160.69it/s]
Validation DataLoader 0: 11%|█ | 35/313 [00:00<00:01, 160.68it/s]
Validation DataLoader 0: 12%|█▏ | 36/313 [00:00<00:01, 160.72it/s]
Validation DataLoader 0: 12%|█▏ | 37/313 [00:00<00:01, 160.58it/s]
Validation DataLoader 0: 12%|█▏ | 38/313 [00:00<00:01, 160.70it/s]
Validation DataLoader 0: 12%|█▏ | 39/313 [00:00<00:01, 160.83it/s]
Validation DataLoader 0: 13%|█▎ | 40/313 [00:00<00:01, 160.84it/s]
Validation DataLoader 0: 13%|█▎ | 41/313 [00:00<00:01, 161.01it/s]
Validation DataLoader 0: 13%|█▎ | 42/313 [00:00<00:01, 161.06it/s]
Validation DataLoader 0: 14%|█▎ | 43/313 [00:00<00:01, 161.14it/s]
Validation DataLoader 0: 14%|█▍ | 44/313 [00:00<00:01, 161.17it/s]
Validation DataLoader 0: 14%|█▍ | 45/313 [00:00<00:01, 161.25it/s]
Validation DataLoader 0: 15%|█▍ | 46/313 [00:00<00:01, 161.41it/s]
Validation DataLoader 0: 15%|█▌ | 47/313 [00:00<00:01, 161.43it/s]
Validation DataLoader 0: 15%|█▌ | 48/313 [00:00<00:01, 161.45it/s]
Validation DataLoader 0: 16%|█▌ | 49/313 [00:00<00:01, 161.52it/s]
Validation DataLoader 0: 16%|█▌ | 50/313 [00:00<00:01, 161.58it/s]
Validation DataLoader 0: 16%|█▋ | 51/313 [00:00<00:01, 161.63it/s]
Validation DataLoader 0: 17%|█▋ | 52/313 [00:00<00:01, 161.54it/s]
Validation DataLoader 0: 17%|█▋ | 53/313 [00:00<00:01, 161.47it/s]
Validation DataLoader 0: 17%|█▋ | 54/313 [00:00<00:01, 161.43it/s]
Validation DataLoader 0: 18%|█▊ | 55/313 [00:00<00:01, 161.26it/s]
Validation DataLoader 0: 18%|█▊ | 56/313 [00:00<00:01, 161.08it/s]
Validation DataLoader 0: 18%|█▊ | 57/313 [00:00<00:01, 160.98it/s]
Validation DataLoader 0: 19%|█▊ | 58/313 [00:00<00:01, 160.97it/s]
Validation DataLoader 0: 19%|█▉ | 59/313 [00:00<00:01, 160.77it/s]
Validation DataLoader 0: 19%|█▉ | 60/313 [00:00<00:01, 160.71it/s]
Validation DataLoader 0: 19%|█▉ | 61/313 [00:00<00:01, 160.54it/s]
Validation DataLoader 0: 20%|█▉ | 62/313 [00:00<00:01, 160.31it/s]
Validation DataLoader 0: 20%|██ | 63/313 [00:00<00:01, 159.90it/s]
Validation DataLoader 0: 20%|██ | 64/313 [00:00<00:01, 159.58it/s]
Validation DataLoader 0: 21%|██ | 65/313 [00:00<00:01, 159.67it/s]
Validation DataLoader 0: 21%|██ | 66/313 [00:00<00:01, 159.40it/s]
Validation DataLoader 0: 21%|██▏ | 67/313 [00:00<00:01, 159.36it/s]
Validation DataLoader 0: 22%|██▏ | 68/313 [00:00<00:01, 159.43it/s]
Validation DataLoader 0: 22%|██▏ | 69/313 [00:00<00:01, 159.39it/s]
Validation DataLoader 0: 22%|██▏ | 70/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 23%|██▎ | 71/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 23%|██▎ | 72/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 23%|██▎ | 73/313 [00:00<00:01, 159.34it/s]
Validation DataLoader 0: 24%|██▎ | 74/313 [00:00<00:01, 159.33it/s]
Validation DataLoader 0: 24%|██▍ | 75/313 [00:00<00:01, 159.33it/s]
Validation DataLoader 0: 24%|██▍ | 76/313 [00:00<00:01, 159.34it/s]
Validation DataLoader 0: 25%|██▍ | 77/313 [00:00<00:01, 159.40it/s]
Validation DataLoader 0: 25%|██▍ | 78/313 [00:00<00:01, 159.34it/s]
Validation DataLoader 0: 25%|██▌ | 79/313 [00:00<00:01, 159.22it/s]
Validation DataLoader 0: 26%|██▌ | 80/313 [00:00<00:01, 159.18it/s]
Validation DataLoader 0: 26%|██▌ | 81/313 [00:00<00:01, 159.23it/s]
Validation DataLoader 0: 26%|██▌ | 82/313 [00:00<00:01, 159.24it/s]
Validation DataLoader 0: 27%|██▋ | 83/313 [00:00<00:01, 159.27it/s]
Validation DataLoader 0: 27%|██▋ | 84/313 [00:00<00:01, 159.13it/s]
Validation DataLoader 0: 27%|██▋ | 85/313 [00:00<00:01, 158.86it/s]
Validation DataLoader 0: 27%|██▋ | 86/313 [00:00<00:01, 158.83it/s]
Validation DataLoader 0: 28%|██▊ | 87/313 [00:00<00:01, 158.82it/s]
Validation DataLoader 0: 28%|██▊ | 88/313 [00:00<00:01, 158.82it/s]
Validation DataLoader 0: 28%|██▊ | 89/313 [00:00<00:01, 158.85it/s]
Validation DataLoader 0: 29%|██▉ | 90/313 [00:00<00:01, 158.89it/s]
Validation DataLoader 0: 29%|██▉ | 91/313 [00:00<00:01, 158.94it/s]
Validation DataLoader 0: 29%|██▉ | 92/313 [00:00<00:01, 158.97it/s]
Validation DataLoader 0: 30%|██▉ | 93/313 [00:00<00:01, 158.96it/s]
Validation DataLoader 0: 30%|███ | 94/313 [00:00<00:01, 158.99it/s]
Validation DataLoader 0: 30%|███ | 95/313 [00:00<00:01, 158.99it/s]
Validation DataLoader 0: 31%|███ | 96/313 [00:00<00:01, 159.01it/s]
Validation DataLoader 0: 31%|███ | 97/313 [00:00<00:01, 159.02it/s]
Validation DataLoader 0: 31%|███▏ | 98/313 [00:00<00:01, 158.94it/s]
Validation DataLoader 0: 32%|███▏ | 99/313 [00:00<00:01, 158.86it/s]
Validation DataLoader 0: 32%|███▏ | 100/313 [00:00<00:01, 158.76it/s]
Validation DataLoader 0: 32%|███▏ | 101/313 [00:00<00:01, 158.63it/s]
Validation DataLoader 0: 33%|███▎ | 102/313 [00:00<00:01, 158.56it/s]
Validation DataLoader 0: 33%|███▎ | 103/313 [00:00<00:01, 158.51it/s]
Validation DataLoader 0: 33%|███▎ | 104/313 [00:00<00:01, 158.49it/s]
Validation DataLoader 0: 34%|███▎ | 105/313 [00:00<00:01, 158.48it/s]
Validation DataLoader 0: 34%|███▍ | 106/313 [00:00<00:01, 158.48it/s]
Validation DataLoader 0: 34%|███▍ | 107/313 [00:00<00:01, 157.33it/s]
Validation DataLoader 0: 35%|███▍ | 108/313 [00:00<00:01, 157.16it/s]
Validation DataLoader 0: 35%|███▍ | 109/313 [00:00<00:01, 157.03it/s]
Validation DataLoader 0: 35%|███▌ | 110/313 [00:00<00:01, 156.97it/s]
Validation DataLoader 0: 35%|███▌ | 111/313 [00:00<00:01, 156.96it/s]
Validation DataLoader 0: 36%|███▌ | 112/313 [00:00<00:01, 156.75it/s]
Validation DataLoader 0: 36%|███▌ | 113/313 [00:00<00:01, 156.52it/s]
Validation DataLoader 0: 36%|███▋ | 114/313 [00:00<00:01, 156.33it/s]
Validation DataLoader 0: 37%|███▋ | 115/313 [00:00<00:01, 156.08it/s]
Validation DataLoader 0: 37%|███▋ | 116/313 [00:00<00:01, 155.92it/s]
Validation DataLoader 0: 37%|███▋ | 117/313 [00:00<00:01, 155.72it/s]
Validation DataLoader 0: 38%|███▊ | 118/313 [00:00<00:01, 155.55it/s]
Validation DataLoader 0: 38%|███▊ | 119/313 [00:00<00:01, 155.39it/s]
Validation DataLoader 0: 38%|███▊ | 120/313 [00:00<00:01, 155.43it/s]
Validation DataLoader 0: 39%|███▊ | 121/313 [00:00<00:01, 155.48it/s]
Validation DataLoader 0: 39%|███▉ | 122/313 [00:00<00:01, 155.30it/s]
Validation DataLoader 0: 39%|███▉ | 123/313 [00:00<00:01, 155.37it/s]
Validation DataLoader 0: 40%|███▉ | 124/313 [00:00<00:01, 155.23it/s]
Validation DataLoader 0: 40%|███▉ | 125/313 [00:00<00:01, 155.10it/s]
Validation DataLoader 0: 40%|████ | 126/313 [00:00<00:01, 155.02it/s]
Validation DataLoader 0: 41%|████ | 127/313 [00:00<00:01, 155.06it/s]
Validation DataLoader 0: 41%|████ | 128/313 [00:00<00:01, 155.14it/s]
Validation DataLoader 0: 41%|████ | 129/313 [00:00<00:01, 155.22it/s]
Validation DataLoader 0: 42%|████▏ | 130/313 [00:00<00:01, 155.33it/s]
Validation DataLoader 0: 42%|████▏ | 131/313 [00:00<00:01, 155.41it/s]
Validation DataLoader 0: 42%|████▏ | 132/313 [00:00<00:01, 155.25it/s]
Validation DataLoader 0: 42%|████▏ | 133/313 [00:00<00:01, 155.20it/s]
Validation DataLoader 0: 43%|████▎ | 134/313 [00:00<00:01, 155.08it/s]
Validation DataLoader 0: 43%|████▎ | 135/313 [00:00<00:01, 155.16it/s]
Validation DataLoader 0: 43%|████▎ | 136/313 [00:00<00:01, 155.22it/s]
Validation DataLoader 0: 44%|████▍ | 137/313 [00:00<00:01, 155.09it/s]
Validation DataLoader 0: 44%|████▍ | 138/313 [00:00<00:01, 155.10it/s]
Validation DataLoader 0: 44%|████▍ | 139/313 [00:00<00:01, 155.17it/s]
Validation DataLoader 0: 45%|████▍ | 140/313 [00:00<00:01, 155.22it/s]
Validation DataLoader 0: 45%|████▌ | 141/313 [00:00<00:01, 155.11it/s]
Validation DataLoader 0: 45%|████▌ | 142/313 [00:00<00:01, 154.91it/s]
Validation DataLoader 0: 46%|████▌ | 143/313 [00:00<00:01, 154.82it/s]
Validation DataLoader 0: 46%|████▌ | 144/313 [00:00<00:01, 154.83it/s]
Validation DataLoader 0: 46%|████▋ | 145/313 [00:00<00:01, 154.86it/s]
Validation DataLoader 0: 47%|████▋ | 146/313 [00:00<00:01, 154.89it/s]
Validation DataLoader 0: 47%|████▋ | 147/313 [00:00<00:01, 154.88it/s]
Validation DataLoader 0: 47%|████▋ | 148/313 [00:00<00:01, 154.88it/s]
Validation DataLoader 0: 48%|████▊ | 149/313 [00:00<00:01, 154.89it/s]
Validation DataLoader 0: 48%|████▊ | 150/313 [00:00<00:01, 154.93it/s]
Validation DataLoader 0: 48%|████▊ | 151/313 [00:00<00:01, 154.94it/s]
Validation DataLoader 0: 49%|████▊ | 152/313 [00:00<00:01, 154.99it/s]
Validation DataLoader 0: 49%|████▉ | 153/313 [00:00<00:01, 155.02it/s]
Validation DataLoader 0: 49%|████▉ | 154/313 [00:00<00:01, 155.05it/s]
Validation DataLoader 0: 50%|████▉ | 155/313 [00:00<00:01, 155.09it/s]
Validation DataLoader 0: 50%|████▉ | 156/313 [00:01<00:01, 155.16it/s]
Validation DataLoader 0: 50%|█████ | 157/313 [00:01<00:01, 155.20it/s]
Validation DataLoader 0: 50%|█████ | 158/313 [00:01<00:00, 155.24it/s]
Validation DataLoader 0: 51%|█████ | 159/313 [00:01<00:00, 155.30it/s]
Validation DataLoader 0: 51%|█████ | 160/313 [00:01<00:00, 155.35it/s]
Validation DataLoader 0: 51%|█████▏ | 161/313 [00:01<00:00, 155.39it/s]
Validation DataLoader 0: 52%|█████▏ | 162/313 [00:01<00:00, 155.40it/s]
Validation DataLoader 0: 52%|█████▏ | 163/313 [00:01<00:00, 155.37it/s]
Validation DataLoader 0: 52%|█████▏ | 164/313 [00:01<00:00, 155.39it/s]
Validation DataLoader 0: 53%|█████▎ | 165/313 [00:01<00:00, 155.41it/s]
Validation DataLoader 0: 53%|█████▎ | 166/313 [00:01<00:00, 155.43it/s]
Validation DataLoader 0: 53%|█████▎ | 167/313 [00:01<00:00, 155.47it/s]
Validation DataLoader 0: 54%|█████▎ | 168/313 [00:01<00:00, 155.50it/s]
Validation DataLoader 0: 54%|█████▍ | 169/313 [00:01<00:00, 155.52it/s]
Validation DataLoader 0: 54%|█████▍ | 170/313 [00:01<00:00, 155.55it/s]
Validation DataLoader 0: 55%|█████▍ | 171/313 [00:01<00:00, 155.60it/s]
Validation DataLoader 0: 55%|█████▍ | 172/313 [00:01<00:00, 155.65it/s]
Validation DataLoader 0: 55%|█████▌ | 173/313 [00:01<00:00, 155.70it/s]
Validation DataLoader 0: 56%|█████▌ | 174/313 [00:01<00:00, 155.75it/s]
Validation DataLoader 0: 56%|█████▌ | 175/313 [00:01<00:00, 155.79it/s]
Validation DataLoader 0: 56%|█████▌ | 176/313 [00:01<00:00, 155.84it/s]
Validation DataLoader 0: 57%|█████▋ | 177/313 [00:01<00:00, 155.89it/s]
Validation DataLoader 0: 57%|█████▋ | 178/313 [00:01<00:00, 155.95it/s]
Validation DataLoader 0: 57%|█████▋ | 179/313 [00:01<00:00, 156.00it/s]
Validation DataLoader 0: 58%|█████▊ | 180/313 [00:01<00:00, 156.00it/s]
Validation DataLoader 0: 58%|█████▊ | 181/313 [00:01<00:00, 156.04it/s]
Validation DataLoader 0: 58%|█████▊ | 182/313 [00:01<00:00, 156.08it/s]
Validation DataLoader 0: 58%|█████▊ | 183/313 [00:01<00:00, 156.11it/s]
Validation DataLoader 0: 59%|█████▉ | 184/313 [00:01<00:00, 156.13it/s]
Validation DataLoader 0: 59%|█████▉ | 185/313 [00:01<00:00, 156.15it/s]
Validation DataLoader 0: 59%|█████▉ | 186/313 [00:01<00:00, 156.18it/s]
Validation DataLoader 0: 60%|█████▉ | 187/313 [00:01<00:00, 156.23it/s]
Validation DataLoader 0: 60%|██████ | 188/313 [00:01<00:00, 156.25it/s]
Validation DataLoader 0: 60%|██████ | 189/313 [00:01<00:00, 156.27it/s]
Validation DataLoader 0: 61%|██████ | 190/313 [00:01<00:00, 156.28it/s]
Validation DataLoader 0: 61%|██████ | 191/313 [00:01<00:00, 156.22it/s]
Validation DataLoader 0: 61%|██████▏ | 192/313 [00:01<00:00, 156.24it/s]
Validation DataLoader 0: 62%|██████▏ | 193/313 [00:01<00:00, 156.28it/s]
Validation DataLoader 0: 62%|██████▏ | 194/313 [00:01<00:00, 156.32it/s]
Validation DataLoader 0: 62%|██████▏ | 195/313 [00:01<00:00, 156.36it/s]
Validation DataLoader 0: 63%|██████▎ | 196/313 [00:01<00:00, 156.38it/s]
Validation DataLoader 0: 63%|██████▎ | 197/313 [00:01<00:00, 156.42it/s]
Validation DataLoader 0: 63%|██████▎ | 198/313 [00:01<00:00, 156.45it/s]
Validation DataLoader 0: 64%|██████▎ | 199/313 [00:01<00:00, 156.47it/s]
Validation DataLoader 0: 64%|██████▍ | 200/313 [00:01<00:00, 156.49it/s]
Validation DataLoader 0: 64%|██████▍ | 201/313 [00:01<00:00, 156.52it/s]
Validation DataLoader 0: 65%|██████▍ | 202/313 [00:01<00:00, 156.55it/s]
Validation DataLoader 0: 65%|██████▍ | 203/313 [00:01<00:00, 156.54it/s]
Validation DataLoader 0: 65%|██████▌ | 204/313 [00:01<00:00, 156.55it/s]
Validation DataLoader 0: 65%|██████▌ | 205/313 [00:01<00:00, 156.58it/s]
Validation DataLoader 0: 66%|██████▌ | 206/313 [00:01<00:00, 156.59it/s]
Validation DataLoader 0: 66%|██████▌ | 207/313 [00:01<00:00, 156.60it/s]
Validation DataLoader 0: 66%|██████▋ | 208/313 [00:01<00:00, 156.60it/s]
Validation DataLoader 0: 67%|██████▋ | 209/313 [00:01<00:00, 156.63it/s]
Validation DataLoader 0: 67%|██████▋ | 210/313 [00:01<00:00, 156.66it/s]
Validation DataLoader 0: 67%|██████▋ | 211/313 [00:01<00:00, 156.69it/s]
Validation DataLoader 0: 68%|██████▊ | 212/313 [00:01<00:00, 156.70it/s]
Validation DataLoader 0: 68%|██████▊ | 213/313 [00:01<00:00, 156.73it/s]
Validation DataLoader 0: 68%|██████▊ | 214/313 [00:01<00:00, 156.75it/s]
Validation DataLoader 0: 69%|██████▊ | 215/313 [00:01<00:00, 156.79it/s]
Validation DataLoader 0: 69%|██████▉ | 216/313 [00:01<00:00, 156.77it/s]
Validation DataLoader 0: 69%|██████▉ | 217/313 [00:01<00:00, 156.74it/s]
Validation DataLoader 0: 70%|██████▉ | 218/313 [00:01<00:00, 156.76it/s]
Validation DataLoader 0: 70%|██████▉ | 219/313 [00:01<00:00, 156.77it/s]
Validation DataLoader 0: 70%|███████ | 220/313 [00:01<00:00, 156.76it/s]
Validation DataLoader 0: 71%|███████ | 221/313 [00:01<00:00, 156.74it/s]
Validation DataLoader 0: 71%|███████ | 222/313 [00:01<00:00, 156.79it/s]
Validation DataLoader 0: 71%|███████ | 223/313 [00:01<00:00, 156.83it/s]
Validation DataLoader 0: 72%|███████▏ | 224/313 [00:01<00:00, 156.86it/s]
Validation DataLoader 0: 72%|███████▏ | 225/313 [00:01<00:00, 156.89it/s]
Validation DataLoader 0: 72%|███████▏ | 226/313 [00:01<00:00, 156.93it/s]
Validation DataLoader 0: 73%|███████▎ | 227/313 [00:01<00:00, 156.98it/s]
Validation DataLoader 0: 73%|███████▎ | 228/313 [00:01<00:00, 157.01it/s]
Validation DataLoader 0: 73%|███████▎ | 229/313 [00:01<00:00, 157.04it/s]
Validation DataLoader 0: 73%|███████▎ | 230/313 [00:01<00:00, 157.08it/s]
Validation DataLoader 0: 74%|███████▍ | 231/313 [00:01<00:00, 157.10it/s]
Validation DataLoader 0: 74%|███████▍ | 232/313 [00:01<00:00, 157.13it/s]
Validation DataLoader 0: 74%|███████▍ | 233/313 [00:01<00:00, 157.17it/s]
Validation DataLoader 0: 75%|███████▍ | 234/313 [00:01<00:00, 157.19it/s]
Validation DataLoader 0: 75%|███████▌ | 235/313 [00:01<00:00, 157.20it/s]
Validation DataLoader 0: 75%|███████▌ | 236/313 [00:01<00:00, 157.18it/s]
Validation DataLoader 0: 76%|███████▌ | 237/313 [00:01<00:00, 157.20it/s]
Validation DataLoader 0: 76%|███████▌ | 238/313 [00:01<00:00, 157.22it/s]
Validation DataLoader 0: 76%|███████▋ | 239/313 [00:01<00:00, 157.23it/s]
Validation DataLoader 0: 77%|███████▋ | 240/313 [00:01<00:00, 157.25it/s]
Validation DataLoader 0: 77%|███████▋ | 241/313 [00:01<00:00, 157.27it/s]
Validation DataLoader 0: 77%|███████▋ | 242/313 [00:01<00:00, 157.29it/s]
Validation DataLoader 0: 78%|███████▊ | 243/313 [00:01<00:00, 157.32it/s]
Validation DataLoader 0: 78%|███████▊ | 244/313 [00:01<00:00, 157.34it/s]
Validation DataLoader 0: 78%|███████▊ | 245/313 [00:01<00:00, 157.36it/s]
Validation DataLoader 0: 79%|███████▊ | 246/313 [00:01<00:00, 157.36it/s]
Validation DataLoader 0: 79%|███████▉ | 247/313 [00:01<00:00, 157.38it/s]
Validation DataLoader 0: 79%|███████▉ | 248/313 [00:01<00:00, 157.34it/s]
Validation DataLoader 0: 80%|███████▉ | 249/313 [00:01<00:00, 157.35it/s]
Validation DataLoader 0: 80%|███████▉ | 250/313 [00:01<00:00, 157.35it/s]
Validation DataLoader 0: 80%|████████ | 251/313 [00:01<00:00, 157.36it/s]
Validation DataLoader 0: 81%|████████ | 252/313 [00:01<00:00, 157.37it/s]
Validation DataLoader 0: 81%|████████ | 253/313 [00:01<00:00, 157.38it/s]
Validation DataLoader 0: 81%|████████ | 254/313 [00:01<00:00, 157.40it/s]
Validation DataLoader 0: 81%|████████▏ | 255/313 [00:01<00:00, 157.41it/s]
Validation DataLoader 0: 82%|████████▏ | 256/313 [00:01<00:00, 157.40it/s]
Validation DataLoader 0: 82%|████████▏ | 257/313 [00:01<00:00, 157.38it/s]
Validation DataLoader 0: 82%|████████▏ | 258/313 [00:01<00:00, 157.38it/s]
Validation DataLoader 0: 83%|████████▎ | 259/313 [00:01<00:00, 157.39it/s]
Validation DataLoader 0: 83%|████████▎ | 260/313 [00:01<00:00, 157.39it/s]
Validation DataLoader 0: 83%|████████▎ | 261/313 [00:01<00:00, 157.40it/s]
Validation DataLoader 0: 84%|████████▎ | 262/313 [00:01<00:00, 157.41it/s]
Validation DataLoader 0: 84%|████████▍ | 263/313 [00:01<00:00, 157.42it/s]
Validation DataLoader 0: 84%|████████▍ | 264/313 [00:01<00:00, 157.39it/s]
Validation DataLoader 0: 85%|████████▍ | 265/313 [00:01<00:00, 157.40it/s]
Validation DataLoader 0: 85%|████████▍ | 266/313 [00:01<00:00, 157.42it/s]
Validation DataLoader 0: 85%|████████▌ | 267/313 [00:01<00:00, 157.44it/s]
Validation DataLoader 0: 86%|████████▌ | 268/313 [00:01<00:00, 157.39it/s]
Validation DataLoader 0: 86%|████████▌ | 269/313 [00:01<00:00, 157.32it/s]
Validation DataLoader 0: 86%|████████▋ | 270/313 [00:01<00:00, 157.26it/s]
Validation DataLoader 0: 87%|████████▋ | 271/313 [00:01<00:00, 157.28it/s]
Validation DataLoader 0: 87%|████████▋ | 272/313 [00:01<00:00, 157.19it/s]
Validation DataLoader 0: 87%|████████▋ | 273/313 [00:01<00:00, 157.20it/s]
Validation DataLoader 0: 88%|████████▊ | 274/313 [00:01<00:00, 157.13it/s]
Validation DataLoader 0: 88%|████████▊ | 275/313 [00:01<00:00, 157.05it/s]
Validation DataLoader 0: 88%|████████▊ | 276/313 [00:01<00:00, 156.96it/s]
Validation DataLoader 0: 88%|████████▊ | 277/313 [00:01<00:00, 156.88it/s]
Validation DataLoader 0: 89%|████████▉ | 278/313 [00:01<00:00, 156.79it/s]
Validation DataLoader 0: 89%|████████▉ | 279/313 [00:01<00:00, 156.71it/s]
Validation DataLoader 0: 89%|████████▉ | 280/313 [00:01<00:00, 156.64it/s]
Validation DataLoader 0: 90%|████████▉ | 281/313 [00:01<00:00, 156.58it/s]
Validation DataLoader 0: 90%|█████████ | 282/313 [00:01<00:00, 156.48it/s]
Validation DataLoader 0: 90%|█████████ | 283/313 [00:01<00:00, 156.49it/s]
Validation DataLoader 0: 91%|█████████ | 284/313 [00:01<00:00, 156.45it/s]
Validation DataLoader 0: 91%|█████████ | 285/313 [00:01<00:00, 156.40it/s]
Validation DataLoader 0: 91%|█████████▏| 286/313 [00:01<00:00, 156.43it/s]
Validation DataLoader 0: 92%|█████████▏| 287/313 [00:01<00:00, 156.46it/s]
Validation DataLoader 0: 92%|█████████▏| 288/313 [00:01<00:00, 156.51it/s]
Validation DataLoader 0: 92%|█████████▏| 289/313 [00:01<00:00, 156.53it/s]
Validation DataLoader 0: 93%|█████████▎| 290/313 [00:01<00:00, 156.51it/s]
Validation DataLoader 0: 93%|█████████▎| 291/313 [00:01<00:00, 156.53it/s]
Validation DataLoader 0: 93%|█████████▎| 292/313 [00:01<00:00, 156.57it/s]
Validation DataLoader 0: 94%|█████████▎| 293/313 [00:01<00:00, 156.60it/s]
Validation DataLoader 0: 94%|█████████▍| 294/313 [00:01<00:00, 156.64it/s]
Validation DataLoader 0: 94%|█████████▍| 295/313 [00:01<00:00, 156.66it/s]
Validation DataLoader 0: 95%|█████████▍| 296/313 [00:01<00:00, 156.68it/s]
Validation DataLoader 0: 95%|█████████▍| 297/313 [00:01<00:00, 156.69it/s]
Validation DataLoader 0: 95%|█████████▌| 298/313 [00:01<00:00, 156.72it/s]
Validation DataLoader 0: 96%|█████████▌| 299/313 [00:01<00:00, 156.74it/s]
Validation DataLoader 0: 96%|█████████▌| 300/313 [00:01<00:00, 156.76it/s]
Validation DataLoader 0: 96%|█████████▌| 301/313 [00:01<00:00, 156.78it/s]
Validation DataLoader 0: 96%|█████████▋| 302/313 [00:01<00:00, 156.81it/s]
Validation DataLoader 0: 97%|█████████▋| 303/313 [00:01<00:00, 156.83it/s]
Validation DataLoader 0: 97%|█████████▋| 304/313 [00:01<00:00, 156.87it/s]
Validation DataLoader 0: 97%|█████████▋| 305/313 [00:01<00:00, 156.90it/s]
Validation DataLoader 0: 98%|█████████▊| 306/313 [00:01<00:00, 156.90it/s]
Validation DataLoader 0: 98%|█████████▊| 307/313 [00:01<00:00, 156.90it/s]
Validation DataLoader 0: 98%|█████████▊| 308/313 [00:01<00:00, 156.92it/s]
Validation DataLoader 0: 99%|█████████▊| 309/313 [00:01<00:00, 156.93it/s]
Validation DataLoader 0: 99%|█████████▉| 310/313 [00:01<00:00, 156.94it/s]
Validation DataLoader 0: 99%|█████████▉| 311/313 [00:01<00:00, 156.94it/s]
Validation DataLoader 0: 100%|█████████▉| 312/313 [00:01<00:00, 156.93it/s]
Validation DataLoader 0: 100%|██████████| 313/313 [00:01<00:00, 157.04it/s]
Epoch 1: 100%|██████████| 3/3 [00:02<00:00, 1.48it/s, v_num=0]
→ creating new artifact version for key='testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8bc.ckpt' (storage: '/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops')
Epoch 1: 100%|██████████| 3/3 [00:02<00:00, 1.45it/s, v_num=0]
Epoch 1: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 2: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 2: 33%|███▎ | 1/3 [00:00<00:00, 121.63it/s, v_num=0]
Epoch 2: 33%|███▎ | 1/3 [00:00<00:00, 112.96it/s, v_num=0]
Epoch 2: 67%|██████▋ | 2/3 [00:00<00:00, 124.19it/s, v_num=0]
Epoch 2: 67%|██████▋ | 2/3 [00:00<00:00, 119.79it/s, v_num=0]
Epoch 2: 100%|██████████| 3/3 [00:00<00:00, 125.16it/s, v_num=0]
Epoch 2: 100%|██████████| 3/3 [00:00<00:00, 121.21it/s, v_num=0]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: | | 0/? [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/313 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 1/313 [00:00<00:01, 215.72it/s]
Validation DataLoader 0: 1%| | 2/313 [00:00<00:01, 186.52it/s]
Validation DataLoader 0: 1%| | 3/313 [00:00<00:01, 177.54it/s]
Validation DataLoader 0: 1%|▏ | 4/313 [00:00<00:01, 172.01it/s]
Validation DataLoader 0: 2%|▏ | 5/313 [00:00<00:01, 170.15it/s]
Validation DataLoader 0: 2%|▏ | 6/313 [00:00<00:01, 168.79it/s]
Validation DataLoader 0: 2%|▏ | 7/313 [00:00<00:01, 167.49it/s]
Validation DataLoader 0: 3%|▎ | 8/313 [00:00<00:01, 166.92it/s]
Validation DataLoader 0: 3%|▎ | 9/313 [00:00<00:01, 166.51it/s]
Validation DataLoader 0: 3%|▎ | 10/313 [00:00<00:01, 166.31it/s]
Validation DataLoader 0: 4%|▎ | 11/313 [00:00<00:01, 165.73it/s]
Validation DataLoader 0: 4%|▍ | 12/313 [00:00<00:01, 164.34it/s]
Validation DataLoader 0: 4%|▍ | 13/313 [00:00<00:01, 164.05it/s]
Validation DataLoader 0: 4%|▍ | 14/313 [00:00<00:01, 163.40it/s]
Validation DataLoader 0: 5%|▍ | 15/313 [00:00<00:01, 160.79it/s]
Validation DataLoader 0: 5%|▌ | 16/313 [00:00<00:01, 158.87it/s]
Validation DataLoader 0: 5%|▌ | 17/313 [00:00<00:01, 158.38it/s]
Validation DataLoader 0: 6%|▌ | 18/313 [00:00<00:01, 158.66it/s]
Validation DataLoader 0: 6%|▌ | 19/313 [00:00<00:01, 158.75it/s]
Validation DataLoader 0: 6%|▋ | 20/313 [00:00<00:01, 159.05it/s]
Validation DataLoader 0: 7%|▋ | 21/313 [00:00<00:01, 159.28it/s]
Validation DataLoader 0: 7%|▋ | 22/313 [00:00<00:01, 159.61it/s]
Validation DataLoader 0: 7%|▋ | 23/313 [00:00<00:01, 159.65it/s]
Validation DataLoader 0: 8%|▊ | 24/313 [00:00<00:01, 158.61it/s]
Validation DataLoader 0: 8%|▊ | 25/313 [00:00<00:01, 159.08it/s]
Validation DataLoader 0: 8%|▊ | 26/313 [00:00<00:01, 159.25it/s]
Validation DataLoader 0: 9%|▊ | 27/313 [00:00<00:01, 159.49it/s]
Validation DataLoader 0: 9%|▉ | 28/313 [00:00<00:01, 159.40it/s]
Validation DataLoader 0: 9%|▉ | 29/313 [00:00<00:01, 159.48it/s]
Validation DataLoader 0: 10%|▉ | 30/313 [00:00<00:01, 159.31it/s]
Validation DataLoader 0: 10%|▉ | 31/313 [00:00<00:01, 159.48it/s]
Validation DataLoader 0: 10%|█ | 32/313 [00:00<00:01, 159.46it/s]
Validation DataLoader 0: 11%|█ | 33/313 [00:00<00:01, 159.07it/s]
Validation DataLoader 0: 11%|█ | 34/313 [00:00<00:01, 158.65it/s]
Validation DataLoader 0: 11%|█ | 35/313 [00:00<00:01, 158.53it/s]
Validation DataLoader 0: 12%|█▏ | 36/313 [00:00<00:01, 158.51it/s]
Validation DataLoader 0: 12%|█▏ | 37/313 [00:00<00:01, 158.50it/s]
Validation DataLoader 0: 12%|█▏ | 38/313 [00:00<00:01, 158.55it/s]
Validation DataLoader 0: 12%|█▏ | 39/313 [00:00<00:01, 158.58it/s]
Validation DataLoader 0: 13%|█▎ | 40/313 [00:00<00:01, 158.68it/s]
Validation DataLoader 0: 13%|█▎ | 41/313 [00:00<00:01, 158.75it/s]
Validation DataLoader 0: 13%|█▎ | 42/313 [00:00<00:01, 158.85it/s]
Validation DataLoader 0: 14%|█▎ | 43/313 [00:00<00:01, 158.89it/s]
Validation DataLoader 0: 14%|█▍ | 44/313 [00:00<00:01, 158.94it/s]
Validation DataLoader 0: 14%|█▍ | 45/313 [00:00<00:01, 158.94it/s]
Validation DataLoader 0: 15%|█▍ | 46/313 [00:00<00:01, 158.81it/s]
Validation DataLoader 0: 15%|█▌ | 47/313 [00:00<00:01, 158.85it/s]
Validation DataLoader 0: 15%|█▌ | 48/313 [00:00<00:01, 158.91it/s]
Validation DataLoader 0: 16%|█▌ | 49/313 [00:00<00:01, 158.97it/s]
Validation DataLoader 0: 16%|█▌ | 50/313 [00:00<00:01, 159.07it/s]
Validation DataLoader 0: 16%|█▋ | 51/313 [00:00<00:01, 159.23it/s]
Validation DataLoader 0: 17%|█▋ | 52/313 [00:00<00:01, 159.23it/s]
Validation DataLoader 0: 17%|█▋ | 53/313 [00:00<00:01, 159.16it/s]
Validation DataLoader 0: 17%|█▋ | 54/313 [00:00<00:01, 159.30it/s]
Validation DataLoader 0: 18%|█▊ | 55/313 [00:00<00:01, 159.38it/s]
Validation DataLoader 0: 18%|█▊ | 56/313 [00:00<00:01, 159.42it/s]
Validation DataLoader 0: 18%|█▊ | 57/313 [00:00<00:01, 159.33it/s]
Validation DataLoader 0: 19%|█▊ | 58/313 [00:00<00:01, 159.39it/s]
Validation DataLoader 0: 19%|█▉ | 59/313 [00:00<00:01, 159.47it/s]
Validation DataLoader 0: 19%|█▉ | 60/313 [00:00<00:01, 159.53it/s]
Validation DataLoader 0: 19%|█▉ | 61/313 [00:00<00:01, 159.60it/s]
Validation DataLoader 0: 20%|█▉ | 62/313 [00:00<00:01, 159.74it/s]
Validation DataLoader 0: 20%|██ | 63/313 [00:00<00:01, 159.84it/s]
Validation DataLoader 0: 20%|██ | 64/313 [00:00<00:01, 159.94it/s]
Validation DataLoader 0: 21%|██ | 65/313 [00:00<00:01, 159.98it/s]
Validation DataLoader 0: 21%|██ | 66/313 [00:00<00:01, 160.01it/s]
Validation DataLoader 0: 21%|██▏ | 67/313 [00:00<00:01, 160.00it/s]
Validation DataLoader 0: 22%|██▏ | 68/313 [00:00<00:01, 160.04it/s]
Validation DataLoader 0: 22%|██▏ | 69/313 [00:00<00:01, 160.09it/s]
Validation DataLoader 0: 22%|██▏ | 70/313 [00:00<00:01, 160.02it/s]
Validation DataLoader 0: 23%|██▎ | 71/313 [00:00<00:01, 160.02it/s]
Validation DataLoader 0: 23%|██▎ | 72/313 [00:00<00:01, 160.05it/s]
Validation DataLoader 0: 23%|██▎ | 73/313 [00:00<00:01, 160.08it/s]
Validation DataLoader 0: 24%|██▎ | 74/313 [00:00<00:01, 160.15it/s]
Validation DataLoader 0: 24%|██▍ | 75/313 [00:00<00:01, 160.21it/s]
Validation DataLoader 0: 24%|██▍ | 76/313 [00:00<00:01, 160.23it/s]
Validation DataLoader 0: 25%|██▍ | 77/313 [00:00<00:01, 160.26it/s]
Validation DataLoader 0: 25%|██▍ | 78/313 [00:00<00:01, 160.25it/s]
Validation DataLoader 0: 25%|██▌ | 79/313 [00:00<00:01, 160.30it/s]
Validation DataLoader 0: 26%|██▌ | 80/313 [00:00<00:01, 160.25it/s]
Validation DataLoader 0: 26%|██▌ | 81/313 [00:00<00:01, 160.27it/s]
Validation DataLoader 0: 26%|██▌ | 82/313 [00:00<00:01, 160.30it/s]
Validation DataLoader 0: 27%|██▋ | 83/313 [00:00<00:01, 160.30it/s]
Validation DataLoader 0: 27%|██▋ | 84/313 [00:00<00:01, 160.36it/s]
Validation DataLoader 0: 27%|██▋ | 85/313 [00:00<00:01, 160.40it/s]
Validation DataLoader 0: 27%|██▋ | 86/313 [00:00<00:01, 160.37it/s]
Validation DataLoader 0: 28%|██▊ | 87/313 [00:00<00:01, 160.44it/s]
Validation DataLoader 0: 28%|██▊ | 88/313 [00:00<00:01, 160.47it/s]
Validation DataLoader 0: 28%|██▊ | 89/313 [00:00<00:01, 160.50it/s]
Validation DataLoader 0: 29%|██▉ | 90/313 [00:00<00:01, 160.56it/s]
Validation DataLoader 0: 29%|██▉ | 91/313 [00:00<00:01, 160.60it/s]
Validation DataLoader 0: 29%|██▉ | 92/313 [00:00<00:01, 160.61it/s]
Validation DataLoader 0: 30%|██▉ | 93/313 [00:00<00:01, 160.68it/s]
Validation DataLoader 0: 30%|███ | 94/313 [00:00<00:01, 160.67it/s]
Validation DataLoader 0: 30%|███ | 95/313 [00:00<00:01, 160.64it/s]
Validation DataLoader 0: 31%|███ | 96/313 [00:00<00:01, 160.50it/s]
Validation DataLoader 0: 31%|███ | 97/313 [00:00<00:01, 160.47it/s]
Validation DataLoader 0: 31%|███▏ | 98/313 [00:00<00:01, 160.46it/s]
Validation DataLoader 0: 32%|███▏ | 99/313 [00:00<00:01, 160.22it/s]
Validation DataLoader 0: 32%|███▏ | 100/313 [00:00<00:01, 159.96it/s]
Validation DataLoader 0: 32%|███▏ | 101/313 [00:00<00:01, 159.82it/s]
Validation DataLoader 0: 33%|███▎ | 102/313 [00:00<00:01, 159.66it/s]
Validation DataLoader 0: 33%|███▎ | 103/313 [00:00<00:01, 159.49it/s]
Validation DataLoader 0: 33%|███▎ | 104/313 [00:00<00:01, 159.58it/s]
Validation DataLoader 0: 34%|███▎ | 105/313 [00:00<00:01, 159.60it/s]
Validation DataLoader 0: 34%|███▍ | 106/313 [00:00<00:01, 159.63it/s]
Validation DataLoader 0: 34%|███▍ | 107/313 [00:00<00:01, 159.65it/s]
Validation DataLoader 0: 35%|███▍ | 108/313 [00:00<00:01, 159.61it/s]
Validation DataLoader 0: 35%|███▍ | 109/313 [00:00<00:01, 159.58it/s]
Validation DataLoader 0: 35%|███▌ | 110/313 [00:00<00:01, 159.63it/s]
Validation DataLoader 0: 35%|███▌ | 111/313 [00:00<00:01, 159.61it/s]
Validation DataLoader 0: 36%|███▌ | 112/313 [00:00<00:01, 159.67it/s]
Validation DataLoader 0: 36%|███▌ | 113/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 36%|███▋ | 114/313 [00:00<00:01, 159.19it/s]
Validation DataLoader 0: 37%|███▋ | 115/313 [00:00<00:01, 158.98it/s]
Validation DataLoader 0: 37%|███▋ | 116/313 [00:00<00:01, 158.76it/s]
Validation DataLoader 0: 37%|███▋ | 117/313 [00:00<00:01, 158.67it/s]
Validation DataLoader 0: 38%|███▊ | 118/313 [00:00<00:01, 158.58it/s]
Validation DataLoader 0: 38%|███▊ | 119/313 [00:00<00:01, 158.62it/s]
Validation DataLoader 0: 38%|███▊ | 120/313 [00:00<00:01, 158.63it/s]
Validation DataLoader 0: 39%|███▊ | 121/313 [00:00<00:01, 158.66it/s]
Validation DataLoader 0: 39%|███▉ | 122/313 [00:00<00:01, 158.68it/s]
Validation DataLoader 0: 39%|███▉ | 123/313 [00:00<00:01, 158.49it/s]
Validation DataLoader 0: 40%|███▉ | 124/313 [00:00<00:01, 158.47it/s]
Validation DataLoader 0: 40%|███▉ | 125/313 [00:00<00:01, 158.47it/s]
Validation DataLoader 0: 40%|████ | 126/313 [00:00<00:01, 158.34it/s]
Validation DataLoader 0: 41%|████ | 127/313 [00:00<00:01, 158.31it/s]
Validation DataLoader 0: 41%|████ | 128/313 [00:00<00:01, 158.12it/s]
Validation DataLoader 0: 41%|████ | 129/313 [00:00<00:01, 157.99it/s]
Validation DataLoader 0: 42%|████▏ | 130/313 [00:00<00:01, 157.85it/s]
Validation DataLoader 0: 42%|████▏ | 131/313 [00:00<00:01, 157.81it/s]
Validation DataLoader 0: 42%|████▏ | 132/313 [00:00<00:01, 157.87it/s]
Validation DataLoader 0: 42%|████▏ | 133/313 [00:00<00:01, 157.93it/s]
Validation DataLoader 0: 43%|████▎ | 134/313 [00:00<00:01, 157.96it/s]
Validation DataLoader 0: 43%|████▎ | 135/313 [00:00<00:01, 158.00it/s]
Validation DataLoader 0: 43%|████▎ | 136/313 [00:00<00:01, 158.03it/s]
Validation DataLoader 0: 44%|████▍ | 137/313 [00:00<00:01, 158.04it/s]
Validation DataLoader 0: 44%|████▍ | 138/313 [00:00<00:01, 158.05it/s]
Validation DataLoader 0: 44%|████▍ | 139/313 [00:00<00:01, 158.05it/s]
Validation DataLoader 0: 45%|████▍ | 140/313 [00:00<00:01, 158.07it/s]
Validation DataLoader 0: 45%|████▌ | 141/313 [00:00<00:01, 158.11it/s]
Validation DataLoader 0: 45%|████▌ | 142/313 [00:00<00:01, 158.12it/s]
Validation DataLoader 0: 46%|████▌ | 143/313 [00:00<00:01, 158.16it/s]
Validation DataLoader 0: 46%|████▌ | 144/313 [00:00<00:01, 158.18it/s]
Validation DataLoader 0: 46%|████▋ | 145/313 [00:00<00:01, 158.20it/s]
Validation DataLoader 0: 47%|████▋ | 146/313 [00:00<00:01, 158.23it/s]
Validation DataLoader 0: 47%|████▋ | 147/313 [00:00<00:01, 158.27it/s]
Validation DataLoader 0: 47%|████▋ | 148/313 [00:00<00:01, 158.30it/s]
Validation DataLoader 0: 48%|████▊ | 149/313 [00:00<00:01, 158.32it/s]
Validation DataLoader 0: 48%|████▊ | 150/313 [00:00<00:01, 158.32it/s]
Validation DataLoader 0: 48%|████▊ | 151/313 [00:00<00:01, 158.34it/s]
Validation DataLoader 0: 49%|████▊ | 152/313 [00:00<00:01, 158.35it/s]
Validation DataLoader 0: 49%|████▉ | 153/313 [00:00<00:01, 158.37it/s]
Validation DataLoader 0: 49%|████▉ | 154/313 [00:00<00:01, 158.39it/s]
Validation DataLoader 0: 50%|████▉ | 155/313 [00:00<00:00, 158.41it/s]
Validation DataLoader 0: 50%|████▉ | 156/313 [00:00<00:00, 158.43it/s]
Validation DataLoader 0: 50%|█████ | 157/313 [00:00<00:00, 158.33it/s]
Validation DataLoader 0: 50%|█████ | 158/313 [00:00<00:00, 158.31it/s]
Validation DataLoader 0: 51%|█████ | 159/313 [00:01<00:00, 158.34it/s]
Validation DataLoader 0: 51%|█████ | 160/313 [00:01<00:00, 158.37it/s]
Validation DataLoader 0: 51%|█████▏ | 161/313 [00:01<00:00, 158.40it/s]
Validation DataLoader 0: 52%|█████▏ | 162/313 [00:01<00:00, 158.44it/s]
Validation DataLoader 0: 52%|█████▏ | 163/313 [00:01<00:00, 158.45it/s]
Validation DataLoader 0: 52%|█████▏ | 164/313 [00:01<00:00, 158.46it/s]
Validation DataLoader 0: 53%|█████▎ | 165/313 [00:01<00:00, 158.49it/s]
Validation DataLoader 0: 53%|█████▎ | 166/313 [00:01<00:00, 158.51it/s]
Validation DataLoader 0: 53%|█████▎ | 167/313 [00:01<00:00, 158.49it/s]
Validation DataLoader 0: 54%|█████▎ | 168/313 [00:01<00:00, 158.50it/s]
Validation DataLoader 0: 54%|█████▍ | 169/313 [00:01<00:00, 158.50it/s]
Validation DataLoader 0: 54%|█████▍ | 170/313 [00:01<00:00, 158.52it/s]
Validation DataLoader 0: 55%|█████▍ | 171/313 [00:01<00:00, 158.57it/s]
Validation DataLoader 0: 55%|█████▍ | 172/313 [00:01<00:00, 158.58it/s]
Validation DataLoader 0: 55%|█████▌ | 173/313 [00:01<00:00, 158.59it/s]
Validation DataLoader 0: 56%|█████▌ | 174/313 [00:01<00:00, 158.63it/s]
Validation DataLoader 0: 56%|█████▌ | 175/313 [00:01<00:00, 158.66it/s]
Validation DataLoader 0: 56%|█████▌ | 176/313 [00:01<00:00, 158.69it/s]
Validation DataLoader 0: 57%|█████▋ | 177/313 [00:01<00:00, 158.72it/s]
Validation DataLoader 0: 57%|█████▋ | 178/313 [00:01<00:00, 158.75it/s]
Validation DataLoader 0: 57%|█████▋ | 179/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 58%|█████▊ | 180/313 [00:01<00:00, 158.83it/s]
Validation DataLoader 0: 58%|█████▊ | 181/313 [00:01<00:00, 158.85it/s]
Validation DataLoader 0: 58%|█████▊ | 182/313 [00:01<00:00, 158.89it/s]
Validation DataLoader 0: 58%|█████▊ | 183/313 [00:01<00:00, 158.91it/s]
Validation DataLoader 0: 59%|█████▉ | 184/313 [00:01<00:00, 158.93it/s]
Validation DataLoader 0: 59%|█████▉ | 185/313 [00:01<00:00, 158.94it/s]
Validation DataLoader 0: 59%|█████▉ | 186/313 [00:01<00:00, 158.96it/s]
Validation DataLoader 0: 60%|█████▉ | 187/313 [00:01<00:00, 158.99it/s]
Validation DataLoader 0: 60%|██████ | 188/313 [00:01<00:00, 159.02it/s]
Validation DataLoader 0: 60%|██████ | 189/313 [00:01<00:00, 159.05it/s]
Validation DataLoader 0: 61%|██████ | 190/313 [00:01<00:00, 159.08it/s]
Validation DataLoader 0: 61%|██████ | 191/313 [00:01<00:00, 159.07it/s]
Validation DataLoader 0: 61%|██████▏ | 192/313 [00:01<00:00, 159.10it/s]
Validation DataLoader 0: 62%|██████▏ | 193/313 [00:01<00:00, 159.12it/s]
Validation DataLoader 0: 62%|██████▏ | 194/313 [00:01<00:00, 159.12it/s]
Validation DataLoader 0: 62%|██████▏ | 195/313 [00:01<00:00, 159.13it/s]
Validation DataLoader 0: 63%|██████▎ | 196/313 [00:01<00:00, 159.13it/s]
Validation DataLoader 0: 63%|██████▎ | 197/313 [00:01<00:00, 159.15it/s]
Validation DataLoader 0: 63%|██████▎ | 198/313 [00:01<00:00, 159.14it/s]
Validation DataLoader 0: 64%|██████▎ | 199/313 [00:01<00:00, 159.13it/s]
Validation DataLoader 0: 64%|██████▍ | 200/313 [00:01<00:00, 159.15it/s]
Validation DataLoader 0: 64%|██████▍ | 201/313 [00:01<00:00, 159.16it/s]
Validation DataLoader 0: 65%|██████▍ | 202/313 [00:01<00:00, 159.18it/s]
Validation DataLoader 0: 65%|██████▍ | 203/313 [00:01<00:00, 159.18it/s]
Validation DataLoader 0: 65%|██████▌ | 204/313 [00:01<00:00, 159.16it/s]
Validation DataLoader 0: 65%|██████▌ | 205/313 [00:01<00:00, 159.15it/s]
Validation DataLoader 0: 66%|██████▌ | 206/313 [00:01<00:00, 159.17it/s]
Validation DataLoader 0: 66%|██████▌ | 207/313 [00:01<00:00, 159.19it/s]
Validation DataLoader 0: 66%|██████▋ | 208/313 [00:01<00:00, 159.12it/s]
Validation DataLoader 0: 67%|██████▋ | 209/313 [00:01<00:00, 159.12it/s]
Validation DataLoader 0: 67%|██████▋ | 210/313 [00:01<00:00, 159.12it/s]
Validation DataLoader 0: 67%|██████▋ | 211/313 [00:01<00:00, 159.11it/s]
Validation DataLoader 0: 68%|██████▊ | 212/313 [00:01<00:00, 159.13it/s]
Validation DataLoader 0: 68%|██████▊ | 213/313 [00:01<00:00, 159.16it/s]
Validation DataLoader 0: 68%|██████▊ | 214/313 [00:01<00:00, 159.18it/s]
Validation DataLoader 0: 69%|██████▊ | 215/313 [00:01<00:00, 159.20it/s]
Validation DataLoader 0: 69%|██████▉ | 216/313 [00:01<00:00, 159.21it/s]
Validation DataLoader 0: 69%|██████▉ | 217/313 [00:01<00:00, 159.25it/s]
Validation DataLoader 0: 70%|██████▉ | 218/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 70%|██████▉ | 219/313 [00:01<00:00, 159.31it/s]
Validation DataLoader 0: 70%|███████ | 220/313 [00:01<00:00, 159.30it/s]
Validation DataLoader 0: 71%|███████ | 221/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 71%|███████ | 222/313 [00:01<00:00, 159.27it/s]
Validation DataLoader 0: 71%|███████ | 223/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 72%|███████▏ | 224/313 [00:01<00:00, 159.25it/s]
Validation DataLoader 0: 72%|███████▏ | 225/313 [00:01<00:00, 159.25it/s]
Validation DataLoader 0: 72%|███████▏ | 226/313 [00:01<00:00, 159.24it/s]
Validation DataLoader 0: 73%|███████▎ | 227/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 73%|███████▎ | 228/313 [00:01<00:00, 159.31it/s]
Validation DataLoader 0: 73%|███████▎ | 229/313 [00:01<00:00, 159.32it/s]
Validation DataLoader 0: 73%|███████▎ | 230/313 [00:01<00:00, 159.32it/s]
Validation DataLoader 0: 74%|███████▍ | 231/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 74%|███████▍ | 232/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 74%|███████▍ | 233/313 [00:01<00:00, 159.28it/s]
Validation DataLoader 0: 75%|███████▍ | 234/313 [00:01<00:00, 159.25it/s]
Validation DataLoader 0: 75%|███████▌ | 235/313 [00:01<00:00, 159.20it/s]
Validation DataLoader 0: 75%|███████▌ | 236/313 [00:01<00:00, 159.20it/s]
Validation DataLoader 0: 76%|███████▌ | 237/313 [00:01<00:00, 159.21it/s]
Validation DataLoader 0: 76%|███████▌ | 238/313 [00:01<00:00, 159.11it/s]
Validation DataLoader 0: 76%|███████▋ | 239/313 [00:01<00:00, 158.95it/s]
Validation DataLoader 0: 77%|███████▋ | 240/313 [00:01<00:00, 158.87it/s]
Validation DataLoader 0: 77%|███████▋ | 241/313 [00:01<00:00, 158.77it/s]
Validation DataLoader 0: 77%|███████▋ | 242/313 [00:01<00:00, 158.73it/s]
Validation DataLoader 0: 78%|███████▊ | 243/313 [00:01<00:00, 158.73it/s]
Validation DataLoader 0: 78%|███████▊ | 244/313 [00:01<00:00, 158.74it/s]
Validation DataLoader 0: 78%|███████▊ | 245/313 [00:01<00:00, 158.76it/s]
Validation DataLoader 0: 79%|███████▊ | 246/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 79%|███████▉ | 247/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 79%|███████▉ | 248/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 80%|███████▉ | 249/313 [00:01<00:00, 158.79it/s]
Validation DataLoader 0: 80%|███████▉ | 250/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 80%|████████ | 251/313 [00:01<00:00, 158.82it/s]
Validation DataLoader 0: 81%|████████ | 252/313 [00:01<00:00, 158.85it/s]
Validation DataLoader 0: 81%|████████ | 253/313 [00:01<00:00, 158.87it/s]
Validation DataLoader 0: 81%|████████ | 254/313 [00:01<00:00, 158.86it/s]
Validation DataLoader 0: 81%|████████▏ | 255/313 [00:01<00:00, 158.88it/s]
Validation DataLoader 0: 82%|████████▏ | 256/313 [00:01<00:00, 158.86it/s]
Validation DataLoader 0: 82%|████████▏ | 257/313 [00:01<00:00, 158.82it/s]
Validation DataLoader 0: 82%|████████▏ | 258/313 [00:01<00:00, 158.75it/s]
Validation DataLoader 0: 83%|████████▎ | 259/313 [00:01<00:00, 158.66it/s]
Validation DataLoader 0: 83%|████████▎ | 260/313 [00:01<00:00, 158.60it/s]
Validation DataLoader 0: 83%|████████▎ | 261/313 [00:01<00:00, 158.60it/s]
Validation DataLoader 0: 84%|████████▎ | 262/313 [00:01<00:00, 158.61it/s]
Validation DataLoader 0: 84%|████████▍ | 263/313 [00:01<00:00, 158.62it/s]
Validation DataLoader 0: 84%|████████▍ | 264/313 [00:01<00:00, 158.62it/s]
Validation DataLoader 0: 85%|████████▍ | 265/313 [00:01<00:00, 158.62it/s]
Validation DataLoader 0: 85%|████████▍ | 266/313 [00:01<00:00, 158.64it/s]
Validation DataLoader 0: 85%|████████▌ | 267/313 [00:01<00:00, 158.64it/s]
Validation DataLoader 0: 86%|████████▌ | 268/313 [00:01<00:00, 158.67it/s]
Validation DataLoader 0: 86%|████████▌ | 269/313 [00:01<00:00, 158.69it/s]
Validation DataLoader 0: 86%|████████▋ | 270/313 [00:01<00:00, 158.72it/s]
Validation DataLoader 0: 87%|████████▋ | 271/313 [00:01<00:00, 158.70it/s]
Validation DataLoader 0: 87%|████████▋ | 272/313 [00:01<00:00, 158.70it/s]
Validation DataLoader 0: 87%|████████▋ | 273/313 [00:01<00:00, 158.71it/s]
Validation DataLoader 0: 88%|████████▊ | 274/313 [00:01<00:00, 158.74it/s]
Validation DataLoader 0: 88%|████████▊ | 275/313 [00:01<00:00, 158.75it/s]
Validation DataLoader 0: 88%|████████▊ | 276/313 [00:01<00:00, 158.77it/s]
Validation DataLoader 0: 88%|████████▊ | 277/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 89%|████████▉ | 278/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 89%|████████▉ | 279/313 [00:01<00:00, 158.79it/s]
Validation DataLoader 0: 89%|████████▉ | 280/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 90%|████████▉ | 281/313 [00:01<00:00, 158.81it/s]
Validation DataLoader 0: 90%|█████████ | 282/313 [00:01<00:00, 158.75it/s]
Validation DataLoader 0: 90%|█████████ | 283/313 [00:01<00:00, 158.77it/s]
Validation DataLoader 0: 91%|█████████ | 284/313 [00:01<00:00, 158.79it/s]
Validation DataLoader 0: 91%|█████████ | 285/313 [00:01<00:00, 158.81it/s]
Validation DataLoader 0: 91%|█████████▏| 286/313 [00:01<00:00, 158.74it/s]
Validation DataLoader 0: 92%|█████████▏| 287/313 [00:01<00:00, 158.66it/s]
Validation DataLoader 0: 92%|█████████▏| 288/313 [00:01<00:00, 158.63it/s]
Validation DataLoader 0: 92%|█████████▏| 289/313 [00:01<00:00, 158.62it/s]
Validation DataLoader 0: 93%|█████████▎| 290/313 [00:01<00:00, 158.62it/s]
Validation DataLoader 0: 93%|█████████▎| 291/313 [00:01<00:00, 158.63it/s]
Validation DataLoader 0: 93%|█████████▎| 292/313 [00:01<00:00, 158.65it/s]
Validation DataLoader 0: 94%|█████████▎| 293/313 [00:01<00:00, 158.67it/s]
Validation DataLoader 0: 94%|█████████▍| 294/313 [00:01<00:00, 158.69it/s]
Validation DataLoader 0: 94%|█████████▍| 295/313 [00:01<00:00, 158.70it/s]
Validation DataLoader 0: 95%|█████████▍| 296/313 [00:01<00:00, 158.71it/s]
Validation DataLoader 0: 95%|█████████▍| 297/313 [00:01<00:00, 158.73it/s]
Validation DataLoader 0: 95%|█████████▌| 298/313 [00:01<00:00, 158.74it/s]
Validation DataLoader 0: 96%|█████████▌| 299/313 [00:01<00:00, 158.76it/s]
Validation DataLoader 0: 96%|█████████▌| 300/313 [00:01<00:00, 158.77it/s]
Validation DataLoader 0: 96%|█████████▌| 301/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 96%|█████████▋| 302/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 97%|█████████▋| 303/313 [00:01<00:00, 158.79it/s]
Validation DataLoader 0: 97%|█████████▋| 304/313 [00:01<00:00, 158.78it/s]
Validation DataLoader 0: 97%|█████████▋| 305/313 [00:01<00:00, 158.77it/s]
Validation DataLoader 0: 98%|█████████▊| 306/313 [00:01<00:00, 158.76it/s]
Validation DataLoader 0: 98%|█████████▊| 307/313 [00:01<00:00, 158.75it/s]
Validation DataLoader 0: 98%|█████████▊| 308/313 [00:01<00:00, 158.76it/s]
Validation DataLoader 0: 99%|█████████▊| 309/313 [00:01<00:00, 158.76it/s]
Validation DataLoader 0: 99%|█████████▉| 310/313 [00:01<00:00, 158.79it/s]
Validation DataLoader 0: 99%|█████████▉| 311/313 [00:01<00:00, 158.80it/s]
Validation DataLoader 0: 100%|█████████▉| 312/313 [00:01<00:00, 158.81it/s]
Validation DataLoader 0: 100%|██████████| 313/313 [00:01<00:00, 158.95it/s]
Epoch 2: 100%|██████████| 3/3 [00:02<00:00, 1.50it/s, v_num=0]
→ creating new artifact version for key='testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8bc.ckpt' (storage: '/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops')
Epoch 2: 100%|██████████| 3/3 [00:02<00:00, 1.47it/s, v_num=0]
Epoch 2: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 3: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 3: 33%|███▎ | 1/3 [00:00<00:00, 119.02it/s, v_num=0]
Epoch 3: 33%|███▎ | 1/3 [00:00<00:00, 109.04it/s, v_num=0]
Epoch 3: 67%|██████▋ | 2/3 [00:00<00:00, 118.39it/s, v_num=0]
Epoch 3: 67%|██████▋ | 2/3 [00:00<00:00, 114.82it/s, v_num=0]
Epoch 3: 100%|██████████| 3/3 [00:00<00:00, 117.27it/s, v_num=0]
Epoch 3: 100%|██████████| 3/3 [00:00<00:00, 114.73it/s, v_num=0]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: | | 0/? [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/313 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 1/313 [00:00<00:01, 180.39it/s]
Validation DataLoader 0: 1%| | 2/313 [00:00<00:01, 170.81it/s]
Validation DataLoader 0: 1%| | 3/313 [00:00<00:01, 168.89it/s]
Validation DataLoader 0: 1%|▏ | 4/313 [00:00<00:01, 165.22it/s]
Validation DataLoader 0: 2%|▏ | 5/313 [00:00<00:01, 164.55it/s]
Validation DataLoader 0: 2%|▏ | 6/313 [00:00<00:01, 157.99it/s]
Validation DataLoader 0: 2%|▏ | 7/313 [00:00<00:01, 158.85it/s]
Validation DataLoader 0: 3%|▎ | 8/313 [00:00<00:01, 159.06it/s]
Validation DataLoader 0: 3%|▎ | 9/313 [00:00<00:01, 157.29it/s]
Validation DataLoader 0: 3%|▎ | 10/313 [00:00<00:01, 157.26it/s]
Validation DataLoader 0: 4%|▎ | 11/313 [00:00<00:01, 156.86it/s]
Validation DataLoader 0: 4%|▍ | 12/313 [00:00<00:01, 156.00it/s]
Validation DataLoader 0: 4%|▍ | 13/313 [00:00<00:01, 155.01it/s]
Validation DataLoader 0: 4%|▍ | 14/313 [00:00<00:01, 154.96it/s]
Validation DataLoader 0: 5%|▍ | 15/313 [00:00<00:01, 155.14it/s]
Validation DataLoader 0: 5%|▌ | 16/313 [00:00<00:01, 155.28it/s]
Validation DataLoader 0: 5%|▌ | 17/313 [00:00<00:01, 155.51it/s]
Validation DataLoader 0: 6%|▌ | 18/313 [00:00<00:01, 155.68it/s]
Validation DataLoader 0: 6%|▌ | 19/313 [00:00<00:01, 156.06it/s]
Validation DataLoader 0: 6%|▋ | 20/313 [00:00<00:01, 156.41it/s]
Validation DataLoader 0: 7%|▋ | 21/313 [00:00<00:01, 156.88it/s]
Validation DataLoader 0: 7%|▋ | 22/313 [00:00<00:01, 157.00it/s]
Validation DataLoader 0: 7%|▋ | 23/313 [00:00<00:01, 157.41it/s]
Validation DataLoader 0: 8%|▊ | 24/313 [00:00<00:01, 157.73it/s]
Validation DataLoader 0: 8%|▊ | 25/313 [00:00<00:01, 157.77it/s]
Validation DataLoader 0: 8%|▊ | 26/313 [00:00<00:01, 157.84it/s]
Validation DataLoader 0: 9%|▊ | 27/313 [00:00<00:01, 157.86it/s]
Validation DataLoader 0: 9%|▉ | 28/313 [00:00<00:01, 158.01it/s]
Validation DataLoader 0: 9%|▉ | 29/313 [00:00<00:01, 158.04it/s]
Validation DataLoader 0: 10%|▉ | 30/313 [00:00<00:01, 158.08it/s]
Validation DataLoader 0: 10%|▉ | 31/313 [00:00<00:01, 158.16it/s]
Validation DataLoader 0: 10%|█ | 32/313 [00:00<00:01, 158.27it/s]
Validation DataLoader 0: 11%|█ | 33/313 [00:00<00:01, 158.44it/s]
Validation DataLoader 0: 11%|█ | 34/313 [00:00<00:01, 158.49it/s]
Validation DataLoader 0: 11%|█ | 35/313 [00:00<00:01, 158.62it/s]
Validation DataLoader 0: 12%|█▏ | 36/313 [00:00<00:01, 158.79it/s]
Validation DataLoader 0: 12%|█▏ | 37/313 [00:00<00:01, 159.01it/s]
Validation DataLoader 0: 12%|█▏ | 38/313 [00:00<00:01, 159.06it/s]
Validation DataLoader 0: 12%|█▏ | 39/313 [00:00<00:01, 159.17it/s]
Validation DataLoader 0: 13%|█▎ | 40/313 [00:00<00:01, 158.40it/s]
Validation DataLoader 0: 13%|█▎ | 41/313 [00:00<00:01, 158.46it/s]
Validation DataLoader 0: 13%|█▎ | 42/313 [00:00<00:01, 158.04it/s]
Validation DataLoader 0: 14%|█▎ | 43/313 [00:00<00:01, 157.89it/s]
Validation DataLoader 0: 14%|█▍ | 44/313 [00:00<00:01, 157.93it/s]
Validation DataLoader 0: 14%|█▍ | 45/313 [00:00<00:01, 158.10it/s]
Validation DataLoader 0: 15%|█▍ | 46/313 [00:00<00:01, 158.16it/s]
Validation DataLoader 0: 15%|█▌ | 47/313 [00:00<00:01, 158.31it/s]
Validation DataLoader 0: 15%|█▌ | 48/313 [00:00<00:01, 158.47it/s]
Validation DataLoader 0: 16%|█▌ | 49/313 [00:00<00:01, 158.52it/s]
Validation DataLoader 0: 16%|█▌ | 50/313 [00:00<00:01, 158.34it/s]
Validation DataLoader 0: 16%|█▋ | 51/313 [00:00<00:01, 157.97it/s]
Validation DataLoader 0: 17%|█▋ | 52/313 [00:00<00:01, 157.94it/s]
Validation DataLoader 0: 17%|█▋ | 53/313 [00:00<00:01, 158.07it/s]
Validation DataLoader 0: 17%|█▋ | 54/313 [00:00<00:01, 157.84it/s]
Validation DataLoader 0: 18%|█▊ | 55/313 [00:00<00:01, 157.99it/s]
Validation DataLoader 0: 18%|█▊ | 56/313 [00:00<00:01, 157.69it/s]
Validation DataLoader 0: 18%|█▊ | 57/313 [00:00<00:01, 157.77it/s]
Validation DataLoader 0: 19%|█▊ | 58/313 [00:00<00:01, 157.53it/s]
Validation DataLoader 0: 19%|█▉ | 59/313 [00:00<00:01, 157.60it/s]
Validation DataLoader 0: 19%|█▉ | 60/313 [00:00<00:01, 157.58it/s]
Validation DataLoader 0: 19%|█▉ | 61/313 [00:00<00:01, 157.24it/s]
Validation DataLoader 0: 20%|█▉ | 62/313 [00:00<00:01, 156.96it/s]
Validation DataLoader 0: 20%|██ | 63/313 [00:00<00:01, 156.61it/s]
Validation DataLoader 0: 20%|██ | 64/313 [00:00<00:01, 156.35it/s]
Validation DataLoader 0: 21%|██ | 65/313 [00:00<00:01, 156.06it/s]
Validation DataLoader 0: 21%|██ | 66/313 [00:00<00:01, 155.98it/s]
Validation DataLoader 0: 21%|██▏ | 67/313 [00:00<00:01, 156.03it/s]
Validation DataLoader 0: 22%|██▏ | 68/313 [00:00<00:01, 155.75it/s]
Validation DataLoader 0: 22%|██▏ | 69/313 [00:00<00:01, 155.67it/s]
Validation DataLoader 0: 22%|██▏ | 70/313 [00:00<00:01, 155.76it/s]
Validation DataLoader 0: 23%|██▎ | 71/313 [00:00<00:01, 155.76it/s]
Validation DataLoader 0: 23%|██▎ | 72/313 [00:00<00:01, 155.81it/s]
Validation DataLoader 0: 23%|██▎ | 73/313 [00:00<00:01, 155.78it/s]
Validation DataLoader 0: 24%|██▎ | 74/313 [00:00<00:01, 155.52it/s]
Validation DataLoader 0: 24%|██▍ | 75/313 [00:00<00:01, 155.55it/s]
Validation DataLoader 0: 24%|██▍ | 76/313 [00:00<00:01, 155.27it/s]
Validation DataLoader 0: 25%|██▍ | 77/313 [00:00<00:01, 155.23it/s]
Validation DataLoader 0: 25%|██▍ | 78/313 [00:00<00:01, 155.02it/s]
Validation DataLoader 0: 25%|██▌ | 79/313 [00:00<00:01, 154.81it/s]
Validation DataLoader 0: 26%|██▌ | 80/313 [00:00<00:01, 154.70it/s]
Validation DataLoader 0: 26%|██▌ | 81/313 [00:00<00:01, 154.54it/s]
Validation DataLoader 0: 26%|██▌ | 82/313 [00:00<00:01, 154.38it/s]
Validation DataLoader 0: 27%|██▋ | 83/313 [00:00<00:01, 154.11it/s]
Validation DataLoader 0: 27%|██▋ | 84/313 [00:00<00:01, 153.94it/s]
Validation DataLoader 0: 27%|██▋ | 85/313 [00:00<00:01, 154.00it/s]
Validation DataLoader 0: 27%|██▋ | 86/313 [00:00<00:01, 153.85it/s]
Validation DataLoader 0: 28%|██▊ | 87/313 [00:00<00:01, 153.71it/s]
Validation DataLoader 0: 28%|██▊ | 88/313 [00:00<00:01, 153.74it/s]
Validation DataLoader 0: 28%|██▊ | 89/313 [00:00<00:01, 153.73it/s]
Validation DataLoader 0: 29%|██▉ | 90/313 [00:00<00:01, 153.61it/s]
Validation DataLoader 0: 29%|██▉ | 91/313 [00:00<00:01, 153.65it/s]
Validation DataLoader 0: 29%|██▉ | 92/313 [00:00<00:01, 153.73it/s]
Validation DataLoader 0: 30%|██▉ | 93/313 [00:00<00:01, 153.56it/s]
Validation DataLoader 0: 30%|███ | 94/313 [00:00<00:01, 153.62it/s]
Validation DataLoader 0: 30%|███ | 95/313 [00:00<00:01, 153.40it/s]
Validation DataLoader 0: 31%|███ | 96/313 [00:00<00:01, 153.25it/s]
Validation DataLoader 0: 31%|███ | 97/313 [00:00<00:01, 153.11it/s]
Validation DataLoader 0: 31%|███▏ | 98/313 [00:00<00:01, 153.06it/s]
Validation DataLoader 0: 32%|███▏ | 99/313 [00:00<00:01, 153.15it/s]
Validation DataLoader 0: 32%|███▏ | 100/313 [00:00<00:01, 153.02it/s]
Validation DataLoader 0: 32%|███▏ | 101/313 [00:00<00:01, 152.93it/s]
Validation DataLoader 0: 33%|███▎ | 102/313 [00:00<00:01, 152.84it/s]
Validation DataLoader 0: 33%|███▎ | 103/313 [00:00<00:01, 152.73it/s]
Validation DataLoader 0: 33%|███▎ | 104/313 [00:00<00:01, 152.54it/s]
Validation DataLoader 0: 34%|███▎ | 105/313 [00:00<00:01, 152.61it/s]
Validation DataLoader 0: 34%|███▍ | 106/313 [00:00<00:01, 152.47it/s]
Validation DataLoader 0: 34%|███▍ | 107/313 [00:00<00:01, 152.35it/s]
Validation DataLoader 0: 35%|███▍ | 108/313 [00:00<00:01, 152.21it/s]
Validation DataLoader 0: 35%|███▍ | 109/313 [00:00<00:01, 152.17it/s]
Validation DataLoader 0: 35%|███▌ | 110/313 [00:00<00:01, 152.10it/s]
Validation DataLoader 0: 35%|███▌ | 111/313 [00:00<00:01, 152.01it/s]
Validation DataLoader 0: 36%|███▌ | 112/313 [00:00<00:01, 151.94it/s]
Validation DataLoader 0: 36%|███▌ | 113/313 [00:00<00:01, 151.98it/s]
Validation DataLoader 0: 36%|███▋ | 114/313 [00:00<00:01, 151.91it/s]
Validation DataLoader 0: 37%|███▋ | 115/313 [00:00<00:01, 151.79it/s]
Validation DataLoader 0: 37%|███▋ | 116/313 [00:00<00:01, 151.65it/s]
Validation DataLoader 0: 37%|███▋ | 117/313 [00:00<00:01, 151.73it/s]
Validation DataLoader 0: 38%|███▊ | 118/313 [00:00<00:01, 151.64it/s]
Validation DataLoader 0: 38%|███▊ | 119/313 [00:00<00:01, 151.57it/s]
Validation DataLoader 0: 38%|███▊ | 120/313 [00:00<00:01, 151.65it/s]
Validation DataLoader 0: 39%|███▊ | 121/313 [00:00<00:01, 151.53it/s]
Validation DataLoader 0: 39%|███▉ | 122/313 [00:00<00:01, 151.57it/s]
Validation DataLoader 0: 39%|███▉ | 123/313 [00:00<00:01, 151.59it/s]
Validation DataLoader 0: 40%|███▉ | 124/313 [00:00<00:01, 151.52it/s]
Validation DataLoader 0: 40%|███▉ | 125/313 [00:00<00:01, 151.47it/s]
Validation DataLoader 0: 40%|████ | 126/313 [00:00<00:01, 151.55it/s]
Validation DataLoader 0: 41%|████ | 127/313 [00:00<00:01, 151.59it/s]
Validation DataLoader 0: 41%|████ | 128/313 [00:00<00:01, 151.66it/s]
Validation DataLoader 0: 41%|████ | 129/313 [00:00<00:01, 151.70it/s]
Validation DataLoader 0: 42%|████▏ | 130/313 [00:00<00:01, 151.75it/s]
Validation DataLoader 0: 42%|████▏ | 131/313 [00:00<00:01, 151.83it/s]
Validation DataLoader 0: 42%|████▏ | 132/313 [00:00<00:01, 151.88it/s]
Validation DataLoader 0: 42%|████▏ | 133/313 [00:00<00:01, 151.98it/s]
Validation DataLoader 0: 43%|████▎ | 134/313 [00:00<00:01, 152.05it/s]
Validation DataLoader 0: 43%|████▎ | 135/313 [00:00<00:01, 152.14it/s]
Validation DataLoader 0: 43%|████▎ | 136/313 [00:00<00:01, 152.23it/s]
Validation DataLoader 0: 44%|████▍ | 137/313 [00:00<00:01, 152.32it/s]
Validation DataLoader 0: 44%|████▍ | 138/313 [00:00<00:01, 152.36it/s]
Validation DataLoader 0: 44%|████▍ | 139/313 [00:00<00:01, 152.43it/s]
Validation DataLoader 0: 45%|████▍ | 140/313 [00:00<00:01, 152.48it/s]
Validation DataLoader 0: 45%|████▌ | 141/313 [00:00<00:01, 152.52it/s]
Validation DataLoader 0: 45%|████▌ | 142/313 [00:00<00:01, 152.37it/s]
Validation DataLoader 0: 46%|████▌ | 143/313 [00:00<00:01, 152.40it/s]
Validation DataLoader 0: 46%|████▌ | 144/313 [00:00<00:01, 152.29it/s]
Validation DataLoader 0: 46%|████▋ | 145/313 [00:00<00:01, 152.16it/s]
Validation DataLoader 0: 47%|████▋ | 146/313 [00:00<00:01, 152.03it/s]
Validation DataLoader 0: 47%|████▋ | 147/313 [00:00<00:01, 151.98it/s]
Validation DataLoader 0: 47%|████▋ | 148/313 [00:00<00:01, 152.01it/s]
Validation DataLoader 0: 48%|████▊ | 149/313 [00:00<00:01, 152.08it/s]
Validation DataLoader 0: 48%|████▊ | 150/313 [00:00<00:01, 152.14it/s]
Validation DataLoader 0: 48%|████▊ | 151/313 [00:00<00:01, 152.20it/s]
Validation DataLoader 0: 49%|████▊ | 152/313 [00:00<00:01, 152.27it/s]
Validation DataLoader 0: 49%|████▉ | 153/313 [00:01<00:01, 152.37it/s]
Validation DataLoader 0: 49%|████▉ | 154/313 [00:01<00:01, 152.42it/s]
Validation DataLoader 0: 50%|████▉ | 155/313 [00:01<00:01, 152.32it/s]
Validation DataLoader 0: 50%|████▉ | 156/313 [00:01<00:01, 152.25it/s]
Validation DataLoader 0: 50%|█████ | 157/313 [00:01<00:01, 152.12it/s]
Validation DataLoader 0: 50%|█████ | 158/313 [00:01<00:01, 152.04it/s]
Validation DataLoader 0: 51%|█████ | 159/313 [00:01<00:01, 151.95it/s]
Validation DataLoader 0: 51%|█████ | 160/313 [00:01<00:01, 151.93it/s]
Validation DataLoader 0: 51%|█████▏ | 161/313 [00:01<00:01, 151.95it/s]
Validation DataLoader 0: 52%|█████▏ | 162/313 [00:01<00:00, 151.97it/s]
Validation DataLoader 0: 52%|█████▏ | 163/313 [00:01<00:00, 152.03it/s]
Validation DataLoader 0: 52%|█████▏ | 164/313 [00:01<00:00, 152.09it/s]
Validation DataLoader 0: 53%|█████▎ | 165/313 [00:01<00:00, 151.94it/s]
Validation DataLoader 0: 53%|█████▎ | 166/313 [00:01<00:00, 151.85it/s]
Validation DataLoader 0: 53%|█████▎ | 167/313 [00:01<00:00, 151.80it/s]
Validation DataLoader 0: 54%|█████▎ | 168/313 [00:01<00:00, 151.74it/s]
Validation DataLoader 0: 54%|█████▍ | 169/313 [00:01<00:00, 151.70it/s]
Validation DataLoader 0: 54%|█████▍ | 170/313 [00:01<00:00, 151.75it/s]
Validation DataLoader 0: 55%|█████▍ | 171/313 [00:01<00:00, 151.66it/s]
Validation DataLoader 0: 55%|█████▍ | 172/313 [00:01<00:00, 151.67it/s]
Validation DataLoader 0: 55%|█████▌ | 173/313 [00:01<00:00, 151.74it/s]
Validation DataLoader 0: 56%|█████▌ | 174/313 [00:01<00:00, 151.79it/s]
Validation DataLoader 0: 56%|█████▌ | 175/313 [00:01<00:00, 151.85it/s]
Validation DataLoader 0: 56%|█████▌ | 176/313 [00:01<00:00, 151.86it/s]
Validation DataLoader 0: 57%|█████▋ | 177/313 [00:01<00:00, 151.85it/s]
Validation DataLoader 0: 57%|█████▋ | 178/313 [00:01<00:00, 151.90it/s]
Validation DataLoader 0: 57%|█████▋ | 179/313 [00:01<00:00, 151.93it/s]
Validation DataLoader 0: 58%|█████▊ | 180/313 [00:01<00:00, 151.99it/s]
Validation DataLoader 0: 58%|█████▊ | 181/313 [00:01<00:00, 152.04it/s]
Validation DataLoader 0: 58%|█████▊ | 182/313 [00:01<00:00, 152.09it/s]
Validation DataLoader 0: 58%|█████▊ | 183/313 [00:01<00:00, 152.16it/s]
Validation DataLoader 0: 59%|█████▉ | 184/313 [00:01<00:00, 152.20it/s]
Validation DataLoader 0: 59%|█████▉ | 185/313 [00:01<00:00, 152.22it/s]
Validation DataLoader 0: 59%|█████▉ | 186/313 [00:01<00:00, 152.23it/s]
Validation DataLoader 0: 60%|█████▉ | 187/313 [00:01<00:00, 152.21it/s]
Validation DataLoader 0: 60%|██████ | 188/313 [00:01<00:00, 152.22it/s]
Validation DataLoader 0: 60%|██████ | 189/313 [00:01<00:00, 152.25it/s]
Validation DataLoader 0: 61%|██████ | 190/313 [00:01<00:00, 152.13it/s]
Validation DataLoader 0: 61%|██████ | 191/313 [00:01<00:00, 152.20it/s]
Validation DataLoader 0: 61%|██████▏ | 192/313 [00:01<00:00, 152.25it/s]
Validation DataLoader 0: 62%|██████▏ | 193/313 [00:01<00:00, 152.30it/s]
Validation DataLoader 0: 62%|██████▏ | 194/313 [00:01<00:00, 152.35it/s]
Validation DataLoader 0: 62%|██████▏ | 195/313 [00:01<00:00, 152.40it/s]
Validation DataLoader 0: 63%|██████▎ | 196/313 [00:01<00:00, 152.46it/s]
Validation DataLoader 0: 63%|██████▎ | 197/313 [00:01<00:00, 152.52it/s]
Validation DataLoader 0: 63%|██████▎ | 198/313 [00:01<00:00, 152.57it/s]
Validation DataLoader 0: 64%|██████▎ | 199/313 [00:01<00:00, 152.59it/s]
Validation DataLoader 0: 64%|██████▍ | 200/313 [00:01<00:00, 152.62it/s]
Validation DataLoader 0: 64%|██████▍ | 201/313 [00:01<00:00, 152.68it/s]
Validation DataLoader 0: 65%|██████▍ | 202/313 [00:01<00:00, 152.70it/s]
Validation DataLoader 0: 65%|██████▍ | 203/313 [00:01<00:00, 152.71it/s]
Validation DataLoader 0: 65%|██████▌ | 204/313 [00:01<00:00, 152.75it/s]
Validation DataLoader 0: 65%|██████▌ | 205/313 [00:01<00:00, 152.80it/s]
Validation DataLoader 0: 66%|██████▌ | 206/313 [00:01<00:00, 152.83it/s]
Validation DataLoader 0: 66%|██████▌ | 207/313 [00:01<00:00, 152.87it/s]
Validation DataLoader 0: 66%|██████▋ | 208/313 [00:01<00:00, 152.90it/s]
Validation DataLoader 0: 67%|██████▋ | 209/313 [00:01<00:00, 152.92it/s]
Validation DataLoader 0: 67%|██████▋ | 210/313 [00:01<00:00, 152.94it/s]
Validation DataLoader 0: 67%|██████▋ | 211/313 [00:01<00:00, 152.96it/s]
Validation DataLoader 0: 68%|██████▊ | 212/313 [00:01<00:00, 153.00it/s]
Validation DataLoader 0: 68%|██████▊ | 213/313 [00:01<00:00, 153.04it/s]
Validation DataLoader 0: 68%|██████▊ | 214/313 [00:01<00:00, 153.05it/s]
Validation DataLoader 0: 69%|██████▊ | 215/313 [00:01<00:00, 153.05it/s]
Validation DataLoader 0: 69%|██████▉ | 216/313 [00:01<00:00, 153.08it/s]
Validation DataLoader 0: 69%|██████▉ | 217/313 [00:01<00:00, 153.04it/s]
Validation DataLoader 0: 70%|██████▉ | 218/313 [00:01<00:00, 153.06it/s]
Validation DataLoader 0: 70%|██████▉ | 219/313 [00:01<00:00, 153.07it/s]
Validation DataLoader 0: 70%|███████ | 220/313 [00:01<00:00, 153.13it/s]
Validation DataLoader 0: 71%|███████ | 221/313 [00:01<00:00, 153.15it/s]
Validation DataLoader 0: 71%|███████ | 222/313 [00:01<00:00, 153.13it/s]
Validation DataLoader 0: 71%|███████ | 223/313 [00:01<00:00, 153.18it/s]
Validation DataLoader 0: 72%|███████▏ | 224/313 [00:01<00:00, 153.23it/s]
Validation DataLoader 0: 72%|███████▏ | 225/313 [00:01<00:00, 153.27it/s]
Validation DataLoader 0: 72%|███████▏ | 226/313 [00:01<00:00, 153.33it/s]
Validation DataLoader 0: 73%|███████▎ | 227/313 [00:01<00:00, 153.37it/s]
Validation DataLoader 0: 73%|███████▎ | 228/313 [00:01<00:00, 153.40it/s]
Validation DataLoader 0: 73%|███████▎ | 229/313 [00:01<00:00, 153.44it/s]
Validation DataLoader 0: 73%|███████▎ | 230/313 [00:01<00:00, 153.48it/s]
Validation DataLoader 0: 74%|███████▍ | 231/313 [00:01<00:00, 153.51it/s]
Validation DataLoader 0: 74%|███████▍ | 232/313 [00:01<00:00, 153.56it/s]
Validation DataLoader 0: 74%|███████▍ | 233/313 [00:01<00:00, 153.61it/s]
Validation DataLoader 0: 75%|███████▍ | 234/313 [00:01<00:00, 153.65it/s]
Validation DataLoader 0: 75%|███████▌ | 235/313 [00:01<00:00, 153.68it/s]
Validation DataLoader 0: 75%|███████▌ | 236/313 [00:01<00:00, 153.72it/s]
Validation DataLoader 0: 76%|███████▌ | 237/313 [00:01<00:00, 153.77it/s]
Validation DataLoader 0: 76%|███████▌ | 238/313 [00:01<00:00, 153.82it/s]
Validation DataLoader 0: 76%|███████▋ | 239/313 [00:01<00:00, 153.83it/s]
Validation DataLoader 0: 77%|███████▋ | 240/313 [00:01<00:00, 153.85it/s]
Validation DataLoader 0: 77%|███████▋ | 241/313 [00:01<00:00, 153.88it/s]
Validation DataLoader 0: 77%|███████▋ | 242/313 [00:01<00:00, 153.93it/s]
Validation DataLoader 0: 78%|███████▊ | 243/313 [00:01<00:00, 153.95it/s]
Validation DataLoader 0: 78%|███████▊ | 244/313 [00:01<00:00, 153.96it/s]
Validation DataLoader 0: 78%|███████▊ | 245/313 [00:01<00:00, 154.02it/s]
Validation DataLoader 0: 79%|███████▊ | 246/313 [00:01<00:00, 154.06it/s]
Validation DataLoader 0: 79%|███████▉ | 247/313 [00:01<00:00, 154.11it/s]
Validation DataLoader 0: 79%|███████▉ | 248/313 [00:01<00:00, 154.16it/s]
Validation DataLoader 0: 80%|███████▉ | 249/313 [00:01<00:00, 154.19it/s]
Validation DataLoader 0: 80%|███████▉ | 250/313 [00:01<00:00, 154.20it/s]
Validation DataLoader 0: 80%|████████ | 251/313 [00:01<00:00, 154.23it/s]
Validation DataLoader 0: 81%|████████ | 252/313 [00:01<00:00, 154.27it/s]
Validation DataLoader 0: 81%|████████ | 253/313 [00:01<00:00, 154.30it/s]
Validation DataLoader 0: 81%|████████ | 254/313 [00:01<00:00, 154.35it/s]
Validation DataLoader 0: 81%|████████▏ | 255/313 [00:01<00:00, 154.38it/s]
Validation DataLoader 0: 82%|████████▏ | 256/313 [00:01<00:00, 154.42it/s]
Validation DataLoader 0: 82%|████████▏ | 257/313 [00:01<00:00, 154.45it/s]
Validation DataLoader 0: 82%|████████▏ | 258/313 [00:01<00:00, 154.46it/s]
Validation DataLoader 0: 83%|████████▎ | 259/313 [00:01<00:00, 154.50it/s]
Validation DataLoader 0: 83%|████████▎ | 260/313 [00:01<00:00, 154.53it/s]
Validation DataLoader 0: 83%|████████▎ | 261/313 [00:01<00:00, 154.57it/s]
Validation DataLoader 0: 84%|████████▎ | 262/313 [00:01<00:00, 154.59it/s]
Validation DataLoader 0: 84%|████████▍ | 263/313 [00:01<00:00, 154.60it/s]
Validation DataLoader 0: 84%|████████▍ | 264/313 [00:01<00:00, 154.65it/s]
Validation DataLoader 0: 85%|████████▍ | 265/313 [00:01<00:00, 154.67it/s]
Validation DataLoader 0: 85%|████████▍ | 266/313 [00:01<00:00, 154.70it/s]
Validation DataLoader 0: 85%|████████▌ | 267/313 [00:01<00:00, 154.69it/s]
Validation DataLoader 0: 86%|████████▌ | 268/313 [00:01<00:00, 154.71it/s]
Validation DataLoader 0: 86%|████████▌ | 269/313 [00:01<00:00, 154.73it/s]
Validation DataLoader 0: 86%|████████▋ | 270/313 [00:01<00:00, 154.77it/s]
Validation DataLoader 0: 87%|████████▋ | 271/313 [00:01<00:00, 154.80it/s]
Validation DataLoader 0: 87%|████████▋ | 272/313 [00:01<00:00, 154.81it/s]
Validation DataLoader 0: 87%|████████▋ | 273/313 [00:01<00:00, 154.84it/s]
Validation DataLoader 0: 88%|████████▊ | 274/313 [00:01<00:00, 154.86it/s]
Validation DataLoader 0: 88%|████████▊ | 275/313 [00:01<00:00, 154.89it/s]
Validation DataLoader 0: 88%|████████▊ | 276/313 [00:01<00:00, 154.93it/s]
Validation DataLoader 0: 88%|████████▊ | 277/313 [00:01<00:00, 154.97it/s]
Validation DataLoader 0: 89%|████████▉ | 278/313 [00:01<00:00, 155.01it/s]
Validation DataLoader 0: 89%|████████▉ | 279/313 [00:01<00:00, 155.03it/s]
Validation DataLoader 0: 89%|████████▉ | 280/313 [00:01<00:00, 155.06it/s]
Validation DataLoader 0: 90%|████████▉ | 281/313 [00:01<00:00, 155.09it/s]
Validation DataLoader 0: 90%|█████████ | 282/313 [00:01<00:00, 155.13it/s]
Validation DataLoader 0: 90%|█████████ | 283/313 [00:01<00:00, 155.15it/s]
Validation DataLoader 0: 91%|█████████ | 284/313 [00:01<00:00, 155.18it/s]
Validation DataLoader 0: 91%|█████████ | 285/313 [00:01<00:00, 155.21it/s]
Validation DataLoader 0: 91%|█████████▏| 286/313 [00:01<00:00, 155.25it/s]
Validation DataLoader 0: 92%|█████████▏| 287/313 [00:01<00:00, 155.28it/s]
Validation DataLoader 0: 92%|█████████▏| 288/313 [00:01<00:00, 155.31it/s]
Validation DataLoader 0: 92%|█████████▏| 289/313 [00:01<00:00, 155.28it/s]
Validation DataLoader 0: 93%|█████████▎| 290/313 [00:01<00:00, 155.30it/s]
Validation DataLoader 0: 93%|█████████▎| 291/313 [00:01<00:00, 155.26it/s]
Validation DataLoader 0: 93%|█████████▎| 292/313 [00:01<00:00, 155.21it/s]
Validation DataLoader 0: 94%|█████████▎| 293/313 [00:01<00:00, 155.17it/s]
Validation DataLoader 0: 94%|█████████▍| 294/313 [00:01<00:00, 155.20it/s]
Validation DataLoader 0: 94%|█████████▍| 295/313 [00:01<00:00, 155.16it/s]
Validation DataLoader 0: 95%|█████████▍| 296/313 [00:01<00:00, 155.18it/s]
Validation DataLoader 0: 95%|█████████▍| 297/313 [00:01<00:00, 155.17it/s]
Validation DataLoader 0: 95%|█████████▌| 298/313 [00:01<00:00, 155.17it/s]
Validation DataLoader 0: 96%|█████████▌| 299/313 [00:01<00:00, 155.16it/s]
Validation DataLoader 0: 96%|█████████▌| 300/313 [00:01<00:00, 155.22it/s]
Validation DataLoader 0: 96%|█████████▌| 301/313 [00:01<00:00, 155.23it/s]
Validation DataLoader 0: 96%|█████████▋| 302/313 [00:01<00:00, 155.22it/s]
Validation DataLoader 0: 97%|█████████▋| 303/313 [00:01<00:00, 155.25it/s]
Validation DataLoader 0: 97%|█████████▋| 304/313 [00:01<00:00, 155.29it/s]
Validation DataLoader 0: 97%|█████████▋| 305/313 [00:01<00:00, 155.32it/s]
Validation DataLoader 0: 98%|█████████▊| 306/313 [00:01<00:00, 155.34it/s]
Validation DataLoader 0: 98%|█████████▊| 307/313 [00:01<00:00, 155.36it/s]
Validation DataLoader 0: 98%|█████████▊| 308/313 [00:01<00:00, 155.38it/s]
Validation DataLoader 0: 99%|█████████▊| 309/313 [00:01<00:00, 155.41it/s]
Validation DataLoader 0: 99%|█████████▉| 310/313 [00:01<00:00, 155.45it/s]
Validation DataLoader 0: 99%|█████████▉| 311/313 [00:02<00:00, 155.49it/s]
Validation DataLoader 0: 100%|█████████▉| 312/313 [00:02<00:00, 155.54it/s]
Validation DataLoader 0: 100%|██████████| 313/313 [00:02<00:00, 155.67it/s]
Epoch 3: 100%|██████████| 3/3 [00:02<00:00, 1.47it/s, v_num=0]
→ creating new artifact version for key='testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8bc.ckpt' (storage: '/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops')
Epoch 3: 100%|██████████| 3/3 [00:02<00:00, 1.44it/s, v_num=0]
Epoch 3: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 4: 0%| | 0/3 [00:00<?, ?it/s, v_num=0]
Epoch 4: 33%|███▎ | 1/3 [00:00<00:00, 114.50it/s, v_num=0]
Epoch 4: 33%|███▎ | 1/3 [00:00<00:00, 107.44it/s, v_num=0]
Epoch 4: 67%|██████▋ | 2/3 [00:00<00:00, 115.25it/s, v_num=0]
Epoch 4: 67%|██████▋ | 2/3 [00:00<00:00, 111.71it/s, v_num=0]
Epoch 4: 100%|██████████| 3/3 [00:00<00:00, 116.15it/s, v_num=0]
Epoch 4: 100%|██████████| 3/3 [00:00<00:00, 113.71it/s, v_num=0]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: | | 0/? [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/313 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 1/313 [00:00<00:01, 200.67it/s]
Validation DataLoader 0: 1%| | 2/313 [00:00<00:01, 169.07it/s]
Validation DataLoader 0: 1%| | 3/313 [00:00<00:01, 167.57it/s]
Validation DataLoader 0: 1%|▏ | 4/313 [00:00<00:01, 159.37it/s]
Validation DataLoader 0: 2%|▏ | 5/313 [00:00<00:01, 159.13it/s]
Validation DataLoader 0: 2%|▏ | 6/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 2%|▏ | 7/313 [00:00<00:01, 159.92it/s]
Validation DataLoader 0: 3%|▎ | 8/313 [00:00<00:01, 160.22it/s]
Validation DataLoader 0: 3%|▎ | 9/313 [00:00<00:01, 160.90it/s]
Validation DataLoader 0: 3%|▎ | 10/313 [00:00<00:01, 161.02it/s]
Validation DataLoader 0: 4%|▎ | 11/313 [00:00<00:01, 160.52it/s]
Validation DataLoader 0: 4%|▍ | 12/313 [00:00<00:01, 159.87it/s]
Validation DataLoader 0: 4%|▍ | 13/313 [00:00<00:01, 159.59it/s]
Validation DataLoader 0: 4%|▍ | 14/313 [00:00<00:01, 159.44it/s]
Validation DataLoader 0: 5%|▍ | 15/313 [00:00<00:01, 159.82it/s]
Validation DataLoader 0: 5%|▌ | 16/313 [00:00<00:01, 160.31it/s]
Validation DataLoader 0: 5%|▌ | 17/313 [00:00<00:01, 160.65it/s]
Validation DataLoader 0: 6%|▌ | 18/313 [00:00<00:01, 161.29it/s]
Validation DataLoader 0: 6%|▌ | 19/313 [00:00<00:01, 160.92it/s]
Validation DataLoader 0: 6%|▋ | 20/313 [00:00<00:01, 161.14it/s]
Validation DataLoader 0: 7%|▋ | 21/313 [00:00<00:01, 161.42it/s]
Validation DataLoader 0: 7%|▋ | 22/313 [00:00<00:01, 161.74it/s]
Validation DataLoader 0: 7%|▋ | 23/313 [00:00<00:01, 160.20it/s]
Validation DataLoader 0: 8%|▊ | 24/313 [00:00<00:01, 158.64it/s]
Validation DataLoader 0: 8%|▊ | 25/313 [00:00<00:01, 157.73it/s]
Validation DataLoader 0: 8%|▊ | 26/313 [00:00<00:01, 156.22it/s]
Validation DataLoader 0: 9%|▊ | 27/313 [00:00<00:01, 155.48it/s]
Validation DataLoader 0: 9%|▉ | 28/313 [00:00<00:01, 154.52it/s]
Validation DataLoader 0: 9%|▉ | 29/313 [00:00<00:01, 153.88it/s]
Validation DataLoader 0: 10%|▉ | 30/313 [00:00<00:01, 152.81it/s]
Validation DataLoader 0: 10%|▉ | 31/313 [00:00<00:01, 151.28it/s]
Validation DataLoader 0: 10%|█ | 32/313 [00:00<00:01, 150.85it/s]
Validation DataLoader 0: 11%|█ | 33/313 [00:00<00:01, 149.87it/s]
Validation DataLoader 0: 11%|█ | 34/313 [00:00<00:01, 149.50it/s]
Validation DataLoader 0: 11%|█ | 35/313 [00:00<00:01, 149.93it/s]
Validation DataLoader 0: 12%|█▏ | 36/313 [00:00<00:01, 150.01it/s]
Validation DataLoader 0: 12%|█▏ | 37/313 [00:00<00:01, 148.75it/s]
Validation DataLoader 0: 12%|█▏ | 38/313 [00:00<00:01, 148.05it/s]
Validation DataLoader 0: 12%|█▏ | 39/313 [00:00<00:01, 147.56it/s]
Validation DataLoader 0: 13%|█▎ | 40/313 [00:00<00:01, 147.24it/s]
Validation DataLoader 0: 13%|█▎ | 41/313 [00:00<00:01, 147.21it/s]
Validation DataLoader 0: 13%|█▎ | 42/313 [00:00<00:01, 147.14it/s]
Validation DataLoader 0: 14%|█▎ | 43/313 [00:00<00:01, 147.11it/s]
Validation DataLoader 0: 14%|█▍ | 44/313 [00:00<00:01, 147.04it/s]
Validation DataLoader 0: 14%|█▍ | 45/313 [00:00<00:01, 146.84it/s]
Validation DataLoader 0: 15%|█▍ | 46/313 [00:00<00:01, 147.18it/s]
Validation DataLoader 0: 15%|█▌ | 47/313 [00:00<00:01, 147.51it/s]
Validation DataLoader 0: 15%|█▌ | 48/313 [00:00<00:01, 147.85it/s]
Validation DataLoader 0: 16%|█▌ | 49/313 [00:00<00:01, 148.21it/s]
Validation DataLoader 0: 16%|█▌ | 50/313 [00:00<00:01, 148.56it/s]
Validation DataLoader 0: 16%|█▋ | 51/313 [00:00<00:01, 148.97it/s]
Validation DataLoader 0: 17%|█▋ | 52/313 [00:00<00:01, 149.29it/s]
Validation DataLoader 0: 17%|█▋ | 53/313 [00:00<00:01, 149.61it/s]
Validation DataLoader 0: 17%|█▋ | 54/313 [00:00<00:01, 149.87it/s]
Validation DataLoader 0: 18%|█▊ | 55/313 [00:00<00:01, 150.13it/s]
Validation DataLoader 0: 18%|█▊ | 56/313 [00:00<00:01, 150.41it/s]
Validation DataLoader 0: 18%|█▊ | 57/313 [00:00<00:01, 150.54it/s]
Validation DataLoader 0: 19%|█▊ | 58/313 [00:00<00:01, 150.67it/s]
Validation DataLoader 0: 19%|█▉ | 59/313 [00:00<00:01, 150.86it/s]
Validation DataLoader 0: 19%|█▉ | 60/313 [00:00<00:01, 151.13it/s]
Validation DataLoader 0: 19%|█▉ | 61/313 [00:00<00:01, 151.29it/s]
Validation DataLoader 0: 20%|█▉ | 62/313 [00:00<00:01, 151.30it/s]
Validation DataLoader 0: 20%|██ | 63/313 [00:00<00:01, 151.38it/s]
Validation DataLoader 0: 20%|██ | 64/313 [00:00<00:01, 151.50it/s]
Validation DataLoader 0: 21%|██ | 65/313 [00:00<00:01, 151.64it/s]
Validation DataLoader 0: 21%|██ | 66/313 [00:00<00:01, 151.84it/s]
Validation DataLoader 0: 21%|██▏ | 67/313 [00:00<00:01, 152.04it/s]
Validation DataLoader 0: 22%|██▏ | 68/313 [00:00<00:01, 152.25it/s]
Validation DataLoader 0: 22%|██▏ | 69/313 [00:00<00:01, 152.37it/s]
Validation DataLoader 0: 22%|██▏ | 70/313 [00:00<00:01, 152.50it/s]
Validation DataLoader 0: 23%|██▎ | 71/313 [00:00<00:01, 152.64it/s]
Validation DataLoader 0: 23%|██▎ | 72/313 [00:00<00:01, 152.37it/s]
Validation DataLoader 0: 23%|██▎ | 73/313 [00:00<00:01, 152.12it/s]
Validation DataLoader 0: 24%|██▎ | 74/313 [00:00<00:01, 151.93it/s]
Validation DataLoader 0: 24%|██▍ | 75/313 [00:00<00:01, 151.98it/s]
Validation DataLoader 0: 24%|██▍ | 76/313 [00:00<00:01, 152.04it/s]
Validation DataLoader 0: 25%|██▍ | 77/313 [00:00<00:01, 152.15it/s]
Validation DataLoader 0: 25%|██▍ | 78/313 [00:00<00:01, 152.22it/s]
Validation DataLoader 0: 25%|██▌ | 79/313 [00:00<00:01, 152.35it/s]
Validation DataLoader 0: 26%|██▌ | 80/313 [00:00<00:01, 152.48it/s]
Validation DataLoader 0: 26%|██▌ | 81/313 [00:00<00:01, 152.60it/s]
Validation DataLoader 0: 26%|██▌ | 82/313 [00:00<00:01, 152.73it/s]
Validation DataLoader 0: 27%|██▋ | 83/313 [00:00<00:01, 152.93it/s]
Validation DataLoader 0: 27%|██▋ | 84/313 [00:00<00:01, 153.10it/s]
Validation DataLoader 0: 27%|██▋ | 85/313 [00:00<00:01, 153.28it/s]
Validation DataLoader 0: 27%|██▋ | 86/313 [00:00<00:01, 153.43it/s]
Validation DataLoader 0: 28%|██▊ | 87/313 [00:00<00:01, 153.56it/s]
Validation DataLoader 0: 28%|██▊ | 88/313 [00:00<00:01, 153.75it/s]
Validation DataLoader 0: 28%|██▊ | 89/313 [00:00<00:01, 153.91it/s]
Validation DataLoader 0: 29%|██▉ | 90/313 [00:00<00:01, 154.07it/s]
Validation DataLoader 0: 29%|██▉ | 91/313 [00:00<00:01, 154.22it/s]
Validation DataLoader 0: 29%|██▉ | 92/313 [00:00<00:01, 154.39it/s]
Validation DataLoader 0: 30%|██▉ | 93/313 [00:00<00:01, 154.51it/s]
Validation DataLoader 0: 30%|███ | 94/313 [00:00<00:01, 154.64it/s]
Validation DataLoader 0: 30%|███ | 95/313 [00:00<00:01, 154.75it/s]
Validation DataLoader 0: 31%|███ | 96/313 [00:00<00:01, 154.85it/s]
Validation DataLoader 0: 31%|███ | 97/313 [00:00<00:01, 155.00it/s]
Validation DataLoader 0: 31%|███▏ | 98/313 [00:00<00:01, 155.12it/s]
Validation DataLoader 0: 32%|███▏ | 99/313 [00:00<00:01, 155.24it/s]
Validation DataLoader 0: 32%|███▏ | 100/313 [00:00<00:01, 155.36it/s]
Validation DataLoader 0: 32%|███▏ | 101/313 [00:00<00:01, 155.47it/s]
Validation DataLoader 0: 33%|███▎ | 102/313 [00:00<00:01, 155.56it/s]
Validation DataLoader 0: 33%|███▎ | 103/313 [00:00<00:01, 155.62it/s]
Validation DataLoader 0: 33%|███▎ | 104/313 [00:00<00:01, 155.68it/s]
Validation DataLoader 0: 34%|███▎ | 105/313 [00:00<00:01, 155.79it/s]
Validation DataLoader 0: 34%|███▍ | 106/313 [00:00<00:01, 155.86it/s]
Validation DataLoader 0: 34%|███▍ | 107/313 [00:00<00:01, 155.97it/s]
Validation DataLoader 0: 35%|███▍ | 108/313 [00:00<00:01, 156.05it/s]
Validation DataLoader 0: 35%|███▍ | 109/313 [00:00<00:01, 156.14it/s]
Validation DataLoader 0: 35%|███▌ | 110/313 [00:00<00:01, 156.23it/s]
Validation DataLoader 0: 35%|███▌ | 111/313 [00:00<00:01, 156.31it/s]
Validation DataLoader 0: 36%|███▌ | 112/313 [00:00<00:01, 156.37it/s]
Validation DataLoader 0: 36%|███▌ | 113/313 [00:00<00:01, 156.44it/s]
Validation DataLoader 0: 36%|███▋ | 114/313 [00:00<00:01, 156.52it/s]
Validation DataLoader 0: 37%|███▋ | 115/313 [00:00<00:01, 156.60it/s]
Validation DataLoader 0: 37%|███▋ | 116/313 [00:00<00:01, 156.64it/s]
Validation DataLoader 0: 37%|███▋ | 117/313 [00:00<00:01, 156.70it/s]
Validation DataLoader 0: 38%|███▊ | 118/313 [00:00<00:01, 156.77it/s]
Validation DataLoader 0: 38%|███▊ | 119/313 [00:00<00:01, 156.87it/s]
Validation DataLoader 0: 38%|███▊ | 120/313 [00:00<00:01, 156.95it/s]
Validation DataLoader 0: 39%|███▊ | 121/313 [00:00<00:01, 157.02it/s]
Validation DataLoader 0: 39%|███▉ | 122/313 [00:00<00:01, 157.08it/s]
Validation DataLoader 0: 39%|███▉ | 123/313 [00:00<00:01, 157.14it/s]
Validation DataLoader 0: 40%|███▉ | 124/313 [00:00<00:01, 157.21it/s]
Validation DataLoader 0: 40%|███▉ | 125/313 [00:00<00:01, 157.18it/s]
Validation DataLoader 0: 40%|████ | 126/313 [00:00<00:01, 157.26it/s]
Validation DataLoader 0: 41%|████ | 127/313 [00:00<00:01, 157.32it/s]
Validation DataLoader 0: 41%|████ | 128/313 [00:00<00:01, 157.40it/s]
Validation DataLoader 0: 41%|████ | 129/313 [00:00<00:01, 157.41it/s]
Validation DataLoader 0: 42%|████▏ | 130/313 [00:00<00:01, 157.48it/s]
Validation DataLoader 0: 42%|████▏ | 131/313 [00:00<00:01, 157.56it/s]
Validation DataLoader 0: 42%|████▏ | 132/313 [00:00<00:01, 157.64it/s]
Validation DataLoader 0: 42%|████▏ | 133/313 [00:00<00:01, 157.72it/s]
Validation DataLoader 0: 43%|████▎ | 134/313 [00:00<00:01, 157.79it/s]
Validation DataLoader 0: 43%|████▎ | 135/313 [00:00<00:01, 157.87it/s]
Validation DataLoader 0: 43%|████▎ | 136/313 [00:00<00:01, 157.89it/s]
Validation DataLoader 0: 44%|████▍ | 137/313 [00:00<00:01, 157.83it/s]
Validation DataLoader 0: 44%|████▍ | 138/313 [00:00<00:01, 157.88it/s]
Validation DataLoader 0: 44%|████▍ | 139/313 [00:00<00:01, 157.93it/s]
Validation DataLoader 0: 45%|████▍ | 140/313 [00:00<00:01, 158.02it/s]
Validation DataLoader 0: 45%|████▌ | 141/313 [00:00<00:01, 158.09it/s]
Validation DataLoader 0: 45%|████▌ | 142/313 [00:00<00:01, 158.16it/s]
Validation DataLoader 0: 46%|████▌ | 143/313 [00:00<00:01, 158.24it/s]
Validation DataLoader 0: 46%|████▌ | 144/313 [00:00<00:01, 158.30it/s]
Validation DataLoader 0: 46%|████▋ | 145/313 [00:00<00:01, 158.38it/s]
Validation DataLoader 0: 47%|████▋ | 146/313 [00:00<00:01, 158.46it/s]
Validation DataLoader 0: 47%|████▋ | 147/313 [00:00<00:01, 158.48it/s]
Validation DataLoader 0: 47%|████▋ | 148/313 [00:00<00:01, 158.56it/s]
Validation DataLoader 0: 48%|████▊ | 149/313 [00:00<00:01, 158.63it/s]
Validation DataLoader 0: 48%|████▊ | 150/313 [00:00<00:01, 158.69it/s]
Validation DataLoader 0: 48%|████▊ | 151/313 [00:00<00:01, 158.76it/s]
Validation DataLoader 0: 49%|████▊ | 152/313 [00:00<00:01, 158.80it/s]
Validation DataLoader 0: 49%|████▉ | 153/313 [00:00<00:01, 158.88it/s]
Validation DataLoader 0: 49%|████▉ | 154/313 [00:00<00:01, 158.91it/s]
Validation DataLoader 0: 50%|████▉ | 155/313 [00:00<00:00, 158.97it/s]
Validation DataLoader 0: 50%|████▉ | 156/313 [00:00<00:00, 159.01it/s]
Validation DataLoader 0: 50%|█████ | 157/313 [00:00<00:00, 159.06it/s]
Validation DataLoader 0: 50%|█████ | 158/313 [00:00<00:00, 159.11it/s]
Validation DataLoader 0: 51%|█████ | 159/313 [00:00<00:00, 159.15it/s]
Validation DataLoader 0: 51%|█████ | 160/313 [00:01<00:00, 159.19it/s]
Validation DataLoader 0: 51%|█████▏ | 161/313 [00:01<00:00, 159.23it/s]
Validation DataLoader 0: 52%|█████▏ | 162/313 [00:01<00:00, 159.27it/s]
Validation DataLoader 0: 52%|█████▏ | 163/313 [00:01<00:00, 159.29it/s]
Validation DataLoader 0: 52%|█████▏ | 164/313 [00:01<00:00, 159.32it/s]
Validation DataLoader 0: 53%|█████▎ | 165/313 [00:01<00:00, 159.34it/s]
Validation DataLoader 0: 53%|█████▎ | 166/313 [00:01<00:00, 159.37it/s]
Validation DataLoader 0: 53%|█████▎ | 167/313 [00:01<00:00, 159.34it/s]
Validation DataLoader 0: 54%|█████▎ | 168/313 [00:01<00:00, 159.38it/s]
Validation DataLoader 0: 54%|█████▍ | 169/313 [00:01<00:00, 159.42it/s]
Validation DataLoader 0: 54%|█████▍ | 170/313 [00:01<00:00, 159.46it/s]
Validation DataLoader 0: 55%|█████▍ | 171/313 [00:01<00:00, 159.50it/s]
Validation DataLoader 0: 55%|█████▍ | 172/313 [00:01<00:00, 159.54it/s]
Validation DataLoader 0: 55%|█████▌ | 173/313 [00:01<00:00, 159.57it/s]
Validation DataLoader 0: 56%|█████▌ | 174/313 [00:01<00:00, 159.61it/s]
Validation DataLoader 0: 56%|█████▌ | 175/313 [00:01<00:00, 159.66it/s]
Validation DataLoader 0: 56%|█████▌ | 176/313 [00:01<00:00, 159.70it/s]
Validation DataLoader 0: 57%|█████▋ | 177/313 [00:01<00:00, 159.72it/s]
Validation DataLoader 0: 57%|█████▋ | 178/313 [00:01<00:00, 159.74it/s]
Validation DataLoader 0: 57%|█████▋ | 179/313 [00:01<00:00, 159.78it/s]
Validation DataLoader 0: 58%|█████▊ | 180/313 [00:01<00:00, 159.80it/s]
Validation DataLoader 0: 58%|█████▊ | 181/313 [00:01<00:00, 159.83it/s]
Validation DataLoader 0: 58%|█████▊ | 182/313 [00:01<00:00, 159.89it/s]
Validation DataLoader 0: 58%|█████▊ | 183/313 [00:01<00:00, 159.92it/s]
Validation DataLoader 0: 59%|█████▉ | 184/313 [00:01<00:00, 159.95it/s]
Validation DataLoader 0: 59%|█████▉ | 185/313 [00:01<00:00, 159.91it/s]
Validation DataLoader 0: 59%|█████▉ | 186/313 [00:01<00:00, 159.93it/s]
Validation DataLoader 0: 60%|█████▉ | 187/313 [00:01<00:00, 159.97it/s]
Validation DataLoader 0: 60%|██████ | 188/313 [00:01<00:00, 160.01it/s]
Validation DataLoader 0: 60%|██████ | 189/313 [00:01<00:00, 160.05it/s]
Validation DataLoader 0: 61%|██████ | 190/313 [00:01<00:00, 160.06it/s]
Validation DataLoader 0: 61%|██████ | 191/313 [00:01<00:00, 160.07it/s]
Validation DataLoader 0: 61%|██████▏ | 192/313 [00:01<00:00, 160.06it/s]
Validation DataLoader 0: 62%|██████▏ | 193/313 [00:01<00:00, 160.01it/s]
Validation DataLoader 0: 62%|██████▏ | 194/313 [00:01<00:00, 160.05it/s]
Validation DataLoader 0: 62%|██████▏ | 195/313 [00:01<00:00, 160.09it/s]
Validation DataLoader 0: 63%|██████▎ | 196/313 [00:01<00:00, 160.11it/s]
Validation DataLoader 0: 63%|██████▎ | 197/313 [00:01<00:00, 160.15it/s]
Validation DataLoader 0: 63%|██████▎ | 198/313 [00:01<00:00, 160.19it/s]
Validation DataLoader 0: 64%|██████▎ | 199/313 [00:01<00:00, 160.24it/s]
Validation DataLoader 0: 64%|██████▍ | 200/313 [00:01<00:00, 160.28it/s]
Validation DataLoader 0: 64%|██████▍ | 201/313 [00:01<00:00, 160.33it/s]
Validation DataLoader 0: 65%|██████▍ | 202/313 [00:01<00:00, 160.35it/s]
Validation DataLoader 0: 65%|██████▍ | 203/313 [00:01<00:00, 160.37it/s]
Validation DataLoader 0: 65%|██████▌ | 204/313 [00:01<00:00, 160.31it/s]
Validation DataLoader 0: 65%|██████▌ | 205/313 [00:01<00:00, 160.31it/s]
Validation DataLoader 0: 66%|██████▌ | 206/313 [00:01<00:00, 160.32it/s]
Validation DataLoader 0: 66%|██████▌ | 207/313 [00:01<00:00, 160.35it/s]
Validation DataLoader 0: 66%|██████▋ | 208/313 [00:01<00:00, 160.40it/s]
Validation DataLoader 0: 67%|██████▋ | 209/313 [00:01<00:00, 160.42it/s]
Validation DataLoader 0: 67%|██████▋ | 210/313 [00:01<00:00, 160.48it/s]
Validation DataLoader 0: 67%|██████▋ | 211/313 [00:01<00:00, 160.52it/s]
Validation DataLoader 0: 68%|██████▊ | 212/313 [00:01<00:00, 160.53it/s]
Validation DataLoader 0: 68%|██████▊ | 213/313 [00:01<00:00, 160.52it/s]
Validation DataLoader 0: 68%|██████▊ | 214/313 [00:01<00:00, 160.57it/s]
Validation DataLoader 0: 69%|██████▊ | 215/313 [00:01<00:00, 160.59it/s]
Validation DataLoader 0: 69%|██████▉ | 216/313 [00:01<00:00, 160.64it/s]
Validation DataLoader 0: 69%|██████▉ | 217/313 [00:01<00:00, 160.67it/s]
Validation DataLoader 0: 70%|██████▉ | 218/313 [00:01<00:00, 160.69it/s]
Validation DataLoader 0: 70%|██████▉ | 219/313 [00:01<00:00, 160.73it/s]
Validation DataLoader 0: 70%|███████ | 220/313 [00:01<00:00, 160.75it/s]
Validation DataLoader 0: 71%|███████ | 221/313 [00:01<00:00, 160.79it/s]
Validation DataLoader 0: 71%|███████ | 222/313 [00:01<00:00, 160.81it/s]
Validation DataLoader 0: 71%|███████ | 223/313 [00:01<00:00, 160.84it/s]
Validation DataLoader 0: 72%|███████▏ | 224/313 [00:01<00:00, 160.87it/s]
Validation DataLoader 0: 72%|███████▏ | 225/313 [00:01<00:00, 160.90it/s]
Validation DataLoader 0: 72%|███████▏ | 226/313 [00:01<00:00, 160.93it/s]
Validation DataLoader 0: 73%|███████▎ | 227/313 [00:01<00:00, 160.94it/s]
Validation DataLoader 0: 73%|███████▎ | 228/313 [00:01<00:00, 160.96it/s]
Validation DataLoader 0: 73%|███████▎ | 229/313 [00:01<00:00, 160.96it/s]
Validation DataLoader 0: 73%|███████▎ | 230/313 [00:01<00:00, 160.97it/s]
Validation DataLoader 0: 74%|███████▍ | 231/313 [00:01<00:00, 160.98it/s]
Validation DataLoader 0: 74%|███████▍ | 232/313 [00:01<00:00, 161.00it/s]
Validation DataLoader 0: 74%|███████▍ | 233/313 [00:01<00:00, 161.03it/s]
Validation DataLoader 0: 75%|███████▍ | 234/313 [00:01<00:00, 161.05it/s]
Validation DataLoader 0: 75%|███████▌ | 235/313 [00:01<00:00, 161.09it/s]
Validation DataLoader 0: 75%|███████▌ | 236/313 [00:01<00:00, 161.11it/s]
Validation DataLoader 0: 76%|███████▌ | 237/313 [00:01<00:00, 161.12it/s]
Validation DataLoader 0: 76%|███████▌ | 238/313 [00:01<00:00, 161.06it/s]
Validation DataLoader 0: 76%|███████▋ | 239/313 [00:01<00:00, 160.78it/s]
Validation DataLoader 0: 77%|███████▋ | 240/313 [00:01<00:00, 160.48it/s]
Validation DataLoader 0: 77%|███████▋ | 241/313 [00:01<00:00, 160.50it/s]
Validation DataLoader 0: 77%|███████▋ | 242/313 [00:01<00:00, 160.51it/s]
Validation DataLoader 0: 78%|███████▊ | 243/313 [00:01<00:00, 160.53it/s]
Validation DataLoader 0: 78%|███████▊ | 244/313 [00:01<00:00, 160.55it/s]
Validation DataLoader 0: 78%|███████▊ | 245/313 [00:01<00:00, 160.59it/s]
Validation DataLoader 0: 79%|███████▊ | 246/313 [00:01<00:00, 160.60it/s]
Validation DataLoader 0: 79%|███████▉ | 247/313 [00:01<00:00, 160.63it/s]
Validation DataLoader 0: 79%|███████▉ | 248/313 [00:01<00:00, 160.65it/s]
Validation DataLoader 0: 80%|███████▉ | 249/313 [00:01<00:00, 160.69it/s]
Validation DataLoader 0: 80%|███████▉ | 250/313 [00:01<00:00, 160.71it/s]
Validation DataLoader 0: 80%|████████ | 251/313 [00:01<00:00, 160.73it/s]
Validation DataLoader 0: 81%|████████ | 252/313 [00:01<00:00, 160.75it/s]
Validation DataLoader 0: 81%|████████ | 253/313 [00:01<00:00, 160.78it/s]
Validation DataLoader 0: 81%|████████ | 254/313 [00:01<00:00, 160.80it/s]
Validation DataLoader 0: 81%|████████▏ | 255/313 [00:01<00:00, 160.83it/s]
Validation DataLoader 0: 82%|████████▏ | 256/313 [00:01<00:00, 160.87it/s]
Validation DataLoader 0: 82%|████████▏ | 257/313 [00:01<00:00, 160.89it/s]
Validation DataLoader 0: 82%|████████▏ | 258/313 [00:01<00:00, 160.92it/s]
Validation DataLoader 0: 83%|████████▎ | 259/313 [00:01<00:00, 160.94it/s]
Validation DataLoader 0: 83%|████████▎ | 260/313 [00:01<00:00, 160.97it/s]
Validation DataLoader 0: 83%|████████▎ | 261/313 [00:01<00:00, 160.99it/s]
Validation DataLoader 0: 84%|████████▎ | 262/313 [00:01<00:00, 161.01it/s]
Validation DataLoader 0: 84%|████████▍ | 263/313 [00:01<00:00, 161.04it/s]
Validation DataLoader 0: 84%|████████▍ | 264/313 [00:01<00:00, 161.06it/s]
Validation DataLoader 0: 85%|████████▍ | 265/313 [00:01<00:00, 161.08it/s]
Validation DataLoader 0: 85%|████████▍ | 266/313 [00:01<00:00, 161.10it/s]
Validation DataLoader 0: 85%|████████▌ | 267/313 [00:01<00:00, 161.11it/s]
Validation DataLoader 0: 86%|████████▌ | 268/313 [00:01<00:00, 161.13it/s]
Validation DataLoader 0: 86%|████████▌ | 269/313 [00:01<00:00, 161.16it/s]
Validation DataLoader 0: 86%|████████▋ | 270/313 [00:01<00:00, 161.18it/s]
Validation DataLoader 0: 87%|████████▋ | 271/313 [00:01<00:00, 161.19it/s]
Validation DataLoader 0: 87%|████████▋ | 272/313 [00:01<00:00, 161.22it/s]
Validation DataLoader 0: 87%|████████▋ | 273/313 [00:01<00:00, 161.25it/s]
Validation DataLoader 0: 88%|████████▊ | 274/313 [00:01<00:00, 161.27it/s]
Validation DataLoader 0: 88%|████████▊ | 275/313 [00:01<00:00, 161.28it/s]
Validation DataLoader 0: 88%|████████▊ | 276/313 [00:01<00:00, 161.30it/s]
Validation DataLoader 0: 88%|████████▊ | 277/313 [00:01<00:00, 161.32it/s]
Validation DataLoader 0: 89%|████████▉ | 278/313 [00:01<00:00, 161.33it/s]
Validation DataLoader 0: 89%|████████▉ | 279/313 [00:01<00:00, 161.35it/s]
Validation DataLoader 0: 89%|████████▉ | 280/313 [00:01<00:00, 161.35it/s]
Validation DataLoader 0: 90%|████████▉ | 281/313 [00:01<00:00, 161.37it/s]
Validation DataLoader 0: 90%|█████████ | 282/313 [00:01<00:00, 161.39it/s]
Validation DataLoader 0: 90%|█████████ | 283/313 [00:01<00:00, 161.41it/s]
Validation DataLoader 0: 91%|█████████ | 284/313 [00:01<00:00, 161.42it/s]
Validation DataLoader 0: 91%|█████████ | 285/313 [00:01<00:00, 161.44it/s]
Validation DataLoader 0: 91%|█████████▏| 286/313 [00:01<00:00, 161.44it/s]
Validation DataLoader 0: 92%|█████████▏| 287/313 [00:01<00:00, 161.45it/s]
Validation DataLoader 0: 92%|█████████▏| 288/313 [00:01<00:00, 161.45it/s]
Validation DataLoader 0: 92%|█████████▏| 289/313 [00:01<00:00, 161.46it/s]
Validation DataLoader 0: 93%|█████████▎| 290/313 [00:01<00:00, 161.48it/s]
Validation DataLoader 0: 93%|█████████▎| 291/313 [00:01<00:00, 161.49it/s]
Validation DataLoader 0: 93%|█████████▎| 292/313 [00:01<00:00, 161.47it/s]
Validation DataLoader 0: 94%|█████████▎| 293/313 [00:01<00:00, 161.48it/s]
Validation DataLoader 0: 94%|█████████▍| 294/313 [00:01<00:00, 161.50it/s]
Validation DataLoader 0: 94%|█████████▍| 295/313 [00:01<00:00, 161.52it/s]
Validation DataLoader 0: 95%|█████████▍| 296/313 [00:01<00:00, 161.54it/s]
Validation DataLoader 0: 95%|█████████▍| 297/313 [00:01<00:00, 161.57it/s]
Validation DataLoader 0: 95%|█████████▌| 298/313 [00:01<00:00, 161.58it/s]
Validation DataLoader 0: 96%|█████████▌| 299/313 [00:01<00:00, 161.61it/s]
Validation DataLoader 0: 96%|█████████▌| 300/313 [00:01<00:00, 161.62it/s]
Validation DataLoader 0: 96%|█████████▌| 301/313 [00:01<00:00, 161.63it/s]
Validation DataLoader 0: 96%|█████████▋| 302/313 [00:01<00:00, 161.65it/s]
Validation DataLoader 0: 97%|█████████▋| 303/313 [00:01<00:00, 161.67it/s]
Validation DataLoader 0: 97%|█████████▋| 304/313 [00:01<00:00, 161.68it/s]
Validation DataLoader 0: 97%|█████████▋| 305/313 [00:01<00:00, 161.70it/s]
Validation DataLoader 0: 98%|█████████▊| 306/313 [00:01<00:00, 161.72it/s]
Validation DataLoader 0: 98%|█████████▊| 307/313 [00:01<00:00, 161.75it/s]
Validation DataLoader 0: 98%|█████████▊| 308/313 [00:01<00:00, 161.77it/s]
Validation DataLoader 0: 99%|█████████▊| 309/313 [00:01<00:00, 161.78it/s]
Validation DataLoader 0: 99%|█████████▉| 310/313 [00:01<00:00, 161.80it/s]
Validation DataLoader 0: 99%|█████████▉| 311/313 [00:01<00:00, 161.79it/s]
Validation DataLoader 0: 100%|█████████▉| 312/313 [00:01<00:00, 161.80it/s]
Validation DataLoader 0: 100%|██████████| 313/313 [00:01<00:00, 161.93it/s]
Epoch 4: 100%|██████████| 3/3 [00:01<00:00, 1.52it/s, v_num=0]
→ creating new artifact version for key='testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8bc.ckpt' (storage: '/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops')
Epoch 4: 100%|██████████| 3/3 [00:02<00:00, 1.49it/s, v_num=0]
INFO:pytorch_lightning.utilities.rank_zero:`Trainer.fit` stopped: `max_epochs=5` reached.
Epoch 4: 100%|██████████| 3/3 [00:02<00:00, 1.48it/s, v_num=0]
2025/10/13 20:32:25 WARNING mlflow.models.model: Model logged without a signature and input example. Please set `input_example` parameter when logging the model to auto infer the model signature.
MLflow and LaminDB user interfaces together¶
MLflow and LaminDB runs:
Both MLflow and LaminDB capture any runs together with run parameters.
MLFlow experiment overview |
LaminHub run overview |
|---|---|
|
|
MLflow run details and LaminDB artifact details:
MLflow and LaminDB complement each other. Whereas MLflow is excellent at capturing metrics over time, LaminDB excells at capturing lineage of input & output data and training checkpoints.
MLFlow run view |
LaminHub lineage view |
|---|---|
|
|
Both frameworks display output artifacts that were generated during the run. LaminDB further captures input artifacts, their origin and the associated source code.
MLFlow artifact view |
LaminHub artifact view |
|---|---|
|
|
All checkpoints are automatically annotated by the specified training metrics and MLflow run ID & name to keep both frameworks in sync:
last_checkpoint_af = ln.Artifact.filter(
key__startswith="testmodels/mlflow/", suffix__endswith="ckpt", is_latest=True
).last()
last_checkpoint_af.describe()
Show code cell output
Artifact .ckpt ├── General │ ├── key: testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8bc.ckpt │ ├── uid: RzsPFAckRW6SWxoP0004 hash: rjJTWHCz-EqmXlog7NgVNg │ ├── size: 621.8 KB transform: mlflow.ipynb │ ├── space: all branch: all │ ├── created_by: anonymous created_at: 2025-10-13 20:32:18 │ └── storage path: │ /home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops/testmodels/mlflow/025f501046bd4fbd93fbc8177c2df8 │ bc.ckpt ├── External features │ └── current_epoch int 4 │ mlflow_run_id str 025f501046bd4fbd93fbc8177c2df8bc │ mlflow_run_name str mercurial-mare-899 │ train_loss float 0.11235281080007553 │ val_loss float 0.11488378047943115 └── Labels └── .projects Project MLflow tutorial
To reuse the checkpoint later:
last_checkpoint_af.cache()
Show code cell output
PosixUPath('/home/runner/work/lamin-mlops/lamin-mlops/docs/lamin-mlops/.lamindb/RzsPFAckRW6SWxoP0004.ckpt')
last_checkpoint_af.view_lineage()
Show code cell output
ln.finish()
Show code cell output
→ finished Run('LSyrIuOBOxaUWNPW') after 18s at 2025-10-13 20:32:25 UTC