mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-12 03:33:33 -05:00
Added integration tests for DataLoader: - test_dataloader_integration.py in tests/integration/ - Training workflow integration - Shuffle consistency across epochs - Memory efficiency verification Updated Module 08: - Added note about optional performance analysis - Clarified that analysis functions can be run manually - Clean flow: text → code → tests Updated datasets/tiny/README.md: - Minor formatting fixes Module 08 is now complete and ready to export: ✅ Dataset abstraction ✅ TensorDataset implementation ✅ DataLoader with batching/shuffling ✅ ASCII visualizations for understanding ✅ Unit tests (in module) ✅ Integration tests (in tests/) ✅ Performance analysis tools (optional) Next: Export with 'bin/tito export 08_dataloader'
TinyTorch Datasets
This directory contains datasets for TinyTorch examples and training.
Directory Structure
datasets/
├── tiny/ ← Tiny datasets shipped with repo (~100KB each)
│ └── digits_8x8.npz (1,797 samples, 67KB)
├── mnist/ ← Full MNIST (downloaded, gitignored)
├── cifar10/ ← Full CIFAR-10 (downloaded, gitignored)
└── download_*.py ← Download scripts for large datasets
Quick Start
For learning (instant, offline):
# Use tiny shipped datasets
import numpy as np
data = np.load('datasets/tiny/digits_8x8.npz')
For serious training (download once):
python datasets/download_mnist.py
MNIST Dataset
The mnist/ directory should contain the MNIST or Fashion-MNIST dataset files:
train-images-idx3-ubyte.gz- Training images (60,000 samples)train-labels-idx1-ubyte.gz- Training labelst10k-images-idx3-ubyte.gz- Test images (10,000 samples)t10k-labels-idx1-ubyte.gz- Test labels
Downloading the Dataset
Run the provided download script:
cd datasets
python download_mnist.py
This will download Fashion-MNIST (which has the same format as MNIST but is more accessible).
Dataset Format
Both MNIST and Fashion-MNIST use the same IDX file format:
- Images: 28x28 grayscale pixels
- Labels: Integer values 0-9
- Gzipped for compression
Fashion-MNIST classes:
- 0: T-shirt/top
- 1: Trouser
- 2: Pullover
- 3: Dress
- 4: Coat
- 5: Sandal
- 6: Shirt
- 7: Sneaker
- 8: Bag
- 9: Ankle boot
The examples will work with either original MNIST digits or Fashion-MNIST items.