+
+
+
Don't just import it. Build it.
-## What is TinyTorch?
+
+Implement every component of a neural network framework yourself—from tensors to transformers to production optimization—and understand exactly how modern ML systems work.
+
-TinyTorch is an educational ML systems course where you **build complete neural networks from scratch**. Instead of using PyTorch or TensorFlow as black boxes, you implement every component yourself—from tensors and gradients to optimizers and attention mechanisms—gaining deep understanding of how modern ML frameworks actually work.
+
+
+
95%+
+
MNIST Accuracy
+
Your neural networks
+
+
+
75%+
+
CIFAR-10 Accuracy
+
Your CNNs
+
+
+
100%
+
Your Code
+
Every implementation
+
+
-**Core Learning Approach**: Build → Profile → Optimize. You'll implement each system component, measure its performance characteristics, and understand the engineering trade-offs that shape production ML systems.
+
+
+
Tensors → Autograd → Training
Achieve: 95%+ MNIST Accuracy"]
+ end
-### Foundation Tier (Modules 01-07)
-Build the mathematical infrastructure: tensors, activations, layers, losses, autograd, optimizers, and training loops. By the end, you'll train neural networks achieving 95%+ accuracy on MNIST using your own implementations.
+ subgraph Architecture["Architecture Tier (08-13)"]
+ A["Implement Modern AI
DataLoader → CNNs → Transformers
Achieve: 75%+ CIFAR-10, Text Generation"]
+ end
-### Architecture Tier (Modules 08-13)
-Implement modern AI architectures: data loading, convolutions for vision, tokenization, embeddings, attention, and transformers for language. Achieve 75%+ accuracy on CIFAR-10 with CNNs and generate coherent text with transformers.
+ subgraph Optimization["Optimization Tier (14-19)"]
+ O["Deploy Production Systems
Profile → Quantize → Benchmark
Achieve: Sub-100ms Inference"]
+ end
-### Optimization Tier (Modules 14-19)
-Deploy production systems: profiling, quantization, compression, memoization, acceleration, and benchmarking. Transform research models into production-ready systems.
+ subgraph Capstone["Capstone (20)"]
+ C["Complete Integration
MLPerf Competition
Compete on Real Hardware"]
+ end
-### Capstone Competition (Module 20)
-Apply all optimizations in the MLPerf® Edu Competition—a standardized benchmark where you optimize models and compete fairly across different hardware platforms.
+ F --> A
+ A --> O
+ O --> C
-## Getting Started
-
-Ready to build ML systems from scratch? Here's your path:
-
-**Quick Setup** (15 minutes):
-1. Clone the repository: `git clone https://github.com/mlsysbook/TinyTorch.git`
-2. Run setup: `./setup-environment.sh`
-3. Activate environment: `source activate.sh`
-4. Verify: `tito system doctor`
-
-**Your First Module**:
-1. Start with Module 01 (Tensor) in `modules/source/01_tensor/`
-2. Implement the required functionality
-3. Export: `tito module complete 01`
-4. Validate: Run milestone scripts to prove your implementation works
-
-See the [Quick Start Guide](quickstart-guide.md) for detailed setup instructions and the [Student Workflow](student-workflow.md) for the complete development cycle.
-
-## The Simple Workflow
-
-TinyTorch follows a simple three-step cycle:
-
-```
-1. Edit modules → 2. Export to package → 3. Validate with milestones
+ style Foundation fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
+ style Architecture fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
+ style Optimization fill:#fce4ec,stroke:#c2185b,stroke-width:2px
+ style Capstone fill:#fff3e0,stroke:#f57c00,stroke-width:2px
```
-**Edit**: Work on module source files in `modules/source/XX_name/`
-**Export**: Run `tito module complete XX` to make your code importable
-**Validate**: Run milestone scripts to prove your implementations work
+**Browse complete module details in the sidebar navigation** — organized by tier with clear learning objectives and implementation guides.
-See [Student Workflow](student-workflow.md) for the complete development cycle, best practices, and troubleshooting.
+**See [Complete Course Structure](chapters/00-introduction.html)** for detailed tier breakdowns, time estimates, and career connections.
## Why Build Instead of Use?
-The difference between using a library and understanding a system is the difference between being limited by tools and being empowered to create them.
+Understanding the difference between using a framework and building one is the difference between being limited by tools and being empowered to create them.
-When you just use PyTorch or TensorFlow, you're stuck when things break—OOM errors, NaN losses, slow training. When you build TinyTorch from scratch, you understand exactly why these issues happen and how to fix them. You know the memory layouts, gradient flows, and performance bottlenecks because you implemented them yourself.
+
-See [FAQ](faq.md) for detailed comparisons with PyTorch, TensorFlow, micrograd, and nanoGPT, including code examples and architectural differences.
+
+
Traditional ML Education
-## Who Is This For?
+```python
+import torch
+model = torch.nn.Linear(784, 10)
+output = model(input)
+# When this breaks, you're stuck
+```
-**Perfect if you're asking these questions:**
+**Problem**: OOM errors, NaN losses, slow training—you can't debug what you don't understand.
+
-**ML Systems Engineers**: "Why does my model training OOM at batch size 32? How do attention mechanisms scale quadratically with sequence length? When does data loading become the bottleneck?" You'll build and profile every component, understanding memory hierarchies, computational complexity, and system bottlenecks that production ML systems face daily.
+
+
TinyTorch Approach
-**Students & Researchers**: "How does that `nn.Linear()` call actually compute gradients? Why does Adam optimizer need 3× the memory of SGD? What's actually happening during a forward pass?" You'll implement the mathematics you learned in class and discover how theoretical concepts become practical systems with real performance implications.
+```python
+from tinytorch import Linear # YOUR code
+model = Linear(784, 10) # YOUR implementation
+output = model(input)
+# You know exactly how this works
+```
-**Performance Engineers**: "Where are the actual bottlenecks in transformer inference? How does KV-cache reduce computation by 10-100×? Why does my CNN use 4GB of memory?" By building these systems from scratch, you'll understand memory access patterns, cache efficiency, and optimization opportunities that profilers alone can't teach.
+**Advantage**: You understand memory layouts, gradient flows, and performance bottlenecks because you implemented them.
+
-**Academics & Educators**: "How can I teach ML systems—not just ML algorithms?" TinyTorch provides a complete pedagogical framework emphasizing systems thinking: memory profiling, performance analysis, and scaling behavior are built into every module, not added as an afterthought.
+
Implement from scratch] --> U[Use
Real data, real problems]
+ U --> R[Reflect
Systems thinking questions]
+ R --> B
-As you complete modules, unlock **historical milestone demonstrations** that prove what you've built works. Each milestone recreates a breakthrough using YOUR implementations—from Rosenblatt's 1957 perceptron to modern transformers and production optimization.
+ style B fill:#FFC107,color:#000
+ style U fill:#4CAF50,color:#fff
+ style R fill:#2196F3,color:#fff
+```
-See [Historical Milestones](chapters/milestones.md) for complete timeline, requirements, and expected results.
+1. **Build**: Implement each component yourself—tensors, autograd, optimizers, attention
+2. **Use**: Apply your implementations to real problems—MNIST, CIFAR-10, text generation
+3. **Reflect**: Answer systems thinking questions—memory usage, scaling behavior, trade-offs
-## Next Steps
+This approach develops not just coding ability, but systems engineering intuition essential for production ML.
-- **New to TinyTorch**: Start with the [Quick Start Guide](quickstart-guide.md) for immediate hands-on experience
-- **Ready to Commit**: Begin Module 01: Tensor (see sidebar navigation) to start building
-- **Understand the Structure**: Read [Course Structure](chapters/00-introduction.md) for detailed tier breakdown and learning outcomes
-- **Teaching a Course**: Review [Instructor Guide](usage-paths/classroom-use.html) for classroom integration
+## Is This For You?
-TinyTorch is more than a course—it's a community of learners building together. Join thousands exploring ML systems from the ground up.
+**Perfect if you want to**:
+- Debug ML systems when frameworks fail (OOM errors, gradient explosions, performance bottlenecks)
+- Implement custom operations for research or production
+- Understand how PyTorch, TensorFlow, and JAX actually work under the hood
+- Transition from ML user to ML systems engineer
+
+**Prerequisites**: Python programming and basic linear algebra (matrix multiplication). No prior ML framework experience required—you'll build your own.
+
+### Start Your Path
+
+
+
+## Essential Resources
+
+**Core Documentation**:
+- **[Quick Start Guide](quickstart-guide.html)** — 15-minute setup and first module
+- **[Course Structure](chapters/00-introduction.html)** — Detailed tier breakdowns and learning outcomes
+- **[Student Workflow](student-workflow.md)** — Day-to-day development cycle
+- **[TITO Essentials](tito-essentials.md)** — Complete CLI command reference
+- **[Historical Milestones](chapters/milestones.md)** — Prove your implementations through ML history
+
+**Learning Support**:
+- **[FAQ](faq.md)** — Comparisons with PyTorch, TensorFlow, micrograd
+- **[Testing Framework](testing-framework.md)** — Quality assurance and validation
+- **[Community](community.md)** — Connect with other builders
+
+---
+
+**Ready to build?** Start with the [Quick Start Guide](quickstart-guide.html) to go from zero to building neural networks in 15 minutes.
+
+**Want context first?** Read the [Course Introduction](chapters/00-introduction.html) to understand the origin story, philosophy, and complete learning progression.
+
+**Teaching a course?** Review the [Instructor Guide](usage-paths/classroom-use.html) for classroom integration, automated grading, and curriculum planning.
diff --git a/site/quickstart-guide.md b/site/quickstart-guide.md
index 237f5786..a03edff1 100644
--- a/site/quickstart-guide.md
+++ b/site/quickstart-guide.md
@@ -54,8 +54,18 @@ See [Essential Commands](tito-essentials.md) for verification commands and troub
Let's build your first neural network component following the **TinyTorch workflow**:
-```
-1. Edit modules → 2. Export to package → 3. Validate with milestones
+```{mermaid}
+graph TD
+ Start[Clone & Setup] --> Edit[Edit Module
tensor_dev.ipynb]
+ Edit --> Export[Export to Package
tito module complete 01]
+ Export --> Test[Test Import
from tinytorch import Tensor]
+ Test --> Next[Continue to Module 02]
+
+ style Start fill:#e3f2fd
+ style Edit fill:#fffbeb
+ style Export fill:#f0fdf4
+ style Test fill:#fef3c7
+ style Next fill:#f3e5f5
```
See [Student Workflow](student-workflow.md) for the complete development cycle.
@@ -72,8 +82,8 @@ See [Student Workflow](student-workflow.md) for the complete development cycle.
```bash
# Step 1: Edit the module source
-cd modules/source/01_tensor
-jupyter lab 01_tensor_dev.py
+cd modules/01_tensor
+jupyter lab tensor_dev.ipynb
```
You'll implement core tensor operations:
@@ -109,8 +119,8 @@ See [Student Workflow](student-workflow.md) for the complete edit → export →
```bash
# Step 1: Edit the module
-cd modules/source/02_activations
-jupyter lab 02_activations_dev.py
+cd modules/02_activations
+jupyter lab activations_dev.ipynb
```
You'll implement essential activation functions:
@@ -146,7 +156,7 @@ tito checkpoint status # View your completion tracking
This is helpful for self-assessment but not required for the core workflow.
-See [Student Workflow](student-workflow.md) for the essential edit → export → validate cycle, and [Track Your Progress](learning-progress.md) for detailed capability tracking.
+See [Student Workflow](student-workflow.md) for the essential edit → export → validate cycle.
@@ -205,7 +215,7 @@ In 15 minutes, you've:
**Master the Workflow:**
- See [Student Workflow](student-workflow.md) for the complete edit → export → validate cycle
- See [Essential Commands](tito-essentials.md) for complete TITO command reference
-- See [Track Your Progress](learning-progress.md) for the full learning path
+- See [Student Workflow](student-workflow.md) for the complete development cycle
**For Instructors:**
- See [Classroom Setup Guide](usage-paths/classroom-use.md) for NBGrader integration (coming soon)
@@ -217,7 +227,7 @@ In 15 minutes, you've:
**The TinyTorch Development Cycle:**
-1. Edit module sources in `modules/source/`
+1. Edit module sources in `modules/NN_name/` (e.g., `modules/01_tensor/tensor_dev.ipynb`)
2. Export with `tito module complete N`
3. Validate by running milestone scripts
diff --git a/site/student-workflow.md b/site/student-workflow.md
index 1f1875a9..67f8eaeb 100644
--- a/site/student-workflow.md
+++ b/site/student-workflow.md
@@ -6,24 +6,32 @@ This guide explains the actual day-to-day workflow for building your ML framewor
TinyTorch follows a simple three-step cycle:
-```
-1. Edit modules → 2. Export to package → 3. Validate with milestones
+```{mermaid}
+graph LR
+ A[Edit Modules
modules/NN_name/] --> B[Export to Package
tito module complete N]
+ B --> C[Validate with Milestones
Run milestone scripts]
+ C --> A
+
+ style A fill:#e3f2fd
+ style B fill:#f0fdf4
+ style C fill:#fef3c7
```
### Step 1: Edit Modules
-Work on module source files in `modules/source/`:
+Work on module notebooks in `modules/`:
```bash
# Example: Working on Module 03 (Layers)
-cd modules/source/03_layers
-# Edit the *_dev.py files with your implementation
+cd modules/03_layers
+jupyter lab layers_dev.ipynb
```
-Each module is a Jupyter notebook in Python format (`.py` files with cell markers). You'll:
+Each module is a Jupyter notebook that you edit interactively. You'll:
- Implement the required functionality
- Add docstrings and comments
-- Include tests within the module
+- Run and test your code inline
+- See immediate feedback
### Step 2: Export to Package
@@ -100,14 +108,15 @@ Here's what a typical session looks like:
```bash
# 1. Work on a module
-cd modules/source/05_autograd
-# Edit 05_autograd_dev.py with your implementation
+cd modules/05_autograd
+jupyter lab autograd_dev.ipynb
+# Edit your implementation interactively
# 2. Export when ready
tito module complete 05
# 3. Validate with existing milestones
-cd ../../milestones/01_1957_perceptron
+cd ../milestones/01_1957_perceptron
python 01_rosenblatt_forward.py # Should still work!
# 4. Continue to next module or milestone
diff --git a/tito/commands/notebooks.py b/tito/commands/notebooks.py
index 22ac5638..2fa663b2 100644
--- a/tito/commands/notebooks.py
+++ b/tito/commands/notebooks.py
@@ -45,38 +45,40 @@ class NotebooksCommand(BaseCommand):
def validate_args(self, args: Namespace) -> None:
"""Validate notebooks command arguments."""
if args.module:
- # Look in modules/ subdirectory
- source_dir = self.config.modules_dir / 'source'
- if not source_dir.exists():
- source_dir = self.config.modules_dir
- module_file = source_dir / args.module / f"{args.module}.py"
- if not module_file.exists():
+ module_dir = self.config.modules_dir / args.module
+ if not module_dir.exists():
+ raise ModuleNotFoundError(f"Module directory '{args.module}' not found")
+
+ # Find *_dev.py file in the module directory
+ dev_files = list(module_dir.glob('*_dev.py'))
+ if not dev_files:
raise ModuleNotFoundError(
- f"Module '{args.module}' not found or no {args.module}.py file"
+ f"No *_dev.py file found in module '{args.module}'"
)
def _find_dev_files(self) -> List[Path]:
- """Find all *.py files in modules directory."""
+ """Find all *_dev.py files in modules directory."""
dev_files = []
- # Look in modules/ subdirectory
- source_dir = self.config.modules_dir / 'source'
- if not source_dir.exists():
- # Fallback to modules_dir directly
- source_dir = self.config.modules_dir
-
- for module_dir in source_dir.iterdir():
- if module_dir.is_dir():
- dev_py = module_dir / f"{module_dir.name}.py"
- if dev_py.exists():
- dev_files.append(dev_py)
- return dev_files
+ # Look in modules/ directory
+ modules_dir = self.config.modules_dir
+
+ for module_dir in modules_dir.iterdir():
+ if module_dir.is_dir() and not module_dir.name.startswith('.'):
+ # Look for *_dev.py files in each module directory
+ for py_file in module_dir.glob('*_dev.py'):
+ dev_files.append(py_file)
+ return sorted(dev_files)
def _convert_file(self, dev_file: Path) -> Tuple[bool, str]:
"""Convert a single Python file to notebook using Jupytext."""
try:
- # Use Jupytext to convert Python file to notebook
+ # Use Jupytext from venv to convert Python file to notebook
+ import sys
+ venv_python = Path(sys.executable)
+ jupytext_cmd = venv_python.parent / "jupytext"
+
result = subprocess.run([
- "jupytext", "--to", "notebook", str(dev_file)
+ str(jupytext_cmd), "--to", "notebook", str(dev_file)
], capture_output=True, text=True, timeout=30, cwd=dev_file.parent)
if result.returncode == 0:
@@ -103,11 +105,9 @@ class NotebooksCommand(BaseCommand):
# Find files to convert
if args.module:
- # Look in modules/ subdirectory
- source_dir = self.config.modules_dir / 'source'
- if not source_dir.exists():
- source_dir = self.config.modules_dir
- dev_files = [source_dir / args.module / f"{args.module}.py"]
+ module_dir = self.config.modules_dir / args.module
+ # Find *_dev.py file(s) in the module directory
+ dev_files = list(module_dir.glob('*_dev.py'))
self.console.print(f"🔄 Building notebook for module: {args.module}")
else:
dev_files = self._find_dev_files()