mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-11 20:13:39 -05:00
3c020f13d1
ML side additions (all actually taught): - GELU, Tanh activations - Xavier initialization - log-sum-exp trick - AdamW optimizer - Cosine scheduling, gradient clipping - Sinusoidal/learned positional encodings - Causal masking - LayerNorm, MLP - Magnitude pruning, knowledge distillation Systems side improvements (more concrete): - Contiguous layout, dtype sizes - Gradient memory multipliers (2x momentum, 3x Adam) - im2col expansion - Sparse gradient updates - Attention score materialization - KV cache sizing, per-layer memory - Cache locality, SIMD utilization - Confidence intervals, warm-up protocols - Pareto optimization Renamed "AI Olympics" to "Olympics" in table. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
TinyTorch Research Paper
Complete LaTeX source for the TinyTorch research paper.
Files
- paper.tex - Main paper (~12-15 pages, two-column format)
- references.bib - Bibliography (22 references)
- compile_paper.sh - Build script (requires LaTeX installation)
Quick Start: Get PDF
Option 1: Overleaf (Recommended)
- Go to Overleaf.com
- Create free account
- Upload
paper.texandreferences.bib - Click "Recompile"
- Download PDF
Option 2: Local Compilation
./compile_paper.sh
Requires LaTeX installation (MacTeX or BasicTeX).
Paper Details
- Format: Two-column LaTeX (conference-standard)
- Length: ~12-15 pages
- Sections: 7 complete sections
- Tables: 3 (framework comparison, learning objectives, performance benchmarks)
- Code listings: 5 (syntax-highlighted Python examples)
- References: 22 citations
Key Contributions
- Progressive disclosure via monkey-patching - Novel pedagogical pattern
- Systems-first curriculum design - Memory/FLOPs from Module 01
- Historical milestone validation - 70 years of ML as learning modules
- Constructionist framework building - Students build complete ML system
Framed as design contribution with empirical validation planned for Fall 2025.
Submission Venues
- ArXiv - Immediate (establish priority)
- SIGCSE 2026 - August deadline (may need 6-page condensed version)
- ICER 2026 - After classroom data (full empirical study)
Ready for submission! Upload to Overleaf to get your PDF.