github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-03-11 17:49:25 -05:00
Code Issues Packages Projects Releases Wiki Activity

Trim abstract to fit arXiv 1920 char limit

Browse Source
This commit is contained in:
Vijay Janapa Reddi
2026-01-26 21:08:58 -05:00
parent e4a664b9c1
commit 0d00348cc8
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tinytorch/paper/paper.tex
View File

@@ -220,7 +220,7 @@
% Abstract - REVISED: Curriculum design focus
\begin{abstract}
Machine learning systems engineering requires understanding framework internals: why optimizers consume memory, when computational complexity becomes prohibitive, how to navigate accuracy-latency-memory tradeoffs. Yet current ML education separates algorithms from systems—students learn gradient descent without measuring memory, attention mechanisms without profiling costs, training without understanding optimizer overhead. This divide leaves graduates unable to debug production failures or make informed engineering decisions, widening the gap between ML research and reliable production deployment. We present TinyTorch, a build-from-scratch curriculum where students implement PyTorch's core components (tensors, autograd, optimizers, neural networks) to gain framework transparency. Three pedagogical patterns address the gap: \textbf{progressive disclosure} gradually reveals complexity (gradient features exist from Module 01, activate in Module 06); \textbf{systems-first curriculum} embeds memory profiling from the start; \textbf{historical milestone validation} recreates nearly 70 years of ML breakthroughs (1958--2025) using exclusively student-implemented code. These patterns are grounded in learning theory (situated cognition, cognitive load theory) but represent testable hypotheses requiring empirical validation. The goal is to prepare the next generation of \emph{AI engineers}: practitioners who understand not just what ML systems do, but why they work and how to make them scale. Complete open-source infrastructure is available at \texttt{mlsysbook.ai/tinytorch}.
Machine learning systems engineering requires understanding framework internals: why optimizers consume memory, when complexity becomes prohibitive, how to navigate accuracy-latency-memory tradeoffs. Yet current ML education separates algorithms from systems—students learn gradient descent without measuring memory, attention without profiling costs. This divide leaves graduates unable to debug production failures, widening the gap between ML research and deployment. We present TinyTorch, a build-from-scratch curriculum where students implement PyTorch's core components (tensors, autograd, optimizers, neural networks). Three pedagogical patterns address this gap: \textbf{progressive disclosure} gradually reveals complexity; \textbf{systems-first curriculum} embeds memory profiling from the start; \textbf{historical milestone validation} recreates ML breakthroughs (1958--2025) using student-implemented code. The goal is to prepare the next generation of \emph{AI engineers}: practitioners who understand not just what ML systems do, but why they work and how to make them scale. Open-source infrastructure at \texttt{mlsysbook.ai/tinytorch}.
\end{abstract}