docs(paper): revise abstract for citation impact

- Name the problem: "algorithm-systems divide"
- Name the approach: "implementation-based systems pedagogy"
- Add concrete systems examples (O(N^2), Adam 3x memory)
- Include MLPerf-style benchmarking in milestones
- Strengthen citable terminology throughout
This commit is contained in:
Vijay Janapa Reddi
2026-01-28 08:27:11 -05:00
parent 2b57102cc6
commit e668c5b700

View File

@@ -201,6 +201,7 @@
\Huge\bfseries Tiny\emoji{fire}Torch\\[0.4em]
\Large\normalfont\itshape Building Machine Learning Systems from First Principles
}
\author{
\fontsize{12}{15}\selectfont
Vijay Janapa Reddi\\[0.2em]
@@ -220,9 +221,8 @@
% Abstract - REVISED: Curriculum design focus
\begin{abstract}
Machine learning systems engineering requires understanding framework internals, yet current education separates algorithms from systems—students learn gradient descent without measuring memory, attention mechanisms without profiling computational costs. This divide leaves graduates unable to debug production failures, widening the gap between ML research and reliable deployment. We present TinyTorch, a 20-module curriculum in which students implement PyTorch's core components (tensors, autograd, optimizers, neural networks) entirely in pure Python. The curriculum employs three pedagogical patterns: \emph{progressive disclosure} that gradually reveals complexity, \emph{systems-first integration} that embeds memory profiling from the start, and \emph{historical milestone validation} that guides students to recreate breakthroughs from the Perceptron (1958) to Transformers using exclusively student-built code. Requiring only a laptop with 4GB RAM and no GPU, TinyTorch makes ML systems education globally accessible. The goal is to prepare the next generation of ``AI engineers''—practitioners who understand not just what ML systems do, but why they work and how to make them scale. The curriculum is available open-source at \texttt{mlsysbook.ai/tinytorch}.
\end{abstract}
Machine learning education faces a fundamental gap: students learn algorithms without understanding the systems that execute them. They study gradient descent without measuring memory, attention mechanisms without analyzing $O(N^2)$ scaling, optimizer theory without knowing why Adam requires $3\times$ the memory of SGD. This \emph{algorithm-systems divide} produces practitioners who can train models but cannot debug memory failures, optimize inference latency, or reason about deployment trade-offs—the very skills industry demands as ``ML systems engineering.'' We present TinyTorch, a 20-module curriculum that closes this gap through \emph{implementation-based systems pedagogy}: students construct PyTorch's core components (tensors, autograd, optimizers, CNNs, transformers) in pure Python, building a complete framework where every operation they invoke is code they wrote. The design employs three patterns: \emph{progressive disclosure} of complexity, \emph{systems-first integration} of profiling from the first module, and \emph{build-to-validate milestones} recreating 67 years of ML breakthroughsfrom Perceptron (1958) through Transformers (2017) to MLPerf-style benchmarking. Requiring only 4GB RAM and no GPU, TinyTorch demonstrates that deep ML systems understanding is achievable without specialized hardware. The curriculum is available open-source at \texttt{mlsysbook.ai/tinytorch}.
\end{abstract}
% Main content
\section{Introduction}