From 4bfd32f345eadadf296d8100233fc5cdcfdedc4b Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Tue, 18 Nov 2025 06:31:39 -0500 Subject: [PATCH] Add energy consumption limitation and restructure future work thematically --- paper/paper.tex | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/paper/paper.tex b/paper/paper.tex index e73ce5e9..a4dea5d4 100644 --- a/paper/paper.tex +++ b/paper/paper.tex @@ -987,32 +987,30 @@ TinyTorch teaches framework internals as foundation for GPU and distributed work \textbf{Performance}: Pure Python executes 100--1000$\times$ slower than PyTorch (\Cref{tab:performance})---deliberate trade-off for transparency. Seven explicit loops reveal convolution complexity; unoptimized operations demonstrate why vectorization matters. Not for production use; students graduate to PyTorch/TensorFlow with internals understanding. +\textbf{Energy consumption}: While TinyTorch covers optimization techniques with significant energy implications (quantization achieving 4$\times$ compression, pruning enabling 10$\times$ model shrinkage), the curriculum does not explicitly measure or quantify energy consumption for students. The optimization modules discuss memory reduction and computational efficiency, implicitly touching on energy-saving benefits, but lack direct energy profiling tools or measurements. This omission means students understand that quantization reduces model size and pruning decreases computation, but may not connect these optimizations to concrete energy savings (joules/inference, watt-hours/training epoch). Future iterations could integrate energy profiling libraries or simulation tools to make energy efficiency an explicit learning objective alongside memory and latency optimization, particularly relevant for edge deployment and sustainable ML practices. + \textbf{Language}: Materials exist exclusively in English. Modular structure facilitates translation; community contributions welcome. \subsection{Future Work} \label{subsec:future-work} -TinyTorch's design enables multiple research and development trajectories maintaining pedagogical principles while extending capability. We organize future work by priority: systems-focused extensions enabling broader ML education coverage, empirical validation and learning science research, curriculum extensions for emerging topics, and community building. +TinyTorch's design enables multiple research and development trajectories maintaining pedagogical principles while extending capability. We organize future work into three thematic categories: (1) \textbf{systems extensions} integrating analytical models and simulators for GPU/distributed awareness, (2) \textbf{empirical validation} measuring learning outcomes and pedagogical effectiveness, and (3) \textbf{curriculum extensions} for emerging ML topics. -\subsubsection{GPU Awareness and Performance Modeling} +\subsubsection{Systems Extensions: Analytical Models and Simulators} -While TinyTorch's CPU-only design prioritizes pedagogical transparency, students benefit from understanding GPU acceleration without implementing CUDA kernels. Future extensions could enable students to compare TinyTorch CPU implementations against PyTorch GPU equivalents, analyzing performance gaps through roofline models~\citep{williams2009roofline}. Rather than writing CUDA code, students would profile existing implementations to understand memory hierarchy differences (CPU cache levels L1/L2/L3 versus GPU global/shared/register memory), parallelism benefits (sequential CPU loops versus massively parallel GPU execution with thousands of threads), roofline analysis techniques (plotting achieved performance against hardware limits to identify compute-bound versus memory-bound operations), and mixed precision advantages (profiling FP32 versus FP16 training speed/memory tradeoffs). Students would run instrumented PyTorch code alongside TinyTorch implementations, measuring wall-clock time, memory usage, and FLOPs utilization. The roofline model visualization shows why GPUs excel at ML workloads: high arithmetic intensity operations (matrix multiplication) approach peak FLOPs, while memory-bound operations (element-wise activations) hit bandwidth limits. This awareness without implementation maintains TinyTorch's accessibility while preparing students for GPU programming courses. +TinyTorch's CPU-only design prioritizes pedagogical transparency, but students benefit from understanding GPU acceleration and distributed training without requiring expensive hardware. We propose integrating \textbf{analytical performance models} and \textbf{systems simulators} to enable hardware-agnostic systems education. -\subsubsection{Distributed Training Fundamentals} +\paragraph{Roofline Models for GPU Performance Analysis} Future extensions could enable students to compare TinyTorch CPU implementations against PyTorch GPU equivalents through roofline models~\citep{williams2009roofline}. Rather than writing CUDA code, students would profile existing implementations to understand: (1) memory hierarchy differences (CPU cache levels L1/L2/L3 versus GPU global/shared/register memory), (2) parallelism benefits (sequential CPU loops versus massively parallel GPU execution with thousands of threads), (3) roofline analysis techniques (plotting achieved performance against hardware limits to identify compute-bound versus memory-bound operations), and (4) mixed precision advantages (profiling FP32 versus FP16 training speed/memory tradeoffs). Students would run instrumented PyTorch code alongside TinyTorch implementations, measuring wall-clock time, memory usage, and FLOPs utilization. The roofline model visualization shows why GPUs excel at ML workloads: high arithmetic intensity operations (matrix multiplication) approach peak FLOPs, while memory-bound operations (element-wise activations) hit bandwidth limits. This awareness without implementation maintains TinyTorch's accessibility while preparing students for GPU programming courses. -Understanding distributed training communication patterns and scalability challenges requires simulation-based pedagogy, not multi-GPU hardware. Future extensions could enable students to explore distributed training concepts through integration with ASTRA-sim~\citep{chakkaravarthy2023astrasim,astrasimsim2020}, a distributed ML training simulator. Rather than requiring 8-GPU clusters, students would simulate multi-device training on single machines, exploring data parallelism basics (gradient synchronization via all-reduce across virtual workers, analyzing communication overhead versus compute time), scalability analysis (measuring weak versus strong scaling, identifying communication bottlenecks as worker count increases), network topology impact (comparing ring all-reduce, tree all-reduce, and hierarchical strategies through ASTRA-sim's topology modeling), and pipeline parallelism introduction (simulating model partitioning across devices, analyzing pipeline bubbles and micro-batching strategies). +\paragraph{ASTRA-sim for Distributed Training Simulation} Understanding distributed training communication patterns and scalability challenges requires simulation-based pedagogy, not multi-GPU clusters. Future extensions could integrate ASTRA-sim~\citep{chakkaravarthy2023astrasim,astrasimsim2020}, a distributed ML training simulator enabling single-machine exploration of multi-device concepts. Rather than requiring 8-GPU clusters, students would simulate multi-device training, exploring: (1) data parallelism basics (gradient synchronization via all-reduce across virtual workers, analyzing communication overhead versus compute time), (2) scalability analysis (measuring weak versus strong scaling, identifying communication bottlenecks as worker count increases), (3) network topology impact (comparing ring all-reduce, tree all-reduce, and hierarchical strategies through ASTRA-sim's topology modeling), and (4) pipeline parallelism introduction (simulating model partitioning across devices, analyzing pipeline bubbles and micro-batching strategies). This simulation-based approach maintains TinyTorch's pedagogical principle: understanding systems through transparent implementation and measurement, not black-box hardware access. Students would understand why gradient synchronization limits distributed training scalability, how network bandwidth affects multi-node training, and when to apply different parallelism strategies based on model and hardware characteristics. -ASTRA-sim integration would enable exploring research questions without hardware barriers. Students could replicate findings from distributed training literature~\citep{astrasimsim2020}, experiencing how collective communication primitives (all-reduce, all-gather) dominate training time at scale. This simulation-based approach maintains TinyTorch's pedagogical principle: understanding systems through transparent implementation and measurement, not black-box hardware access. Students would understand why gradient synchronization limits distributed training scalability, how network bandwidth affects multi-node training, what communication patterns different parallelism strategies require, and when to apply data versus model versus pipeline parallelism based on model and hardware characteristics. - -\subsubsection{Empirical Validation} +\subsubsection{Empirical Validation and Learning Science Research} The most immediate priority involves deploying TinyTorch in university CS curricula and measuring learning outcomes through controlled comparison. Planned experimental design compares learning outcomes between traditional ML courses (focusing on algorithms, using PyTorch as black box) and TinyTorch courses (embedding systems concepts, building frameworks). Pre/post assessments would measure systems thinking competency (memory profiling, complexity reasoning, optimization analysis), framework comprehension (autograd mechanics, layer composition, training dynamics), and production readiness (debugging gradient flows, profiling performance, deployment decisions). Key research questions include: Does embedding systems concepts from the start improve production ML readiness compared to algorithm-only approaches? Do students who build frameworks transfer knowledge to PyTorch/TensorFlow more effectively than students who only use these frameworks? Does progressive disclosure reduce cognitive load compared to introducing separate gradient classes? Do historical milestones increase motivation and learning compared to equivalent technical validation? -\subsubsection{Learning Science Research} - -TinyTorch's design enables controlled studies of pedagogical questions in ML education: +\paragraph{Pedagogical Mechanisms} TinyTorch's design enables controlled studies of specific pedagogical questions in ML education: \textbf{Threshold Concepts}: Is autograd transformative (Meyer \& Land's criteria: troublesome, irreversible, integrative)? Do students exhibit conceptual breakthroughs or gradual understanding? @@ -1024,7 +1022,7 @@ TinyTorch's design enables controlled studies of pedagogical questions in ML edu These studies require validated instruments, control groups, and longitudinal data collection as part of future research programs. -\subsubsection{Advanced Curriculum Extensions} +\subsubsection{Curriculum Extensions for Emerging Topics} Maintaining modular structure, TinyTorch can incorporate emerging ML systems topics through community contributions: