Restructure Discussion with 3 subsections: Scope, Pedagogical Spiral, Limitations

Added back "Scope: What's NOT Covered" section to clearly state what TinyTorch
deliberately omits (GPU programming, distributed training, production deployment).

Added new "Pedagogical Spiral" subsection discussing how concepts revisit and
reinforce across tiers:
- Memory reasoning: tensor.nbytes → Conv2d memory → attention O(N²) → quantization
- Computational complexity: matrix multiply FLOPs → convolution → attention → optimization
- Backward connections: later modules illuminate why earlier abstractions matter

Renamed final subsection to "Limitations and Future Directions" with focused
discussion of assessment validation, performance tradeoffs, energy measurement gaps,
and accessibility constraints.

This 3-section structure provides clearer organization:
1. What we deliberately excluded (scope boundaries)
2. What we learned about spiral reinforcement (pedagogical observations)
3. What needs improvement (honest limitations)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

paper/paper.tex

@@ -1003,15 +1003,38 @@ TinyTorch embraces productive failure \citep{kapur2008productive}---learning thr
 \section{Discussion and Limitations}
 \label{sec:discussion}
 
-TinyTorch prioritizes framework internals understanding over production completeness, creating three categories of limitations that reflect deliberate pedagogical trade-offs rather than technical barriers.
+This section positions TinyTorch's pedagogical approach through three lenses: deliberate scope decisions that focus curriculum on framework internals, observations about spiral reinforcement across modules, and honest assessment of limitations requiring future work.
 
-\noindent\textbf{Production Systems Beyond Scope.} TinyTorch teaches framework internals as foundation for advanced topics, not as replacement. The CPU-only design omits GPU programming (CUDA kernels, tensor cores, mixed precision), distributed training (data/model parallelism, gradient synchronization), and production deployment (model serving, compilation, MLOps tooling). These topics require substantial complexity---parallel programming, hardware knowledge, deployment infrastructure---that would shift focus from framework understanding. Complete ML engineer preparation requires TinyTorch (internals foundation) followed by PyTorch tutorials (GPU acceleration), distributed training courses (multi-node systems), and production experience. The CPU-only scope offers three pedagogical benefits: \textbf{accessibility} (works on modest hardware in regions with limited cloud access), \textbf{reproducibility} (consistent performance across institutions), and \textbf{pedagogical focus} (internals learning not confounded with hardware optimization).
+\subsection{Scope: What's NOT Covered}
+\label{subsec:scope}
 
-\noindent\textbf{Infrastructure Maturity Gaps.} NBGrader scaffolding works in development but remains unvalidated for large-scale deployment---grading validity requires investigation through item analysis and transfer task correlation. Performance benchmarks exist (Table 4) but pure Python executes 100--1000$\times$ slower than PyTorch, a deliberate trade-off for transparency where seven explicit convolution loops reveal algorithmic complexity better than optimized C++ kernels. Energy profiling remains implicit: students understand quantization's 4$\times$ compression and pruning's computational reduction but don't measure concrete energy savings (joules/inference, watt-hours/training). Future iterations could integrate energy profiling tools to make sustainability an explicit learning objective alongside memory and latency optimization.
+TinyTorch prioritizes framework internals understanding over production ML completeness. The curriculum explicitly omits several critical production skills that require substantial additional complexity orthogonal to framework internals. GPU programming and hardware acceleration---including CUDA kernel optimization, memory hierarchies, tensor cores, mixed precision training with FP16/BF16/INT8 and gradient scaling, and hardware-specific optimization for TPUs and Apple Neural Engine---represent substantial domains requiring parallel programming expertise that would overwhelm students still mastering tensor operations and automatic differentiation.
 
-\noindent\textbf{Accessibility Constraints.} Materials exist exclusively in English (modular structure facilitates community translation). Code examples omit type annotations (\texttt{def forward(self, x: Tensor) -> Tensor}) to reduce visual complexity for students simultaneously learning ML concepts and Python implementation, though this means students don't practice type-driven development standard in production codebases. Future iterations could introduce type hints progressively---omitting them in foundation modules (01--05), adding them in optimization modules (14--18) where interface contracts become critical. Advanced systems concepts (gradient checkpointing, operator fusion, Flash Attention) remain unaddressed, as these optimization techniques would obscure pedagogical transparency that makes TinyTorch's implementations understandable.
+Distributed training fundamentals including data parallelism (DistributedDataParallel, gradient synchronization), model parallelism (pipeline parallelism, tensor parallelism), and large-scale training systems (FSDP, DeepSpeed, Megatron-LM) similarly introduce communication complexity that extends beyond framework internals. Production deployment and serving skills---model compilation through TorchScript, ONNX, and TensorRT, serving infrastructure including batching, load balancing, and latency optimization, and MLOps tooling for experiment tracking, model versioning, and A/B testing---represent deployment infrastructure concerns distinct from framework understanding.
 
-These limitations are addressable through community contribution and curriculum evolution. The empirical validation roadmap (Section~\ref{sec:future-work}) details specific plans for validating pedagogical effectiveness and identifying high-priority extensions based on classroom deployment feedback.
+Advanced systems techniques such as gradient checkpointing (trading computation for memory), operator fusion and graph compilation, Flash Attention and memory-efficient attention variants, and dynamic versus static computation graphs, while important for production systems, introduce optimization complexity that obscures pedagogical transparency. These topics require substantial additional complexity---parallel programming semantics, hardware knowledge, deployment infrastructure---that would shift focus away from framework internals understanding.
+
+TinyTorch teaches framework internals as foundation for GPU and distributed work, not as replacement. Complete production ML engineer preparation requires TinyTorch (internals) followed by PyTorch Distributed (GPU/multi-node), deployment courses (serving), and on-the-job experience. Students completing TinyTorch should pursue GPU and distributed training through PyTorch tutorials, NVIDIA Deep Learning Institute courses, or advanced ML systems courses. The CPU-only design offers three pedagogical benefits: accessibility (students in regions with limited cloud computing access can complete curriculum on modest hardware), reproducibility (no GPU availability variability across institutions), and pedagogical focus (internals learning not confounded with hardware optimization).
+
+\subsection{Pedagogical Spiral: Concepts Revisited Across Tiers}
+
+TinyTorch's 20-module curriculum exhibits spiral design: core concepts introduced in foundation modules reappear with increasing sophistication across architecture and optimization tiers. This recursive reinforcement serves dual purposes---validating earlier understanding while deepening systems intuition through application at scale.
+
+Memory reasoning spirals across all tiers. Module 01 students calculate tensor footprints (\texttt{tensor.nbytes}), Module 09 extends this to convolutional parameter counts and activation memory (output spatial dimensions $\times$ channels), Module 12 reveals attention's quadratic memory scaling ($O(N^2)$ for sequence length $N$), and Module 15 quantization demonstrates 4$\times$ compression through INT8 versus FP32 representation. Each encounter builds on prior knowledge: students who manually calculated Conv2d memory in Module 09 immediately recognize why quantization matters when profiling transformer memory in Module 15.
+
+Computational complexity follows similar progression. Module 03 introduces FLOPs counting for matrix multiplication ($O(N^3)$ for $N\times N$ matrices), Module 09 applies this to convolution (sliding window creates $O(W \times H \times C_{in} \times C_{out} \times K^2)$ complexity for image dimensions $W,H$, channels, and kernel size $K$), Module 12 analyzes attention's $O(N^2 \cdot d)$ complexity for embedding dimension $d$, and Module 18 acceleration targets these measured bottlenecks through algorithmic improvements. Students develop intuition for complexity analysis not through abstract notation but through profiling actual implementations across architectures of increasing sophistication.
+
+The integration testing pattern (Build $\rightarrow$ Use $\rightarrow$ Reflect) creates backward connections: later modules illuminate earlier implementations. Students building Module 13 transformers discover why Module 03 layer abstractions matter---without clean interfaces, composing embeddings, attention, and MLPs becomes intractable. This backward reinforcement differs from forward prerequisites: students \emph{could} implement transformers without prior layer experience, but doing so reveals \emph{why} abstraction boundaries exist. Spiral design makes implicit knowledge explicit through repeated encounter at increasing depth.
+
+\subsection{Limitations and Future Directions}
+
+TinyTorch's current implementation contains gaps requiring future work. \textbf{Assessment infrastructure}: NBGrader scaffolding works in development but remains unvalidated for large-scale deployment. Grading validity requires investigation: Do tests measure conceptual understanding or syntax? Future work should validate through item analysis and transfer task correlation.
+
+\textbf{Performance transparency tradeoff}: Pure Python executes 100--1000$\times$ slower than PyTorch (\Cref{tab:performance})---deliberate choice for pedagogical clarity. Seven explicit convolution loops reveal algorithmic complexity better than optimized C++ kernels, but slow execution limits practical experimentation. Students complete milestones (75\%+ CIFAR-10 accuracy, transformer text generation) but cannot iterate rapidly on architecture search.
+
+\textbf{Energy consumption measurement}: While TinyTorch covers optimization techniques with significant energy implications (quantization achieving 4$\times$ compression, pruning enabling 10$\times$ model shrinkage), the curriculum does not explicitly measure or quantify energy consumption. Students understand that quantization reduces model size and pruning decreases computation, but may not connect these optimizations to concrete energy savings (joules/inference, watt-hours/training epoch). Future iterations could integrate energy profiling libraries to make sustainability an explicit learning objective alongside memory and latency optimization, particularly relevant for edge deployment.
+
+\textbf{Language and accessibility}: Materials exist exclusively in English. Modular structure facilitates translation; community contributions welcome. Code examples omit type annotations (e.g., \texttt{def forward(self, x: Tensor) -> Tensor:}) to reduce visual complexity for students learning ML concepts simultaneously. While this prioritizes pedagogical clarity, it means students don't practice type-driven development increasingly standard in production ML codebases. Future iterations could introduce type hints progressively: omitting them in early modules (01--05), then adding them in optimization modules (14--18) where interface contracts become critical.
 
 \section{Future Directions}
 \label{sec:future-work}