TinyTorch/site/development/module-rules.md

TinyTorch Module Development Rules

Version: 2.0
Date: January 2025
Status: Complete Reference Guide
Reference Implementation: src/08_optimizers/08_optimizers.py

This document defines the complete set of rules, patterns, and conventions for developing TinyTorch modules. Instead of maintaining separate documentation, use 08_optimizers as your reference implementation - it follows all current patterns perfectly.

Development Structure

Key Paths:

• Source: src/NN_modulename/NN_modulename.py ← Developers edit here
• Generated: modules/NN_modulename/NN_modulename.ipynb ← Auto-generated for students
• Package: tinytorch/ ← Auto-generated package code

📚 Educational Philosophy

Core Principles

1. Educational First: Every module is designed for learning, not just functionality
2. Progressive Complexity: Start simple, build complexity step by step
3. Real-World Connection: Connect concepts to practical ML applications
4. Standalone Learning: Each module should be self-contained
5. Professional Standards: Use industry-standard patterns and practices

"Build → Use → [Understand/Reflect/Analyze/Optimize]" Framework

Each module follows this pedagogical pattern:

• Build: Implement the component from scratch
• Use: Apply it to real data and problems
• Third Stage: Varies by module (Understand/Reflect/Analyze/Optimize)

📁 File Structure and Organization

1. File Naming Convention

Developer Workflow (Source):

src/NN_modulename/
├── NN_modulename.py           # SOURCE OF TRUTH (developers edit)
├── module.yaml                # Module configuration
├── ABOUT.md                   # Module documentation
└── README.md                  # Optional additional docs

Student Workflow (Generated):

modules/NN_modulename/
└── NN_modulename.ipynb        # AUTO-GENERATED notebook (students use)

2. File Format: Jupytext Percent Format

All source Python files in src/ MUST use Jupytext percent format:

# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: percent
#       format_version: '1.3'
#       jupytext_version: 1.17.1
# ---

Build Process:

1. Developer edits src/NN_name/NN_name.py
2. Run tito source export NN_name
3. Generates modules/NN_name/NN_name.ipynb (notebook)
4. Exports to tinytorch/ (package code)

🏗️ Module Template Structure

Follow this exact structure (see 08_optimizers for reference):

A. Header Section

# %% [markdown]
"""
# Module N: Title - Brief Description

## Learning Goals
- Goal 1: Specific outcome
- Goal 2: Another objective  
- Goal 3: Connection to ML concepts

## Build → Use → [Understand/Reflect/Analyze/Optimize]
1. **Build**: What students implement
2. **Use**: How they apply it
3. **[Third Stage]**: Deeper engagement
"""

B. Setup and Imports

# %% nbgrader={"grade": false, "grade_id": "modulename-imports", "locked": false, "schema_version": 3, "solution": false, "task": false}
#| default_exp core.modulename

#| export
import numpy as np
import sys
from typing import Union, List, Tuple, Optional, Any

# %% nbgrader={"grade": false, "grade_id": "modulename-setup", "locked": false, "schema_version": 3, "solution": false, "task": false}
print("🔥 TinyTorch [Module] Module")
print(f"Python version: {sys.version_info.major}.{sys.version_info.minor}")
print("Ready to [action]!")

C. Package Location

# %% [markdown]
"""
## 📦 Where This Code Lives in the Final Package

**Learning Side:** You work in `modules/NN_modulename/modulename_dev.py`  
**Building Side:** Code exports to `tinytorch.core.modulename`

```python
# Final package structure:
from tinytorch.core.modulename import ComponentName  # Main functionality!
from tinytorch.core.tensor import Tensor  # Foundation

Why this matters:

• Learning: Focused modules for deep understanding
• Production: Proper organization like PyTorch's structure
• Consistency: All [module] operations live together
• Foundation: Connection to broader ML systems """

### D. **Educational Content Structure**
```python
# %% [markdown]
"""
## What Are [Components]?

### The Problem/Motivation
Explain why this module exists and what problem it solves.

### The Solution
Describe the approach and key insights.

### Real-World Impact
Show concrete applications and industry relevance.

### What We'll Build
1. **Component 1**: Brief description
2. **Component 2**: Brief description
3. **Integration**: How components work together
"""

E. ASCII Diagrams and Tables

ASCII diagrams are powerful educational tools that visualize concepts, architectures, and data flows. Follow these guidelines to ensure they render correctly across all environments.

Box Alignment Rules

1. Border Width Consistency: The top border (┌───┐) defines the box width. All content lines must match this width exactly.

✅ CORRECT:
┌───────────────────────────────┐
│ Content fits within borders  │
│ All lines same width         │
└───────────────────────────────┘

❌ WRONG:
┌─────────────────┐
│ Content extends beyond│
│ Border mismatch      │
└─────────────────┘

2. Nested Boxes: Inner boxes must align with the outer box's right border. Use consistent spacing.

✅ CORRECT:
┌───────────────────────────────────────────────────────┐
│ OUTER BOX TITLE                                       │
├───────────────────────────────────────────────────────┤
│                                                       │
│ Section Header                                        │
│ ┌───────────────────────────────────────────────────┐ │
│ │ Inner content properly aligned                    │ │
│ │ All borders match outer box width                 │ │
│ └───────────────────────────────────────────────────┘ │
│                                                       │
└───────────────────────────────────────────────────────┘

❌ WRONG (misaligned nested boxes):
┌──────────────────────────────────┐
│ OUTER BOX                        │
│ ┌──────────────────────────────┐          │
│ │ Inner box extends beyond     │          │
│ └──────────────────────────────┘          │
└──────────────────────────────────────────────────┘

3. Side-by-Side Boxes: Both boxes should have equal or proportional widths with consistent spacing.

✅ CORRECT:
┌───────────────────────────────┐    ┌───────────────────────────────┐
│ Left Box                      │    │ Right Box                     │
│ Content here                  │ →  │ Content here                  │
│ Same width as right           │    │ Same width as left            │
└───────────────────────────────┘    └───────────────────────────────┘

❌ WRONG (unequal widths, misaligned):
┌─────────────────┐         ┌─────────────────────────┐
│ Left Box          │         │ Right Box            │
│ Width mismatch   │    →    │ Different size      │
└─────────────────┘         └─────────────────────────┘

Table Formatting Rules

1. Column Alignment: All column separators (│) must align vertically. Use consistent column widths.

✅ CORRECT:
┌─────────────────┬─────────────────┬─────────────────┐
│ Column 1        │ Column 2        │ Column 3        │
├─────────────────┼─────────────────┼─────────────────┤
│ Data aligned    │ Same width      │ Consistent      │
│ Short           │ Medium content  │ Longer content  │
└─────────────────┴─────────────────┴─────────────────┘

❌ WRONG (separators don't align):
┌─────────────────┬─────────────────┬─────────────────┐
│ Column 1        │ Column 2        │ Column 3        │
├─────────────────┼─────────────────┼─────────────────┤
│ Misaligned│ Content      │ Wrong widths     │
└─────────────────┴─────────────────┴─────────────────┘

2. Multi-Column Tables: Header separators (┬, ┼, ┴) must align with content separators.

Special Characters

Emoji Width: Emojis display as ~2 characters wide but count as 1 in string length. Add extra padding after emojis.

✅ CORRECT (emoji padding):
│ ✅ Success message here      │
│ ❌ Error message here        │

❌ WRONG (no padding compensation):
│ ✅ Success message here│
│ ❌ Error message here │

Box-Drawing Characters Reference:

• Corners: ┌ ┐ └ ┘
• Lines: ─ (horizontal), │ (vertical)
• Intersections: ┬ ┴ ├ ┤ ┼
• Arrows: → ← ↑ ↓ ▼ ▲

Quality Checklist for Diagrams

Before committing, verify each diagram:

• Top border width matches bottom border width
• All content lines have consistent right-edge │ alignment
• Nested boxes align with outer box boundaries
• Table columns have uniform widths
• Side-by-side boxes are visually balanced
• Emojis have appropriate padding

F. Implementation Sections

# %% [markdown]
"""
## Step N: [Component Name]

### Mathematical Foundation
Mathematical explanation with formulas and intuition.

### Implementation Strategy
Step-by-step approach to building the component.
"""

# %% nbgrader={"grade": false, "grade_id": "component-implementation", "locked": false, "schema_version": 3, "solution": true, "task": false}
#| export
class ComponentName:
    """
    Brief description of the component.
    
    TODO: Student implementation guidance
    
    APPROACH:
    1. [First step with specific guidance]
    2. [Second step with specific guidance]
    3. [Third step with specific guidance]
    
    EXAMPLE:
    Input: [concrete example]
    Expected: [concrete expected output]
    """
    def __init__(self, parameter1, parameter2):
        ### BEGIN SOLUTION
        # Complete implementation (hidden from students)
        ### END SOLUTION
        raise NotImplementedError("Student implementation required")

G. Test Functions

# %% [markdown]
"""
### 🧪 Unit Test: Component Name

**Description**: Brief explanation of what is tested

**This is a unit test** - it tests [specific functionality] in isolation.
"""

# %% nbgrader={"grade": true, "grade_id": "test-component", "locked": true, "points": 10, "schema_version": 3, "solution": false, "task": false}
def test_component_function():
    """Test the component functionality."""
    print("🔬 Unit Test: Component Function...")
    
    # Test implementation
    try:
        # Test logic
        assert condition, "Error message"
        print("✅ Component test works")
        print("📈 Progress: Component ✓")
        return True
    except Exception as e:
        print(f"❌ Component test failed: {e}")
        return False

# Test execution
if __name__ == "__main__":
    test_component_function()

H. Module Summary (CRITICAL)

# %% [markdown]
"""
## 🎯 Module Summary

Congratulations! You've successfully implemented [module description]:

### ✅ What You've Built
- **Component 1**: Description of accomplishment
- **Component 2**: Description of accomplishment  
- **Integration**: How components work together
- **Complete System**: End-to-end functionality

### ✅ Key Learning Outcomes
- **Understanding**: Core concepts mastered
- **Implementation**: Technical skills developed
- **Mathematical mastery**: Formulas and algorithms implemented
- **Real-world application**: Practical applications understood

### ✅ Mathematical Foundations Mastered
- **Formula 1**: Mathematical concept with notation
- **Formula 2**: Another key mathematical insight
- **Algorithm**: Implementation of key algorithm

### ✅ Professional Skills Developed
- **Skill 1**: Technical capability gained
- **Skill 2**: Another professional competency
- **Integration**: Systems thinking and design

### ✅ Ready for Advanced Applications
Your implementations now enable:
- **Application 1**: What students can build next
- **Application 2**: Another capability unlocked
- **Real Systems**: Connection to production applications

### 🔗 Connection to Real ML Systems
Your implementations mirror production systems:
- **PyTorch**: `torch.component` provides identical functionality
- **TensorFlow**: Similar concepts in TensorFlow
- **Industry Standard**: Used in major ML frameworks

### 🎯 The Power of [Technology]
You've unlocked the key technology that [impact description]:
- **Capability 1**: What this enables
- **Capability 2**: Another important capability
- **Scale**: How this technology scales

### 🧠 Deep Learning Revolution/Impact
You now understand the technology that [revolutionary impact]:
- **Historical context**: Before/after this technology
- **Modern applications**: Current uses
- **Future implications**: What this enables

### 🚀 What's Next
Your implementations are the foundation for:
- **Next Module**: Natural progression
- **Advanced Topics**: Related advanced concepts
- **Research**: Opportunities for exploration

**Next Module**: [Description of next module and its connection]

[Motivational closing emphasizing what students have accomplished]
"""

🧪 Testing Standards

1. Test Function Naming

All test functions MUST follow this pattern:

def test_component_name():
    """Test the component functionality."""

2. Test Function Structure

def test_component_function():
    """Test description."""
    print("🔬 Unit Test: Component Function...")
    
    ### 🧪 Unit Test: Component Function
    
    **Description**: Brief explanation of what is tested
    
    **This is a unit test** - it tests [specific functionality] in isolation.
    
    try:
        # Test logic
        print("✅ [check] works")
        print("📈 Progress: Component ✓")
        return True
    except Exception as e:
        print(f"❌ Test failed: {e}")
        return False

3. Test Execution

if __name__ == "__main__":
    test_function_1()
    test_function_2()
    test_function_3()

📦 NBDev Integration

1. Export Directives

#| export
def function_to_export():
    """Function that becomes part of tinytorch package."""
    pass

2. Default Export Target

#| default_exp core.modulename

3. NBGrader Integration

# %% nbgrader={"grade": false, "grade_id": "unique-id", "locked": false, "schema_version": 3, "solution": true, "task": false}

4. Solution Hiding (NBGrader)

def student_function():
    """
    Student implementation function.
    
    TODO: Implementation guidance for students.
    """
    ### BEGIN SOLUTION
    # Complete implementation hidden from students
    ### END SOLUTION
    raise NotImplementedError("Student implementation required")

🔧 Development Workflow

1. Python-First Development

• Work in .py files (source of truth)
• Generate .ipynb with tito nbgrader generate
• Never commit .ipynb files to version control

2. Testing Integration

• Use inline tests for immediate feedback
• All tests must pass before module completion
• Use pytest for any external testing

📋 Module Metadata (module.yaml)

name: "modulename"
title: "Module Title"
description: "Brief description of module functionality"
version: "1.0.0"
author: "TinyTorch Team"

learning_objectives:
  - "Objective 1"
  - "Objective 2"

prerequisites:
  - "prerequisite_module"

metadata:
  difficulty: "intermediate"
  time_estimate: "4-6 hours"
  pedagogical_framework: "Build → Use → Understand"

concepts:
  - "concept1"
  - "concept2"

exports:
  - "ComponentName"
  - "helper_function"

files:
  main: "modulename_dev.py"
  readme: "README.md"

assessment:
  total_points: 50
  breakdown:
    component1: 20
    component2: 20
    integration: 10

next_modules:
  - "next_module"

✅ Quality Checklist

Before completing a module:

Content Requirements

• Jupytext percent format header
• Educational content with clear explanations
• Step-by-step implementation guidance
• Mathematical foundations explained
• Real-world applications discussed
• Complete module summary (following 08_optimizers pattern)

Technical Requirements

• All functions have docstrings
• NBGrader cells properly configured
• NBDev export directives in place
• Solution blocks use ### BEGIN SOLUTION / ### END SOLUTION
• Error handling implemented
• Type hints where appropriate

Testing Requirements

• All inline tests pass
• Test functions use standard naming (test_*)
• Test output follows emoji standards
• if __name__ == "__main__": block present
• Tests provide educational feedback

Documentation Requirements

• module.yaml properly configured
• README.md updated
• Learning objectives clear
• Prerequisites documented
• Export list accurate

📚 Additional Resources

• Reference Implementation: modules/08_optimizers/optimizers_dev.py
• NBGrader Documentation: NBGrader docs
• NBDev Documentation: NBDev docs
• TinyTorch CLI: Use tito --help for development commands

---

Remember: When in doubt, reference 08_optimizers - it follows all these patterns perfectly and serves as the living example of proper module structure.