github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-04-29 18:35:59 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

Cleanup: Remove old/unused files

Browse Source 
- Remove datasets analysis and download scripts (replaced by updated README)
- Remove archived book development documentation
- Remove module review reports (16_compression, 17_memoization)
This commit is contained in:
Vijay Janapa Reddi
2025-11-11 19:04:56 -05:00
parent aeb6638975
commit cb5ad9ccf1
14 changed files with 0 additions and 3923 deletions
Download Patch File Download Diff File Expand all files Collapse all files

127
docs/archive/book-development/THEME_DESIGN.md
View File

@@ -1,127 +0,0 @@
# TinyTorch Flame-Inspired Design System
## Design Philosophy
The TinyTorch website design is inspired by the flame logo, creating a warm, professional academic environment that reflects the educational nature of the framework while maintaining credibility and accessibility.
## Color Palette
### Primary Flame Colors (Extracted from Logo)
- **Flame Primary**: `#E85A34` - Main orange from the flame
- **Flame Secondary**: `#F97316` - Secondary warm orange
- **Flame Light**: `#FED7AA` - Light warm orange for backgrounds
- **Flame Yellow**: `#FCD34D` - Warm yellow from flame core
- **Flame Deep**: `#DC2626` - Deep red from flame base
### Professional Text Colors
- **Text Dark**: `#1F2937` - Primary text color
- **Text Medium**: `#4B5563` - Secondary text
- **Text Light**: `#6B7280` - Tertiary text
### Background System
- **Background Main**: `#F8F9FA` - Matches logo background
- **Background White**: `#FFFFFF` - Content areas
- **Background Warm**: `#FEF7F0` - Subtle warm backgrounds
- **Accent Gradient**: Subtle flame-inspired gradient
## Design Principles
### 1. Warm Professionalism
- Flame colors provide warmth without sacrificing academic credibility
- Subtle gradients and warm backgrounds create inviting learning environment
- Professional typography maintains educational standards
### 2. Clean Academic Lines
- **No curved borders** - maintains academic formality
- Clean rectangular layouts with flame-colored accents
- Consistent spacing and typography hierarchy
### 3. Flame-Inspired Accents
- **Left borders**: Flame gradients on content blocks, code, and admonitions
- **Progress indicators**: Flame gradient progress bars
- **Interactive elements**: Flame colors for hover states and focus
### 4. Subtle Visual Hierarchy
- **H1 headers**: Flame gradient underlines
- **H3 headers**: Flame primary color
- **Links**: Flame primary with deeper red hover
- **Buttons**: Flame primary background with professional styling
## Component Styling
### Navigation
- **Sidebar**: Flame primary accents for current/hover states
- **Header**: Clean white with flame-colored interactive elements
- **TOC**: No curves, flame-colored indicators
### Content Areas
- **Code blocks**: Warm background with flame gradient left border
- **Admonitions**: Flame-colored borders with warm backgrounds
- **Blockquotes**: Flame left border with warm background
### Interactive Elements
- **Buttons**: Flame primary background, clean professional styling
- **Focus states**: Flame-colored outlines
- **Selection**: Flame background for text selection
- **Hover effects**: Subtle flame-colored shadows and transforms
### Special Components
- **Achievement cards**: Flame left borders with hover animations
- **Learning path steps**: Flame indicators with warm backgrounds
- **Module badges**: Flame-colored completion indicators
- **CTA boxes**: Flame gradient backgrounds with flame borders
## Accessibility Features
### High Contrast Support
- Darker flame colors in high contrast mode
- Maintained readability standards
- WCAG AA compliance for color contrast
### Reduced Motion Support
- Disabled animations for users with motion sensitivity
- Static alternatives for all animated elements
### Focus Management
- Clear flame-colored focus indicators
- Keyboard navigation support
- Screen reader friendly markup
## Usage Guidelines
### Do's
- Use flame colors for accents and interactive elements
- Maintain warm, professional tone
- Keep backgrounds subtle and readable
- Use gradients sparingly for emphasis
### Don'ts
- Avoid intense orange that overwhelms content
- Don't use flame colors for large background areas
- Avoid curved borders (academic requirement)
- Don't compromise text readability for visual appeal
## Implementation Notes
### CSS Custom Properties
All flame colors are defined as CSS custom properties for consistent theming and easy maintenance.
### Browser Compatibility
- Gradient fallbacks for older browsers
- Progressive enhancement for modern features
- Mobile-responsive design
### Performance
- Minimal use of animations
- Optimized gradients and shadows
- Efficient CSS organization
## Relationship to TinyTorch Logo
The design system directly extracts colors from the TinyTorch flame logo:
- Orange/red flame colors for primary accents
- Yellow core colors for highlights and progress
- Maintains visual consistency with brand identity
- Creates cohesive experience from logo to full website
This creates a unified brand experience where the logo naturally fits within the overall design language.

452
docs/archive/book-development/convert_modules.py
View File

@@ -1,452 +0,0 @@
#!/usr/bin/env python3
"""
Convert TinyTorch modules to Jupyter Book chapters.
This script processes modules/source/*_dev.py files and converts them to
student-ready notebooks for the Jupyter Book, stripping solutions manually.
"""
import os
import sys
import json
import subprocess
import tempfile
from pathlib import Path
from typing import Dict, List, Any, Optional
# Add project root to path for imports
project_root = Path(__file__).parent.parent
sys.path.insert(0, str(project_root))
class ModuleConverter:
"""Convert TinyTorch modules to Jupyter Book chapters."""
def __init__(self):
# Use absolute paths relative to project root
project_root = Path(__file__).parent.parent
self.modules_dir = project_root / "modules/source"
self.book_dir = project_root / "book"
self.chapters_dir = self.book_dir / "chapters"
# Module to chapter mapping
self.module_mapping = {
"": {"title": "Development Environment", "filename": "01-setup"},
"01_tensor": {"title": "Tensors", "filename": "02-tensor"},
"02_activations": {"title": "Activations", "filename": "03-activations"},
"03_layers": {"title": "Layers", "filename": "04-layers"},
"05_networks": {"title": "Networks", "filename": "05-networks"},
"06_cnn": {"title": "CNNs", "filename": "06-cnn"},
"07_dataloader": {"title": "DataLoader", "filename": "07-dataloader"},
"08_autograd": {"title": "Autograd", "filename": "08-autograd"},
"09_optimizers": {"title": "Optimizers", "filename": "09-optimizers"},
"10_training": {"title": "Training", "filename": "10-training"},
"11_compression": {"title": "Compression", "filename": "11-compression"},
"12_kernels": {"title": "Kernels", "filename": "12-kernels"},
"13_benchmarking": {"title": "Benchmarking", "filename": "13-benchmarking"},
"14_mlops": {"title": "MLOps", "filename": "14-mlops"},
}
# Mapping from directory name to dev file name
self.dev_file_mapping = {
"": "setup_dev.py",
"01_tensor": "tensor_dev.py",
"02_activations": "activations_dev.py",
"03_layers": "layers_dev.py",
"05_networks": "networks_dev.py",
"06_cnn": "cnn_dev.py",
"07_dataloader": "dataloader_dev.py",
"08_autograd": "autograd_dev.py",
"09_optimizers": "optimizers_dev.py",
"10_training": "training_dev.py",
"11_compression": "compression_dev.py",
"12_kernels": "kernels_dev.py",
"13_benchmarking": "benchmarking_dev.py",
"14_mlops": "mlops_dev.py",
}
def convert_to_notebook(self, dev_file: Path) -> Optional[Path]:
"""Convert dev file to notebook using Jupytext."""
print(f"📝 Converting {dev_file.name} to notebook")
# Create temporary output file
temp_notebook = dev_file.with_suffix('.temp.ipynb')
# Use jupytext to convert .py to .ipynb
cmd = ["jupytext", "--to", "ipynb", str(dev_file.absolute()), "--output", str(temp_notebook.absolute())]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
print(f"❌ Failed to convert {dev_file} to notebook: {result.stderr}")
return None
return temp_notebook
def remove_solutions(self, notebook_path: Path) -> Path:
"""Remove solutions from notebook."""
with open(notebook_path, 'r') as f:
notebook = json.load(f)
# Process each cell
for cell in notebook.get('cells', []):
if cell.get('cell_type') == 'code':
source = cell.get('source', [])
new_source = []
in_solution = False
for line in source:
if '### BEGIN SOLUTION' in line:
in_solution = True
new_source.append(line)
new_source.append(' # YOUR CODE HERE\n')
new_source.append(' raise NotImplementedError()\n')
continue
elif '### END SOLUTION' in line:
in_solution = False
new_source.append(line)
continue
elif in_solution:
# Skip solution lines
continue
else:
new_source.append(line)
cell['source'] = new_source
# Save processed notebook
output_path = notebook_path.with_suffix('.student.ipynb')
with open(output_path, 'w') as f:
json.dump(notebook, f, indent=2)
return output_path
def add_binder_config(self, notebook: Dict[str, Any], module_name: str) -> Dict[str, Any]:
"""Add Binder configuration to notebook metadata."""
if 'metadata' not in notebook:
notebook['metadata'] = {}
notebook['metadata'].update({
'kernelspec': {
'display_name': 'Python 3',
'language': 'python',
'name': 'python3'
},
'language_info': {
'name': 'python',
'version': '3.8+'
},
'mystnb': {
'execution_mode': 'auto'
}
})
return notebook
def extract_learning_goals(self, dev_file: Path) -> str:
"""Extract learning goals from source file and format as admonition block."""
with open(dev_file, 'r') as f:
content = f.read()
# Find the Learning Goals section
goals_start = content.find('## Learning Goals\n')
if goals_start == -1:
return ""
# Find the end of the goals section (next ## heading)
goals_content_start = goals_start + len('## Learning Goals\n')
next_section = content.find('\n## ', goals_content_start)
if next_section == -1:
# If no next section found, look for next markdown cell
next_section = content.find('\n# %%', goals_content_start)
if next_section == -1:
goals_text = content[goals_content_start:].strip()
else:
goals_text = content[goals_content_start:next_section].strip()
# Format as admonition block
admonition = ['```{admonition} 🎯 Learning Goals\n']
admonition.append(':class: tip\n')
for line in goals_text.split('\n'):
if line.strip():
admonition.append(f'{line}\n')
admonition.append('```\n\n')
return ''.join(admonition)
def extract_module_overview(self, dev_file: Path) -> str:
"""Extract first markdown cell content for book overview."""
with open(dev_file, 'r') as f:
content = f.read()
# Find first markdown cell
start = content.find('# %% [markdown]\n"""')
if start == -1:
return ""
end = content.find('"""', start + 20)
if end == -1:
return ""
# Extract and clean the content
overview = content[start + len('# %% [markdown]\n"""'):end].strip()
# Replace Learning Goals section with admonition block
learning_goals = self.extract_learning_goals(dev_file)
if learning_goals and '## Learning Goals' in overview:
# Find and replace the Learning Goals section
goals_start = overview.find('## Learning Goals')
if goals_start != -1:
# Find end of goals section
next_section = overview.find('\n## ', goals_start + 1)
if next_section == -1:
# Goals are at the end
overview = overview[:goals_start] + learning_goals
else:
# Replace goals section with admonition
overview = (overview[:goals_start] +
learning_goals +
overview[next_section:])
return overview
def create_module_overview_page(self, module_name: str) -> bool:
"""Create a module overview page for the book (hybrid approach)."""
if module_name not in self.module_mapping:
return False
module_dir = self.modules_dir / module_name
dev_file_name = self.dev_file_mapping.get(module_name)
if not dev_file_name:
return False
dev_file = module_dir / dev_file_name
if not dev_file.exists():
return False
module_info = self.module_mapping[module_name]
# Extract overview content
overview = self.extract_module_overview(dev_file)
# Create interactive launch buttons
github_url = f"https://github.com/mlsysbook/TinyTorch/blob/main/modules/source/{module_name}/{dev_file_name}"
binder_url = f"https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/source/{module_name}/{dev_file_name.replace('.py', '.ipynb')}"
colab_url = f"https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/source/{module_name}/{dev_file_name.replace('.py', '.ipynb')}"
interactive_section = f"""
## 🚀 Interactive Learning
Choose your preferred way to engage with this module:
````{{grid}} 1 2 3 3
```{{grid-item-card}} 🚀 Launch Binder
:link: {binder_url}
:class-header: bg-light
Run this module interactively in your browser. No installation required!
```
```{{grid-item-card}} ⚡ Open in Colab
:link: {colab_url}
:class-header: bg-light
Use Google Colab for GPU access and cloud compute power.
```
```{{grid-item-card}} 📖 View Source
:link: {github_url}
:class-header: bg-light
Browse the Python source code and understand the implementation.
```
````
```{{admonition}} 💾 Save Your Progress
:class: tip
**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
Ready for serious development? → [🏗️ Local Setup Guide](../usage-paths/serious-development.md)
```
"""
# Combine everything
page_content = overview + interactive_section
# Save to chapters directory
self.chapters_dir.mkdir(parents=True, exist_ok=True)
output_file = self.chapters_dir / f"{module_info['filename']}.md"
with open(output_file, 'w') as f:
f.write(page_content)
print(f"✅ Created overview page: {output_file}")
return True
def add_book_frontmatter(self, notebook: Dict[str, Any], module_name: str, title: str) -> Dict[str, Any]:
"""Add Jupyter Book frontmatter to the notebook."""
# Create interactive learning admonition
interactive_cell = {
'cell_type': 'markdown',
'metadata': {},
'source': [
'```{admonition} Interactive Learning\n',
':class: tip\n',
'🚀 **Launch Binder**: Click the rocket icon above to run this chapter interactively!\n',
'\n',
'💾 **Save Your Work**: Download your completed notebook when done.\n',
'\n',
'🏗️ **Build Locally**: Ready for serious development? [Fork the repo](https://github.com/your-org/tinytorch) and work locally with the full `tito` workflow.\n',
'```\n',
'\n'
]
}
# Insert interactive cell after the first title cell
cells = notebook.get('cells', [])
# Find the first title cell and add interactive cell after it
title_found = False
for i, cell in enumerate(cells):
if cell.get('cell_type') == 'markdown':
source = ''.join(cell.get('source', []))
if source.startswith('# '):
# Insert interactive cell after the title
cells.insert(i + 1, interactive_cell)
title_found = True
break
if not title_found:
cells.insert(0, interactive_cell)
notebook['cells'] = cells
return notebook
def convert_module(self, module_name: str) -> bool:
"""Convert a single module to a chapter."""
if module_name not in self.module_mapping:
print(f"❌ Unknown module: {module_name}")
return False
module_dir = self.modules_dir / module_name
if not module_dir.exists():
print(f"❌ Module directory not found: {module_dir}")
return False
# Get the dev file name for this module
dev_file_name = self.dev_file_mapping.get(module_name)
if not dev_file_name:
print(f"❌ No dev file mapping for {module_name}")
return False
dev_file = module_dir / dev_file_name
if not dev_file.exists():
print(f"❌ Dev file not found: {dev_file}")
return False
print(f"🔄 Converting {module_name}: {dev_file}")
try:
# Convert to notebook
notebook_path = self.convert_to_notebook(dev_file)
if not notebook_path:
return False
# Keep solutions (no NBGrader processing)
# student_notebook_path = self.remove_solutions(notebook_path) # Disabled - keep solutions
# Load the full notebook with solutions
with open(notebook_path, 'r') as f:
notebook = json.load(f)
# Add book-specific enhancements
module_info = self.module_mapping[module_name]
notebook = self.add_binder_config(notebook, module_name)
# notebook = self.add_book_frontmatter(notebook, module_name, module_info['title']) # Disabled for raw export
# Save to chapters directory
self.chapters_dir.mkdir(parents=True, exist_ok=True)
output_file = self.chapters_dir / f"{module_info['filename']}.ipynb"
with open(output_file, 'w') as f:
json.dump(notebook, f, indent=2)
print(f"✅ Created chapter: {output_file}")
# Clean up temporary files
notebook_path.unlink(missing_ok=True)
return True
except Exception as e:
print(f"❌ Error converting {module_name}: {e}")
return False
def convert_all_modules(self) -> bool:
"""Convert all available modules."""
print("🔄 Converting all TinyTorch modules to Jupyter Book chapters...")
success_count = 0
total_count = 0
for module_name in self.module_mapping.keys():
total_count += 1
if self.convert_module(module_name):
success_count += 1
print(f"\n📊 Conversion Summary:")
print(f" ✅ Success: {success_count}/{total_count} modules")
print(f" 📁 Output: {self.chapters_dir}")
return success_count == total_count
def main():
"""Main conversion script."""
import argparse
parser = argparse.ArgumentParser(description="Convert TinyTorch modules to Jupyter Book")
parser.add_argument('--module', help='Convert specific module (e.g., )')
parser.add_argument('--all', action='store_true', help='Convert all modules')
parser.add_argument('--overview', action='store_true', help='Create overview pages instead of full notebooks')
parser.add_argument('--overview-module', help='Create overview page for specific module')
args = parser.parse_args()
converter = ModuleConverter()
if args.overview_module:
success = converter.create_module_overview_page(args.overview_module)
sys.exit(0 if success else 1)
elif args.overview:
# Create overview pages for all modules
print("🔄 Creating module overview pages for Jupyter Book...")
success_count = 0
total_count = 0
for module_name in converter.module_mapping.keys():
total_count += 1
if converter.create_module_overview_page(module_name):
success_count += 1
print(f"\n📊 Overview Creation Summary:")
print(f" ✅ Success: {success_count}/{total_count} modules")
print(f" 📁 Output: {converter.chapters_dir}")
success = success_count == total_count
sys.exit(0 if success else 1)
elif args.module:
success = converter.convert_module(args.module)
sys.exit(0 if success else 1)
elif args.all:
success = converter.convert_all_modules()
sys.exit(0 if success else 1)
else:
parser.print_help()
sys.exit(1)
if __name__ == "__main__":
main()

298
docs/archive/book-development/convert_readmes.py
View File

@@ -1,298 +0,0 @@
#!/usr/bin/env python3
"""
Convert module READMEs to Jupyter Book chapters.
This script takes README files from modules/source/*/README.md and converts them
to Jupyter Book chapters in book/chapters/ with proper frontmatter and web optimization.
"""
import os
import re
import yaml
from pathlib import Path
from typing import Dict, List, Optional
def get_module_info(module_path: Path) -> Dict[str, str]:
"""Extract module information from module.yaml file."""
yaml_path = module_path / "module.yaml"
if yaml_path.exists():
with open(yaml_path, 'r') as f:
module_data = yaml.safe_load(f)
return {
'title': module_data.get('title', module_path.name.replace('_', ' ').title()),
'description': module_data.get('description', ''),
'difficulty': module_data.get('difficulty', 'Intermediate'),
'time_estimate': module_data.get('time_estimate', '2-4 hours'),
'prerequisites': module_data.get('prerequisites', []),
'next_steps': module_data.get('next_steps', [])
}
return {}
def extract_learning_objectives(content: str) -> List[str]:
"""Extract learning objectives from README content."""
objectives = []
# Look for common patterns in READMEs
patterns = [
r'By the end of this module, you will:?\s*\n((?:- [^\n]+\n?)+)',
r'Learning Goals?:?\s*\n((?:- [^\n]+\n?)+)',
r'Learning Objectives?:?\s*\n((?:- [^\n]+\n?)+)'
]
for pattern in patterns:
match = re.search(pattern, content, re.IGNORECASE | re.MULTILINE)
if match:
objectives_text = match.group(1)
objectives = [line.strip('- ').strip() for line in objectives_text.split('\n') if line.strip().startswith('-')]
break
return objectives
def create_frontmatter(module_name: str, module_info: Dict[str, str], objectives: List[str]) -> str:
"""Create Jupyter Book frontmatter for the chapter."""
# Clean up module name for title
title = module_info.get('title', module_name.replace('_', ' ').title())
frontmatter = f"""---
title: "{title}"
description: "{module_info.get('description', '')}"
difficulty: "{module_info.get('difficulty', 'Intermediate')}"
time_estimate: "{module_info.get('time_estimate', '2-4 hours')}"
prerequisites: {module_info.get('prerequisites', [])}
next_steps: {module_info.get('next_steps', [])}
learning_objectives: {objectives}
---
"""
return frontmatter
def enhance_content_for_web(content: str, module_name: str, module_num: int) -> str:
"""Enhance README content for web presentation."""
# Remove existing grid cards to prevent conflicts with new interactive elements
# Pattern to match grid sections (from ```{grid} to closing ```)
grid_pattern = r'```\{grid\}[^`]*?```'
content = re.sub(grid_pattern, '', content, flags=re.DOTALL)
# Also remove individual grid-item-card patterns that might be floating
grid_item_pattern = r'\{grid-item-card\}[^`]*?```'
content = re.sub(grid_item_pattern, '', content, flags=re.DOTALL)
# Clean up any remaining grid-related patterns
content = re.sub(r'\{grid-item-card\}[^\n]*\n', '', content)
content = re.sub(r':link:[^\n]*\n', '', content)
content = re.sub(r':class-[^:]*:[^\n]*\n', '', content)
# Clean up multiple newlines that result from removals
content = re.sub(r'\n{3,}', '\n\n', content)
# Add badges for difficulty and time
difficulty = get_difficulty_stars(module_name)
time_estimate = get_time_estimate(module_name)
badges = f"\n```{{div}} badges\n{difficulty} | ⏱️ {time_estimate}\n```\n"
# Get previous and next module names for navigation
prev_module = f"{module_num-1:02d}_{get_prev_module_name(module_num)}" if module_num > 1 else None
# Add interactive learning elements and navigation at the end
interactive_elements = f"""
Choose your preferred way to engage with this module:
````{{grid}} 1 2 3 3
```{{grid-item-card}} 🚀 Launch Binder
:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/source/{module_name}/{module_name.split('_', 1)[1]}_dev.ipynb
:class-header: bg-light
Run this module interactively in your browser. No installation required!
```
```{{grid-item-card}} ⚡ Open in Colab
:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/source/{module_name}/{module_name.split('_', 1)[1]}_dev.ipynb
:class-header: bg-light
Use Google Colab for GPU access and cloud compute power.
```
```{{grid-item-card}} 📖 View Source
:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/source/{module_name}/{module_name.split('_', 1)[1]}_dev.py
:class-header: bg-light
Browse the Python source code and understand the implementation.
```
````
```{{admonition}} 💾 Save Your Progress
:class: tip
**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
Ready for serious development? → [🏗️ Local Setup Guide](../usage-paths/serious-development.md)
```
---
"""
# Add navigation links
nav_links = "<div class=\"prev-next-area\">\n"
if prev_module:
nav_links += f'<a class="left-prev" href="../chapters/{prev_module}.html" title="previous page">← Previous Module</a>\n'
# Get total number of modules dynamically
module_names = get_module_names()
if module_num < len(module_names):
next_module = f"{module_num+1:02d}_{get_next_module_name(module_num)}"
nav_links += f'<a class="right-next" href="../chapters/{next_module}.html" title="next page">Next Module →</a>\n'
nav_links += "</div>\n"
# Combine interactive elements with navigation
nav_links = interactive_elements + nav_links
# Insert badges after the first heading
lines = content.split('\n')
enhanced_lines = []
added_badges = False
for i, line in enumerate(lines):
# Keep the meaningful module headers but clean up the breadcrumb reference
if line.startswith('# ') and not added_badges:
# Keep "Module: CNN" format, just remove emoji for clean display
if '🔥 Module:' in line:
line = line.replace('🔥 ', '') # Remove emoji, keep "Module: CNN"
enhanced_lines.append(line)
# Add badges after first heading
if not added_badges and line.startswith('# '):
enhanced_lines.append(badges)
added_badges = True
# Add navigation at the end
enhanced_lines.append(nav_links)
return '\n'.join(enhanced_lines)
def get_difficulty_stars(module_name: str) -> str:
"""Get difficulty stars from module.yaml file."""
# Map module number to module folder name
module_path = Path(f'../modules/source/{module_name}')
module_info = get_module_info(module_path)
return module_info.get('difficulty', '⭐⭐')
def get_time_estimate(module_name: str) -> str:
"""Get time estimate from module.yaml file."""
# Map module number to module folder name
module_path = Path(f'../modules/source/{module_name}')
module_info = get_module_info(module_path)
return module_info.get('time_estimate', '3-4 hours')
def get_module_names() -> List[str]:
"""Get actual module names from module.yaml files."""
modules_dir = Path("../modules/source")
module_names = []
# Get all module directories (sorted by number)
module_dirs = []
for item in modules_dir.iterdir():
if item.is_dir() and item.name != 'utils':
# Extract module number from directory name
match = re.match(r'(\d+)_(.+)', item.name)
if match:
module_num = int(match.group(1))
module_dirs.append((module_num, item))
# Sort by module number
module_dirs.sort(key=lambda x: x[0])
# Read module names from module.yaml files
for module_num, module_dir in module_dirs:
module_yaml_path = module_dir / "module.yaml"
if module_yaml_path.exists():
module_info = get_module_info(module_dir)
module_names.append(module_info.get('name', module_dir.name.split('_', 1)[1]))
else:
# Fallback to directory name
module_names.append(module_dir.name.split('_', 1)[1])
return module_names
def get_prev_module_name(module_num: int) -> str:
"""Get previous module name."""
module_names = get_module_names()
return module_names[module_num - 2] if module_num > 1 and module_num - 2 < len(module_names) else 'setup'
def get_next_module_name(module_num: int) -> str:
"""Get next module name."""
module_names = get_module_names()
return module_names[module_num] if module_num < len(module_names) else module_names[-1] if module_names else 'setup'
def convert_readme_to_chapter(readme_path: Path, chapter_path: Path, module_num: int):
"""Convert a single README to a Jupyter Book chapter."""
print(f"Converting {readme_path} to {chapter_path}")
# Read README content
with open(readme_path, 'r', encoding='utf-8') as f:
content = f.read()
# Get module information
module_path = readme_path.parent
module_name = module_path.name
module_info = get_module_info(module_path)
# Extract learning objectives
objectives = extract_learning_objectives(content)
# Create frontmatter
frontmatter = create_frontmatter(module_name, module_info, objectives)
# Enhance content for web
enhanced_content = enhance_content_for_web(content, module_name, module_num)
# Write chapter file
with open(chapter_path, 'w', encoding='utf-8') as f:
f.write(frontmatter)
f.write(enhanced_content)
print(f"✅ Created {chapter_path}")
def main():
"""Convert all module READMEs to Jupyter Book chapters."""
# Setup paths
modules_dir = Path("../modules/source")
chapters_dir = Path("chapters")
# Ensure chapters directory exists
chapters_dir.mkdir(exist_ok=True)
# Get all module directories (sorted by number)
module_dirs = []
for item in modules_dir.iterdir():
if item.is_dir() and item.name != 'utils':
# Extract module number from directory name
match = re.match(r'(\d+)_(.+)', item.name)
if match:
module_num = int(match.group(1))
module_dirs.append((module_num, item))
# Sort by module number
module_dirs.sort(key=lambda x: x[0])
print(f"Found {len(module_dirs)} modules to convert")
# Convert each README
for module_num, module_dir in module_dirs:
readme_path = module_dir / "README.md"
if readme_path.exists():
# Create chapter filename (just module number and name, no duplicate)
chapter_filename = f"{module_num:02d}-{module_dir.name.split('_', 1)[1]}.md"
chapter_path = chapters_dir / chapter_filename
convert_readme_to_chapter(readme_path, chapter_path, module_num)
else:
print(f"⚠️ No README.md found in {module_dir}")
print(f"\n🎉 Converted {len(module_dirs)} modules to chapters in {chapters_dir}")
if __name__ == "__main__":
main()

663
docs/archive/book-development/faq.md
View File

@@ -1,663 +0,0 @@
# Frequently Asked Questions
## 🤔 Getting Started Questions
### **Installation & Setup**
**Q: I'm getting "tito: command not found" - what's wrong?**
A: This usually means your virtual environment isn't activated or TinyTorch isn't installed:
```bash
# 1. Activate virtual environment
source .venv/bin/activate # Windows: .venv\Scripts\activate
# 2. Install TinyTorch
pip install -e .
# 3. Verify installation
tito system doctor
```
**Q: What Python version do I need?**
A: Python 3.8 or higher. Check with:
```bash
python --version # Should show 3.8+
```
**Q: Can I use conda instead of venv?**
A: Yes! Replace the venv setup with:
```bash
conda create -n tinytorch python=3.9
conda activate tinytorch
pip install -r requirements.txt && pip install -e .
```
**Q: The installation is taking forever - is this normal?**
A: Initial setup typically takes 2-5 minutes depending on your connection. The main time is downloading NumPy, Jupyter, and other scientific packages.
---
## 📚 Learning Questions
### **Course Structure**
**Q: How long does TinyTorch take to complete?**
A: Depends on your goals and pace:
| **Goal** | **Time** | **Coverage** | **What You'll Build** |
|----------|----------|--------------|----------------------|
| **Quick Taste** | 15 minutes | Demo + overview | See framework in action |
| **Weekend Project** | 8-12 hours | Modules 1-6 | Neural network solver |
| **Neural Networks** | 4 weeks | Modules 1-8 | MNIST classifier |
| **Computer Vision** | 6 weeks | Modules 1-10 | CIFAR-10 CNN |
| **Language Models** | 8 weeks | Modules 1-14 | TinyGPT generator |
| **Full Framework** | 12 weeks | All 20 modules | Production-ready system |
**Q: Do I need machine learning experience to start?**
A: **No!** TinyTorch teaches ML systems from fundamentals. You need:
**✅ Required:**
- Basic Python (functions, classes, imports)
- High school math (multiplication, basic algebra)
- Curiosity about how things work
**❌ Not Required:**
- Previous ML experience
- Deep learning knowledge
- Advanced mathematics
- PyTorch/TensorFlow experience
**Q: Can I skip modules or do them out of order?**
A: **No** - the progression is carefully designed:
- Each module builds on previous implementations
- Later modules import code from earlier ones
- Checkpoints verify prerequisites are met
- Skipping creates import errors and broken functionality
**Example:** Module 6 (Autograd) requires your Tensor class from Module 2. Skipping Module 2 breaks everything that follows.
**Q: What if I get stuck on a difficult concept?**
A: Multiple support options:
1. **Interactive Help**: `tito help --interactive` for personalized guidance
2. **Module README**: Each module has detailed explanations
3. **Community Support**: Join leaderboard for peer help
4. **Troubleshooting**: `tito help troubleshooting` for common issues
5. **Office Hours**: If taking as a course, use instructor support
### **Learning Methods**
**Q: Should I read everything before coding, or jump right into coding?**
A: **Jump into coding!** TinyTorch uses active learning:
- Read just enough to understand the task
- Start implementing immediately
- Learn through building and testing
- Explanations become clearer after you've tried the code
**Q: How much time should I spend on each module?**
A: Varies by module and experience:
| **Module Type** | **Typical Time** | **Examples** |
|----------------|------------------|--------------|
| **Foundation** | 2-4 hours | Tensors, Activations |
| **Architecture** | 3-5 hours | Layers, Training |
| **Advanced** | 4-6 hours | Attention, Transformers |
| **Optimization** | 2-3 hours | Profiling, Benchmarking |
**Don't rush!** Deep understanding matters more than speed.
**Q: What's the difference between modules and checkpoints?**
A: **Modules** = Building, **Checkpoints** = Validating
| **Modules** | **Checkpoints** |
|-------------|-----------------|
| 20 hands-on coding sessions | 16 capability assessments |
| You build implementations | Tests verify understanding |
| `tito module complete 05` | `tito checkpoint test 05` |
| Export code to framework | Validate you achieved capability |
**Workflow:** Complete module → Export implementation → Checkpoint test validates learning
---
## 🛠️ Technical Questions
### **Development Workflow**
**Q: Why can't I edit files in the `tinytorch/` directory?**
A: Those files are **auto-generated** from your source modules:
**✅ Edit Here:**
```
modules/02_tensor/tensor_dev.py ← Your source code
```
**❌ Don't Edit:**
```
tinytorch/core/tensor.py ← Generated from source
```
**Workflow:**
1. Edit source: `modules/0X_name/name_dev.py`
2. Export: `tito module complete 0X_name`
3. Uses your code: `from tinytorch.core.name import Component`
**Q: What's the difference between .py and .ipynb files?**
A: **TinyTorch uses .py files only** for all development:
- **Source**: `tensor_dev.py` (edit this)
- **Generated**: `tensor_dev.ipynb` (auto-created from .py)
- **Never edit**: `.ipynb` files directly
**Why .py only?**
- Clean version control (no JSON metadata)
- Professional development practices
- Consistent environment across contributors
- Easy code review and collaboration
**Q: My tests are failing after implementing a function - what's wrong?**
A: Common debugging steps:
1. **Check syntax**: Run the module file directly
```bash
python modules/03_activations/activations_dev.py
```
2. **Verify function signature**: Make sure your function matches the expected interface
```python
# Expected
def relu(x: np.ndarray) -> np.ndarray:
# Not this
def relu(x): # Missing type hints
```
3. **Test incrementally**: Run tests after each function
```bash
tito checkpoint test 02 --verbose
```
4. **Check imports**: Ensure NumPy is imported as `np`
**Q: How do I run just one test instead of all tests?**
A: Use specific test commands:
```bash
# Test specific checkpoint
tito checkpoint test 03
# Test specific module export
tito module complete 03_activations --dry-run
# Run module file directly
python modules/03_activations/activations_dev.py
```
### **System Issues**
**Q: Jupyter Lab won't start - what's wrong?**
A: Common solutions:
1. **Check installation**:
```bash
pip install jupyterlab jupyter
jupyter lab --version
```
2. **Port conflict**:
```bash
jupyter lab --port 8889 # Try different port
```
3. **Virtual environment**:
```bash
source .venv/bin/activate # Ensure activated
which jupyter # Should show .venv path
```
**Q: I'm getting import errors when testing - help!**
A: Import errors usually mean:
1. **Virtual environment not activated**:
```bash
source .venv/bin/activate
```
2. **TinyTorch not installed in development mode**:
```bash
pip install -e . --force-reinstall
```
3. **Module not exported**:
```bash
tito module complete 0X_module_name
```
4. **Check your export directive**:
```python
#| default_exp tinytorch.core.module_name # At top of file
```
---
## 🌍 Community Questions
### **Leaderboard & Community**
**Q: Is the leaderboard competitive or supportive?**
A: **Both!** We designed it to be inclusive and encouraging:
**🏆 Multiple Ways to Excel:**
- **Progress**: Checkpoint completion (everyone can achieve)
- **Speed**: Fast learners (if that's your style)
- **Innovation**: Creative optimizations (for advanced users)
- **Community**: Helping others (valuable contribution)
**🤝 Supportive Culture:**
- Celebrate all achievements, not just "first place"
- Anonymous participation options available
- Community helps each other learn
- No shame in taking time to understand concepts
**Q: Do I have to share my progress publicly?**
A: **No!** Participation is entirely optional:
- All learning features work without leaderboard
- Checkpoint system tracks progress locally
- Join community only when/if you want to
- Privacy controls let you share what you're comfortable with
**Q: What information is shared when I join the leaderboard?**
A: You control what's shared:
**Always Shared:**
- Display name (you choose - can be pseudonymous)
- Checkpoint completion status
- Module completion dates
**Optionally Shared:**
- Real name (if you choose)
- Institution/company
- Achievement celebrations
- Optimization benchmarks
**Never Shared:**
- Personal information
- Email addresses
- Code implementations
- Detailed progress metrics (unless you opt in)
### **Competition & Olympics**
**Q: What are the Olympics and how are they different from the leaderboard?**
A: **Leaderboard** = Learning Progress, **Olympics** = Performance Competition
| **Leaderboard** | **Olympics** |
|-----------------|--------------|
| Track learning progress | Compete on optimization |
| Checkpoint completion | Benchmark performance |
| Supportive community | Competitive challenges |
| All experience levels | Advanced optimization |
**Olympics Events:**
- **MLP Sprint**: Fastest matrix operations
- **CNN Marathon**: Memory-efficient convolutions
- **Transformer Decathlon**: Complete language model optimization
**Q: Do I need to be an expert to participate in Olympics?**
A: **No!** Olympics have multiple categories:
- **Beginner**: Just-working implementations compete
- **Intermediate**: Solid optimizations
- **Advanced**: Cutting-edge techniques
- **Innovation**: Novel approaches
**Everyone can contribute and learn from others' solutions.**
---
## 🎓 Instructor Questions
### **Classroom Setup**
**Q: How much setup is required to use TinyTorch in my class?**
A: **Minimal!** TinyTorch includes complete teaching infrastructure:
**One-time Setup (30 minutes):**
```bash
tito nbgrader setup-instructor
tito grade setup-course
```
**Per-semester Setup (10 minutes):**
```bash
tito nbgrader create-student-repos
tito grade release-module 01_setup
```
**Everything Included:**
- NBGrader integration works out-of-the-box
- Student progress tracking built-in
- Automated grading workflow
- Assignment release/collection system
**Q: Can I customize the curriculum for my specific course?**
A: **Absolutely!** TinyTorch is designed for flexibility:
**Duration Options:**
- **4 weeks**: Neural network foundations (Modules 1-8)
- **8 weeks**: Add computer vision (Modules 1-10)
- **12 weeks**: Include language models (Modules 1-14)
- **16 weeks**: Complete system optimization (All 20)
**Difficulty Customization:**
- **Beginner**: Additional scaffolding and explanations
- **Advanced**: Extra optimization challenges
- **Research**: Custom project integration
**Q: How do I track student progress across the class?**
A: Multiple tracking tools built-in:
```bash
# Class overview
tito grade class-overview
# Individual student
tito grade student-progress student_name
# Checkpoint statistics
tito checkpoint class-stats
# Module completion rates
tito grade module-stats 05_losses
```
**Visual dashboards show:**
- Who's completed which modules
- Where students are getting stuck
- Average completion times
- Achievement distributions
### **Grading & Assessment**
**Q: How does automated grading work?**
A: **Three-layer validation system:**
1. **Functional Tests**: Does the code work correctly?
2. **Interface Tests**: Does it match expected function signatures?
3. **Checkpoint Tests**: Can student use their implementation?
```bash
# Grade student submission
tito nbgrader autograde 05_losses student_name
# Results show:
# ✅ Function implementation (40 points)
# ✅ Interface compliance (30 points)
# ✅ Integration test (30 points)
# Total: 100/100
```
**Q: What if a student's implementation works but doesn't match the test exactly?**
A: **Flexible grading system:**
- **Core functionality**: Must work correctly (non-negotiable)
- **Implementation details**: Multiple valid approaches accepted
- **Code style**: Guidance provided, not penalized
- **Performance**: Bonus points for optimization, not required
**Manual review system** catches edge cases and provides personalized feedback.
**Q: How do I handle students working at different paces?**
A: **Built-in flexibility:**
**Self-paced Options:**
- Students can work ahead through modules
- Checkpoint system validates readiness for advanced topics
- Extra credit opportunities for early finishers
**Support for Struggling Students:**
- Extended deadlines through system configuration
- Additional scaffolding materials included
- Peer tutoring through community features
- Office hours integration with progress tracking
---
## 🔧 Troubleshooting
### **Common Error Messages**
**Error: `ModuleNotFoundError: No module named 'tinytorch'`**
**Solutions:**
```bash
# 1. Activate virtual environment
source .venv/bin/activate
# 2. Install in development mode
pip install -e .
# 3. Verify installation
python -c "import tinytorch; print('Success!')"
```
**Error: `AttributeError: module 'tinytorch.core.tensor' has no attribute 'Tensor'`**
**Cause:** Module not exported or export failed
**Solutions:**
```bash
# 1. Check export status
tito module status 02_tensor
# 2. Re-export module
tito module complete 02_tensor
# 3. Verify export worked
python -c "from tinytorch.core.tensor import Tensor; print('Success!')"
```
**Error: Tests pass individually but fail in checkpoint**
**Cause:** Integration issues between modules
**Solutions:**
```bash
# 1. Test integration
tito checkpoint test 05 --verbose
# 2. Check all dependencies exported
tito module status --all
# 3. Re-export dependency chain
tito module complete 02_tensor
tito module complete 03_activations
# ... up to current module
```
### **Performance Issues**
**Q: Training is really slow - is this normal?**
A: **Some slowness is expected** (you're building from scratch!), but here's how to optimize:
**Expected Performance:**
- **Pure NumPy**: 10-100x slower than PyTorch
- **Simple examples**: Should complete in seconds
- **CIFAR-10 training**: 5-10 minutes per epoch
**Optimization Tips:**
```python
# Use vectorized operations
result = np.dot(x, weights) # Fast
# Avoid Python loops
for i in range(len(x)): # Slow
result[i] = x[i] * weights[i]
```
**Q: My computer is running out of memory during training**
A: **Memory management strategies:**
1. **Reduce batch size**:
```python
batch_size = 32 # Instead of 256
```
2. **Use gradient accumulation**:
```python
# Accumulate gradients over mini-batches
optimizer.step_every_n_batches(4)
```
3. **Profile memory usage**:
```bash
tito checkpoint test 10 --profile-memory
```
---
## 💡 Best Practices
### **Learning Effectively**
**Q: What's the best way to approach each module?**
A: **Follow the Build → Use → Reflect pattern:**
**1. Build (Implementation)**
- Read the introduction to understand the goal
- Implement functions one at a time
- Test each function immediately after writing it
**2. Use (Integration)**
- Export your module: `tito module complete 0X_name`
- Test the integration with checkpoint
- Use your component in examples
**3. Reflect (Understanding)**
- Answer the ML Systems Thinking questions
- Consider memory usage and performance
- Connect to production ML systems
**Q: How do I know if I really understand a concept?**
A: **True understanding means you can:**
1. **Implement from memory**: Re-write the function without looking
2. **Explain to others**: Describe how and why it works
3. **Debug problems**: Fix issues when something breaks
4. **Optimize performance**: Improve memory or speed
5. **Connect to production**: Relate to PyTorch/TensorFlow internals
**Checkpoint tests verify some of this, but self-reflection is crucial.**
### **Time Management**
**Q: I'm spending too much time on implementation details - should I move on?**
A: **Balance depth with progress:**
**When to Push Through:**
- Core concepts not clicking yet
- Function doesn't work correctly
- Tests are failing
**When to Move On:**
- Function works and passes tests
- You understand the main concept
- You're optimizing minor details
**Remember:** You can always return to optimize later. The goal is understanding systems, not perfect code.
**Q: Should I complete all modules before starting real projects?**
A: **No!** Start projects as soon as you have the basics:
- **After Module 6**: Build XOR solver
- **After Module 8**: Train MNIST classifier
- **After Module 10**: CIFAR-10 CNN
- **After Module 14**: TinyGPT language model
**Real projects reinforce learning and show practical applications.**
---
## 🚀 Getting More Help
### **When These FAQs Don't Help**
**1. Interactive CLI Help**
```bash
tito help --interactive # Personalized guidance
tito help troubleshooting # Common technical issues
```
**2. System Diagnostics**
```bash
tito system doctor # Comprehensive system check
```
**3. Community Support**
- Join leaderboard for peer help and discussion
- Share specific error messages for targeted assistance
- Learn from others' solutions and approaches
**4. Documentation Resources**
- **Module README files**: Detailed explanations for each topic
- **User Manual**: Comprehensive guide to all features
- **Instructor Guide**: Teaching resources and classroom management
**5. Course Support (if applicable)**
- Office hours with instructor
- Class discussion forums
- Teaching assistant support
### **Reporting Issues**
**Found a bug or unclear documentation?**
Please include:
- **System info**: Output of `tito system doctor`
- **Error message**: Complete traceback if available
- **Steps to reproduce**: What commands led to the issue
- **Expected vs actual**: What you expected to happen
**Contact through:**
- Course instructor (if taking as class)
- Community leaderboard (for peer support)
- GitHub issues (for bug reports)
---
**Still have questions? Try `tito help --interactive` for personalized guidance! 🚀**

232
docs/archive/book-development/kiss-principle.md
View File

@@ -1,232 +0,0 @@
# KISS Principle in TinyTorch
## Keep It Simple, Stupid
The KISS principle is at the core of TinyTorch's design philosophy. Every component, interface, and implementation follows this fundamental rule: **simplicity enables understanding**.
## Why KISS Matters in ML Education
### Traditional ML Frameworks: Complexity by Default
Most production ML frameworks prioritize performance and features over clarity:
```python
# PyTorch: Multiple ways to do everything
torch.nn.Conv2d(3, 64, kernel_size=3, padding=1) # Object-oriented
F.conv2d(x, weight, bias, padding=1) # Functional
torch.conv2d(x, weight, bias, padding=[1,1]) # Low-level
# Result: Students learn APIs, not concepts
```
### TinyTorch: Clarity by Design
TinyTorch chooses the simplest approach that teaches the concept:
```python
# TinyTorch: One clear way to do each operation
Conv2D(in_channels=3, out_channels=64, kernel_size=3, padding=1)
# Result: Students understand the operation itself
```
## KISS in Practice
### 1. Single Responsibility Components
Every class has one clear purpose:
```python
# ✅ GOOD: Clear, single responsibility
class ReLU:
def forward(self, x):
return np.maximum(0, x)
def backward(self, grad_output):
return grad_output * (self.last_input > 0)
# ❌ AVOID: Multiple responsibilities
class ActivationWithDropoutAndNormalization:
# Too many concerns in one class
```
### 2. Minimal Interfaces
Functions do one thing with clear inputs/outputs:
```python
# ✅ GOOD: Simple, predictable interface
def conv2d(input, weight, bias=None, stride=1, padding=0):
# Implementation...
return output
# ❌ AVOID: Complex, unclear interface
def conv2d_advanced(input, weight, bias=None, stride=1, padding=0,
dilation=1, groups=1, padding_mode='zeros',
output_padding=0, **kwargs):
# Too many options obscure the core concept
```
### 3. Explicit Over Implicit
Make the "magic" visible:
```python
# ✅ GOOD: Shows what's happening
def train_step(model, loss_fn, optimizer, batch_x, batch_y):
# Forward pass
pred = model(batch_x)
loss = loss_fn(pred, batch_y)
# Backward pass
loss.backward()
optimizer.step()
optimizer.zero_grad()
return loss.data
# ❌ AVOID: Hidden complexity
def train_step(trainer, batch):
return trainer.step(batch) # What actually happens?
```
## KISS Design Decisions
### File Organization
```
# ✅ Simple structure
tinytorch/
├── core/ # Core implementations
├── utils/ # Utilities
└── datasets/ # Data handling
# vs. complex hierarchies with deep nesting
```
### Module Design
- **One concept per module**: Tensors, Activations, Layers, etc.
- **Progressive complexity**: Each module builds on previous ones
- **Self-contained**: Each module can be understood independently
### Code Style
- **No magic methods**: `__add__` is clear, `__radd__` is confusing
- **Explicit names**: `conv2d` not `conv`, `ReLU` not `R`
- **Minimal inheritance**: Composition over complex hierarchies
## Educational Benefits
### 1. Cognitive Load Reduction
Simple code means students focus on concepts, not syntax:
```python
# Cognitive load: LOW - focus on the math
def sigmoid(x):
return 1 / (1 + np.exp(-x))
# Cognitive load: HIGH - distracted by implementation details
def sigmoid(x, inplace=False, dtype=None, device=None, memory_format=None):
# Complex implementation with many edge cases
```
### 2. Debugging Clarity
When something breaks, simple code is easy to debug:
```python
# ✅ Easy to debug: clear execution path
def forward(self, x):
self.last_input = x
return np.maximum(0, x)
# ❌ Hard to debug: hidden state and side effects
def forward(self, x):
return self._apply_with_state_management(x, self._relu_impl)
```
### 3. Modification Confidence
Simple code invites experimentation:
```python
# Students think: "I can modify this!"
def adam_update(param, grad, m, v, lr=0.001, beta1=0.9, beta2=0.999):
m = beta1 * m + (1 - beta1) * grad
v = beta2 * v + (1 - beta2) * grad * grad
param -= lr * m / (np.sqrt(v) + 1e-8)
return param, m, v
# Students think: "I better not touch this..."
# [100 lines of optimized, abstracted update logic]
```
## KISS vs. Performance
### The Trade-off
KISS sometimes means choosing clarity over peak performance:
```python
# TinyTorch: Clear but not optimized
def conv2d_simple(input, kernel):
output = np.zeros(output_shape)
for i in range(output_height):
for j in range(output_width):
# Clear nested loops show the operation
output[i, j] = np.sum(input[i:i+k_h, j:j+k_w] * kernel)
return output
# Production: Optimized but opaque
def conv2d_optimized(input, kernel):
# BLAS calls, memory optimization, SIMD instructions
return torch._C._nn.conv2d(input, kernel, ...)
```
### When We Optimize
We add optimization layers **after** establishing clarity:
1. **First**: Implement the clearest possible version
2. **Then**: Profile and identify bottlenecks
3. **Finally**: Add optimizations with clear documentation
### Documentation of Trade-offs
Every optimization is explained:
```python
def conv2d_vectorized(input, kernel):
"""Vectorized convolution implementation.
This version uses im2col transformation for speed.
For the clear, educational version, see conv2d_simple().
Trade-off: 10x faster, but obscures the sliding window concept.
"""
```
## KISS Guidelines for Contributors
### Before Adding Complexity
Ask these questions:
1. **Is this essential for understanding the concept?**
2. **Can students modify this confidently?**
3. **Does this make debugging easier or harder?**
4. **Is there a simpler way to achieve the same goal?**
### Code Review Checklist
- [ ] Single responsibility per function/class
- [ ] Clear, explicit names
- [ ] Minimal parameter lists
- [ ] No hidden state or side effects
- [ ] Students can understand the implementation
- [ ] Debugging is straightforward
### Refactoring Triggers
Refactor when:
- Functions have more than 3-4 parameters
- Classes have more than one clear responsibility
- Students ask "what does this do?" frequently
- Debugging requires deep knowledge of implementation details
## The KISS Promise
TinyTorch promises that every component follows KISS principles:
- **You can understand any implementation in 5 minutes**
- **You can modify any component confidently**
- **When something breaks, you can debug it yourself**
- **The simplest solution is always preferred**
This isn't just about code - it's about **empowering learners** to become confident ML systems engineers who understand their tools completely.
Remember: **Complex problems often have simple solutions. Simple solutions enable deep understanding.**

89
docs/archive/book-development/quick-exploration.md
View File

@@ -1,89 +0,0 @@
# Quick Exploration Path
**Perfect for:** "I want to see what this is about" • "Can I try this without installing anything?"
**Time Commitment:** 5-30 minutes • **Setup Required:** None
---
## Launch Immediately (0 Setup Required)
Click the **Launch Binder** button on any chapter to get:
- Live Jupyter environment in your browser
- Pre-configured TinyTorch development setup
- Ability to run and modify all code immediately
- No installation, no account creation needed
```{admonition} What You'll Experience in 5-30 Minutes
:class: tip
**Immediate implementation experience** with real ML components:
- **5 min**: ReLU activation function from scratch
- **10 min**: Tensor operations that power neural networks
- **15 min**: Dense layers that transform data
- **20 min**: Complete neural networks for image classification
- **30 min**: See how language models use the same foundations
All running live in your browser with zero setup!
```
---
## Recommended Exploration Path
### Start Here: Chapter 1 - Setup
- Understand the TinyTorch development workflow
- Get familiar with the educational approach
- See how components fit together
**[Launch Setup Chapter](../chapters/01-setup.md)**
### Then Try: Chapter 3 - Activations
- Implement your first ML function (ReLU)
- See immediate visual results
- Understand why nonlinearity matters
**[Launch Activations Chapter](../chapters/03-activations.md)**
### Build Up: Chapter 4 - Layers
- Create the building blocks of neural networks
- Combine your ReLU with matrix operations
- See how simple math becomes powerful AI
**[Launch Layers Chapter](../chapters/04-layers.md)**
---
## Important Limitations
**Sessions are temporary:**
- Binder sessions timeout after ~20 minutes of inactivity
- Your work is **not saved** when the session ends
- Great for exploration, not for ongoing projects
**For persistent work:** Ready to build your own TinyTorch? → **[Serious Development Path](serious-development.md)**
---
## What You'll Understand
After exploring 2-3 chapters, you'll have hands-on understanding of:
- **How ML frameworks work under the hood**
- **Why activation functions are crucial**
- **How matrix multiplication powers neural networks**
- **The relationship between layers, networks, and learning**
- **Real implementation vs. high-level APIs**
- **Why vision and language models share the same foundations**
---
## Next Steps
**Satisfied with exploration?** You've gained valuable insight into ML systems!
**Want to build more?** → **[Fork the repo and work locally](serious-development.md)**
**Teaching a class?** → **[Classroom setup guide](classroom-use.md)**
---
*No commitment required - just click and explore!*

244
docs/archive/book-development/serious-development.md
View File

@@ -1,244 +0,0 @@
# Serious Development Path
**Perfect for:** "I want to build this myself" • "This is my class assignment" • "I want to understand ML frameworks deeply"
---
## What You'll Build
A complete ML framework from scratch, including:
- **Your own tensor library** with operations and autograd
- **Neural network components** (layers, activations, optimizers)
- **Training systems** that work on real datasets (CIFAR-10)
- **Production features** (compression, monitoring, benchmarking)
- **Language models** that extend your vision framework to TinyGPT
**End result:** A working ML framework that powers both computer vision AND language models.
---
## Quick Start (5 minutes)
### Step 1: Get the Code
```bash
git clone https://github.com/your-org/tinytorch.git
cd TinyTorch
```
### Step 2: Setup Environment
```bash
# Activate virtual environment
source bin/activate-tinytorch.sh
# Install dependencies
make install
# Verify everything works
tito system doctor
```
### Step 3: Start Building
```bash
# Open first assignment
cd modules/01_setup
jupyter lab setup_dev.py
```
### Step 4: Build → Test → Export → Use
```bash
# After implementing code in the notebook:
tito export # Export your code to tinytorch package
tito test setup # Test your implementation
# Now use YOUR own code:
python -c "from tinytorch.core.setup import hello_tinytorch; hello_tinytorch()"
# 🔥 TinyTorch! Built by: [Your Name]
```
---
## Learning Path (Progressive Complexity)
### Foundation (Weeks 1-2)
Build the core infrastructure:
**Module 01: Setup & CLI**
- Professional development workflow with `tito` CLI
- Understanding package architecture and exports
- Quality assurance with automated testing
**Module 01: Tensors**
- Multi-dimensional arrays and operations
- Memory management and data types
- Foundation for all ML operations
**Module 02: Activations**
- ReLU, Sigmoid, Tanh, Softmax functions
- Understanding nonlinearity in neural networks
- Mathematical foundations of deep learning
---
### 🧱 Building Blocks (Weeks 3-4)
Create neural network components:
**Module 03: Layers**
- Dense (linear) layers with matrix multiplication
- Weight initialization strategies
- Building blocks that stack together
**Module 04: Networks**
- Sequential model architecture
- Composition patterns and forward propagation
- Creating complete neural networks
**Module 05: CNNs**
- Convolutional operations for computer vision
- Understanding spatial processing
- Building blocks for image classification
---
### Training Systems (Weeks 5-6)
Complete training infrastructure:
**Module 06: DataLoader**
- Efficient data loading and preprocessing
- Real dataset handling (CIFAR-10)
- Batching, shuffling, and memory management
**Module 07: Autograd**
- Automatic differentiation engine
- Computational graphs and backpropagation
- The magic that makes training possible
**Module 08: Optimizers**
- SGD, Adam, and learning rate scheduling
- Understanding gradient descent variants
- Convergence and training dynamics
**Module 09: Training**
- Complete training loops and loss functions
- Model evaluation and metrics
- Checkpointing and persistence
---
### Production & Performance (Weeks 7-8)
Real-world deployment:
**Module 10: Compression**
- Model pruning and quantization
- Reducing model size by 75%+
- Deployment optimization
**Module 11: Kernels**
- High-performance custom operations
- Hardware-aware optimization
- Understanding framework internals
**Module 12: Benchmarking**
- Systematic performance measurement
- Statistical validation and reporting
- MLPerf-style evaluation
**Module 13: MLOps**
- Production deployment and monitoring
- Continuous learning and model updates
- Complete production pipeline
**Module 16: TinyGPT 🔥**
- Extend vision framework to language models
- GPT-style transformers with 95% component reuse
- Autoregressive text generation
- Framework generalization mastery
---
## Development Workflow
### The `tito` CLI System
TinyTorch includes a complete CLI for professional development:
```bash
# System management
tito system doctor # Check environment health
tito system info # Show module status
# Module development
tito export # Export dev code to package
tito test setup # Test specific module
tito test --all # Test everything
# NBGrader integration
tito nbgrader generate setup # Create assignments
tito nbgrader release setup # Release to students
tito nbgrader autograde setup # Auto-grade submissions
```
### Quality Assurance
Every module includes comprehensive testing:
- **100+ automated tests** ensure correctness
- **Inline tests** provide immediate feedback
- **Integration tests** verify cross-module functionality
- **Performance benchmarks** track optimization
---
## Proven Student Outcomes
```{admonition} Real Results
:class: success
**After 6-8 weeks, students consistently:**
✅ Build multi-layer perceptrons that classify CIFAR-10 images
✅ Implement automatic differentiation from scratch
✅ Create custom optimizers (SGD, Adam) that converge reliably
✅ Optimize models with pruning and quantization
✅ Deploy production ML systems with monitoring
✅ Understand framework internals better than most ML engineers
🔥 **Extend their vision framework to language models with 95% reuse**
**Test Coverage:** 200+ tests across all modules ensure student implementations work
```
---
## Why This Approach Works
### Build → Use → Understand
Every component follows this pattern:
1. **🔧 Build**: Implement `ReLU()` from scratch
2. **🚀 Use**: `from tinytorch.core.activations import ReLU` - your code!
3. **💡 Understand**: See how it enables complex pattern learning
### Real Data, Real Systems
- Work with CIFAR-10 (not toy datasets)
- Production-style code organization
- Performance and engineering considerations
- Professional development practices
### Immediate Feedback
- Code works immediately after implementation
- Visual progress indicators and success messages
- Comprehensive error handling and guidance
- Professional-quality development experience
---
## Ready to Start?
### Choose Your Module
**New to ML frameworks?** → Start with [Setup](../chapters/01-setup.md)
**Have ML experience?** → Jump to [Tensors](../chapters/01-tensor.md)
**Want to see the vision?** → Try [Activations](../chapters/02-activations.md)
### Get Help
- **💬 Discussions**: GitHub Discussions for questions
- **🐛 Issues**: Report bugs or suggest improvements
- **📧 Support**: Direct contact with TinyTorch team
---
*🎉 Ready to build your own ML framework? Your unified vision+language framework is 8 weeks away!*

103
docs/archive/book-development/verify_build.py
View File

@@ -1,103 +0,0 @@
#!/usr/bin/env python3
"""
Verify that the Jupyter Book build is complete and all pages are present.
"""
import os
from pathlib import Path
from rich.console import Console
from rich.table import Table
from rich.panel import Panel
console = Console()
def verify_book_build():
"""Verify the book build is complete."""
build_dir = Path("book/_build/html")
if not build_dir.exists():
console.print("❌ Build directory not found! Run 'tito book build' first.")
return False
# Pages that must exist
required_pages = {
"Main Pages": [
"index.html",
"intro.html",
"setup.html",
"instructor-guide.html",
"system-architecture.html"
],
"Module Chapters": [
f"chapters/{i:02d}-{name}.html" for i, name in enumerate([
"introduction", "setup", "tensor", "activations", "layers",
"dense", "spatial", "attention", "dataloader", "autograd",
"optimizers", "training", "compression", "kernels",
"benchmarking", "mlops", "tinygpt"
], 0)
],
"New Documentation": [
"testing-framework.html",
"kiss-principle.html"
],
"Usage Paths": [
"usage-paths/quick-start.html",
"usage-paths/browse-online.html",
"usage-paths/serious-development.html"
]
}
# Check each category
results = {}
for category, pages in required_pages.items():
results[category] = []
for page in pages:
full_path = build_dir / page
exists = full_path.exists()
size = full_path.stat().st_size if exists else 0
results[category].append({
'page': page,
'exists': exists,
'size': size
})
# Display results
console.print(Panel.fit(
"📚 [bold blue]TinyTorch Jupyter Book Verification[/bold blue]",
border_style="blue"
))
all_good = True
for category, checks in results.items():
console.print(f"\n[bold]{category}[/bold]")
for check in checks:
if check['exists']:
if check['size'] > 100: # More than just a redirect
console.print(f" ✅ {check['page']} ({check['size']:,} bytes)")
else:
console.print(f" ⚠️ {check['page']} (small: {check['size']} bytes)")
else:
console.print(f" ❌ {check['page']} (missing)")
all_good = False
# Summary
if all_good:
console.print(Panel.fit(
"✨ [bold green]All documentation pages built successfully![/bold green]\n"
f"🌐 View at: file://{build_dir.absolute()}/index.html",
border_style="green"
))
else:
console.print(Panel.fit(
"⚠️ [bold yellow]Some pages are missing![/bold yellow]\n"
"Run 'tito book build' to rebuild the documentation.",
border_style="yellow"
))
return all_good
if __name__ == "__main__":
os.chdir(Path(__file__).parent.parent) # Go to project root
success = verify_book_build()
exit(0 if success else 1)

213
docs/archive/book-development/vision.md
View File

@@ -1,213 +0,0 @@
# The TinyTorch Vision
**Training ML Systems Engineers: From Computer Vision to Language Models**
---
## The Problem We're Solving
The ML field has a critical gap: **most education teaches you to use frameworks, not build them.**
### Traditional ML Education:
```python
import torch
import torch.nn as nn
model = nn.Linear(784, 10)
optimizer = torch.optim.Adam(model.parameters())
```
**Questions students can't answer:**
- Why does Adam use 3× more memory than SGD?
- How does `loss.backward()` actually compute gradients?
- When should you use gradient accumulation vs larger batch sizes?
- Why do attention mechanisms limit context length?
### The TinyTorch Difference:
```python
class Linear:
def __init__(self, in_features, out_features):
self.weight = Tensor(np.random.randn(in_features, out_features))
self.bias = Tensor(np.zeros(out_features))
def forward(self, x):
return x @ self.weight + self.bias # YOU implemented @
def backward(self, grad_output):
# YOU understand exactly how gradients flow
self.weight.grad = x.T @ grad_output
return grad_output @ self.weight.T
```
**Questions students CAN answer:**
- Exactly how automatic differentiation works
- Why certain optimizers use more memory
- How to debug training instability
- When to make performance vs accuracy trade-offs
---
## What We Teach: Systems Thinking
### Beyond Algorithms: System-Level Understanding
**Memory Management:**
- Why Adam needs 3× parameter memory (parameters + momentum + variance)
- How attention matrices scale O(N²) with sequence length
- When gradient accumulation saves memory vs compute trade-offs
**Performance Analysis:**
- Why naive convolution is 100× slower than optimized versions
- How cache misses destroy performance in matrix operations
- When vectorization provides 10-100× speedups
**Production Trade-offs:**
- SGD vs Adam: convergence speed vs memory constraints
- Gradient checkpointing: trading compute for memory
- Mixed precision: 2× memory savings with accuracy considerations
**Hardware Awareness:**
- How memory bandwidth limits ML performance
- Why GPU utilization matters more than peak FLOPS
- When distributed training becomes necessary
---
## Target Audience: Future ML Systems Engineers
### Perfect For:
**Computer Science Students**
- Going beyond "use PyTorch" to "understand PyTorch"
- Building portfolio projects that demonstrate deep system knowledge
- Preparing for ML engineering roles (not just data science)
**Software Engineers → ML Engineers**
- Leveraging existing programming skills for ML systems
- Understanding performance, debugging, and optimization
- Learning production ML patterns and infrastructure
**ML Practitioners**
- Moving from model users to model builders
- Debugging training issues at the systems level
- Optimizing models for production deployment
**Researchers & Advanced Users**
- Implementing custom operations and architectures
- Understanding framework limitations and workarounds
- Building specialized ML systems for unique domains
### Career Transformation:
**Before TinyTorch:** "I can train models with PyTorch"
**After TinyTorch:** "I can build and optimize ML systems"
You become the person your team asks:
- *"Why is our training bottlenecked?"*
- *"Can we fit this model in memory?"*
- *"How do we implement this research paper?"*
- *"What's the best architecture for our constraints?"*
---
## Pedagogical Philosophy: Build → Use → Understand
### 1. Build First
Every component implemented from scratch:
- Tensors with broadcasting and memory management
- Automatic differentiation with computational graphs
- Optimizers with state management and memory profiling
- Complete training loops with checkpointing and monitoring
### 2. Use Immediately
No toy examples - recreate ML history with real results:
- **MLP Era**: Train MLPs to 52.7% CIFAR-10 accuracy (the baseline that motivated CNNs)
- **CNN Revolution**: Build LeNet-1 (39.4%) and LeNet-5 (47.5%) - witness the breakthrough
- **Modern CNNs**: Push beyond MLPs with optimized architectures (75%+ achievable)
- **Transformer Era**: Language models using 95% vision framework reuse
### 3. Understand Systems
Connect implementations to production reality:
- How your tensor maps to PyTorch's memory model
- Why your optimizer choices affect GPU utilization
- How your autograd compares to production frameworks
- When your implementations would need modification at scale
### 4. Reflect on Trade-offs
ML Systems Thinking sections in every module:
- Memory vs compute trade-offs in different architectures
- Accuracy vs efficiency considerations for deployment
- Debugging strategies for common production issues
- Framework design principles and their implications
---
## Unique Value Proposition
### What Makes TinyTorch Different:
**Systems-First Approach**
- Not just "how does attention work" but "why does attention scale O(N²) and how do production systems handle this?"
- Not just "implement SGD" but "when do you choose SGD vs Adam in production?"
**Production Relevance**
- Memory profiling, performance optimization, deployment patterns
- Real datasets, realistic scale, professional development workflow
- Connection to industry practices and framework design decisions
**Framework Generalization**
- 20 modules that build ONE cohesive ML framework supporting vision AND language
- 95% component reuse from computer vision to language models
- Professional package structure with CLI tools and testing
**Proven Pedagogy**
- Build → Use → Understand cycle creates deep intuition
- Immediate testing and feedback for every component
- Progressive complexity with solid foundations
- NBGrader integration for classroom deployment
---
## Learning Outcomes: Becoming an ML Systems Engineer
### Technical Mastery
- **Implement any ML paper** from first principles
- **Debug training issues** at the systems level
- **Optimize models** for production deployment
- **Profile and improve** ML system performance
- **Design custom architectures** for specialized domains
- **Understand framework generalization** across vision and language
### Systems Understanding
- **Memory management** in ML frameworks
- **Computational complexity** vs real-world performance
- **Hardware utilization** patterns and optimization
- **Distributed training** challenges and solutions
- **Production deployment** considerations and trade-offs
### Professional Skills
- **Test-driven development** for ML systems
- **Performance profiling** and optimization techniques
- **Code organization** and package development
- **Documentation** and API design
- **MLOps** and production monitoring
### Career Impact
- **Technical interviews**: Demonstrate deep ML systems knowledge
- **Job opportunities**: Qualify for ML engineer (not just data scientist) roles
- **Team leadership**: Become the go-to person for ML systems questions
- **Research ability**: Implement cutting-edge papers independently
- **Entrepreneurship**: Build ML products with full-stack understanding
---
## Ready to Become an ML Systems Engineer?
**TinyTorch transforms ML users into ML builders.**
Stop wondering how frameworks work. Start building them.
**[Begin Your Journey →](chapters/00-introduction.md)**
---
*TinyTorch: Because understanding how to build ML systems makes you a more effective ML engineer.*