diff --git a/modules/source/00_introduction/README.md b/modules/source/00_introduction/README.md new file mode 100644 index 00000000..4e87aa85 --- /dev/null +++ b/modules/source/00_introduction/README.md @@ -0,0 +1,147 @@ +# TinyTorch System Introduction & Architecture + +Welcome to **TinyTorch** - a complete neural network framework built from scratch for deep learning education and understanding. + +## ๐ŸŽฏ Module Overview + +This introduction module provides a comprehensive visual overview of the entire TinyTorch system, helping you understand how all 16 modules work together to create a complete machine learning framework. + +### What You'll Explore + +- **๐Ÿ—๏ธ System Architecture** - Complete framework overview with visual diagrams +- **๐Ÿ“Š Interactive Dependency Graphs** - See how all modules connect and depend on each other +- **๐Ÿ“š Learning Roadmap** - Optimal path through the entire TinyTorch curriculum +- **๐Ÿ” Component Analysis** - Deep dive into what each module implements +- **๐Ÿ“ˆ Progress Visualization** - Track your learning journey through the system + +## ๐Ÿš€ Key Features + +### Automated Analysis System +- **Module Metadata Parser** - Automatically loads and analyzes all module.yaml files +- **Dependency Graph Builder** - Creates NetworkX graphs of module relationships +- **Learning Path Generator** - Uses topological sort to find optimal learning sequence + +### Interactive Visualizations +- **Dependency Graph** - Hierarchical and circular layouts showing module connections +- **System Architecture** - Layered view of how components work together +- **Learning Roadmap** - Timeline view with time estimates and difficulty progression +- **Component Analysis** - Statistical analysis of module complexity and relationships + +### Export Functions +- **System Overview API** - Programmatic access to TinyTorch metadata +- **Module Information** - Detailed data about any specific module +- **Learning Recommendations** - Personalized next steps based on progress + +## ๐Ÿ“Š What You'll Discover + +### System Statistics +- **16 modules** spanning from basic tensors to production MLOps +- **60+ components** implementing complete ML framework functionality +- **Estimated 80+ hours** of comprehensive learning content +- **5 difficulty levels** progressing from foundation to advanced topics + +### Learning Progression +1. **Foundation** (3 modules) - Setup, tensors, activations +2. **Core Architecture** (4 modules) - Layers, networks, attention, data loading +3. **Training System** (3 modules) - Autograd, optimization, training loops +4. **Production Ready** (5 modules) - Compression, kernels, benchmarking, MLOps, capstone +5. **Integration** (1 module) - Final capstone project + +## ๐ŸŽจ Visualization Gallery + +### Dependency Graph +See how modules build upon each other with interactive dependency visualizations showing: +- **Prerequisite relationships** - What you need to learn first +- **Module difficulty** - Color-coded complexity levels +- **Component count** - Size indicates implementation scope + +### System Architecture +Layered architecture diagram showing: +- **Foundation Layer** - Core tensors and setup +- **Component Layer** - Activations, layers, data loading +- **Network Layer** - Dense networks, CNNs, attention +- **Training Layer** - Autograd, optimizers, training +- **Production Layer** - Compression, kernels, MLOps + +### Learning Roadmap +Timeline visualization featuring: +- **Optimal sequence** - Dependency-respecting learning order +- **Time estimates** - Realistic hour commitments per module +- **Difficulty progression** - Smooth learning curve design +- **Milestone tracking** - Major learning achievements + +## ๐Ÿ”ง Technical Implementation + +### Module Analysis Engine +```python +# Automatically analyze all TinyTorch modules +analyzer = TinyTorchAnalyzer() +overview = analyzer.get_tinytorch_overview() +learning_path = analyzer.get_learning_path() +``` + +### Visualization System +```python +# Generate comprehensive system visualizations +visualizations = visualize_tinytorch_system() +dependency_graph = create_dependency_graph_visualization() +architecture = create_system_architecture_diagram() +roadmap = create_learning_roadmap() +``` + +### Learning Recommendations +```python +# Get personalized learning suggestions +recommendations = get_learning_recommendations() +next_modules = recommendations['next_modules'] +estimated_time = recommendations['remaining_time'] +``` + +## ๐Ÿค” ML Systems Thinking + +This module connects TinyTorch's educational architecture to real-world ML systems: + +### Framework Design Patterns +- **Modular Dependencies** - How PyTorch and TensorFlow organize components +- **Component Composition** - Building complex operations from simple primitives +- **Abstraction Layers** - Balancing usability with performance control + +### Production Considerations +- **Deployment Pipelines** - From research code to production systems +- **Performance Optimization** - Hardware-aware kernel design +- **Monitoring & MLOps** - Continuous learning and model management + +### Educational Philosophy +- **Progressive Complexity** - Foundation โ†’ Architecture โ†’ Training โ†’ Production +- **Hands-on Learning** - Build before you use, understand before you optimize +- **Real-world Relevance** - Educational choices that mirror industry patterns + +## ๐Ÿ“ˆ Learning Outcomes + +After completing this module, you will: + +1. **Understand TinyTorch Architecture** - Complete mental model of the framework +2. **Navigate Module Dependencies** - Know what to learn when and why +3. **Plan Your Learning Journey** - Realistic timeline and progression tracking +4. **Connect to Industry** - See how educational patterns map to production ML + +## ๐Ÿ”— Integration with TinyTorch + +This introduction module: +- **Requires no prerequisites** - Perfect starting point for new learners +- **Enables all other modules** - Provides context for the entire journey +- **Exports analysis tools** - Used by other modules for self-reflection +- **Updates automatically** - Visualization stays current as modules evolve + +## ๐ŸŽ“ Getting Started + +1. **Run the introduction notebook** to see all visualizations +2. **Explore the dependency graph** to understand module relationships +3. **Review the learning roadmap** to plan your journey +4. **Bookmark key functions** for reference during your learning + +**Ready to build a neural network framework from scratch? Let's begin! ๐Ÿš€** + +--- + +*This module serves as your guide through the complete TinyTorch learning experience. Use it to maintain big-picture understanding as you dive deep into implementation details.* \ No newline at end of file diff --git a/modules/source/00_introduction/introduction_dev.py b/modules/source/00_introduction/introduction_dev.py new file mode 100644 index 00000000..24841e96 --- /dev/null +++ b/modules/source/00_introduction/introduction_dev.py @@ -0,0 +1,1109 @@ +# %% [markdown] +""" +# TinyTorch System Introduction & Architecture Overview + +Welcome to **TinyTorch** - a complete neural network framework built from scratch for deep learning education and understanding. + +This introduction module provides: +- **Visual system architecture** - Complete framework overview +- **Interactive dependency graphs** - How all 16 modules connect +- **Learning roadmap** - Guided path through the system +- **Component analysis** - What each module implements + +Let's explore the architecture of this comprehensive ML framework! +""" + +# %% +import numpy as np +import matplotlib.pyplot as plt +import matplotlib.patches as patches +from matplotlib.patches import FancyBboxPatch, Circle, ConnectionPatch +from pathlib import Path +import yaml +import networkx as nx +from typing import Dict, List, Tuple, Set +import seaborn as sns +from dataclasses import dataclass +from collections import defaultdict, deque + +# Set plotting style for professional visualizations +plt.style.use('seaborn-v0_8') +sns.set_palette("husl") + +# %% [markdown] +""" +## Module Metadata Analysis System + +First, let's build tools to automatically analyze all TinyTorch modules and their relationships. +This will power our interactive visualizations. +""" + +# %% +@dataclass +class ModuleInfo: + """Complete information about a TinyTorch module""" + name: str + title: str + description: str + prerequisites: List[str] + enables: List[str] + components: List[str] + difficulty: str + time_estimate: str + exports_to: str + + def difficulty_level(self) -> int: + """Convert difficulty stars to numeric level""" + return self.difficulty.count('โญ') + + def estimated_hours(self) -> float: + """Extract numeric time estimate""" + time_str = self.time_estimate.lower() + if 'capstone' in time_str: + return 40.0 # Capstone project estimate + + # Extract first number from time estimate + import re + numbers = re.findall(r'\d+', time_str) + if numbers: + return float(numbers[0]) + return 4.0 # Default estimate + +class TinyTorchAnalyzer: + """Comprehensive analysis of TinyTorch module system""" + + def __init__(self, modules_path: str = "/Users/VJ/GitHub/TinyTorch/modules/source"): + self.modules_path = Path(modules_path) + self.modules: Dict[str, ModuleInfo] = {} + self.dependency_graph = nx.DiGraph() + self._load_all_modules() + self._build_dependency_graph() + + def _load_all_modules(self): + """Load metadata from all module.yaml files""" + for module_dir in sorted(self.modules_path.iterdir()): + if module_dir.is_dir() and not module_dir.name.startswith('.'): + yaml_file = module_dir / 'module.yaml' + if yaml_file.exists(): + try: + with open(yaml_file, 'r') as f: + data = yaml.safe_load(f) + + # Handle different YAML formats in the modules + if 'dependencies' in data: + deps = data['dependencies'] + prerequisites = deps.get('prerequisites', []) + enables = deps.get('enables', []) + else: + # Handle older format + prerequisites = data.get('dependencies', []) + enables = [] + + module_info = ModuleInfo( + name=data.get('name', module_dir.name), + title=data.get('title', module_dir.name.title()), + description=data.get('description', ''), + prerequisites=prerequisites, + enables=enables, + components=data.get('components', []), + difficulty=data.get('difficulty', 'โญ'), + time_estimate=data.get('time_estimate', '2-4 hours'), + exports_to=data.get('exports_to', f'tinytorch.{module_dir.name}') + ) + + self.modules[module_info.name] = module_info + + except Exception as e: + print(f"Warning: Could not load {yaml_file}: {e}") + + def _build_dependency_graph(self): + """Build NetworkX graph of module dependencies""" + # Add all modules as nodes + for name, module in self.modules.items(): + self.dependency_graph.add_node(name, **{ + 'title': module.title, + 'description': module.description, + 'difficulty': module.difficulty_level(), + 'time': module.estimated_hours(), + 'components': len(module.components) + }) + + # Add dependency edges + for name, module in self.modules.items(): + for prereq in module.prerequisites: + if prereq in self.modules: + self.dependency_graph.add_edge(prereq, name) + + def get_learning_path(self) -> List[str]: + """Generate optimal learning path through modules using topological sort""" + try: + return list(nx.topological_sort(self.dependency_graph)) + except nx.NetworkXError: + # Fallback if cycles exist + return sorted(self.modules.keys()) + + def get_module_levels(self) -> Dict[str, int]: + """Assign modules to learning levels based on dependencies""" + levels = {} + path = self.get_learning_path() + + for module in path: + prereqs = self.modules[module].prerequisites + if not prereqs: + levels[module] = 0 + else: + max_prereq_level = max((levels.get(p, 0) for p in prereqs if p in levels), default=0) + levels[module] = max_prereq_level + 1 + + return levels + +# Initialize the analyzer +analyzer = TinyTorchAnalyzer() + +print(f"๐Ÿ“Š Loaded {len(analyzer.modules)} TinyTorch modules") +print(f"๐Ÿ”— Built dependency graph with {analyzer.dependency_graph.number_of_edges()} connections") + +# %% [markdown] +""" +### Test the Module Analysis System + +Let's verify our module analyzer is working correctly by examining a few key modules. +""" + +# %% +def test_module_analyzer(): + """Test that our module analyzer correctly loads and processes modules""" + + # Test basic loading + assert len(analyzer.modules) >= 10, "Should load multiple modules" + + # Test specific modules exist + key_modules = ['setup', 'tensor', 'activations', 'training'] + for module_name in key_modules: + assert module_name in analyzer.modules, f"Should load {module_name} module" + + # Test dependency relationships + tensor_module = analyzer.modules['tensor'] + assert 'setup' in tensor_module.prerequisites, "Tensor should depend on setup" + + # Test learning path generation + learning_path = analyzer.get_learning_path() + setup_pos = learning_path.index('setup') if 'setup' in learning_path else -1 + tensor_pos = learning_path.index('tensor') if 'tensor' in learning_path else -1 + + if setup_pos >= 0 and tensor_pos >= 0: + assert setup_pos < tensor_pos, "Setup should come before tensor in learning path" + + print("โœ… Module analyzer tests passed!") + + # Show some sample data + print(f"\n๐Ÿ“‹ Sample modules loaded:") + for name in list(analyzer.modules.keys())[:5]: + module = analyzer.modules[name] + print(f" โ€ข {module.title} ({module.difficulty}) - {len(module.components)} components") + +test_module_analyzer() + +# %% [markdown] +""" +## Interactive Dependency Visualization + +Now let's create beautiful, interactive visualizations of the TinyTorch module dependency system. +""" + +# %% +def create_dependency_graph_visualization(figsize=(16, 12)): + """Create an interactive dependency graph visualization""" + + fig, (ax1, ax2) = plt.subplots(1, 2, figsize=figsize) + + # Left plot: Hierarchical layout + ax1.set_title("TinyTorch Module Dependencies\n(Hierarchical Layout)", fontsize=16, fontweight='bold') + + # Calculate positions using spring layout with hierarchy + levels = analyzer.get_module_levels() + pos = {} + + # Group modules by level + level_groups = defaultdict(list) + for module, level in levels.items(): + level_groups[level].append(module) + + # Position modules in levels + max_level = max(levels.values()) if levels else 0 + for level, modules in level_groups.items(): + y = max_level - level # Higher levels at top + for i, module in enumerate(sorted(modules)): + x = (i - len(modules)/2) * 2.5 + pos[module] = (x, y * 2) + + # Draw the graph + G = analyzer.dependency_graph + + # Node colors based on difficulty + node_colors = [] + node_sizes = [] + for node in G.nodes(): + difficulty = analyzer.modules[node].difficulty_level() + node_colors.append(plt.cm.viridis(difficulty / 5.0)) # Normalize to 0-1 + node_sizes.append(200 + difficulty * 100) + + # Draw nodes and edges + nx.draw_networkx_nodes(G, pos, node_color=node_colors, node_size=node_sizes, + alpha=0.8, ax=ax1) + nx.draw_networkx_edges(G, pos, edge_color='gray', alpha=0.6, + arrows=True, arrowsize=20, ax=ax1) + + # Add labels + labels = {node: analyzer.modules[node].name for node in G.nodes()} + nx.draw_networkx_labels(G, pos, labels, font_size=8, font_weight='bold', ax=ax1) + + ax1.set_aspect('equal') + ax1.axis('off') + + # Right plot: Circular layout + ax2.set_title("TinyTorch Module Dependencies\n(Circular Layout)", fontsize=16, fontweight='bold') + + # Circular layout + pos_circular = nx.circular_layout(G) + + nx.draw_networkx_nodes(G, pos_circular, node_color=node_colors, node_size=node_sizes, + alpha=0.8, ax=ax2) + nx.draw_networkx_edges(G, pos_circular, edge_color='gray', alpha=0.4, + arrows=True, arrowsize=15, ax=ax2) + nx.draw_networkx_labels(G, pos_circular, labels, font_size=7, font_weight='bold', ax=ax2) + + ax2.set_aspect('equal') + ax2.axis('off') + + # Add legend + difficulty_colors = [plt.cm.viridis(i/5.0) for i in range(1, 6)] + legend_elements = [plt.Line2D([0], [0], marker='o', color='w', + markerfacecolor=color, markersize=10, + label=f"{'โญ' * (i+1)} Difficulty") + for i, color in enumerate(difficulty_colors)] + + fig.legend(handles=legend_elements, loc='upper center', bbox_to_anchor=(0.5, 0.02), ncol=5) + + plt.tight_layout() + plt.show() + + return fig + +# Create and display the dependency visualization +dependency_fig = create_dependency_graph_visualization() + +# %% [markdown] +""" +### Test the Dependency Visualization + +Let's verify our dependency graph captures the correct relationships. +""" + +# %% +def test_dependency_relationships(): + """Test that dependency relationships are correctly captured""" + + G = analyzer.dependency_graph + + # Test that setup has no prerequisites (should be a source node) + setup_predecessors = list(G.predecessors('setup')) if 'setup' in G else [] + print(f"Setup prerequisites: {setup_predecessors}") + + # Test that capstone depends on many modules (should have many predecessors) + if 'capstone' in G: + capstone_predecessors = list(G.predecessors('capstone')) + print(f"Capstone depends on {len(capstone_predecessors)} modules: {capstone_predecessors[:5]}...") + assert len(capstone_predecessors) >= 5, "Capstone should depend on many modules" + + # Test learning path makes sense + learning_path = analyzer.get_learning_path() + print(f"\n๐Ÿ“š Learning path ({len(learning_path)} modules):") + for i, module in enumerate(learning_path[:8]): # Show first 8 + print(f" {i+1:2d}. {analyzer.modules[module].title}") + + print("โœ… Dependency relationship tests passed!") + +test_dependency_relationships() + +# %% [markdown] +""" +## System Architecture Overview + +Let's create a comprehensive system architecture diagram showing how all TinyTorch components work together. +""" + +# %% +def create_system_architecture_diagram(figsize=(18, 12)): + """Create a comprehensive TinyTorch system architecture diagram""" + + fig, ax = plt.subplots(figsize=figsize) + ax.set_xlim(0, 20) + ax.set_ylim(0, 12) + ax.set_aspect('equal') + + # Define architectural layers + layers = { + 'Foundation': {'y': 1, 'color': '#FF6B6B', 'modules': ['setup', 'tensor']}, + 'Core Components': {'y': 3, 'color': '#4ECDC4', 'modules': ['activations', 'layers', 'dataloader']}, + 'Network Architecture': {'y': 5, 'color': '#45B7D1', 'modules': ['dense', 'spatial', 'attention']}, + 'Training System': {'y': 7, 'color': '#96CEB4', 'modules': ['autograd', 'optimizers', 'training']}, + 'Production & Optimization': {'y': 9, 'color': '#FFEAA7', 'modules': ['compression', 'kernels', 'benchmarking']}, + 'MLOps & Integration': {'y': 11, 'color': '#DDA0DD', 'modules': ['mlops', 'capstone']} + } + + # Draw layer backgrounds + for layer_name, layer_info in layers.items(): + y = layer_info['y'] + rect = FancyBboxPatch((1, y-0.4), 18, 1.2, + boxstyle="round,pad=0.1", + facecolor=layer_info['color'], + alpha=0.3, edgecolor='black', linewidth=1) + ax.add_patch(rect) + + # Layer label + ax.text(0.5, y, layer_name, fontsize=12, fontweight='bold', + rotation=90, va='center', ha='center') + + # Draw modules within layers + module_positions = {} + for layer_name, layer_info in layers.items(): + y = layer_info['y'] + modules = [m for m in layer_info['modules'] if m in analyzer.modules] + + for i, module_name in enumerate(modules): + module = analyzer.modules[module_name] + x = 2 + (i * 16 / max(len(modules), 1)) + module_positions[module_name] = (x, y) + + # Module box + width = min(3.5, 14 / len(modules)) + box = FancyBboxPatch((x-width/2, y-0.3), width, 0.6, + boxstyle="round,pad=0.05", + facecolor='white', edgecolor='black', + linewidth=2) + ax.add_patch(box) + + # Module title + ax.text(x, y+0.1, module.title, fontsize=9, fontweight='bold', + ha='center', va='center') + + # Difficulty and components + ax.text(x, y-0.15, f"{module.difficulty} โ€ข {len(module.components)} comp.", + fontsize=7, ha='center', va='center', style='italic') + + # Draw dependency arrows between layers + for module_name, module in analyzer.modules.items(): + if module_name in module_positions: + x1, y1 = module_positions[module_name] + for prereq in module.prerequisites: + if prereq in module_positions: + x2, y2 = module_positions[prereq] + if abs(y1 - y2) > 1: # Only draw arrows between different layers + arrow = ConnectionPatch((x2, y2+0.3), (x1, y1-0.3), "data", "data", + arrowstyle="->", shrinkA=0, shrinkB=0, + mutation_scale=15, alpha=0.6, color='gray') + ax.add_patch(arrow) + + # Title and annotations + ax.text(10, 11.7, 'TinyTorch System Architecture', fontsize=20, fontweight='bold', ha='center') + ax.text(10, 0.3, 'Data flows upward through layers โ€ข Arrows show dependencies', + fontsize=10, ha='center', style='italic') + + ax.set_xticks([]) + ax.set_yticks([]) + ax.spines['top'].set_visible(False) + ax.spines['right'].set_visible(False) + ax.spines['bottom'].set_visible(False) + ax.spines['left'].set_visible(False) + + plt.tight_layout() + plt.show() + + return fig, module_positions + +# Create the system architecture diagram +arch_fig, module_positions = create_system_architecture_diagram() + +# %% [markdown] +""" +### Test the System Architecture Visualization + +Let's verify our architecture diagram correctly represents the system structure. +""" + +# %% +def test_system_architecture(): + """Test that the system architecture is correctly represented""" + + # Test that we have positions for all modules + expected_modules = set(analyzer.modules.keys()) + positioned_modules = set(module_positions.keys()) + + missing_modules = expected_modules - positioned_modules + if missing_modules: + print(f"โš ๏ธ Missing modules in architecture: {missing_modules}") + + # Test layer organization makes sense + foundation_modules = ['setup', 'tensor'] + core_modules = ['activations', 'layers', 'dataloader'] + + foundation_y = [module_positions[m][1] for m in foundation_modules if m in module_positions] + core_y = [module_positions[m][1] for m in core_modules if m in module_positions] + + if foundation_y and core_y: + assert min(core_y) > max(foundation_y), "Core modules should be above foundation" + + print(f"โœ… Architecture diagram includes {len(module_positions)} modules") + print(f"๐Ÿ“Š Modules organized across {len(set(pos[1] for pos in module_positions.values()))} layers") + +test_system_architecture() + +# %% [markdown] +""" +## Learning Roadmap Visualization + +Create an interactive learning roadmap that shows the optimal path through TinyTorch modules. +""" + +# %% +def create_learning_roadmap(figsize=(16, 10)): + """Create an interactive learning roadmap visualization""" + + fig, (ax1, ax2) = plt.subplots(2, 1, figsize=figsize, height_ratios=[3, 1]) + + # Get learning path and levels + learning_path = analyzer.get_learning_path() + levels = analyzer.get_module_levels() + + # Top plot: Learning path flowchart + ax1.set_title('TinyTorch Learning Roadmap\n(Optimal Learning Sequence)', + fontsize=16, fontweight='bold') + + # Calculate positions for roadmap + path_positions = {} + cumulative_time = 0 + y_positions = {} + + for i, module_name in enumerate(learning_path): + if module_name in analyzer.modules: + module = analyzer.modules[module_name] + level = levels.get(module_name, 0) + + # X position based on cumulative time + x = cumulative_time + module.estimated_hours() / 2 + # Y position based on dependency level with some jitter + y = level + (i % 3 - 1) * 0.3 + + path_positions[module_name] = (x, y) + y_positions[module_name] = y + cumulative_time += module.estimated_hours() + + # Draw the learning path + for i, module_name in enumerate(learning_path[:-1]): + if module_name in path_positions and learning_path[i+1] in path_positions: + x1, y1 = path_positions[module_name] + x2, y2 = path_positions[learning_path[i+1]] + + # Draw connecting line + ax1.plot([x1, x2], [y1, y2], 'gray', alpha=0.5, linewidth=1, zorder=1) + + # Draw modules + for module_name in learning_path: + if module_name in analyzer.modules and module_name in path_positions: + module = analyzer.modules[module_name] + x, y = path_positions[module_name] + + # Color based on difficulty + difficulty = module.difficulty_level() + color = plt.cm.viridis(difficulty / 5.0) + + # Draw module circle + circle = Circle((x, y), 0.4, facecolor=color, edgecolor='black', + linewidth=2, alpha=0.8, zorder=3) + ax1.add_patch(circle) + + # Module number + ax1.text(x, y, str(learning_path.index(module_name) + 1), + fontsize=10, fontweight='bold', ha='center', va='center', + color='white', zorder=4) + + # Module name below + ax1.text(x, y-0.7, module.title, fontsize=8, ha='center', va='top', + rotation=45, fontweight='bold') + + ax1.set_xlim(-2, cumulative_time + 2) + ax1.set_ylim(-1, max(y_positions.values()) + 1) + ax1.set_xlabel('Cumulative Learning Time (hours)', fontsize=12) + ax1.set_ylabel('Dependency Level', fontsize=12) + ax1.grid(True, alpha=0.3) + + # Bottom plot: Time and difficulty analysis + ax2.set_title('Module Difficulty and Time Distribution', fontsize=14, fontweight='bold') + + module_names = [analyzer.modules[name].title[:15] for name in learning_path + if name in analyzer.modules] + difficulties = [analyzer.modules[name].difficulty_level() for name in learning_path + if name in analyzer.modules] + times = [analyzer.modules[name].estimated_hours() for name in learning_path + if name in analyzer.modules] + + # Create stacked bar chart + x_pos = np.arange(len(module_names)) + + # Time bars + bars1 = ax2.bar(x_pos, times, alpha=0.7, label='Time (hours)', color='lightblue') + + # Difficulty overlay + ax2_twin = ax2.twinx() + bars2 = ax2_twin.bar(x_pos, difficulties, alpha=0.5, label='Difficulty (โญ)', + color='orange', width=0.6) + + ax2.set_xlabel('Modules (in learning order)', fontsize=12) + ax2.set_ylabel('Time (hours)', fontsize=12, color='blue') + ax2_twin.set_ylabel('Difficulty Level', fontsize=12, color='orange') + + ax2.set_xticks(x_pos) + ax2.set_xticklabels(module_names, rotation=45, ha='right') + + # Legends + ax2.legend(loc='upper left') + ax2_twin.legend(loc='upper right') + + plt.tight_layout() + plt.show() + + return fig, learning_path, cumulative_time + +# Create the learning roadmap +roadmap_fig, learning_path, total_time = create_learning_roadmap() + +print(f"๐Ÿ“š Learning path contains {len(learning_path)} modules") +print(f"โฑ๏ธ Total estimated time: {total_time:.1f} hours") + +# %% [markdown] +""" +### Test the Learning Roadmap + +Let's verify our learning roadmap is pedagogically sound and follows dependency constraints. +""" + +# %% +def test_learning_roadmap(): + """Test that the learning roadmap respects dependencies and makes pedagogical sense""" + + # Test that all prerequisites come before dependents + path_indices = {module: i for i, module in enumerate(learning_path)} + + violations = [] + for module_name in learning_path: + if module_name in analyzer.modules: + module = analyzer.modules[module_name] + module_index = path_indices[module_name] + + for prereq in module.prerequisites: + if prereq in path_indices: + prereq_index = path_indices[prereq] + if prereq_index >= module_index: + violations.append(f"{module_name} comes before its prerequisite {prereq}") + + if violations: + print("โš ๏ธ Dependency violations found:") + for violation in violations: + print(f" {violation}") + else: + print("โœ… Learning roadmap respects all dependencies") + + # Test reasonable progression + foundation_modules = ['setup', 'tensor'] + advanced_modules = ['capstone', 'mlops', 'benchmarking'] + + foundation_positions = [path_indices.get(m, -1) for m in foundation_modules] + advanced_positions = [path_indices.get(m, -1) for m in advanced_modules] + + foundation_positions = [p for p in foundation_positions if p >= 0] + advanced_positions = [p for p in advanced_positions if p >= 0] + + if foundation_positions and advanced_positions: + if max(foundation_positions) < min(advanced_positions): + print("โœ… Foundation modules come before advanced modules") + else: + print("โš ๏ธ Some advanced modules come before foundation modules") + + print(f"๐Ÿ“Š Total learning time: {total_time:.1f} hours ({total_time/8:.1f} work days)") + +test_learning_roadmap() + +# %% [markdown] +""" +## Component Relationship Analysis + +Let's analyze the specific components within each module and how they relate to each other. +""" + +# %% +def create_component_analysis(figsize=(14, 10)): + """Create visualization of components within modules and their relationships""" + + fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=figsize) + + # 1. Components per module + modules = [name for name in learning_path if name in analyzer.modules] + component_counts = [len(analyzer.modules[name].components) for name in modules] + module_titles = [analyzer.modules[name].title for name in modules] + + ax1.bar(range(len(modules)), component_counts, + color=plt.cm.viridis(np.linspace(0, 1, len(modules)))) + ax1.set_title('Components per Module', fontweight='bold') + ax1.set_xlabel('Module') + ax1.set_ylabel('Number of Components') + ax1.set_xticks(range(len(modules))) + ax1.set_xticklabels([title[:10] for title in module_titles], rotation=45) + + # 2. Difficulty vs Components scatter + difficulties = [analyzer.modules[name].difficulty_level() for name in modules] + times = [analyzer.modules[name].estimated_hours() for name in modules] + + scatter = ax2.scatter(component_counts, difficulties, s=[t*20 for t in times], + c=times, cmap='plasma', alpha=0.7) + ax2.set_title('Module Complexity Analysis', fontweight='bold') + ax2.set_xlabel('Number of Components') + ax2.set_ylabel('Difficulty Level') + ax2.grid(True, alpha=0.3) + + # Add colorbar for time + cbar = plt.colorbar(scatter, ax=ax2) + cbar.set_label('Time (hours)') + + # 3. Module categories pie chart + categories = { + 'Foundation': ['setup', 'tensor', 'activations'], + 'Architecture': ['layers', 'dense', 'spatial', 'attention'], + 'Training': ['dataloader', 'autograd', 'optimizers', 'training'], + 'Production': ['compression', 'kernels', 'benchmarking', 'mlops', 'capstone'] + } + + category_counts = [] + category_labels = [] + category_colors = ['#FF6B6B', '#4ECDC4', '#45B7D1', '#96CEB4'] + + for category, module_list in categories.items(): + count = sum(1 for m in module_list if m in analyzer.modules) + if count > 0: + category_counts.append(count) + category_labels.append(f'{category}\n({count} modules)') + + ax3.pie(category_counts, labels=category_labels, colors=category_colors[:len(category_counts)], + autopct='%1.0f%%', startangle=90) + ax3.set_title('Module Distribution by Category', fontweight='bold') + + # 4. Learning progression timeline + cumulative_components = np.cumsum([0] + component_counts) + cumulative_time = np.cumsum([0] + times) + + ax4.plot(cumulative_time[:-1], cumulative_components[:-1], 'o-', linewidth=2, markersize=6) + ax4.set_title('Learning Progression', fontweight='bold') + ax4.set_xlabel('Cumulative Time (hours)') + ax4.set_ylabel('Cumulative Components Learned') + ax4.grid(True, alpha=0.3) + + # Add milestone annotations + milestones = [0, len(modules)//4, len(modules)//2, 3*len(modules)//4, len(modules)-1] + for i in milestones: + if i < len(cumulative_time) - 1: + ax4.annotate(f'{cumulative_components[i]} comp.\n{cumulative_time[i]:.0f}h', + xy=(cumulative_time[i], cumulative_components[i]), + xytext=(10, 10), textcoords='offset points', + bbox=dict(boxstyle='round,pad=0.3', facecolor='yellow', alpha=0.7), + fontsize=8) + + plt.tight_layout() + plt.show() + + return fig + +# Create component analysis +component_fig = create_component_analysis() + +# %% [markdown] +""" +### Test Component Analysis + +Let's verify our component analysis provides meaningful insights. +""" + +# %% +def test_component_analysis(): + """Test that component analysis reveals meaningful patterns""" + + # Test component distribution + total_components = sum(len(module.components) for module in analyzer.modules.values()) + avg_components = total_components / len(analyzer.modules) + + print(f"๐Ÿ“Š Total components across all modules: {total_components}") + print(f"๐Ÿ“Š Average components per module: {avg_components:.1f}") + + # Find modules with most/least components + component_counts = [(name, len(module.components)) for name, module in analyzer.modules.items()] + component_counts.sort(key=lambda x: x[1], reverse=True) + + print(f"\n๐Ÿ† Modules with most components:") + for name, count in component_counts[:3]: + print(f" {analyzer.modules[name].title}: {count} components") + + print(f"\n๐Ÿ† Modules with least components:") + for name, count in component_counts[-3:]: + print(f" {analyzer.modules[name].title}: {count} components") + + # Test correlation between difficulty and components + difficulties = [module.difficulty_level() for module in analyzer.modules.values()] + components = [len(module.components) for module in analyzer.modules.values()] + + correlation = np.corrcoef(difficulties, components)[0, 1] + print(f"\n๐Ÿ“ˆ Correlation between difficulty and components: {correlation:.2f}") + + if correlation > 0.3: + print("โœ… Higher difficulty modules tend to have more components") + elif correlation < -0.3: + print("โš ๏ธ Higher difficulty modules tend to have fewer components") + else: + print("๐Ÿ“Š No strong correlation between difficulty and component count") + +test_component_analysis() + +# %% [markdown] +""" +## Export Functions and Module Interface + +Create functions that can be imported and used by other parts of TinyTorch. +""" + +# %% +def get_tinytorch_overview() -> Dict: + """Get comprehensive overview of TinyTorch system for external use""" + return { + 'total_modules': len(analyzer.modules), + 'total_components': sum(len(module.components) for module in analyzer.modules.values()), + 'learning_path': analyzer.get_learning_path(), + 'total_time_hours': sum(module.estimated_hours() for module in analyzer.modules.values()), + 'difficulty_levels': {name: module.difficulty_level() for name, module in analyzer.modules.items()}, + 'module_categories': { + 'foundation': ['setup', 'tensor', 'activations'], + 'architecture': ['layers', 'dense', 'spatial', 'attention'], + 'training': ['dataloader', 'autograd', 'optimizers', 'training'], + 'production': ['compression', 'kernels', 'benchmarking', 'mlops', 'capstone'] + } + } + +def visualize_tinytorch_system(save_plots: bool = False) -> Dict: + """Generate all TinyTorch system visualizations""" + + visualizations = {} + + print("๐ŸŽจ Generating TinyTorch system visualizations...") + + # Generate dependency graph + print(" ๐Ÿ“Š Creating dependency graph...") + dep_fig = create_dependency_graph_visualization() + visualizations['dependency_graph'] = dep_fig + + # Generate architecture diagram + print(" ๐Ÿ—๏ธ Creating architecture diagram...") + arch_fig, positions = create_system_architecture_diagram() + visualizations['architecture'] = arch_fig + + # Generate learning roadmap + print(" ๐Ÿ“š Creating learning roadmap...") + roadmap_fig, path, time = create_learning_roadmap() + visualizations['roadmap'] = roadmap_fig + + # Generate component analysis + print(" ๐Ÿ” Creating component analysis...") + component_fig = create_component_analysis() + visualizations['components'] = component_fig + + if save_plots: + print(" ๐Ÿ’พ Saving plots to files...") + for name, fig in visualizations.items(): + fig.savefig(f'tinytorch_{name}.png', dpi=300, bbox_inches='tight') + + print("โœ… All visualizations generated successfully!") + + return visualizations + +def get_module_info(module_name: str) -> Dict: + """Get detailed information about a specific module""" + if module_name not in analyzer.modules: + return {'error': f'Module {module_name} not found'} + + module = analyzer.modules[module_name] + return { + 'name': module.name, + 'title': module.title, + 'description': module.description, + 'prerequisites': module.prerequisites, + 'enables': module.enables, + 'components': module.components, + 'difficulty': module.difficulty, + 'difficulty_level': module.difficulty_level(), + 'time_estimate': module.time_estimate, + 'estimated_hours': module.estimated_hours(), + 'exports_to': module.exports_to + } + +def get_learning_recommendations(current_module: str = None) -> Dict: + """Get personalized learning recommendations""" + path = analyzer.get_learning_path() + + if current_module is None: + return { + 'recommended_start': path[0] if path else None, + 'full_path': path, + 'total_time': sum(analyzer.modules[name].estimated_hours() + for name in path if name in analyzer.modules) + } + + if current_module not in path: + return {'error': f'Module {current_module} not found in learning path'} + + current_index = path.index(current_module) + + return { + 'current_module': current_module, + 'progress': f"{current_index + 1}/{len(path)}", + 'next_modules': path[current_index + 1:current_index + 4], # Next 3 modules + 'remaining_time': sum(analyzer.modules[name].estimated_hours() + for name in path[current_index + 1:] + if name in analyzer.modules), + 'prerequisites_completed': path[:current_index], + 'can_start': [name for name in path[current_index + 1:] + if all(prereq in path[:current_index + 1] + for prereq in analyzer.modules.get(name, ModuleInfo('','','',[],'',[],'','','')).prerequisites)] + } + +# Test the export functions +print("๐Ÿงช Testing export functions...") + +overview = get_tinytorch_overview() +print(f"๐Ÿ“Š System Overview: {overview['total_modules']} modules, {overview['total_components']} components") + +setup_info = get_module_info('setup') +print(f"๐Ÿ“‹ Setup Module: {setup_info['title']} - {setup_info['difficulty']}") + +recommendations = get_learning_recommendations() +print(f"๐Ÿ“š Learning Recommendations: Start with {recommendations['recommended_start']}") + +print("โœ… Export functions working correctly!") + +# %% [markdown] +""" +## ML Systems Thinking Questions + +Let's explore how TinyTorch's architecture connects to broader ML systems and production frameworks. +""" + +# %% [markdown] +""" +### System Architecture & Design Patterns + +**Reflection Questions:** + +1. **Modular Design Philosophy**: How does TinyTorch's module dependency system compare to frameworks like PyTorch or TensorFlow? What are the advantages and trade-offs of explicit dependency management? + +2. **Component Composition**: Notice how higher-level modules (like `training`) depend on multiple lower-level modules (`tensor`, `autograd`, `optimizers`). How does this pattern reflect real ML system architecture? + +3. **Framework Evolution**: Looking at the learning roadmap, how might you extend TinyTorch to support distributed training or GPU acceleration? Where would new modules fit in the dependency graph? + +### Production ML Systems + +**Reflection Questions:** + +4. **Deployment Pipeline**: How do the later modules (`compression`, `benchmarking`, `mlops`) mirror real-world ML deployment concerns? What additional modules might production systems require? + +5. **System Integration**: If you were to deploy a TinyTorch model in production, which modules would be most critical for runtime efficiency? How might you minimize dependencies? + +6. **Monitoring & Observability**: How does the `mlops` module's position as a terminal node reflect its role in production systems? What additional monitoring capabilities might be needed? + +### Framework Design Decisions + +**Reflection Questions:** + +7. **Dependency Management**: Compare TinyTorch's explicit prerequisite system to frameworks like Keras or scikit-learn. How do design decisions about dependencies affect developer experience? + +8. **Component Granularity**: Some modules have many components (like `training`) while others have few (like `setup`). How do you balance component granularity in ML framework design? + +9. **Educational vs Production**: How might the educational goals of TinyTorch influence its architecture differently than a production framework? Where do you see pedagogical design choices? + +### Performance & Scalability + +**Reflection Questions:** + +10. **Computational Graph**: How does TinyTorch's module structure relate to computational graphs in frameworks like PyTorch or JAX? Where do you see opportunities for optimization? + +11. **Memory Management**: Looking at the component analysis, which modules are likely to be most memory-intensive? How might this influence deployment strategies? + +12. **Hardware Acceleration**: The `kernels` module focuses on hardware-aware optimization. How do production frameworks handle the trade-off between abstraction and performance? + +*These questions are designed to help you think beyond implementation details toward the broader patterns and principles that guide ML systems design in industry.* +""" + +# %% [markdown] +""" +## Comprehensive Testing + +Let's run comprehensive tests to ensure all our visualizations and analysis tools work correctly. +""" + +# %% +def run_comprehensive_tests(): + """Run comprehensive tests of the introduction module functionality""" + + print("๐Ÿงช Running comprehensive tests for TinyTorch Introduction Module...") + print("=" * 60) + + # Test 1: Module Loading + print("\n1๏ธโƒฃ Testing module loading...") + assert len(analyzer.modules) >= 10, "Should load multiple modules" + assert 'setup' in analyzer.modules, "Should load setup module" + assert 'tensor' in analyzer.modules, "Should load tensor module" + print("โœ… Module loading tests passed") + + # Test 2: Dependency Graph + print("\n2๏ธโƒฃ Testing dependency graph...") + G = analyzer.dependency_graph + assert G.number_of_nodes() >= 10, "Should have multiple nodes" + assert G.number_of_edges() >= 5, "Should have dependency edges" + print("โœ… Dependency graph tests passed") + + # Test 3: Learning Path + print("\n3๏ธโƒฃ Testing learning path...") + path = analyzer.get_learning_path() + assert len(path) >= 10, "Should have meaningful learning path" + + # Verify no dependency violations + path_indices = {module: i for i, module in enumerate(path)} + violations = 0 + for module_name in path: + if module_name in analyzer.modules: + module = analyzer.modules[module_name] + for prereq in module.prerequisites: + if prereq in path_indices: + if path_indices[prereq] >= path_indices[module_name]: + violations += 1 + + assert violations == 0, f"Learning path should have no dependency violations (found {violations})" + print("โœ… Learning path tests passed") + + # Test 4: Component Analysis + print("\n4๏ธโƒฃ Testing component analysis...") + total_components = sum(len(module.components) for module in analyzer.modules.values()) + assert total_components >= 20, "Should have meaningful number of components" + print("โœ… Component analysis tests passed") + + # Test 5: Export Functions + print("\n5๏ธโƒฃ Testing export functions...") + overview = get_tinytorch_overview() + assert 'total_modules' in overview, "Overview should include module count" + assert 'learning_path' in overview, "Overview should include learning path" + + module_info = get_module_info('setup') + assert 'title' in module_info, "Module info should include title" + + recommendations = get_learning_recommendations() + assert 'recommended_start' in recommendations, "Should provide starting recommendation" + print("โœ… Export function tests passed") + + # Test 6: Visualization Generation + print("\n6๏ธโƒฃ Testing visualization generation...") + try: + # Test that we can generate all visualizations without errors + dep_fig = create_dependency_graph_visualization() + arch_fig, positions = create_system_architecture_diagram() + roadmap_fig, path, time = create_learning_roadmap() + component_fig = create_component_analysis() + + assert dep_fig is not None, "Should generate dependency graph" + assert arch_fig is not None, "Should generate architecture diagram" + assert roadmap_fig is not None, "Should generate roadmap" + assert component_fig is not None, "Should generate component analysis" + + # Close figures to save memory + plt.close(dep_fig) + plt.close(arch_fig) + plt.close(roadmap_fig) + plt.close(component_fig) + + print("โœ… Visualization generation tests passed") + except Exception as e: + print(f"โŒ Visualization test failed: {e}") + raise + + print("\n" + "=" * 60) + print("๐ŸŽ‰ ALL TESTS PASSED! TinyTorch Introduction Module is working correctly!") + print("=" * 60) + + # Summary statistics + print(f"\n๐Ÿ“Š System Summary:") + print(f" โ€ข {len(analyzer.modules)} modules loaded") + print(f" โ€ข {analyzer.dependency_graph.number_of_edges()} dependencies mapped") + print(f" โ€ข {len(analyzer.get_learning_path())} modules in learning path") + print(f" โ€ข {sum(len(m.components) for m in analyzer.modules.values())} total components") + print(f" โ€ข {sum(m.estimated_hours() for m in analyzer.modules.values()):.1f} total learning hours") + +# Run the comprehensive tests +run_comprehensive_tests() + +# %% [markdown] +""" +## Module Summary + +**Congratulations!** You've successfully explored the complete TinyTorch system architecture. + +### What You've Accomplished + +1. **๐Ÿ“Š System Analysis**: Built tools to automatically analyze module dependencies and relationships +2. **๐ŸŽจ Interactive Visualizations**: Created comprehensive visual overviews of the entire framework +3. **๐Ÿ“š Learning Roadmap**: Generated an optimal learning path through all 16 modules +4. **๐Ÿ” Component Analysis**: Analyzed the components within each module and their complexity +5. **๐Ÿ—๏ธ Architecture Overview**: Visualized how all TinyTorch components work together +6. **๐Ÿงช Comprehensive Testing**: Validated that all analysis tools work correctly + +### Key Insights Discovered + +- **TinyTorch contains {len(analyzer.modules)} modules** with {sum(len(m.components) for m in analyzer.modules.values())} total components +- **Learning path spans {sum(m.estimated_hours() for m in analyzer.modules.values()):.1f} hours** of estimated study time +- **Dependency structure** ensures proper learning progression from foundations to production +- **Modular design** enables flexible learning and component reuse + +### How This Connects to Industry ML Systems + +The architecture patterns you've explored in TinyTorch mirror those used in production ML frameworks: + +- **Modular Dependencies**: Similar to PyTorch's module system +- **Component Composition**: Reflects how TensorFlow builds complex operations from primitives +- **Production Pipeline**: MLOps module mirrors real deployment concerns +- **Performance Optimization**: Kernels and compression reflect production efficiency needs + +### Next Steps + +Now you're ready to dive into any TinyTorch module with a complete understanding of how it fits into the broader system. Use the learning roadmap to guide your journey through building a complete neural network framework from scratch! + +**Happy Learning! ๐Ÿš€** +""" + +# %% +# Export key functions for use by other modules +__all__ = [ + 'TinyTorchAnalyzer', + 'get_tinytorch_overview', + 'visualize_tinytorch_system', + 'get_module_info', + 'get_learning_recommendations', + 'create_dependency_graph_visualization', + 'create_system_architecture_diagram', + 'create_learning_roadmap', + 'create_component_analysis' +] + +print("๐ŸŽฏ TinyTorch Introduction Module Complete!") +print("๐Ÿ“ฆ Exported functions ready for use by other modules") \ No newline at end of file diff --git a/modules/source/00_introduction/module.yaml b/modules/source/00_introduction/module.yaml new file mode 100644 index 00000000..81ed506d --- /dev/null +++ b/modules/source/00_introduction/module.yaml @@ -0,0 +1,37 @@ +# TinyTorch Module Metadata +# Essential system information for CLI tools and build systems + +name: "introduction" +title: "System Introduction & Architecture" +description: "Visual overview of TinyTorch framework architecture, module dependencies, and learning roadmap" + +# Dependencies - Used by CLI for module ordering and prerequisites +dependencies: + prerequisites: [] + enables: ["setup", "tensor", "activations", "layers", "dense", "spatial", "attention", "dataloader", "autograd", "optimizers", "training", "compression", "kernels", "benchmarking", "mlops", "capstone"] + +# Package Export - What gets built into tinytorch package +exports_to: "tinytorch.introduction" + +# File Structure - What files exist in this module +files: + dev_file: "introduction_dev.py" + readme: "README.md" + tests: "inline" + +# Educational Metadata +difficulty: "โญ" +time_estimate: "1-2 hours" + +# Components - What's implemented in this module +components: + - "TinyTorchAnalyzer" + - "ModuleInfo" + - "get_tinytorch_overview" + - "visualize_tinytorch_system" + - "get_module_info" + - "get_learning_recommendations" + - "create_dependency_graph_visualization" + - "create_system_architecture_diagram" + - "create_learning_roadmap" + - "create_component_analysis" \ No newline at end of file