mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-11 19:53:33 -05:00
cleanup: remove redundant files and directories
- Delete _proc/ directory (duplicate of modules/ with processed files) - Delete bin/tito.py (old CLI implementation, replaced by tito/ structure) - Delete temporary log files (tito-cli.log, tinytorch-cli.log) - Update bin/tito wrapper to use new CLI structure - All CLI functionality preserved with new architecture
This commit is contained in:
Vijay Janapa Reddi
@@ -1,22 +0,0 @@
|
||||
project:
|
||||
type: website
|
||||
|
||||
format:
|
||||
html:
|
||||
theme: cosmo
|
||||
css: styles.css
|
||||
toc: true
|
||||
keep-md: true
|
||||
commonmark: default
|
||||
|
||||
website:
|
||||
twitter-card: true
|
||||
open-graph: true
|
||||
repo-actions: [issue]
|
||||
navbar:
|
||||
background: primary
|
||||
search: true
|
||||
sidebar:
|
||||
style: floating
|
||||
|
||||
metadata-files: [nbdev.yml, sidebar.yml]
|
||||
@@ -1,237 +0,0 @@
|
||||
# 🔥 TinyTorch Activations Module
|
||||
|
||||
Welcome to the **Activations** module! This is where you'll implement the mathematical functions that give neural networks their power to learn complex patterns.
|
||||
|
||||
## 🎯 Learning Objectives
|
||||
|
||||
By the end of this module, you will:
|
||||
1. **Understand** why activation functions are essential for neural networks
|
||||
2. **Implement** the three most important activation functions: ReLU, Sigmoid, and Tanh
|
||||
3. **Test** your functions with various inputs to understand their behavior
|
||||
4. **Grasp** the mathematical properties that make each function useful
|
||||
|
||||
## 🧠 Why This Module Matters
|
||||
|
||||
**Without activation functions, neural networks are just linear transformations!**
|
||||
|
||||
```
|
||||
Linear → Linear → Linear = Still just Linear
|
||||
Linear → Activation → Linear = Can learn complex patterns!
|
||||
```
|
||||
|
||||
This module teaches you the mathematical foundations that make deep learning possible.
|
||||
|
||||
## 📚 What You'll Build
|
||||
|
||||
### 1. **ReLU** (Rectified Linear Unit)
|
||||
- **Formula**: `f(x) = max(0, x)`
|
||||
- **Properties**: Simple, sparse, unbounded
|
||||
- **Use case**: Hidden layers (most common)
|
||||
|
||||
### 2. **Sigmoid**
|
||||
- **Formula**: `f(x) = 1 / (1 + e^(-x))`
|
||||
- **Properties**: Bounded to (0,1), smooth, probabilistic
|
||||
- **Use case**: Binary classification, gates
|
||||
|
||||
### 3. **Tanh** (Hyperbolic Tangent)
|
||||
- **Formula**: `f(x) = tanh(x)`
|
||||
- **Properties**: Bounded to (-1,1), zero-centered, smooth
|
||||
- **Use case**: Hidden layers, RNNs
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Development Workflow
|
||||
|
||||
1. **Open the development file**:
|
||||
```bash
|
||||
python bin/tito.py jupyter
|
||||
# Then open modules/activations/activations_dev.py
|
||||
```
|
||||
|
||||
2. **Implement the functions**:
|
||||
- Start with ReLU (simplest)
|
||||
- Move to Sigmoid (numerical stability challenge)
|
||||
- Finish with Tanh (symmetry properties)
|
||||
|
||||
3. **Visualize your functions**:
|
||||
- Each function has plotting sections
|
||||
- See how your implementation transforms inputs
|
||||
- Compare all functions side-by-side
|
||||
|
||||
4. **Test as you go**:
|
||||
```bash
|
||||
python bin/tito.py test --module activations
|
||||
```
|
||||
|
||||
5. **Export to package**:
|
||||
```bash
|
||||
python bin/tito.py sync
|
||||
```
|
||||
|
||||
### 📊 Visual Learning Features
|
||||
|
||||
This module includes comprehensive plotting sections to help you understand:
|
||||
|
||||
- **Individual Function Plots**: See each activation function's curve
|
||||
- **Implementation Comparison**: Your implementation vs ideal side-by-side
|
||||
- **Mathematical Explanations**: Visual breakdown of function properties
|
||||
- **Error Analysis**: Quantitative feedback on implementation accuracy
|
||||
- **Comprehensive Comparison**: All functions analyzed together
|
||||
|
||||
**Enhanced Features**:
|
||||
- **4-Panel Plots**: Implementation vs ideal, mathematical definition, properties, error analysis
|
||||
- **Real-time Feedback**: Immediate accuracy scores with color-coded status
|
||||
- **Mathematical Insights**: Detailed explanations of function properties
|
||||
- **Numerical Stability Testing**: Verification with extreme values
|
||||
- **Property Verification**: Symmetry, monotonicity, and zero-centering tests
|
||||
|
||||
**Why enhanced plots matter**:
|
||||
- **Visual Debugging**: See exactly where your implementation differs
|
||||
- **Quantitative Feedback**: Get precise error measurements
|
||||
- **Mathematical Understanding**: Connect formulas to visual behavior
|
||||
- **Implementation Confidence**: Know immediately if your code is correct
|
||||
- **Learning Reinforcement**: Multiple visual perspectives of the same concept
|
||||
|
||||
### Implementation Tips
|
||||
|
||||
#### ReLU Implementation
|
||||
```python
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
return Tensor(np.maximum(0, x.data))
|
||||
```
|
||||
|
||||
#### Sigmoid Implementation (Numerical Stability)
|
||||
```python
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
# For x >= 0: sigmoid(x) = 1 / (1 + exp(-x))
|
||||
# For x < 0: sigmoid(x) = exp(x) / (1 + exp(x))
|
||||
x_data = x.data
|
||||
result = np.zeros_like(x_data)
|
||||
|
||||
positive_mask = x_data >= 0
|
||||
result[positive_mask] = 1.0 / (1.0 + np.exp(-x_data[positive_mask]))
|
||||
result[~positive_mask] = np.exp(x_data[~positive_mask]) / (1.0 + np.exp(x_data[~positive_mask]))
|
||||
|
||||
return Tensor(result)
|
||||
```
|
||||
|
||||
#### Tanh Implementation
|
||||
```python
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
return Tensor(np.tanh(x.data))
|
||||
```
|
||||
|
||||
## 🧪 Testing Your Implementation
|
||||
|
||||
### Unit Tests
|
||||
```bash
|
||||
python bin/tito.py test --module activations
|
||||
```
|
||||
|
||||
**Test Coverage**:
|
||||
- ✅ Mathematical correctness
|
||||
- ✅ Numerical stability
|
||||
- ✅ Shape preservation
|
||||
- ✅ Edge cases
|
||||
- ✅ Function properties
|
||||
|
||||
### Manual Testing
|
||||
```python
|
||||
# Test all activations
|
||||
from tinytorch.core.tensor import Tensor
|
||||
from modules.activations.activations_dev import ReLU, Sigmoid, Tanh
|
||||
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
print("Input:", x.data)
|
||||
print("ReLU:", relu(x).data)
|
||||
print("Sigmoid:", sigmoid(x).data)
|
||||
print("Tanh:", tanh(x).data)
|
||||
```
|
||||
|
||||
## 📊 Understanding Function Properties
|
||||
|
||||
### Range Comparison
|
||||
| Function | Input Range | Output Range | Zero Point |
|
||||
|----------|-------------|--------------|------------|
|
||||
| ReLU | (-∞, ∞) | [0, ∞) | f(0) = 0 |
|
||||
| Sigmoid | (-∞, ∞) | (0, 1) | f(0) = 0.5 |
|
||||
| Tanh | (-∞, ∞) | (-1, 1) | f(0) = 0 |
|
||||
|
||||
### Key Properties
|
||||
- **ReLU**: Sparse (zeros out negatives), unbounded, simple
|
||||
- **Sigmoid**: Probabilistic (0-1 range), smooth, saturating
|
||||
- **Tanh**: Zero-centered, symmetric, stronger gradients than sigmoid
|
||||
|
||||
## 🔧 Integration with TinyTorch
|
||||
|
||||
After implementation, your activations will be available as:
|
||||
|
||||
```python
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh
|
||||
|
||||
# Use in neural networks
|
||||
relu = ReLU()
|
||||
output = relu(input_tensor)
|
||||
```
|
||||
|
||||
## 🎯 Common Issues & Solutions
|
||||
|
||||
### Issue 1: Sigmoid Overflow
|
||||
**Problem**: `exp()` overflow with large inputs
|
||||
**Solution**: Use numerically stable implementation (see code above)
|
||||
|
||||
### Issue 2: Wrong Output Range
|
||||
**Problem**: Sigmoid/Tanh outputs outside expected range
|
||||
**Solution**: Check your mathematical implementation
|
||||
|
||||
### Issue 3: Shape Mismatch
|
||||
**Problem**: Output shape differs from input shape
|
||||
**Solution**: Ensure element-wise operations preserve shape
|
||||
|
||||
### Issue 4: Import Errors
|
||||
**Problem**: Cannot import after implementation
|
||||
**Solution**: Run `python bin/tito.py sync` to export to package
|
||||
|
||||
## 📈 Performance Considerations
|
||||
|
||||
- **ReLU**: Fastest (simple max operation)
|
||||
- **Sigmoid**: Moderate (exponential computation)
|
||||
- **Tanh**: Moderate (hyperbolic function)
|
||||
|
||||
All implementations use NumPy for vectorized operations.
|
||||
|
||||
## 🚀 What's Next
|
||||
|
||||
After mastering activations, you'll use them in:
|
||||
1. **Layers Module**: Building neural network layers
|
||||
2. **Loss Functions**: Computing training objectives
|
||||
3. **Advanced Architectures**: CNNs, RNNs, and more
|
||||
|
||||
These functions are the mathematical foundation for everything that follows!
|
||||
|
||||
## 📚 Further Reading
|
||||
|
||||
**Mathematical Background**:
|
||||
- [Activation Functions in Neural Networks](https://en.wikipedia.org/wiki/Activation_function)
|
||||
- [Deep Learning Book - Chapter 6](http://www.deeplearningbook.org/)
|
||||
|
||||
**Advanced Topics**:
|
||||
- ReLU variants (Leaky ReLU, ELU, Swish)
|
||||
- Activation function choice and impact
|
||||
- Gradient flow and vanishing gradients
|
||||
|
||||
## 🎉 Success Criteria
|
||||
|
||||
You've mastered this module when:
|
||||
- [ ] All tests pass (`python bin/tito.py test --module activations`)
|
||||
- [ ] You understand why each function is useful
|
||||
- [ ] You can explain the mathematical properties
|
||||
- [ ] You can use activations in neural networks
|
||||
- [ ] You appreciate the importance of nonlinearity
|
||||
|
||||
**Great work! You've built the mathematical foundation of neural networks!** 🎉
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,345 +0,0 @@
|
||||
"""
|
||||
Test suite for the TinyTorch Activations module.
|
||||
|
||||
This test suite validates the mathematical correctness of activation functions:
|
||||
- ReLU: f(x) = max(0, x)
|
||||
- Sigmoid: f(x) = 1 / (1 + e^(-x))
|
||||
- Tanh: f(x) = tanh(x)
|
||||
|
||||
Tests focus on:
|
||||
1. Mathematical correctness
|
||||
2. Numerical stability
|
||||
3. Edge cases
|
||||
4. Shape preservation
|
||||
5. Type consistency
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import math
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
# Import the activation functions
|
||||
import sys
|
||||
import os
|
||||
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
|
||||
from activations_dev import ReLU, Sigmoid, Tanh
|
||||
|
||||
|
||||
class TestReLU:
|
||||
"""Test the ReLU activation function."""
|
||||
|
||||
def test_relu_basic_functionality(self):
|
||||
"""Test basic ReLU behavior: max(0, x)"""
|
||||
relu = ReLU()
|
||||
|
||||
# Test mixed positive/negative values
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
y = relu(x)
|
||||
expected = np.array([[0.0, 0.0, 0.0, 1.0, 2.0]])
|
||||
|
||||
assert np.allclose(y.data, expected), f"Expected {expected}, got {y.data}"
|
||||
|
||||
def test_relu_all_positive(self):
|
||||
"""Test ReLU with all positive values (should be unchanged)"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[1.0, 2.5, 3.7, 10.0]])
|
||||
y = relu(x)
|
||||
|
||||
assert np.allclose(y.data, x.data), "ReLU should preserve positive values"
|
||||
|
||||
def test_relu_all_negative(self):
|
||||
"""Test ReLU with all negative values (should be zeros)"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[-1.0, -2.5, -3.7, -10.0]])
|
||||
y = relu(x)
|
||||
expected = np.zeros_like(x.data)
|
||||
|
||||
assert np.allclose(y.data, expected), "ReLU should zero out negative values"
|
||||
|
||||
def test_relu_zero_input(self):
|
||||
"""Test ReLU with zero input"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[0.0]])
|
||||
y = relu(x)
|
||||
|
||||
assert y.data[0, 0] == 0.0, "ReLU(0) should be 0"
|
||||
|
||||
def test_relu_shape_preservation(self):
|
||||
"""Test that ReLU preserves tensor shape"""
|
||||
relu = ReLU()
|
||||
|
||||
# Test different shapes
|
||||
shapes = [(1, 5), (2, 3), (4, 1), (3, 3)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = relu(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_relu_callable(self):
|
||||
"""Test that ReLU can be called directly"""
|
||||
relu = ReLU()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = relu(x)
|
||||
y2 = relu.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestSigmoid:
|
||||
"""Test the Sigmoid activation function."""
|
||||
|
||||
def test_sigmoid_basic_functionality(self):
|
||||
"""Test basic Sigmoid behavior"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test known values
|
||||
x = Tensor([[0.0]])
|
||||
y = sigmoid(x)
|
||||
assert abs(y.data[0, 0] - 0.5) < 1e-6, "Sigmoid(0) should be 0.5"
|
||||
|
||||
def test_sigmoid_range(self):
|
||||
"""Test that Sigmoid outputs are in (0, 1)"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test wide range of inputs
|
||||
x = Tensor([[-10.0, -5.0, -1.0, 0.0, 1.0, 5.0, 10.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
assert np.all(y.data > 0), "Sigmoid outputs should be > 0"
|
||||
assert np.all(y.data < 1), "Sigmoid outputs should be < 1"
|
||||
|
||||
def test_sigmoid_numerical_stability(self):
|
||||
"""Test Sigmoid with extreme values (numerical stability)"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test extreme values that could cause overflow
|
||||
x = Tensor([[-100.0, -50.0, 50.0, 100.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Should not contain NaN or inf
|
||||
assert not np.any(np.isnan(y.data)), "Sigmoid should not produce NaN"
|
||||
assert not np.any(np.isinf(y.data)), "Sigmoid should not produce inf"
|
||||
|
||||
# Should be close to 0 for very negative, close to 1 for very positive
|
||||
assert y.data[0, 0] < 1e-10, "Sigmoid(-100) should be very close to 0"
|
||||
assert y.data[0, 1] < 1e-10, "Sigmoid(-50) should be very close to 0"
|
||||
assert y.data[0, 2] > 1 - 1e-10, "Sigmoid(50) should be very close to 1"
|
||||
assert y.data[0, 3] > 1 - 1e-10, "Sigmoid(100) should be very close to 1"
|
||||
|
||||
def test_sigmoid_monotonicity(self):
|
||||
"""Test that Sigmoid is monotonically increasing"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
x = Tensor([[-3.0, -1.0, 0.0, 1.0, 3.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Check that outputs are increasing
|
||||
for i in range(len(y.data[0]) - 1):
|
||||
assert y.data[0, i] < y.data[0, i + 1], "Sigmoid should be monotonically increasing"
|
||||
|
||||
def test_sigmoid_shape_preservation(self):
|
||||
"""Test that Sigmoid preserves tensor shape"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
shapes = [(1, 5), (2, 3), (4, 1)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = sigmoid(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_sigmoid_callable(self):
|
||||
"""Test that Sigmoid can be called directly"""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = sigmoid(x)
|
||||
y2 = sigmoid.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestTanh:
|
||||
"""Test the Tanh activation function."""
|
||||
|
||||
def test_tanh_basic_functionality(self):
|
||||
"""Test basic Tanh behavior"""
|
||||
tanh = Tanh()
|
||||
|
||||
# Test known values
|
||||
x = Tensor([[0.0]])
|
||||
y = tanh(x)
|
||||
assert abs(y.data[0, 0] - 0.0) < 1e-6, "Tanh(0) should be 0"
|
||||
|
||||
def test_tanh_range(self):
|
||||
"""Test that Tanh outputs are in [-1, 1]"""
|
||||
tanh = Tanh()
|
||||
|
||||
# Test wide range of inputs
|
||||
x = Tensor([[-10.0, -5.0, -1.0, 0.0, 1.0, 5.0, 10.0]])
|
||||
y = tanh(x)
|
||||
|
||||
assert np.all(y.data >= -1), "Tanh outputs should be >= -1"
|
||||
assert np.all(y.data <= 1), "Tanh outputs should be <= 1"
|
||||
|
||||
def test_tanh_symmetry(self):
|
||||
"""Test that Tanh is symmetric: tanh(-x) = -tanh(x)"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
x_neg = Tensor([[-1.0, -2.0, -3.0]])
|
||||
|
||||
y_pos = tanh(x)
|
||||
y_neg = tanh(x_neg)
|
||||
|
||||
assert np.allclose(y_neg.data, -y_pos.data), "Tanh should be symmetric"
|
||||
|
||||
def test_tanh_monotonicity(self):
|
||||
"""Test that Tanh is monotonically increasing"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-3.0, -1.0, 0.0, 1.0, 3.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Check that outputs are increasing
|
||||
for i in range(len(y.data[0]) - 1):
|
||||
assert y.data[0, i] < y.data[0, i + 1], "Tanh should be monotonically increasing"
|
||||
|
||||
def test_tanh_extreme_values(self):
|
||||
"""Test Tanh with extreme values"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-100.0, 100.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Should be close to -1 and 1 respectively
|
||||
assert abs(y.data[0, 0] - (-1.0)) < 1e-10, "Tanh(-100) should be very close to -1"
|
||||
assert abs(y.data[0, 1] - 1.0) < 1e-10, "Tanh(100) should be very close to 1"
|
||||
|
||||
def test_tanh_shape_preservation(self):
|
||||
"""Test that Tanh preserves tensor shape"""
|
||||
tanh = Tanh()
|
||||
|
||||
shapes = [(1, 5), (2, 3), (4, 1)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = tanh(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_tanh_callable(self):
|
||||
"""Test that Tanh can be called directly"""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = tanh(x)
|
||||
y2 = tanh.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestActivationComparison:
|
||||
"""Test interactions and comparisons between activation functions."""
|
||||
|
||||
def test_activation_consistency(self):
|
||||
"""Test that all activations work with the same input"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
|
||||
# All should process without error
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
# All should preserve shape
|
||||
assert y_relu.shape == x.shape
|
||||
assert y_sigmoid.shape == x.shape
|
||||
assert y_tanh.shape == x.shape
|
||||
|
||||
def test_activation_ranges(self):
|
||||
"""Test that activations have expected output ranges"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-5.0, -2.0, 0.0, 2.0, 5.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
# ReLU: [0, inf)
|
||||
assert np.all(y_relu.data >= 0), "ReLU should be non-negative"
|
||||
|
||||
# Sigmoid: (0, 1)
|
||||
assert np.all(y_sigmoid.data > 0), "Sigmoid should be positive"
|
||||
assert np.all(y_sigmoid.data < 1), "Sigmoid should be less than 1"
|
||||
|
||||
# Tanh: (-1, 1)
|
||||
assert np.all(y_tanh.data > -1), "Tanh should be greater than -1"
|
||||
assert np.all(y_tanh.data < 1), "Tanh should be less than 1"
|
||||
|
||||
|
||||
# Integration tests with edge cases
|
||||
class TestActivationEdgeCases:
|
||||
"""Test edge cases and boundary conditions."""
|
||||
|
||||
def test_zero_tensor(self):
|
||||
"""Test all activations with zero tensor"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[0.0, 0.0, 0.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert np.allclose(y_relu.data, [0.0, 0.0, 0.0]), "ReLU(0) should be 0"
|
||||
assert np.allclose(y_sigmoid.data, [0.5, 0.5, 0.5]), "Sigmoid(0) should be 0.5"
|
||||
assert np.allclose(y_tanh.data, [0.0, 0.0, 0.0]), "Tanh(0) should be 0"
|
||||
|
||||
def test_single_element_tensor(self):
|
||||
"""Test all activations with single element tensor"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[1.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert y_relu.shape == (1, 1)
|
||||
assert y_sigmoid.shape == (1, 1)
|
||||
assert y_tanh.shape == (1, 1)
|
||||
|
||||
def test_large_tensor(self):
|
||||
"""Test activations with larger tensors"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
# Create a 10x10 tensor
|
||||
x = Tensor(np.random.randn(10, 10))
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert y_relu.shape == (10, 10)
|
||||
assert y_sigmoid.shape == (10, 10)
|
||||
assert y_tanh.shape == (10, 10)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run tests with pytest
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -1,206 +0,0 @@
|
||||
# 🧱 Module 2: Layers - Neural Network Building Blocks
|
||||
|
||||
**Build the fundamental transformations that compose into neural networks**
|
||||
|
||||
## 🎯 Learning Objectives
|
||||
|
||||
After completing this module, you will:
|
||||
- Understand layers as functions that transform tensors: `y = f(x)`
|
||||
- Implement Dense layers with linear transformations: `y = Wx + b`
|
||||
- Add activation functions for nonlinearity (ReLU, Sigmoid, Tanh)
|
||||
- See how neural networks are just function composition
|
||||
- Build intuition for neural network architecture before diving into training
|
||||
|
||||
## 🧱 Build → Use → Understand
|
||||
|
||||
This module follows the TinyTorch pedagogical framework:
|
||||
|
||||
1. **Build**: Dense layers and activation functions from scratch
|
||||
2. **Use**: Transform tensors and see immediate results
|
||||
3. **Understand**: How neural networks transform information
|
||||
|
||||
## 📚 What You'll Build
|
||||
|
||||
### **Dense Layer**
|
||||
```python
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
y = layer(x) # Shape: (1, 2)
|
||||
```
|
||||
|
||||
### **Activation Functions**
|
||||
```python
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-1.0, 0.0, 1.0]])
|
||||
y_relu = relu(x) # [0.0, 0.0, 1.0]
|
||||
y_sigmoid = sigmoid(x) # [0.27, 0.5, 0.73]
|
||||
y_tanh = tanh(x) # [-0.76, 0.0, 0.76]
|
||||
```
|
||||
|
||||
### **Neural Networks**
|
||||
```python
|
||||
# 3 → 4 → 2 network
|
||||
layer1 = Dense(input_size=3, output_size=4)
|
||||
activation1 = ReLU()
|
||||
layer2 = Dense(input_size=4, output_size=2)
|
||||
activation2 = Sigmoid()
|
||||
|
||||
# Forward pass
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
h1 = layer1(x)
|
||||
h1_activated = activation1(h1)
|
||||
h2 = layer2(h1_activated)
|
||||
output = activation2(h2)
|
||||
```
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Prerequisites
|
||||
- Complete Module 1: Tensor ✅
|
||||
- Understand basic linear algebra (matrix multiplication)
|
||||
- Familiar with Python classes and methods
|
||||
|
||||
### Quick Start
|
||||
```bash
|
||||
# Navigate to the layers module
|
||||
cd modules/layers
|
||||
|
||||
# Work in the development notebook
|
||||
jupyter notebook layers_dev.ipynb
|
||||
|
||||
# Or work in the Python file
|
||||
code layers_dev.py
|
||||
```
|
||||
|
||||
## 📖 Module Structure
|
||||
|
||||
```
|
||||
modules/layers/
|
||||
├── layers_dev.py # Main development file (work here!)
|
||||
├── layers_dev.ipynb # Jupyter notebook version
|
||||
├── tests/
|
||||
│ └── test_layers.py # Comprehensive tests
|
||||
├── README.md # This file
|
||||
└── solutions/ # Reference implementations (if stuck)
|
||||
```
|
||||
|
||||
## 🎓 Learning Path
|
||||
|
||||
### Step 1: Dense Layer (Linear Transformation)
|
||||
- Understand `y = Wx + b`
|
||||
- Implement weight initialization
|
||||
- Handle matrix multiplication and bias addition
|
||||
- Test with single examples and batches
|
||||
|
||||
### Step 2: Activation Functions
|
||||
- Implement ReLU: `max(0, x)`
|
||||
- Implement Sigmoid: `1 / (1 + e^(-x))`
|
||||
- Implement Tanh: `tanh(x)`
|
||||
- Understand why nonlinearity is crucial
|
||||
|
||||
### Step 3: Layer Composition
|
||||
- Chain layers together
|
||||
- Build complete neural networks
|
||||
- See how simple layers create complex functions
|
||||
|
||||
### Step 4: Real-World Application
|
||||
- Build an image classification network
|
||||
- Understand how architecture affects capability
|
||||
|
||||
## 🧪 Testing Your Implementation
|
||||
|
||||
### Module-Level Tests
|
||||
```bash
|
||||
# Run comprehensive tests
|
||||
python -m pytest tests/test_layers.py -v
|
||||
|
||||
# Quick test
|
||||
python -c "from layers_dev import Dense, ReLU; print('✅ Layers working!')"
|
||||
```
|
||||
|
||||
### Package-Level Tests
|
||||
```bash
|
||||
# Export to package
|
||||
python ../../bin/tito.py sync
|
||||
|
||||
# Test integration
|
||||
python ../../bin/tito.py test --module layers
|
||||
```
|
||||
|
||||
## 🎯 Key Concepts
|
||||
|
||||
### **Layers as Functions**
|
||||
- Input: Tensor with some shape
|
||||
- Transformation: Mathematical operation
|
||||
- Output: Tensor with possibly different shape
|
||||
|
||||
### **Linear vs Nonlinear**
|
||||
- Dense layers: Linear transformations
|
||||
- Activation functions: Nonlinear transformations
|
||||
- Composition: Linear + Nonlinear = Complex functions
|
||||
|
||||
### **Neural Networks = Function Composition**
|
||||
```
|
||||
Input → Dense → ReLU → Dense → Sigmoid → Output
|
||||
```
|
||||
|
||||
### **Why This Matters**
|
||||
- **Modularity**: Build complex networks from simple parts
|
||||
- **Reusability**: Same layers work for different problems
|
||||
- **Understanding**: Know how each part contributes to the whole
|
||||
|
||||
## 🔍 Common Issues
|
||||
|
||||
### **Import Errors**
|
||||
```python
|
||||
# Make sure you're in the right directory
|
||||
import sys
|
||||
sys.path.append('../../')
|
||||
from modules.tensor.tensor_dev import Tensor
|
||||
```
|
||||
|
||||
### **Shape Mismatches**
|
||||
```python
|
||||
# Check input/output sizes match
|
||||
layer1 = Dense(input_size=3, output_size=4)
|
||||
layer2 = Dense(input_size=4, output_size=2) # 4 matches output of layer1
|
||||
```
|
||||
|
||||
### **Gradient Issues (Later)**
|
||||
```python
|
||||
# Use proper weight initialization
|
||||
limit = math.sqrt(6.0 / (input_size + output_size))
|
||||
weights = np.random.uniform(-limit, limit, (input_size, output_size))
|
||||
```
|
||||
|
||||
## 🎉 Success Criteria
|
||||
|
||||
You've successfully completed this module when:
|
||||
- ✅ All tests pass (`pytest tests/test_layers.py`)
|
||||
- ✅ You can build a 2-layer neural network
|
||||
- ✅ You understand how layers transform tensors
|
||||
- ✅ You see the connection between layers and neural networks
|
||||
- ✅ Package export works (`tito test --module layers`)
|
||||
|
||||
## 🚀 What's Next
|
||||
|
||||
After completing this module, you're ready for:
|
||||
- **Module 3: Networks** - Compose layers into common architectures
|
||||
- **Module 4: Training** - Learn how networks improve through experience
|
||||
- **Module 5: Applications** - Use networks for real problems
|
||||
|
||||
## 🤝 Getting Help
|
||||
|
||||
- Check the tests for examples of expected behavior
|
||||
- Look at the solutions/ directory if you're stuck
|
||||
- Review the pedagogical principles in `docs/pedagogy/`
|
||||
- Remember: Build → Use → Understand!
|
||||
|
||||
---
|
||||
|
||||
**Great job building the foundation of neural networks!** 🎉
|
||||
|
||||
*This module implements the core insight: neural networks are just function composition of simple building blocks.*
|
||||
@@ -1,469 +0,0 @@
|
||||
# ---
|
||||
# jupyter:
|
||||
# jupytext:
|
||||
# text_representation:
|
||||
# extension: .py
|
||||
# format_name: percent
|
||||
# format_version: '1.3'
|
||||
# jupytext_version: 1.17.1
|
||||
# ---
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
# Module 2: Layers - Neural Network Building Blocks
|
||||
|
||||
Welcome to the Layers module! This is where neural networks begin. You'll implement the fundamental building blocks that transform tensors.
|
||||
|
||||
## Learning Goals
|
||||
- Understand layers as functions that transform tensors: `y = f(x)`
|
||||
- Implement Dense layers with linear transformations: `y = Wx + b`
|
||||
- Use activation functions from the activations module for nonlinearity
|
||||
- See how neural networks are just function composition
|
||||
- Build intuition before diving into training
|
||||
|
||||
## Build → Use → Understand
|
||||
1. **Build**: Dense layers using activation functions as building blocks
|
||||
2. **Use**: Transform tensors and see immediate results
|
||||
3. **Understand**: How neural networks transform information
|
||||
|
||||
## Module Dependencies
|
||||
This module builds on the **activations** module:
|
||||
- **activations** → **layers** → **networks**
|
||||
- Clean separation of concerns: math functions → layer building blocks → full networks
|
||||
|
||||
## Module → Package Structure
|
||||
**🎓 Teaching vs. 🔧 Building**:
|
||||
- **Learning side**: Work in `modules/layers/layers_dev.py`
|
||||
- **Building side**: Exports to `tinytorch/core/layers.py`
|
||||
|
||||
This module builds the fundamental transformations that compose into neural networks.
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| default_exp core.layers
|
||||
|
||||
# Setup and imports
|
||||
import numpy as np
|
||||
import sys
|
||||
from typing import Union, Optional, Callable
|
||||
import math
|
||||
|
||||
# %%
|
||||
#| export
|
||||
import numpy as np
|
||||
import math
|
||||
import sys
|
||||
from typing import Union, Optional, Callable
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
# Import activation functions from the activations module
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh
|
||||
|
||||
# Import our Tensor class
|
||||
# sys.path.append('../../')
|
||||
# from modules.tensor.tensor_dev import Tensor
|
||||
|
||||
# print("🔥 TinyTorch Layers Module")
|
||||
# print(f"NumPy version: {np.__version__}")
|
||||
# print(f"Python version: {sys.version_info.major}.{sys.version_info.minor}")
|
||||
# print("Ready to build neural network layers!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1: What is a Layer?
|
||||
|
||||
A **layer** is a function that transforms tensors. Think of it as:
|
||||
- **Input**: Tensor with some shape
|
||||
- **Transformation**: Mathematical operation (linear, nonlinear, etc.)
|
||||
- **Output**: Tensor with possibly different shape
|
||||
|
||||
**The fundamental insight**: Neural networks are just function composition!
|
||||
```
|
||||
x → Layer1 → Layer2 → Layer3 → y
|
||||
```
|
||||
|
||||
**Why layers matter**:
|
||||
- They're the building blocks of all neural networks
|
||||
- Each layer learns a different transformation
|
||||
- Composing layers creates complex functions
|
||||
- Understanding layers = understanding neural networks
|
||||
|
||||
Let's start with the most important layer: **Dense** (also called Linear or Fully Connected).
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
class Dense:
|
||||
"""
|
||||
Dense (Linear) Layer: y = Wx + b
|
||||
|
||||
The fundamental building block of neural networks.
|
||||
Performs linear transformation: matrix multiplication + bias addition.
|
||||
|
||||
Args:
|
||||
input_size: Number of input features
|
||||
output_size: Number of output features
|
||||
use_bias: Whether to include bias term (default: True)
|
||||
|
||||
TODO: Implement the Dense layer with weight initialization and forward pass.
|
||||
"""
|
||||
|
||||
def __init__(self, input_size: int, output_size: int, use_bias: bool = True):
|
||||
"""
|
||||
Initialize Dense layer with random weights.
|
||||
|
||||
TODO:
|
||||
1. Store layer parameters (input_size, output_size, use_bias)
|
||||
2. Initialize weights with small random values
|
||||
3. Initialize bias to zeros (if use_bias=True)
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
"""
|
||||
Forward pass: y = Wx + b
|
||||
|
||||
Args:
|
||||
x: Input tensor of shape (batch_size, input_size)
|
||||
|
||||
Returns:
|
||||
Output tensor of shape (batch_size, output_size)
|
||||
|
||||
TODO: Implement matrix multiplication and bias addition
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __call__(self, x: Tensor) -> Tensor:
|
||||
"""Make layer callable: layer(x) same as layer.forward(x)"""
|
||||
return self.forward(x)
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
class Dense:
|
||||
"""
|
||||
Dense (Linear) Layer: y = Wx + b
|
||||
|
||||
The fundamental building block of neural networks.
|
||||
Performs linear transformation: matrix multiplication + bias addition.
|
||||
"""
|
||||
|
||||
def __init__(self, input_size: int, output_size: int, use_bias: bool = True):
|
||||
"""Initialize Dense layer with random weights."""
|
||||
self.input_size = input_size
|
||||
self.output_size = output_size
|
||||
self.use_bias = use_bias
|
||||
|
||||
# Initialize weights with Xavier/Glorot initialization
|
||||
# This helps with gradient flow during training
|
||||
limit = math.sqrt(6.0 / (input_size + output_size))
|
||||
self.weights = Tensor(
|
||||
np.random.uniform(-limit, limit, (input_size, output_size)).astype(np.float32)
|
||||
)
|
||||
|
||||
# Initialize bias to zeros
|
||||
if use_bias:
|
||||
self.bias = Tensor(np.zeros(output_size, dtype=np.float32))
|
||||
else:
|
||||
self.bias = None
|
||||
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
"""Forward pass: y = Wx + b"""
|
||||
# Matrix multiplication: x @ weights
|
||||
# x shape: (batch_size, input_size)
|
||||
# weights shape: (input_size, output_size)
|
||||
# result shape: (batch_size, output_size)
|
||||
output = Tensor(x.data @ self.weights.data)
|
||||
|
||||
# Add bias if present
|
||||
if self.bias is not None:
|
||||
output = Tensor(output.data + self.bias.data)
|
||||
|
||||
return output
|
||||
|
||||
def __call__(self, x: Tensor) -> Tensor:
|
||||
"""Make layer callable: layer(x) same as layer.forward(x)"""
|
||||
return self.forward(x)
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Dense Layer
|
||||
|
||||
Once you implement the Dense layer above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test the Dense layer
|
||||
try:
|
||||
print("=== Testing Dense Layer ===")
|
||||
|
||||
# Create a simple Dense layer: 3 inputs → 2 outputs
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
print(f"Created Dense layer: {layer.input_size} → {layer.output_size}")
|
||||
print(f"Weights shape: {layer.weights.shape}")
|
||||
print(f"Bias shape: {layer.bias.shape if layer.bias else 'No bias'}")
|
||||
|
||||
# Test with a single example
|
||||
x = Tensor([[1.0, 2.0, 3.0]]) # Shape: (1, 3)
|
||||
y = layer(x)
|
||||
print(f"Input shape: {x.shape}")
|
||||
print(f"Output shape: {y.shape}")
|
||||
print(f"Input: {x.data}")
|
||||
print(f"Output: {y.data}")
|
||||
|
||||
# Test with batch
|
||||
x_batch = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) # Shape: (2, 3)
|
||||
y_batch = layer(x_batch)
|
||||
print(f"\nBatch input shape: {x_batch.shape}")
|
||||
print(f"Batch output shape: {y_batch.shape}")
|
||||
|
||||
print("✅ Dense layer working!")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error: {e}")
|
||||
print("Make sure to implement the Dense layer above!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2: Activation Functions - Adding Nonlinearity
|
||||
|
||||
Now we'll use the activation functions from the **activations** module!
|
||||
|
||||
**Clean Architecture**: We import the activation functions rather than redefining them:
|
||||
```python
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh
|
||||
```
|
||||
|
||||
**Why this matters**:
|
||||
- **Separation of concerns**: Math functions vs. layer building blocks
|
||||
- **Reusability**: Activations can be used anywhere in the system
|
||||
- **Maintainability**: One place to update activation implementations
|
||||
- **Composability**: Clean imports make neural networks easier to build
|
||||
|
||||
**Why nonlinearity matters**: Without it, stacking layers is pointless!
|
||||
```
|
||||
Linear → Linear → Linear = Just one big Linear transformation
|
||||
Linear → NonLinear → Linear = Can learn complex patterns
|
||||
```
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Activation Functions from Activations Module
|
||||
|
||||
Let's test that we can use the activation functions from the activations module:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test activation functions from activations module
|
||||
try:
|
||||
print("=== Testing Activation Functions from Activations Module ===")
|
||||
|
||||
# Test data: mix of positive, negative, and zero
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
print(f"Input: {x.data}")
|
||||
|
||||
# Test ReLU from activations module
|
||||
relu = ReLU()
|
||||
y_relu = relu(x)
|
||||
print(f"ReLU output: {y_relu.data}")
|
||||
|
||||
# Test Sigmoid from activations module
|
||||
sigmoid = Sigmoid()
|
||||
y_sigmoid = sigmoid(x)
|
||||
print(f"Sigmoid output: {y_sigmoid.data}")
|
||||
|
||||
# Test Tanh from activations module
|
||||
tanh = Tanh()
|
||||
y_tanh = tanh(x)
|
||||
print(f"Tanh output: {y_tanh.data}")
|
||||
|
||||
print("✅ Activation functions from activations module working!")
|
||||
print("🎉 Clean architecture: layers module uses activations module!")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error: {e}")
|
||||
print("Make sure the activations module is properly exported!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 3: Layer Composition - Building Neural Networks
|
||||
|
||||
Now comes the magic! We can **compose** layers to build neural networks:
|
||||
|
||||
```
|
||||
Input → Dense → ReLU → Dense → Sigmoid → Output
|
||||
```
|
||||
|
||||
This is a 2-layer neural network that can learn complex nonlinear patterns!
|
||||
|
||||
**Notice the clean architecture**:
|
||||
- Dense layers handle linear transformations
|
||||
- Activation functions (from activations module) handle nonlinearity
|
||||
- Composition creates complex behaviors from simple building blocks
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Build a simple 2-layer neural network
|
||||
try:
|
||||
print("=== Building a 2-Layer Neural Network ===")
|
||||
|
||||
# Network architecture: 3 → 4 → 2
|
||||
# Input: 3 features
|
||||
# Hidden: 4 neurons with ReLU
|
||||
# Output: 2 neurons with Sigmoid
|
||||
|
||||
layer1 = Dense(input_size=3, output_size=4)
|
||||
activation1 = ReLU() # From activations module
|
||||
layer2 = Dense(input_size=4, output_size=2)
|
||||
activation2 = Sigmoid() # From activations module
|
||||
|
||||
print("Network architecture:")
|
||||
print(f" Input: 3 features")
|
||||
print(f" Hidden: {layer1.input_size} → {layer1.output_size} (Dense + ReLU)")
|
||||
print(f" Output: {layer2.input_size} → {layer2.output_size} (Dense + Sigmoid)")
|
||||
|
||||
# Test with sample data
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) # 2 examples, 3 features each
|
||||
print(f"\nInput shape: {x.shape}")
|
||||
print(f"Input data: {x.data}")
|
||||
|
||||
# Forward pass through the network
|
||||
h1 = layer1(x) # Dense layer 1
|
||||
h1_activated = activation1(h1) # ReLU activation
|
||||
h2 = layer2(h1_activated) # Dense layer 2
|
||||
output = activation2(h2) # Sigmoid activation
|
||||
|
||||
print(f"\nAfter layer 1: {h1.shape}")
|
||||
print(f"After ReLU: {h1_activated.shape}")
|
||||
print(f"After layer 2: {h2.shape}")
|
||||
print(f"Final output: {output.shape}")
|
||||
print(f"Output values: {output.data}")
|
||||
|
||||
print("\n🎉 Neural network working! You just built your first neural network!")
|
||||
print("🏗️ Clean architecture: Dense layers + Activations module = Neural Network")
|
||||
print("Notice how the network transforms 3D input into 2D output through learned transformations.")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error: {e}")
|
||||
print("Make sure to implement the layers and check activations module!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 4: Understanding What We Built
|
||||
|
||||
Congratulations! You just implemented a clean, modular neural network architecture:
|
||||
|
||||
### 🧱 **What You Built**
|
||||
1. **Dense Layer**: Linear transformation `y = Wx + b`
|
||||
2. **Activation Functions**: Imported from activations module (ReLU, Sigmoid, Tanh)
|
||||
3. **Layer Composition**: Chaining layers to build networks
|
||||
|
||||
### 🏗️ **Clean Architecture Benefits**
|
||||
- **Separation of concerns**: Math functions vs. layer building blocks
|
||||
- **Reusability**: Activations can be used across different modules
|
||||
- **Maintainability**: One place to update activation implementations
|
||||
- **Composability**: Clean imports make complex networks easier to build
|
||||
|
||||
### 🎯 **Key Insights**
|
||||
- **Layers are functions**: They transform tensors from one space to another
|
||||
- **Composition creates complexity**: Simple layers → complex networks
|
||||
- **Nonlinearity is crucial**: Without it, deep networks are just linear transformations
|
||||
- **Neural networks are function approximators**: They learn to map inputs to outputs
|
||||
- **Modular design**: Building blocks can be combined in many ways
|
||||
|
||||
### 🚀 **What's Next**
|
||||
In the next modules, you'll learn:
|
||||
- **Training**: How networks learn from data (backpropagation, optimizers)
|
||||
- **Architectures**: Specialized layers for different problems (CNNs, RNNs)
|
||||
- **Applications**: Using networks for real problems
|
||||
|
||||
### 🔧 **Export to Package**
|
||||
Run this to export your layers to the TinyTorch package:
|
||||
```bash
|
||||
python bin/tito.py sync
|
||||
```
|
||||
|
||||
Then test your implementation:
|
||||
```bash
|
||||
python bin/tito.py test --module layers
|
||||
```
|
||||
|
||||
**Great job! You've built a clean, modular foundation for neural networks!** 🎉
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Final demonstration: A more complex example
|
||||
try:
|
||||
print("=== Final Demo: Image Classification Network ===")
|
||||
|
||||
# Simulate a small image: 28x28 pixels flattened to 784 features
|
||||
# This is like a tiny MNIST digit
|
||||
image_size = 28 * 28 # 784 pixels
|
||||
num_classes = 10 # 10 digits (0-9)
|
||||
|
||||
# Build a 3-layer network for digit classification
|
||||
# 784 → 128 → 64 → 10
|
||||
layer1 = Dense(input_size=image_size, output_size=128)
|
||||
relu1 = ReLU() # From activations module
|
||||
layer2 = Dense(input_size=128, output_size=64)
|
||||
relu2 = ReLU() # From activations module
|
||||
layer3 = Dense(input_size=64, output_size=num_classes)
|
||||
softmax = Sigmoid() # Using Sigmoid as a simple "probability-like" output
|
||||
|
||||
print(f"Image classification network:")
|
||||
print(f" Input: {image_size} pixels (28x28 image)")
|
||||
print(f" Hidden 1: {layer1.input_size} → {layer1.output_size} (Dense + ReLU)")
|
||||
print(f" Hidden 2: {layer2.input_size} → {layer2.output_size} (Dense + ReLU)")
|
||||
print(f" Output: {layer3.input_size} → {layer3.output_size} (Dense + Sigmoid)")
|
||||
|
||||
# Simulate a batch of 5 images
|
||||
batch_size = 5
|
||||
fake_images = Tensor(np.random.randn(batch_size, image_size).astype(np.float32))
|
||||
|
||||
# Forward pass
|
||||
h1 = relu1(layer1(fake_images))
|
||||
h2 = relu2(layer2(h1))
|
||||
predictions = softmax(layer3(h2))
|
||||
|
||||
print(f"\nBatch processing:")
|
||||
print(f" Input batch shape: {fake_images.shape}")
|
||||
print(f" Predictions shape: {predictions.shape}")
|
||||
print(f" Sample predictions: {predictions.data[0]}") # First image predictions
|
||||
|
||||
print("\n🎉 You built a neural network that could classify images!")
|
||||
print("🏗️ Clean architecture: Dense layers + Activations module = Image Classifier")
|
||||
print("With training, this network could learn to recognize handwritten digits!")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error: {e}")
|
||||
print("Check your layer implementations and activations module!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## 🎓 Module Summary
|
||||
|
||||
### What You Learned
|
||||
1. **Layer Architecture**: Dense layers as linear transformations
|
||||
2. **Clean Dependencies**: Layers module uses activations module
|
||||
3. **Function Composition**: Simple building blocks → complex networks
|
||||
4. **Modular Design**: Separation of concerns for maintainable code
|
||||
|
||||
### Key Architectural Insight
|
||||
```
|
||||
activations (math functions) → layers (building blocks) → networks (applications)
|
||||
```
|
||||
|
||||
This clean dependency graph makes the system:
|
||||
- **Understandable**: Each module has a clear purpose
|
||||
- **Testable**: Each module can be tested independently
|
||||
- **Reusable**: Components can be used across different contexts
|
||||
- **Maintainable**: Changes are localized to appropriate modules
|
||||
|
||||
### Next Steps
|
||||
- **Training**: Learn how networks learn from data
|
||||
- **Advanced Architectures**: CNNs, RNNs, Transformers
|
||||
- **Applications**: Real-world machine learning problems
|
||||
|
||||
**Congratulations on building a clean, modular neural network foundation!** 🚀
|
||||
"""
|
||||
@@ -1,347 +0,0 @@
|
||||
"""
|
||||
Tests for TinyTorch Layers module.
|
||||
|
||||
Tests the core layer functionality including Dense layers, activation functions,
|
||||
and layer composition.
|
||||
|
||||
These tests work with the current implementation and provide stretch goals
|
||||
for students to implement additional features.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import pytest
|
||||
import numpy as np
|
||||
|
||||
# Add the parent directory to path to import layers_dev
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Import from the module's development file
|
||||
# Note: This imports the instructor version with full implementation
|
||||
from layers_dev import Dense, Tensor
|
||||
|
||||
# Import activation functions from the activations module
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), '..', 'activations'))
|
||||
from activations_dev import ReLU, Sigmoid, Tanh
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .numpy() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'numpy'):
|
||||
return tensor.numpy()
|
||||
else:
|
||||
return tensor.data
|
||||
|
||||
class TestDenseLayer:
|
||||
"""Test Dense (Linear) layer functionality."""
|
||||
|
||||
def test_dense_creation(self):
|
||||
"""Test creating Dense layers with different configurations."""
|
||||
# Basic dense layer
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
assert layer.input_size == 3
|
||||
assert layer.output_size == 2
|
||||
assert layer.use_bias == True
|
||||
assert layer.weights.shape == (3, 2)
|
||||
assert layer.bias.shape == (2,)
|
||||
|
||||
# Dense layer without bias
|
||||
layer_no_bias = Dense(input_size=4, output_size=3, use_bias=False)
|
||||
assert layer_no_bias.use_bias == False
|
||||
assert layer_no_bias.bias is None
|
||||
|
||||
def test_dense_forward_single(self):
|
||||
"""Test Dense layer forward pass with single input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Single input
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 2)
|
||||
assert isinstance(y, Tensor)
|
||||
|
||||
def test_dense_forward_batch(self):
|
||||
"""Test Dense layer forward pass with batch input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Batch input
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (2, 2)
|
||||
assert isinstance(y, Tensor)
|
||||
|
||||
def test_dense_no_bias(self):
|
||||
"""Test Dense layer without bias."""
|
||||
layer = Dense(input_size=2, output_size=1, use_bias=False)
|
||||
|
||||
x = Tensor([[1.0, 2.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 1)
|
||||
# Should be just matrix multiplication without bias
|
||||
expected = safe_numpy(x) @ safe_numpy(layer.weights)
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), expected)
|
||||
|
||||
def test_dense_callable(self):
|
||||
"""Test that Dense layer is callable."""
|
||||
layer = Dense(input_size=2, output_size=1)
|
||||
x = Tensor([[1.0, 2.0]])
|
||||
|
||||
# Both should work
|
||||
y1 = layer.forward(x)
|
||||
y2 = layer(x)
|
||||
|
||||
np.testing.assert_array_equal(safe_numpy(y1), safe_numpy(y2))
|
||||
|
||||
class TestActivationFunctions:
|
||||
"""Test activation function implementations."""
|
||||
|
||||
def test_relu_basic(self):
|
||||
"""Test ReLU activation function."""
|
||||
relu = ReLU()
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
y = relu(x)
|
||||
|
||||
expected = [[0.0, 0.0, 0.0, 1.0, 2.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(y), expected)
|
||||
|
||||
def test_relu_callable(self):
|
||||
"""Test that ReLU is callable."""
|
||||
relu = ReLU()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = relu.forward(x)
|
||||
y2 = relu(x)
|
||||
|
||||
np.testing.assert_array_equal(safe_numpy(y1), safe_numpy(y2))
|
||||
|
||||
def test_sigmoid_basic(self):
|
||||
"""Test Sigmoid activation function."""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[0.0]]) # sigmoid(0) = 0.5
|
||||
y = sigmoid(x)
|
||||
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), [[0.5]])
|
||||
|
||||
def test_sigmoid_range(self):
|
||||
"""Test Sigmoid output range."""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[-10.0, 0.0, 10.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Should be in range [0, 1] - use reasonable bounds
|
||||
assert np.all(safe_numpy(y) >= 0)
|
||||
assert np.all(safe_numpy(y) <= 1)
|
||||
# Check that extreme values are close to bounds
|
||||
assert safe_numpy(y)[0][0] < 0.01 # Very small for -10
|
||||
assert safe_numpy(y)[0][2] > 0.99 # Very large for 10
|
||||
|
||||
def test_tanh_basic(self):
|
||||
"""Test Tanh activation function."""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[0.0]]) # tanh(0) = 0
|
||||
y = tanh(x)
|
||||
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), [[0.0]])
|
||||
|
||||
def test_tanh_range(self):
|
||||
"""Test Tanh output range."""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[-10.0, 0.0, 10.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Should be in range [-1, 1] - use reasonable bounds
|
||||
assert np.all(safe_numpy(y) >= -1)
|
||||
assert np.all(safe_numpy(y) <= 1)
|
||||
# Check that extreme values are close to bounds
|
||||
assert safe_numpy(y)[0][0] < -0.99 # Very negative for -10
|
||||
assert safe_numpy(y)[0][2] > 0.99 # Very positive for 10
|
||||
|
||||
class TestLayerComposition:
|
||||
"""Test composing layers into neural networks."""
|
||||
|
||||
def test_simple_network(self):
|
||||
"""Test a simple 2-layer network."""
|
||||
# 3 → 4 → 2 network
|
||||
layer1 = Dense(input_size=3, output_size=4)
|
||||
relu = ReLU()
|
||||
layer2 = Dense(input_size=4, output_size=2)
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Forward pass
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
h1 = layer1(x)
|
||||
h1_activated = relu(h1)
|
||||
h2 = layer2(h1_activated)
|
||||
output = sigmoid(h2)
|
||||
|
||||
assert h1.shape == (1, 4)
|
||||
assert h1_activated.shape == (1, 4)
|
||||
assert h2.shape == (1, 2)
|
||||
assert output.shape == (1, 2)
|
||||
|
||||
# Output should be in sigmoid range
|
||||
assert np.all(safe_numpy(output) >= 0)
|
||||
assert np.all(safe_numpy(output) <= 1)
|
||||
|
||||
def test_batch_network(self):
|
||||
"""Test network with batch processing."""
|
||||
layer1 = Dense(input_size=2, output_size=3)
|
||||
relu = ReLU()
|
||||
layer2 = Dense(input_size=3, output_size=1)
|
||||
|
||||
# Batch of 4 examples
|
||||
x = Tensor([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0], [7.0, 8.0]])
|
||||
|
||||
h1 = layer1(x)
|
||||
h1_activated = relu(h1)
|
||||
output = layer2(h1_activated)
|
||||
|
||||
assert output.shape == (4, 1)
|
||||
|
||||
def test_deep_network(self):
|
||||
"""Test deeper network composition."""
|
||||
# 5-layer network
|
||||
layers = [
|
||||
Dense(input_size=10, output_size=8),
|
||||
ReLU(),
|
||||
Dense(input_size=8, output_size=6),
|
||||
ReLU(),
|
||||
Dense(input_size=6, output_size=4),
|
||||
ReLU(),
|
||||
Dense(input_size=4, output_size=2),
|
||||
Sigmoid()
|
||||
]
|
||||
|
||||
x = Tensor([[1.0] * 10]) # 10 features
|
||||
|
||||
# Forward pass through all layers
|
||||
current = x
|
||||
for layer in layers:
|
||||
current = layer(current)
|
||||
|
||||
assert current.shape == (1, 2)
|
||||
# Final output should be in sigmoid range
|
||||
assert np.all(safe_numpy(current) >= 0)
|
||||
assert np.all(safe_numpy(current) <= 1)
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and error conditions."""
|
||||
|
||||
def test_zero_input(self):
|
||||
"""Test layers with zero input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[0.0, 0.0, 0.0]])
|
||||
y = layer(x)
|
||||
y_relu = relu(y)
|
||||
|
||||
assert y.shape == (1, 2)
|
||||
assert y_relu.shape == (1, 2)
|
||||
|
||||
def test_large_input(self):
|
||||
"""Test layers with large input values."""
|
||||
layer = Dense(input_size=2, output_size=1)
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
x = Tensor([[1000.0, -1000.0]])
|
||||
y = layer(x)
|
||||
y_sigmoid = sigmoid(y)
|
||||
|
||||
# Should not overflow
|
||||
assert not np.any(np.isnan(safe_numpy(y_sigmoid)))
|
||||
assert not np.any(np.isinf(safe_numpy(y_sigmoid)))
|
||||
|
||||
def test_single_neuron(self):
|
||||
"""Test single neuron layers."""
|
||||
layer = Dense(input_size=1, output_size=1)
|
||||
x = Tensor([[5.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 1)
|
||||
|
||||
# Stretch goal tests (these will be skipped if methods don't exist)
|
||||
class TestStretchGoals:
|
||||
"""Stretch goal tests for advanced features."""
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Weight initialization methods")
|
||||
def test_weight_initialization_methods(self):
|
||||
"""Test different weight initialization strategies."""
|
||||
# Xavier initialization
|
||||
layer_xavier = Dense(input_size=100, output_size=50, init_method='xavier')
|
||||
weights_xavier = safe_numpy(layer_xavier.weights)
|
||||
|
||||
# He initialization
|
||||
layer_he = Dense(input_size=100, output_size=50, init_method='he')
|
||||
weights_he = safe_numpy(layer_he.weights)
|
||||
|
||||
# Check initialization ranges
|
||||
xavier_limit = np.sqrt(6.0 / (100 + 50))
|
||||
assert np.all(np.abs(weights_xavier) <= xavier_limit)
|
||||
|
||||
he_limit = np.sqrt(2.0 / 100)
|
||||
assert np.std(weights_he) <= he_limit * 1.5 # Some tolerance
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Layer parameter access")
|
||||
def test_layer_parameters(self):
|
||||
"""Test accessing and modifying layer parameters."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Should be able to access parameters
|
||||
assert hasattr(layer, 'parameters')
|
||||
params = layer.parameters()
|
||||
assert len(params) == 2 # weights and bias
|
||||
|
||||
# Should be able to set parameters
|
||||
new_weights = Tensor(np.ones((3, 2)))
|
||||
layer.set_weights(new_weights)
|
||||
np.testing.assert_array_equal(safe_numpy(layer.weights), safe_numpy(new_weights))
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Additional activation functions")
|
||||
def test_additional_activations(self):
|
||||
"""Test additional activation functions."""
|
||||
# Leaky ReLU
|
||||
leaky_relu = LeakyReLU(alpha=0.1)
|
||||
x = Tensor([[-1.0, 0.0, 1.0]])
|
||||
y = leaky_relu(x)
|
||||
expected = [[-0.1, 0.0, 1.0]]
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), expected)
|
||||
|
||||
# Softmax
|
||||
softmax = Softmax()
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
y = softmax(x)
|
||||
# Should sum to 1
|
||||
assert np.allclose(np.sum(safe_numpy(y)), 1.0)
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Dropout layer")
|
||||
def test_dropout_layer(self):
|
||||
"""Test dropout layer implementation."""
|
||||
dropout = Dropout(p=0.5)
|
||||
x = Tensor([[1.0, 2.0, 3.0, 4.0]])
|
||||
|
||||
# Training mode
|
||||
dropout.train()
|
||||
y_train = dropout(x)
|
||||
|
||||
# Inference mode
|
||||
dropout.eval()
|
||||
y_eval = dropout(x)
|
||||
|
||||
# In eval mode, should be same as input
|
||||
np.testing.assert_array_equal(safe_numpy(y_eval), safe_numpy(x))
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Batch normalization")
|
||||
def test_batch_normalization(self):
|
||||
"""Test batch normalization layer."""
|
||||
bn = BatchNorm1d(num_features=3)
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
y = bn(x)
|
||||
|
||||
# Should normalize across batch dimension
|
||||
assert y.shape == x.shape
|
||||
# Mean should be close to 0, std close to 1
|
||||
assert np.allclose(np.mean(safe_numpy(y), axis=0), 0.0, atol=1e-6)
|
||||
assert np.allclose(np.std(safe_numpy(y), axis=0), 1.0, atol=1e-6)
|
||||
@@ -1,9 +0,0 @@
|
||||
project:
|
||||
output-dir: _docs
|
||||
|
||||
website:
|
||||
title: "TinyTorch"
|
||||
site-url: "https://tinytorch.github.io/TinyTorch/"
|
||||
description: "Build ML Systems from Scratch - A hands-on systems course"
|
||||
repo-branch: main
|
||||
repo-url: "https://github.com/tinytorch/TinyTorch/"
|
||||
@@ -1,141 +0,0 @@
|
||||
# Setup Module
|
||||
|
||||
Welcome to TinyTorch! This is your first module in the Machine Learning Systems course.
|
||||
|
||||
## Overview
|
||||
|
||||
The setup module teaches you the complete TinyTorch development workflow while introducing fundamental programming concepts. You'll learn to write code with NBDev directives, implement classes and functions, and understand the module-to-package export system.
|
||||
|
||||
## Learning Goals
|
||||
|
||||
- Understand the nbdev notebook-to-Python workflow
|
||||
- Write your first TinyTorch code with `#| export` directives
|
||||
- Implement system information collection and developer profiles
|
||||
- Run tests and use the CLI tools
|
||||
- Get comfortable with the development rhythm
|
||||
|
||||
## Files
|
||||
|
||||
- `setup_dev.py` - Main development file (Jupytext format with full educational content)
|
||||
- `setup_dev.ipynb` - Jupyter notebook version (auto-generated and executed)
|
||||
- `tinytorch_flame.txt` - ASCII art file containing the TinyTorch flame design
|
||||
- `tests/test_setup.py` - Comprehensive pytest test suite
|
||||
- `README.md` - This file
|
||||
|
||||
## What You'll Implement
|
||||
|
||||
### 1. Basic Functions
|
||||
- `hello_tinytorch()` - Display ASCII art and welcome message
|
||||
- `add_numbers()` - Basic arithmetic (foundation of ML operations)
|
||||
|
||||
### 2. System Information Class
|
||||
- `SystemInfo` - Collect and display Python version, platform, and machine info
|
||||
- Compatibility checking for minimum requirements
|
||||
|
||||
### 3. Developer Profile Class
|
||||
- `DeveloperProfile` - Personalized developer information and signatures
|
||||
- ASCII art customization and file loading
|
||||
- Professional code attribution system
|
||||
|
||||
## Usage
|
||||
|
||||
### Python Script
|
||||
```python
|
||||
from setup_dev import hello_tinytorch, add_numbers, SystemInfo, DeveloperProfile
|
||||
|
||||
# Display welcome message
|
||||
hello_tinytorch()
|
||||
|
||||
# Basic arithmetic
|
||||
result = add_numbers(2, 3)
|
||||
|
||||
# System information
|
||||
info = SystemInfo()
|
||||
print(f"System: {info}")
|
||||
print(f"Compatible: {info.is_compatible()}")
|
||||
|
||||
# Developer profile
|
||||
profile = DeveloperProfile()
|
||||
print(profile.get_full_profile())
|
||||
```
|
||||
|
||||
### Jupyter Notebook
|
||||
Open `setup_dev.ipynb` and work through the educational content step by step.
|
||||
|
||||
## Testing
|
||||
|
||||
Run the comprehensive test suite using pytest:
|
||||
|
||||
```bash
|
||||
# Using the TinyTorch CLI (recommended)
|
||||
python bin/tito.py test --module setup
|
||||
|
||||
# Or directly with pytest
|
||||
python -m pytest modules/setup/tests/test_setup.py -v
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
|
||||
The test suite includes **20 comprehensive tests** covering:
|
||||
- ✅ **Function execution** - All functions run without errors
|
||||
- ✅ **Output validation** - Correct content and formatting
|
||||
- ✅ **Arithmetic operations** - Basic, negative, and floating-point math
|
||||
- ✅ **System information** - Platform detection and compatibility
|
||||
- ✅ **Developer profiles** - Default and custom configurations
|
||||
- ✅ **ASCII art handling** - File loading and fallback behavior
|
||||
- ✅ **Error recovery** - Graceful handling of missing files
|
||||
- ✅ **Integration testing** - All components work together
|
||||
|
||||
## Development Workflow
|
||||
|
||||
This module teaches the core TinyTorch development cycle:
|
||||
|
||||
1. **Write code** in the notebook using `#| export` directives
|
||||
2. **Export code** with `python bin/tito.py sync --module setup`
|
||||
3. **Run tests** with `python bin/tito.py test --module setup`
|
||||
4. **Check progress** with `python bin/tito.py info`
|
||||
|
||||
## Key Concepts
|
||||
|
||||
- **NBDev workflow** - Write in notebooks, export to Python packages
|
||||
- **Export directives** - Use `#| export` to mark code for export
|
||||
- **Module → Package mapping** - This module exports to `tinytorch/core/utils.py`
|
||||
- **Teaching vs. Building** - Learn by modules, build by function
|
||||
- **Student implementation** - TODO sections with instructor solutions hidden
|
||||
|
||||
## Personalization Features
|
||||
|
||||
### ASCII Art Customization
|
||||
The ASCII art is loaded from `tinytorch_flame.txt`. You can customize it by:
|
||||
|
||||
1. **Edit the file directly** - Modify `tinytorch_flame.txt` with your own ASCII art
|
||||
2. **Custom parameter** - Pass your own ASCII art to `DeveloperProfile`
|
||||
3. **Create your own design** - Your initials, logo, or motivational art
|
||||
|
||||
### Developer Profile Customization
|
||||
```python
|
||||
my_profile = DeveloperProfile(
|
||||
name="Your Name",
|
||||
affiliation="Your University",
|
||||
email="your.email@example.com",
|
||||
github_username="yourgithub",
|
||||
ascii_art="Your custom ASCII art here!"
|
||||
)
|
||||
```
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
This comprehensive module introduces:
|
||||
- **NBDev educational patterns** - `#| export`, `#| hide` directives
|
||||
- **File I/O operations** - Loading ASCII art with error handling
|
||||
- **Object-oriented programming** - Classes, methods, and properties
|
||||
- **System programming** - Platform detection and compatibility
|
||||
- **Testing with pytest** - Professional test structure and assertions
|
||||
- **Code organization** - Module structure and package exports
|
||||
- **The TinyTorch development workflow** - Complete cycle from code to tests
|
||||
|
||||
## Next Steps
|
||||
|
||||
Once you've completed this module and all tests pass, you're ready to move on to the **tensor module** where you'll build the core data structures that power TinyTorch neural networks!
|
||||
|
||||
The skills you learn here - the development workflow, testing patterns, and code organization - will be used throughout every module in TinyTorch.
|
||||
@@ -1,535 +0,0 @@
|
||||
# ---
|
||||
# jupyter:
|
||||
# jupytext:
|
||||
# text_representation:
|
||||
# extension: .py
|
||||
# format_name: percent
|
||||
# format_version: '1.3'
|
||||
# jupytext_version: 1.17.1
|
||||
# ---
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
# Module 0: Setup - Tiny🔥Torch Development Workflow
|
||||
|
||||
Welcome to TinyTorch! This module teaches you the development workflow you'll use throughout the course.
|
||||
|
||||
## Learning Goals
|
||||
- Understand the nbdev notebook-to-Python workflow
|
||||
- Write your first TinyTorch code
|
||||
- Run tests and use the CLI tools
|
||||
- Get comfortable with the development rhythm
|
||||
|
||||
## The TinyTorch Development Cycle
|
||||
|
||||
1. **Write code** in this notebook using `#| export`
|
||||
2. **Export code** with `python bin/tito.py sync --module setup`
|
||||
3. **Run tests** with `python bin/tito.py test --module setup`
|
||||
4. **Check progress** with `python bin/tito.py info`
|
||||
|
||||
Let's get started!
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| default_exp core.utils
|
||||
|
||||
# Setup imports and environment
|
||||
import sys
|
||||
import platform
|
||||
from datetime import datetime
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
print("🔥 TinyTorch Development Environment")
|
||||
print(f"Python {sys.version}")
|
||||
print(f"Platform: {platform.system()} {platform.release()}")
|
||||
print(f"Started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1: Understanding the Module → Package Structure
|
||||
|
||||
**🎓 Teaching vs. 🔧 Building**: This course has two sides:
|
||||
- **Teaching side**: You work in `modules/setup/setup_dev.ipynb` (learning-focused)
|
||||
- **Building side**: Your code exports to `tinytorch/core/utils.py` (production package)
|
||||
|
||||
**Key Concept**: The `#| default_exp core.utils` directive at the top tells nbdev to export all `#| export` cells to `tinytorch/core/utils.py`.
|
||||
|
||||
This separation allows us to:
|
||||
- Organize learning by **concepts** (modules)
|
||||
- Organize code by **function** (package structure)
|
||||
- Build a real ML framework while learning systematically
|
||||
|
||||
Let's write a simple "Hello World" function with the `#| export` directive:
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
def hello_tinytorch():
|
||||
"""
|
||||
A simple hello world function for TinyTorch.
|
||||
|
||||
TODO: Implement this function to display TinyTorch ASCII art and welcome message.
|
||||
Load the flame art from tinytorch_flame.txt file with graceful fallback.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def add_numbers(a, b):
|
||||
"""
|
||||
Add two numbers together.
|
||||
|
||||
TODO: Implement addition of two numbers.
|
||||
This is the foundation of all mathematical operations in ML.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
def hello_tinytorch():
|
||||
"""Display the TinyTorch ASCII art and welcome message."""
|
||||
try:
|
||||
# Get the directory containing this file
|
||||
current_dir = Path(__file__).parent
|
||||
art_file = current_dir / "tinytorch_flame.txt"
|
||||
|
||||
if art_file.exists():
|
||||
with open(art_file, 'r') as f:
|
||||
ascii_art = f.read()
|
||||
print(ascii_art)
|
||||
print("Tiny🔥Torch")
|
||||
print("Build ML Systems from Scratch!")
|
||||
else:
|
||||
print("🔥 TinyTorch 🔥")
|
||||
print("Build ML Systems from Scratch!")
|
||||
except NameError:
|
||||
# Handle case when running in notebook where __file__ is not defined
|
||||
try:
|
||||
art_file = Path(os.getcwd()) / "tinytorch_flame.txt"
|
||||
if art_file.exists():
|
||||
with open(art_file, 'r') as f:
|
||||
ascii_art = f.read()
|
||||
print(ascii_art)
|
||||
print("Tiny🔥Torch")
|
||||
print("Build ML Systems from Scratch!")
|
||||
else:
|
||||
print("🔥 TinyTorch 🔥")
|
||||
print("Build ML Systems from Scratch!")
|
||||
except:
|
||||
print("🔥 TinyTorch 🔥")
|
||||
print("Build ML Systems from Scratch!")
|
||||
|
||||
def add_numbers(a, b):
|
||||
"""Add two numbers together."""
|
||||
return a + b
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Implementation
|
||||
|
||||
Once you implement the functions above, run this cell to test them:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test the functions in the notebook (will fail until implemented)
|
||||
try:
|
||||
print("Testing hello_tinytorch():")
|
||||
hello_tinytorch()
|
||||
print()
|
||||
print("Testing add_numbers():")
|
||||
print(f"2 + 3 = {add_numbers(2, 3)}")
|
||||
except NotImplementedError as e:
|
||||
print(f"⚠️ {e}")
|
||||
print("Implement the functions above first!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2: A Simple Class
|
||||
|
||||
Let's create a simple class that will help us understand system information. This is still basic, but shows how to structure classes in TinyTorch.
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
class SystemInfo:
|
||||
"""
|
||||
Simple system information class.
|
||||
|
||||
TODO: Implement this class to collect and display system information.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""
|
||||
Initialize system information collection.
|
||||
|
||||
TODO: Collect Python version, platform, and machine information.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __str__(self):
|
||||
"""
|
||||
Return human-readable system information.
|
||||
|
||||
TODO: Format system info as a readable string.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def is_compatible(self):
|
||||
"""
|
||||
Check if system meets minimum requirements.
|
||||
|
||||
TODO: Check if Python version is >= 3.8
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
class SystemInfo:
|
||||
"""Simple system information class."""
|
||||
|
||||
def __init__(self):
|
||||
self.python_version = sys.version_info
|
||||
self.platform = platform.system()
|
||||
self.machine = platform.machine()
|
||||
|
||||
def __str__(self):
|
||||
return f"Python {self.python_version.major}.{self.python_version.minor} on {self.platform} ({self.machine})"
|
||||
|
||||
def is_compatible(self):
|
||||
"""Check if system meets minimum requirements."""
|
||||
return self.python_version >= (3, 8)
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your SystemInfo Class
|
||||
|
||||
Once you implement the SystemInfo class above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test the SystemInfo class (will fail until implemented)
|
||||
try:
|
||||
print("Testing SystemInfo class:")
|
||||
info = SystemInfo()
|
||||
print(f"System: {info}")
|
||||
print(f"Compatible: {info.is_compatible()}")
|
||||
except NotImplementedError as e:
|
||||
print(f"⚠️ {e}")
|
||||
print("Implement the SystemInfo class above first!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 3: Developer Personalization
|
||||
|
||||
Let's make TinyTorch yours! Create a developer profile that will identify you throughout your ML systems journey.
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
class DeveloperProfile:
|
||||
"""
|
||||
Developer profile for personalizing TinyTorch experience.
|
||||
|
||||
TODO: Implement this class to store and display developer information.
|
||||
Default to course instructor but allow students to personalize.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _load_default_flame():
|
||||
"""
|
||||
Load the default TinyTorch flame ASCII art from file.
|
||||
|
||||
TODO: Implement file loading for tinytorch_flame.txt with fallback.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __init__(self, name="Vijay Janapa Reddi", affiliation="Harvard University",
|
||||
email="vj@eecs.harvard.edu", github_username="profvjreddi", ascii_art=None):
|
||||
"""
|
||||
Initialize developer profile.
|
||||
|
||||
TODO: Store developer information with sensible defaults.
|
||||
Students should be able to customize this with their own info and ASCII art.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __str__(self):
|
||||
"""
|
||||
Return formatted developer information.
|
||||
|
||||
TODO: Format developer info as a professional signature with optional ASCII art.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def get_signature(self):
|
||||
"""
|
||||
Get a short signature for code headers.
|
||||
|
||||
TODO: Return a concise signature like "Built by Name (@github)"
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def get_ascii_art(self):
|
||||
"""
|
||||
Get ASCII art for the profile.
|
||||
|
||||
TODO: Return custom ASCII art or default flame loaded from file.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
class DeveloperProfile:
|
||||
"""Developer profile for personalizing TinyTorch experience."""
|
||||
|
||||
@staticmethod
|
||||
def _load_default_flame():
|
||||
"""Load the default TinyTorch flame ASCII art from file."""
|
||||
try:
|
||||
# Try to load from the same directory as this module
|
||||
try:
|
||||
# Try to get the directory of the current file
|
||||
current_dir = os.path.dirname(__file__)
|
||||
except NameError:
|
||||
# If __file__ is not defined (e.g., in notebook), use current directory
|
||||
current_dir = os.getcwd()
|
||||
|
||||
flame_path = os.path.join(current_dir, 'tinytorch_flame.txt')
|
||||
|
||||
with open(flame_path, 'r', encoding='utf-8') as f:
|
||||
flame_art = f.read()
|
||||
|
||||
# Add the Tiny🔥Torch text below the flame
|
||||
return f"""{flame_art}
|
||||
|
||||
Tiny🔥Torch
|
||||
Build ML Systems from Scratch!
|
||||
"""
|
||||
except (FileNotFoundError, IOError):
|
||||
# Fallback to simple flame if file not found
|
||||
return """
|
||||
🔥 TinyTorch Developer 🔥
|
||||
. . . . . .
|
||||
. . . . . .
|
||||
. . . . . . .
|
||||
. . . . . . . .
|
||||
. . . . . . . . .
|
||||
. . . . . . . . . .
|
||||
. . . . . . . . . . .
|
||||
. . . . . . . . . . . .
|
||||
. . . . . . . . . . . . .
|
||||
. . . . . . . . . . . . . .
|
||||
\\ \\ \\ \\ \\ \\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ \\ \\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ \\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ \\ / / / / / /
|
||||
\\ \\ \\ / / / / / /
|
||||
\\ \\ / / / / / /
|
||||
\\ / / / / / /
|
||||
\\/ / / / / /
|
||||
\\/ / / / /
|
||||
\\/ / / /
|
||||
\\/ / /
|
||||
\\/ /
|
||||
\\/
|
||||
|
||||
Tiny🔥Torch
|
||||
Build ML Systems from Scratch!
|
||||
"""
|
||||
|
||||
def __init__(self, name="Vijay Janapa Reddi", affiliation="Harvard University",
|
||||
email="vj@eecs.harvard.edu", github_username="profvjreddi", ascii_art=None):
|
||||
self.name = name
|
||||
self.affiliation = affiliation
|
||||
self.email = email
|
||||
self.github_username = github_username
|
||||
self.ascii_art = ascii_art or self._load_default_flame()
|
||||
|
||||
def __str__(self):
|
||||
return f"👨💻 {self.name} | {self.affiliation} | @{self.github_username}"
|
||||
|
||||
def get_signature(self):
|
||||
"""Get a short signature for code headers."""
|
||||
return f"Built by {self.name} (@{self.github_username})"
|
||||
|
||||
def get_ascii_art(self):
|
||||
"""Get ASCII art for the profile."""
|
||||
return self.ascii_art
|
||||
|
||||
def get_full_profile(self):
|
||||
"""Get complete profile with ASCII art."""
|
||||
return f"""{self.ascii_art}
|
||||
|
||||
👨💻 Developer: {self.name}
|
||||
🏛️ Affiliation: {self.affiliation}
|
||||
📧 Email: {self.email}
|
||||
🐙 GitHub: @{self.github_username}
|
||||
🔥 Ready to build ML systems from scratch!
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Developer Profile
|
||||
|
||||
Customize your developer profile! Replace the default information with your own:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test the DeveloperProfile class
|
||||
try:
|
||||
print("Testing DeveloperProfile (with defaults):")
|
||||
# Default profile (instructor)
|
||||
default_profile = DeveloperProfile()
|
||||
print(f"Profile: {default_profile}")
|
||||
print(f"Signature: {default_profile.get_signature()}")
|
||||
print()
|
||||
|
||||
print("🎨 ASCII Art Preview:")
|
||||
print(default_profile.get_ascii_art())
|
||||
print()
|
||||
|
||||
print("🔥 Full Profile Display:")
|
||||
print(default_profile.get_full_profile())
|
||||
print()
|
||||
|
||||
# TODO: Students should customize this with their own information!
|
||||
print("🎯 YOUR TURN: Create your own profile!")
|
||||
print("Uncomment and modify the lines below:")
|
||||
print("# my_profile = DeveloperProfile(")
|
||||
print("# name='Your Name',")
|
||||
print("# affiliation='Your University/Company',")
|
||||
print("# email='your.email@example.com',")
|
||||
print("# github_username='yourgithub',")
|
||||
print("# ascii_art='''")
|
||||
print("# Your Custom ASCII Art Here!")
|
||||
print("# Maybe your initials, a logo, or something fun!")
|
||||
print("# '''")
|
||||
print("# )")
|
||||
print("# print(f'My Profile: {my_profile}')")
|
||||
print("# print(f'My Signature: {my_profile.get_signature()}')")
|
||||
print("# print(my_profile.get_full_profile())")
|
||||
|
||||
except NotImplementedError as e:
|
||||
print(f"⚠️ {e}")
|
||||
print("Implement the DeveloperProfile class above first!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🎨 Personalization Challenge
|
||||
|
||||
**For Students**: Make TinyTorch truly yours by:
|
||||
|
||||
1. **Update your profile** in the cell above with your real information
|
||||
2. **Create custom ASCII art** - your initials, a simple logo, or something that represents you
|
||||
3. **Customize the flame file** - edit `tinytorch_flame.txt` to create your own default art
|
||||
4. **Add your signature** to code you write throughout the course
|
||||
5. **Show off your full profile** with the `get_full_profile()` method
|
||||
|
||||
This isn't just about customization - it's about taking ownership of your learning journey in ML systems!
|
||||
|
||||
**ASCII Art Customization Options:**
|
||||
|
||||
**Option 1: Custom ASCII Art Parameter**
|
||||
```python
|
||||
my_profile = DeveloperProfile(
|
||||
name="Your Name",
|
||||
ascii_art='''
|
||||
Your Custom ASCII Art Here!
|
||||
Maybe your initials, a logo, or something fun!
|
||||
'''
|
||||
)
|
||||
```
|
||||
|
||||
**Option 2: Edit the Default Flame File**
|
||||
- Edit `tinytorch_flame.txt` in this directory
|
||||
- Replace with your own ASCII art design
|
||||
- All students using defaults will see your custom art!
|
||||
|
||||
**ASCII Art Ideas:**
|
||||
- Your initials in block letters
|
||||
- A simple logo or symbol that represents you
|
||||
- Your university mascot in ASCII
|
||||
- A coding-themed design
|
||||
- Something that motivates you!
|
||||
|
||||
**Pro Tip**: The `tinytorch_flame.txt` file contains the beautiful default flame art. You can:
|
||||
- Edit it directly for a personalized default
|
||||
- Create your own `.txt` file and modify the code to load it
|
||||
- Use online ASCII art generators for inspiration
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 4: Try the Export Process
|
||||
|
||||
Now let's export our code! In your terminal, run:
|
||||
|
||||
```bash
|
||||
python bin/tito.py sync --module setup
|
||||
```
|
||||
|
||||
This will export the code marked with `#| export` to `tinytorch/core/utils.py`.
|
||||
|
||||
**What happens during export:**
|
||||
1. nbdev scans this notebook for `#| export` cells
|
||||
2. Extracts the Python code
|
||||
3. Writes it to `tinytorch/core/utils.py` (because of `#| default_exp core.utils`)
|
||||
4. Handles imports and dependencies automatically
|
||||
|
||||
**🔍 Verification**: After export, check `tinytorch/core/utils.py` - you'll see your functions there with auto-generated headers pointing back to this notebook!
|
||||
|
||||
**Note**: The export process will use the instructor solutions (from `#|hide` cells) so the package will have working implementations even if you haven't completed the exercises yet.
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 5: Run Tests
|
||||
|
||||
After exporting, run the tests:
|
||||
|
||||
```bash
|
||||
python bin/tito.py test --module setup
|
||||
```
|
||||
|
||||
This will run all tests for the setup module and verify your implementation works correctly.
|
||||
|
||||
## Step 6: Check Your Progress
|
||||
|
||||
See your overall progress:
|
||||
|
||||
```bash
|
||||
python bin/tito.py info
|
||||
```
|
||||
|
||||
This shows which modules are complete and which are pending.
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## 🎉 Congratulations!
|
||||
|
||||
You've learned the TinyTorch development workflow:
|
||||
|
||||
1. ✅ Write code in notebooks with `#| export`
|
||||
2. ✅ Export with `tito sync --module setup`
|
||||
3. ✅ Test with `tito test --module setup`
|
||||
4. ✅ Check progress with `tito info`
|
||||
|
||||
**This is the rhythm you'll use for every module in TinyTorch.**
|
||||
|
||||
### Next Steps
|
||||
|
||||
Ready for the real work? Head to **Module 1: Tensor** where you'll build the core data structures that power everything else in TinyTorch.
|
||||
|
||||
**Development Tips:**
|
||||
- Always test your code in the notebook first
|
||||
- Export frequently to catch issues early
|
||||
- Read error messages carefully - they're designed to help
|
||||
- When stuck, check if your code exports cleanly first
|
||||
|
||||
Happy building! 🔥
|
||||
"""
|
||||
@@ -1,703 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "raw",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"output-file: setup_dev.html\n",
|
||||
"title: \"Module 0: Setup - Tiny\\U0001F525Torch Development Workflow\"\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a1a9c143",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\"",
|
||||
"lines_to_next_cell": 1
|
||||
},
|
||||
"source": [
|
||||
"## Step 1: Understanding the Module → Package Structure\n",
|
||||
"\n",
|
||||
"**🎓 Teaching vs. 🔧 Building**: This course has two sides:\n",
|
||||
"- **Teaching side**: You work in `modules/setup/setup_dev.ipynb` (learning-focused)\n",
|
||||
"- **Building side**: Your code exports to `tinytorch/core/utils.py` (production package)\n",
|
||||
"\n",
|
||||
"**Key Concept**: The `#| default_exp core.utils` directive at the top tells nbdev to export all `#| export` cells to `tinytorch/core/utils.py`.\n",
|
||||
"\n",
|
||||
"This separation allows us to:\n",
|
||||
"- Organize learning by **concepts** (modules) \n",
|
||||
"- Organize code by **function** (package structure)\n",
|
||||
"- Build a real ML framework while learning systematically\n",
|
||||
"\n",
|
||||
"Let's write a simple \"Hello World\" function with the `#| export` directive:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 0,
|
||||
"has_sd": true,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/markdown": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### add_numbers\n",
|
||||
"\n",
|
||||
"> add_numbers (a, b)\n",
|
||||
"\n",
|
||||
"*Add two numbers together.\n",
|
||||
"\n",
|
||||
"TODO: Implement addition of two numbers.\n",
|
||||
"This is the foundation of all mathematical operations in ML.*"
|
||||
],
|
||||
"text/plain": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### add_numbers\n",
|
||||
"\n",
|
||||
"> add_numbers (a, b)\n",
|
||||
"\n",
|
||||
"*Add two numbers together.\n",
|
||||
"\n",
|
||||
"TODO: Implement addition of two numbers.\n",
|
||||
"This is the foundation of all mathematical operations in ML.*"
|
||||
]
|
||||
},
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"#| echo: false\n",
|
||||
"#| output: asis\n",
|
||||
"show_doc(add_numbers)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 0,
|
||||
"has_sd": true,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/markdown": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### hello_tinytorch\n",
|
||||
"\n",
|
||||
"> hello_tinytorch ()\n",
|
||||
"\n",
|
||||
"*A simple hello world function for TinyTorch.\n",
|
||||
"\n",
|
||||
"TODO: Implement this function to display TinyTorch ASCII art and welcome message.\n",
|
||||
"Load the flame art from tinytorch_flame.txt file with graceful fallback.*"
|
||||
],
|
||||
"text/plain": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### hello_tinytorch\n",
|
||||
"\n",
|
||||
"> hello_tinytorch ()\n",
|
||||
"\n",
|
||||
"*A simple hello world function for TinyTorch.\n",
|
||||
"\n",
|
||||
"TODO: Implement this function to display TinyTorch ASCII art and welcome message.\n",
|
||||
"Load the flame art from tinytorch_flame.txt file with graceful fallback.*"
|
||||
]
|
||||
},
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"#| echo: false\n",
|
||||
"#| output: asis\n",
|
||||
"show_doc(hello_tinytorch)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b28103af",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"### 🧪 Test Your Implementation\n",
|
||||
"\n",
|
||||
"Once you implement the functions above, run this cell to test them:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "a1beca72",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2025-07-10T23:28:59.088616Z",
|
||||
"iopub.status.busy": "2025-07-10T23:28:59.088506Z",
|
||||
"iopub.status.idle": "2025-07-10T23:28:59.091981Z",
|
||||
"shell.execute_reply": "2025-07-10T23:28:59.091554Z"
|
||||
},
|
||||
"language": "python"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Testing hello_tinytorch():\n",
|
||||
". . ... ....... .... ... . . .. .... . .. . . . . . .... \n",
|
||||
". . .. .++. .. . . . .. ... . . . .. ... .. \n",
|
||||
" . . . .=++=.. . . . .. . . .. . ... .. . . \n",
|
||||
". .. ... .++++=. . . . . .. . .. .\n",
|
||||
". . . . ....-+++++.... ... .. . .... .. . . . . . . . . . . . . \n",
|
||||
" . .. ...-++++++-...... .. . ..... ..-:.. .. . .... .. . . .. . .. . . . \n",
|
||||
" .. .. ..++++++++-.. . . ..##... -%#. . . . . . \n",
|
||||
". .. .:+++++++++.... ... . ...:%%:............:-:. ..... ...... . . ....... .. . . \n",
|
||||
" ..+++++++++++. ... . .. .=#%%##+.-##..#%####%%=.=%%. .*%+.. . . . ... \n",
|
||||
" . ..++++++++++++...-++..... . .%%... -##..##=...=%#..*%*..=%#.. . .. ... . . . . .. . ...\n",
|
||||
" ..-+++++++++++++..=++++... .....%#.. -##..#%-.. -##. .%%=.%%.. . . . . . ... .\n",
|
||||
". .=++++++++++++++-+++++++.... . ...%%:...-##..#%-. .-%#. ..#%#%=.. . .. ... . . . .\n",
|
||||
"..=+++++++++++++++++++++++-. . ..=%%%+.-%#..##-. .-%#....-%%*.. . .. . .. .. .. \n",
|
||||
".:+++++++++++=+++++++++++++. . ................ .......-%%... . .. . . .. . \n",
|
||||
".++++++++++===+++++++++++++: . .................... . ...%%%#:........ . .. ..... ......... ....\n",
|
||||
":+++++++++====+++++++++++++=.. ...-----------.....-+#*=:.....-------:.......:=*#+-.. ..--:.....--=.\n",
|
||||
":++++++++======++++++++++++=.. ...#%%%%%%%%%#..-#%%###%%#=...#%####%%%=...+%%%###%%#...#%+.. ..#%%.\n",
|
||||
".+++++++========+++++++++++- .. .#%%.. ..-%%+.. ..-%%+..#%*.. .*%%..*%%:. ..#%*..#%+... .#%%.\n",
|
||||
".=++++++==========+++++++++: . .#%%.....#%#.... .*%#..#%*...-%%*..#%+. ... . ..##%#####%%%.\n",
|
||||
"..++++++===========+++++++-. . ...#%%. . .#%#. . .*%#..#%%%%%%#-. .#%+. . ....#%*-----#%%.\n",
|
||||
"...+++++===========++++++=. . . . .#%%... -%%+.....=%%+..#%*..+%%-. .*%%-.....#%*..%%+.. ..%%%.\n",
|
||||
". ..-+++===========+++++.. . .. ..#%%. .:%%%###%%%=...#%*...+%%=...+%%####%%#...%%+.. ..%%%.\n",
|
||||
" . ...-++==========+++:.... ... . .===. ... ..-+++=.. ..-=-....-==: ..:=+++-.. ..==-... .===.\n",
|
||||
" ....-+=======+-...... .. . . ... . . .. ... . . .... . . . . ..... . ... ..... .\n",
|
||||
" .... . ......:..... ... . .. . ... . . ... . . . ... . . . ... .. ..... . . \n",
|
||||
"\n",
|
||||
"Tiny🔥Torch\n",
|
||||
"Build ML Systems from Scratch!\n",
|
||||
"\n",
|
||||
"Testing add_numbers():\n",
|
||||
"2 + 3 = 5\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Test the functions in the notebook (will fail until implemented)\n",
|
||||
"try:\n",
|
||||
" print(\"Testing hello_tinytorch():\")\n",
|
||||
" hello_tinytorch()\n",
|
||||
" print()\n",
|
||||
" print(\"Testing add_numbers():\")\n",
|
||||
" print(f\"2 + 3 = {add_numbers(2, 3)}\")\n",
|
||||
"except NotImplementedError as e:\n",
|
||||
" print(f\"⚠️ {e}\")\n",
|
||||
" print(\"Implement the functions above first!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "887b9723",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\"",
|
||||
"lines_to_next_cell": 1
|
||||
},
|
||||
"source": [
|
||||
"## Step 2: A Simple Class\n",
|
||||
"\n",
|
||||
"Let's create a simple class that will help us understand system information. This is still basic, but shows how to structure classes in TinyTorch."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 0,
|
||||
"has_sd": true,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/markdown": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### SystemInfo\n",
|
||||
"\n",
|
||||
"> SystemInfo ()\n",
|
||||
"\n",
|
||||
"*Simple system information class.\n",
|
||||
"\n",
|
||||
"TODO: Implement this class to collect and display system information.*"
|
||||
],
|
||||
"text/plain": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### SystemInfo\n",
|
||||
"\n",
|
||||
"> SystemInfo ()\n",
|
||||
"\n",
|
||||
"*Simple system information class.\n",
|
||||
"\n",
|
||||
"TODO: Implement this class to collect and display system information.*"
|
||||
]
|
||||
},
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"#| echo: false\n",
|
||||
"#| output: asis\n",
|
||||
"show_doc(SystemInfo)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a182b8ad",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"### 🧪 Test Your SystemInfo Class\n",
|
||||
"\n",
|
||||
"Once you implement the SystemInfo class above, run this cell to test it:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "bfd7d3c4",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2025-07-10T23:28:59.101366Z",
|
||||
"iopub.status.busy": "2025-07-10T23:28:59.101221Z",
|
||||
"iopub.status.idle": "2025-07-10T23:28:59.103476Z",
|
||||
"shell.execute_reply": "2025-07-10T23:28:59.103228Z"
|
||||
},
|
||||
"language": "python"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Testing SystemInfo class:\n",
|
||||
"System: Python 3.13 on Darwin (arm64)\n",
|
||||
"Compatible: True\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Test the SystemInfo class (will fail until implemented)\n",
|
||||
"try:\n",
|
||||
" print(\"Testing SystemInfo class:\")\n",
|
||||
" info = SystemInfo()\n",
|
||||
" print(f\"System: {info}\")\n",
|
||||
" print(f\"Compatible: {info.is_compatible()}\")\n",
|
||||
"except NotImplementedError as e:\n",
|
||||
" print(f\"⚠️ {e}\")\n",
|
||||
" print(\"Implement the SystemInfo class above first!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9a14de41",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\"",
|
||||
"lines_to_next_cell": 1
|
||||
},
|
||||
"source": [
|
||||
"## Step 3: Developer Personalization\n",
|
||||
"\n",
|
||||
"Let's make TinyTorch yours! Create a developer profile that will identify you throughout your ML systems journey."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 0,
|
||||
"has_sd": true,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/markdown": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### DeveloperProfile\n",
|
||||
"\n",
|
||||
"> DeveloperProfile (name='Vijay Janapa Reddi', affiliation='Harvard\n",
|
||||
"> University', email='vj@eecs.harvard.edu',\n",
|
||||
"> github_username='profvjreddi', ascii_art=None)\n",
|
||||
"\n",
|
||||
"*Developer profile for personalizing TinyTorch experience.\n",
|
||||
"\n",
|
||||
"TODO: Implement this class to store and display developer information.\n",
|
||||
"Default to course instructor but allow students to personalize.*"
|
||||
],
|
||||
"text/plain": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"### DeveloperProfile\n",
|
||||
"\n",
|
||||
"> DeveloperProfile (name='Vijay Janapa Reddi', affiliation='Harvard\n",
|
||||
"> University', email='vj@eecs.harvard.edu',\n",
|
||||
"> github_username='profvjreddi', ascii_art=None)\n",
|
||||
"\n",
|
||||
"*Developer profile for personalizing TinyTorch experience.\n",
|
||||
"\n",
|
||||
"TODO: Implement this class to store and display developer information.\n",
|
||||
"Default to course instructor but allow students to personalize.*"
|
||||
]
|
||||
},
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"#| echo: false\n",
|
||||
"#| output: asis\n",
|
||||
"show_doc(DeveloperProfile)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b848981d",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"### 🧪 Test Your Developer Profile\n",
|
||||
"\n",
|
||||
"Customize your developer profile! Replace the default information with your own:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"id": "5d80e79c",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2025-07-10T23:28:59.114628Z",
|
||||
"iopub.status.busy": "2025-07-10T23:28:59.114540Z",
|
||||
"iopub.status.idle": "2025-07-10T23:28:59.118055Z",
|
||||
"shell.execute_reply": "2025-07-10T23:28:59.117792Z"
|
||||
},
|
||||
"language": "python"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Testing DeveloperProfile (with defaults):\n",
|
||||
"Profile: 👨💻 Vijay Janapa Reddi | Harvard University | @profvjreddi\n",
|
||||
"Signature: Built by Vijay Janapa Reddi (@profvjreddi)\n",
|
||||
"\n",
|
||||
"🎨 ASCII Art Preview:\n",
|
||||
". . ... ....... .... ... . . .. .... . .. . . . . . .... \n",
|
||||
". . .. .++. .. . . . .. ... . . . .. ... .. \n",
|
||||
" . . . .=++=.. . . . .. . . .. . ... .. . . \n",
|
||||
". .. ... .++++=. . . . . .. . .. .\n",
|
||||
". . . . ....-+++++.... ... .. . .... .. . . . . . . . . . . . . \n",
|
||||
" . .. ...-++++++-...... .. . ..... ..-:.. .. . .... .. . . .. . .. . . . \n",
|
||||
" .. .. ..++++++++-.. . . ..##... -%#. . . . . . \n",
|
||||
". .. .:+++++++++.... ... . ...:%%:............:-:. ..... ...... . . ....... .. . . \n",
|
||||
" ..+++++++++++. ... . .. .=#%%##+.-##..#%####%%=.=%%. .*%+.. . . . ... \n",
|
||||
" . ..++++++++++++...-++..... . .%%... -##..##=...=%#..*%*..=%#.. . .. ... . . . . .. . ...\n",
|
||||
" ..-+++++++++++++..=++++... .....%#.. -##..#%-.. -##. .%%=.%%.. . . . . . ... .\n",
|
||||
". .=++++++++++++++-+++++++.... . ...%%:...-##..#%-. .-%#. ..#%#%=.. . .. ... . . . .\n",
|
||||
"..=+++++++++++++++++++++++-. . ..=%%%+.-%#..##-. .-%#....-%%*.. . .. . .. .. .. \n",
|
||||
".:+++++++++++=+++++++++++++. . ................ .......-%%... . .. . . .. . \n",
|
||||
".++++++++++===+++++++++++++: . .................... . ...%%%#:........ . .. ..... ......... ....\n",
|
||||
":+++++++++====+++++++++++++=.. ...-----------.....-+#*=:.....-------:.......:=*#+-.. ..--:.....--=.\n",
|
||||
":++++++++======++++++++++++=.. ...#%%%%%%%%%#..-#%%###%%#=...#%####%%%=...+%%%###%%#...#%+.. ..#%%.\n",
|
||||
".+++++++========+++++++++++- .. .#%%.. ..-%%+.. ..-%%+..#%*.. .*%%..*%%:. ..#%*..#%+... .#%%.\n",
|
||||
".=++++++==========+++++++++: . .#%%.....#%#.... .*%#..#%*...-%%*..#%+. ... . ..##%#####%%%.\n",
|
||||
"..++++++===========+++++++-. . ...#%%. . .#%#. . .*%#..#%%%%%%#-. .#%+. . ....#%*-----#%%.\n",
|
||||
"...+++++===========++++++=. . . . .#%%... -%%+.....=%%+..#%*..+%%-. .*%%-.....#%*..%%+.. ..%%%.\n",
|
||||
". ..-+++===========+++++.. . .. ..#%%. .:%%%###%%%=...#%*...+%%=...+%%####%%#...%%+.. ..%%%.\n",
|
||||
" . ...-++==========+++:.... ... . .===. ... ..-+++=.. ..-=-....-==: ..:=+++-.. ..==-... .===.\n",
|
||||
" ....-+=======+-...... .. . . ... . . .. ... . . .... . . . . ..... . ... ..... .\n",
|
||||
" .... . ......:..... ... . .. . ... . . ... . . . ... . . . ... .. ..... . . \n",
|
||||
"\n",
|
||||
"\n",
|
||||
" Tiny🔥Torch\n",
|
||||
" Build ML Systems from Scratch!\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"🔥 Full Profile Display:\n",
|
||||
". . ... ....... .... ... . . .. .... . .. . . . . . .... \n",
|
||||
". . .. .++. .. . . . .. ... . . . .. ... .. \n",
|
||||
" . . . .=++=.. . . . .. . . .. . ... .. . . \n",
|
||||
". .. ... .++++=. . . . . .. . .. .\n",
|
||||
". . . . ....-+++++.... ... .. . .... .. . . . . . . . . . . . . \n",
|
||||
" . .. ...-++++++-...... .. . ..... ..-:.. .. . .... .. . . .. . .. . . . \n",
|
||||
" .. .. ..++++++++-.. . . ..##... -%#. . . . . . \n",
|
||||
". .. .:+++++++++.... ... . ...:%%:............:-:. ..... ...... . . ....... .. . . \n",
|
||||
" ..+++++++++++. ... . .. .=#%%##+.-##..#%####%%=.=%%. .*%+.. . . . ... \n",
|
||||
" . ..++++++++++++...-++..... . .%%... -##..##=...=%#..*%*..=%#.. . .. ... . . . . .. . ...\n",
|
||||
" ..-+++++++++++++..=++++... .....%#.. -##..#%-.. -##. .%%=.%%.. . . . . . ... .\n",
|
||||
". .=++++++++++++++-+++++++.... . ...%%:...-##..#%-. .-%#. ..#%#%=.. . .. ... . . . .\n",
|
||||
"..=+++++++++++++++++++++++-. . ..=%%%+.-%#..##-. .-%#....-%%*.. . .. . .. .. .. \n",
|
||||
".:+++++++++++=+++++++++++++. . ................ .......-%%... . .. . . .. . \n",
|
||||
".++++++++++===+++++++++++++: . .................... . ...%%%#:........ . .. ..... ......... ....\n",
|
||||
":+++++++++====+++++++++++++=.. ...-----------.....-+#*=:.....-------:.......:=*#+-.. ..--:.....--=.\n",
|
||||
":++++++++======++++++++++++=.. ...#%%%%%%%%%#..-#%%###%%#=...#%####%%%=...+%%%###%%#...#%+.. ..#%%.\n",
|
||||
".+++++++========+++++++++++- .. .#%%.. ..-%%+.. ..-%%+..#%*.. .*%%..*%%:. ..#%*..#%+... .#%%.\n",
|
||||
".=++++++==========+++++++++: . .#%%.....#%#.... .*%#..#%*...-%%*..#%+. ... . ..##%#####%%%.\n",
|
||||
"..++++++===========+++++++-. . ...#%%. . .#%#. . .*%#..#%%%%%%#-. .#%+. . ....#%*-----#%%.\n",
|
||||
"...+++++===========++++++=. . . . .#%%... -%%+.....=%%+..#%*..+%%-. .*%%-.....#%*..%%+.. ..%%%.\n",
|
||||
". ..-+++===========+++++.. . .. ..#%%. .:%%%###%%%=...#%*...+%%=...+%%####%%#...%%+.. ..%%%.\n",
|
||||
" . ...-++==========+++:.... ... . .===. ... ..-+++=.. ..-=-....-==: ..:=+++-.. ..==-... .===.\n",
|
||||
" ....-+=======+-...... .. . . ... . . .. ... . . .... . . . . ..... . ... ..... .\n",
|
||||
" .... . ......:..... ... . .. . ... . . ... . . . ... . . . ... .. ..... . . \n",
|
||||
"\n",
|
||||
"\n",
|
||||
" Tiny🔥Torch\n",
|
||||
" Build ML Systems from Scratch!\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"👨💻 Developer: Vijay Janapa Reddi\n",
|
||||
"🏛️ Affiliation: Harvard University\n",
|
||||
"📧 Email: vj@eecs.harvard.edu\n",
|
||||
"🐙 GitHub: @profvjreddi\n",
|
||||
"🔥 Ready to build ML systems from scratch!\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"🎯 YOUR TURN: Create your own profile!\n",
|
||||
"Uncomment and modify the lines below:\n",
|
||||
"# my_profile = DeveloperProfile(\n",
|
||||
"# name='Your Name',\n",
|
||||
"# affiliation='Your University/Company',\n",
|
||||
"# email='your.email@example.com',\n",
|
||||
"# github_username='yourgithub',\n",
|
||||
"# ascii_art='''\n",
|
||||
"# Your Custom ASCII Art Here!\n",
|
||||
"# Maybe your initials, a logo, or something fun!\n",
|
||||
"# '''\n",
|
||||
"# )\n",
|
||||
"# print(f'My Profile: {my_profile}')\n",
|
||||
"# print(f'My Signature: {my_profile.get_signature()}')\n",
|
||||
"# print(my_profile.get_full_profile())\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Test the DeveloperProfile class\n",
|
||||
"try:\n",
|
||||
" print(\"Testing DeveloperProfile (with defaults):\")\n",
|
||||
" # Default profile (instructor)\n",
|
||||
" default_profile = DeveloperProfile()\n",
|
||||
" print(f\"Profile: {default_profile}\")\n",
|
||||
" print(f\"Signature: {default_profile.get_signature()}\")\n",
|
||||
" print()\n",
|
||||
" \n",
|
||||
" print(\"🎨 ASCII Art Preview:\")\n",
|
||||
" print(default_profile.get_ascii_art())\n",
|
||||
" print()\n",
|
||||
" \n",
|
||||
" print(\"🔥 Full Profile Display:\")\n",
|
||||
" print(default_profile.get_full_profile())\n",
|
||||
" print()\n",
|
||||
" \n",
|
||||
" # TODO: Students should customize this with their own information!\n",
|
||||
" print(\"🎯 YOUR TURN: Create your own profile!\")\n",
|
||||
" print(\"Uncomment and modify the lines below:\")\n",
|
||||
" print(\"# my_profile = DeveloperProfile(\")\n",
|
||||
" print(\"# name='Your Name',\")\n",
|
||||
" print(\"# affiliation='Your University/Company',\")\n",
|
||||
" print(\"# email='your.email@example.com',\")\n",
|
||||
" print(\"# github_username='yourgithub',\")\n",
|
||||
" print(\"# ascii_art='''\")\n",
|
||||
" print(\"# Your Custom ASCII Art Here!\")\n",
|
||||
" print(\"# Maybe your initials, a logo, or something fun!\")\n",
|
||||
" print(\"# '''\")\n",
|
||||
" print(\"# )\")\n",
|
||||
" print(\"# print(f'My Profile: {my_profile}')\")\n",
|
||||
" print(\"# print(f'My Signature: {my_profile.get_signature()}')\")\n",
|
||||
" print(\"# print(my_profile.get_full_profile())\")\n",
|
||||
" \n",
|
||||
"except NotImplementedError as e:\n",
|
||||
" print(f\"⚠️ {e}\")\n",
|
||||
" print(\"Implement the DeveloperProfile class above first!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4f117574",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"### 🎨 Personalization Challenge\n",
|
||||
"\n",
|
||||
"**For Students**: Make TinyTorch truly yours by:\n",
|
||||
"\n",
|
||||
"1. **Update your profile** in the cell above with your real information\n",
|
||||
"2. **Create custom ASCII art** - your initials, a simple logo, or something that represents you\n",
|
||||
"3. **Customize the flame file** - edit `tinytorch_flame.txt` to create your own default art\n",
|
||||
"4. **Add your signature** to code you write throughout the course\n",
|
||||
"5. **Show off your full profile** with the `get_full_profile()` method\n",
|
||||
"\n",
|
||||
"This isn't just about customization - it's about taking ownership of your learning journey in ML systems!\n",
|
||||
"\n",
|
||||
"**ASCII Art Customization Options:**\n",
|
||||
"\n",
|
||||
"**Option 1: Custom ASCII Art Parameter**\n",
|
||||
"```python\n",
|
||||
"my_profile = DeveloperProfile(\n",
|
||||
" name=\"Your Name\",\n",
|
||||
" ascii_art='''\n",
|
||||
" Your Custom ASCII Art Here!\n",
|
||||
" Maybe your initials, a logo, or something fun!\n",
|
||||
" '''\n",
|
||||
")\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"**Option 2: Edit the Default Flame File**\n",
|
||||
"- Edit `tinytorch_flame.txt` in this directory\n",
|
||||
"- Replace with your own ASCII art design\n",
|
||||
"- All students using defaults will see your custom art!\n",
|
||||
"\n",
|
||||
"**ASCII Art Ideas:**\n",
|
||||
"- Your initials in block letters\n",
|
||||
"- A simple logo or symbol that represents you\n",
|
||||
"- Your university mascot in ASCII\n",
|
||||
"- A coding-themed design\n",
|
||||
"- Something that motivates you!\n",
|
||||
"\n",
|
||||
"**Pro Tip**: The `tinytorch_flame.txt` file contains the beautiful default flame art. You can:\n",
|
||||
"- Edit it directly for a personalized default\n",
|
||||
"- Create your own `.txt` file and modify the code to load it\n",
|
||||
"- Use online ASCII art generators for inspiration"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3e34c7fe",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"## Step 4: Try the Export Process\n",
|
||||
"\n",
|
||||
"Now let's export our code! In your terminal, run:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"python bin/tito.py sync --module setup\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"This will export the code marked with `#| export` to `tinytorch/core/utils.py`.\n",
|
||||
"\n",
|
||||
"**What happens during export:**\n",
|
||||
"1. nbdev scans this notebook for `#| export` cells\n",
|
||||
"2. Extracts the Python code \n",
|
||||
"3. Writes it to `tinytorch/core/utils.py` (because of `#| default_exp core.utils`)\n",
|
||||
"4. Handles imports and dependencies automatically\n",
|
||||
"\n",
|
||||
"**🔍 Verification**: After export, check `tinytorch/core/utils.py` - you'll see your functions there with auto-generated headers pointing back to this notebook!\n",
|
||||
"\n",
|
||||
"**Note**: The export process will use the instructor solutions (from `#|hide` cells) so the package will have working implementations even if you haven't completed the exercises yet."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "641ad5d7",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"## Step 5: Run Tests\n",
|
||||
"\n",
|
||||
"After exporting, run the tests:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"python bin/tito.py test --module setup\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"This will run all tests for the setup module and verify your implementation works correctly.\n",
|
||||
"\n",
|
||||
"## Step 6: Check Your Progress\n",
|
||||
"\n",
|
||||
"See your overall progress:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"python bin/tito.py info\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"This shows which modules are complete and which are pending."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7a09b00d",
|
||||
"metadata": {
|
||||
"cell_marker": "\"\"\""
|
||||
},
|
||||
"source": [
|
||||
"## 🎉 Congratulations!\n",
|
||||
"\n",
|
||||
"You've learned the TinyTorch development workflow:\n",
|
||||
"\n",
|
||||
"1. ✅ Write code in notebooks with `#| export`\n",
|
||||
"2. ✅ Export with `tito sync --module setup` \n",
|
||||
"3. ✅ Test with `tito test --module setup`\n",
|
||||
"4. ✅ Check progress with `tito info`\n",
|
||||
"\n",
|
||||
"**This is the rhythm you'll use for every module in TinyTorch.**\n",
|
||||
"\n",
|
||||
"### Next Steps\n",
|
||||
"\n",
|
||||
"Ready for the real work? Head to **Module 1: Tensor** where you'll build the core data structures that power everything else in TinyTorch.\n",
|
||||
"\n",
|
||||
"**Development Tips:**\n",
|
||||
"- Always test your code in the notebook first\n",
|
||||
"- Export frequently to catch issues early \n",
|
||||
"- Read error messages carefully - they're designed to help\n",
|
||||
"- When stuck, check if your code exports cleanly first\n",
|
||||
"\n",
|
||||
"Happy building! 🔥"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"jupytext": {
|
||||
"main_language": "python"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.13.3"
|
||||
},
|
||||
"widgets": {
|
||||
"application/vnd.jupyter.widget-state+json": {
|
||||
"state": {},
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -1,251 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests for the setup module using pytest.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
from io import StringIO
|
||||
|
||||
# Add the parent directory to the path so we can import setup_dev
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from setup_dev import hello_tinytorch, add_numbers, SystemInfo, DeveloperProfile
|
||||
|
||||
|
||||
class TestSetupFunctions:
|
||||
"""Test setup module functions."""
|
||||
|
||||
def test_hello_tinytorch_executes(self):
|
||||
"""Test that hello_tinytorch runs without error."""
|
||||
# Should not raise any exceptions
|
||||
hello_tinytorch()
|
||||
|
||||
def test_hello_tinytorch_prints_content(self, capsys):
|
||||
"""Test that hello_tinytorch prints the expected content."""
|
||||
hello_tinytorch()
|
||||
captured = capsys.readouterr()
|
||||
|
||||
# Should print the branding text
|
||||
assert "Tiny🔥Torch" in captured.out
|
||||
assert "Build ML Systems from Scratch!" in captured.out
|
||||
|
||||
def test_add_numbers_basic(self):
|
||||
"""Test basic addition functionality."""
|
||||
assert add_numbers(2, 3) == 5
|
||||
assert add_numbers(10, 15) == 25
|
||||
assert add_numbers(0, 0) == 0
|
||||
|
||||
def test_add_numbers_negative(self):
|
||||
"""Test addition with negative numbers."""
|
||||
assert add_numbers(-5, 3) == -2
|
||||
assert add_numbers(-10, -15) == -25
|
||||
assert add_numbers(10, -5) == 5
|
||||
|
||||
def test_add_numbers_floats(self):
|
||||
"""Test addition with floating point numbers."""
|
||||
assert abs(add_numbers(2.5, 3.7) - 6.2) < 1e-9
|
||||
assert abs(add_numbers(1.1, 2.2) - 3.3) < 1e-9
|
||||
|
||||
|
||||
class TestSystemInfo:
|
||||
"""Test SystemInfo class."""
|
||||
|
||||
def test_system_info_creation(self):
|
||||
"""Test SystemInfo class instantiation."""
|
||||
info = SystemInfo()
|
||||
assert hasattr(info, 'python_version')
|
||||
assert hasattr(info, 'platform')
|
||||
assert hasattr(info, 'machine')
|
||||
|
||||
def test_system_info_properties(self):
|
||||
"""Test SystemInfo properties."""
|
||||
info = SystemInfo()
|
||||
|
||||
# Check python_version is a version tuple
|
||||
assert hasattr(info.python_version, 'major')
|
||||
assert hasattr(info.python_version, 'minor')
|
||||
assert isinstance(info.python_version.major, int)
|
||||
assert isinstance(info.python_version.minor, int)
|
||||
|
||||
# Check platform is a string
|
||||
assert isinstance(info.platform, str)
|
||||
assert len(info.platform) > 0
|
||||
|
||||
# Check machine is a string
|
||||
assert isinstance(info.machine, str)
|
||||
assert len(info.machine) > 0
|
||||
|
||||
def test_system_info_str(self):
|
||||
"""Test SystemInfo string representation."""
|
||||
info = SystemInfo()
|
||||
str_repr = str(info)
|
||||
|
||||
assert isinstance(str_repr, str)
|
||||
assert "Python" in str_repr
|
||||
assert str(info.python_version.major) in str_repr
|
||||
assert str(info.python_version.minor) in str_repr
|
||||
assert info.platform in str_repr
|
||||
assert info.machine in str_repr
|
||||
|
||||
def test_is_compatible(self):
|
||||
"""Test SystemInfo compatibility check."""
|
||||
info = SystemInfo()
|
||||
compatible = info.is_compatible()
|
||||
|
||||
# Should return a boolean
|
||||
assert isinstance(compatible, bool)
|
||||
|
||||
# Since we're running this test, Python should be >= 3.8
|
||||
assert compatible is True
|
||||
|
||||
|
||||
class TestDeveloperProfile:
|
||||
"""Test DeveloperProfile class."""
|
||||
|
||||
def test_developer_profile_creation_defaults(self):
|
||||
"""Test DeveloperProfile with default values."""
|
||||
profile = DeveloperProfile()
|
||||
|
||||
# Check default values
|
||||
assert profile.name == "Vijay Janapa Reddi"
|
||||
assert profile.affiliation == "Harvard University"
|
||||
assert profile.email == "vj@eecs.harvard.edu"
|
||||
assert profile.github_username == "profvjreddi"
|
||||
assert profile.ascii_art is not None # Should have default flame
|
||||
|
||||
def test_developer_profile_creation_custom(self):
|
||||
"""Test DeveloperProfile with custom values."""
|
||||
custom_art = """
|
||||
Custom ASCII Art
|
||||
****************
|
||||
"""
|
||||
profile = DeveloperProfile(
|
||||
name="Test Student",
|
||||
affiliation="Test University",
|
||||
email="test@example.com",
|
||||
github_username="teststudent",
|
||||
ascii_art=custom_art
|
||||
)
|
||||
|
||||
assert profile.name == "Test Student"
|
||||
assert profile.affiliation == "Test University"
|
||||
assert profile.email == "test@example.com"
|
||||
assert profile.github_username == "teststudent"
|
||||
assert profile.ascii_art == custom_art
|
||||
|
||||
def test_developer_profile_str(self):
|
||||
"""Test DeveloperProfile string representation."""
|
||||
profile = DeveloperProfile()
|
||||
str_repr = str(profile)
|
||||
|
||||
assert isinstance(str_repr, str)
|
||||
assert "👨💻" in str_repr
|
||||
assert "Vijay Janapa Reddi" in str_repr
|
||||
assert "Harvard University" in str_repr
|
||||
assert "@profvjreddi" in str_repr
|
||||
|
||||
def test_developer_profile_signature(self):
|
||||
"""Test DeveloperProfile signature method."""
|
||||
profile = DeveloperProfile()
|
||||
signature = profile.get_signature()
|
||||
|
||||
assert isinstance(signature, str)
|
||||
assert "Built by" in signature
|
||||
assert "Vijay Janapa Reddi" in signature
|
||||
assert "@profvjreddi" in signature
|
||||
|
||||
def test_developer_profile_ascii_art(self):
|
||||
"""Test DeveloperProfile ASCII art functionality."""
|
||||
# Test default ASCII art
|
||||
profile = DeveloperProfile()
|
||||
ascii_art = profile.get_ascii_art()
|
||||
|
||||
assert isinstance(ascii_art, str)
|
||||
assert "Tiny🔥Torch" in ascii_art
|
||||
assert "Build ML Systems from Scratch!" in ascii_art
|
||||
assert len(ascii_art) > 100 # Should be substantial ASCII art
|
||||
|
||||
# Test custom ASCII art
|
||||
custom_art = "My Custom Art!"
|
||||
custom_profile = DeveloperProfile(ascii_art=custom_art)
|
||||
assert custom_profile.get_ascii_art() == custom_art
|
||||
|
||||
def test_developer_profile_full_profile(self):
|
||||
"""Test DeveloperProfile full profile display."""
|
||||
profile = DeveloperProfile()
|
||||
full_profile = profile.get_full_profile()
|
||||
|
||||
assert isinstance(full_profile, str)
|
||||
assert "Tiny🔥Torch" in full_profile
|
||||
assert "Build ML Systems from Scratch!" in full_profile
|
||||
assert "👨💻 Developer: Vijay Janapa Reddi" in full_profile
|
||||
assert "🏛️ Affiliation: Harvard University" in full_profile
|
||||
assert "📧 Email: vj@eecs.harvard.edu" in full_profile
|
||||
assert "🐙 GitHub: @profvjreddi" in full_profile
|
||||
assert "🔥 Ready to build ML systems from scratch!" in full_profile
|
||||
|
||||
|
||||
class TestFileOperations:
|
||||
"""Test file-related operations."""
|
||||
|
||||
def test_ascii_art_file_exists(self):
|
||||
"""Test that the ASCII art file exists."""
|
||||
art_file = Path(__file__).parent.parent / "tinytorch_flame.txt"
|
||||
assert art_file.exists(), "ASCII art file should exist"
|
||||
assert art_file.is_file(), "ASCII art should be a file"
|
||||
|
||||
def test_ascii_art_file_has_content(self):
|
||||
"""Test that the ASCII art file has content."""
|
||||
art_file = Path(__file__).parent.parent / "tinytorch_flame.txt"
|
||||
content = art_file.read_text()
|
||||
|
||||
assert len(content) > 0, "ASCII art file should not be empty"
|
||||
assert len(content.splitlines()) > 10, "ASCII art should have multiple lines"
|
||||
|
||||
def test_hello_tinytorch_handles_missing_file(self, monkeypatch, capsys):
|
||||
"""Test that hello_tinytorch handles missing ASCII art file gracefully."""
|
||||
# Mock Path.exists to return False
|
||||
def mock_exists(self):
|
||||
return False
|
||||
|
||||
monkeypatch.setattr(Path, "exists", mock_exists)
|
||||
|
||||
# Should still work without the file
|
||||
hello_tinytorch()
|
||||
captured = capsys.readouterr()
|
||||
|
||||
# Should still print the branding text
|
||||
assert "🔥 TinyTorch 🔥" in captured.out
|
||||
assert "Build ML Systems from Scratch!" in captured.out
|
||||
|
||||
|
||||
class TestModuleIntegration:
|
||||
"""Test integration between different parts of the setup module."""
|
||||
|
||||
def test_all_functions_work_together(self):
|
||||
"""Test that all setup functions work without conflicts."""
|
||||
# Test functions
|
||||
hello_tinytorch() # Should not raise
|
||||
sum_result = add_numbers(5, 10)
|
||||
|
||||
# Test classes
|
||||
info = SystemInfo()
|
||||
profile = DeveloperProfile()
|
||||
|
||||
# All should work without errors
|
||||
assert sum_result == 15
|
||||
assert str(info) # Should not be empty
|
||||
assert str(profile) # Should not be empty
|
||||
assert profile.get_signature() # Should not be empty
|
||||
assert profile.get_ascii_art() # Should not be empty
|
||||
|
||||
def test_no_import_errors(self):
|
||||
"""Test that imports work correctly."""
|
||||
# If we got here, imports worked
|
||||
assert callable(hello_tinytorch)
|
||||
assert callable(add_numbers)
|
||||
assert callable(SystemInfo)
|
||||
assert callable(DeveloperProfile)
|
||||
@@ -1,25 +0,0 @@
|
||||
. . ... ....... .... ... . . .. .... . .. . . . . . ....
|
||||
. . .. .++. .. . . . .. ... . . . .. ... ..
|
||||
. . . .=++=.. . . . .. . . .. . ... .. . .
|
||||
. .. ... .++++=. . . . . .. . .. .
|
||||
. . . . ....-+++++.... ... .. . .... .. . . . . . . . . . . . .
|
||||
. .. ...-++++++-...... .. . ..... ..-:.. .. . .... .. . . .. . .. . . .
|
||||
.. .. ..++++++++-.. . . ..##... -%#. . . . . .
|
||||
. .. .:+++++++++.... ... . ...:%%:............:-:. ..... ...... . . ....... .. . .
|
||||
..+++++++++++. ... . .. .=#%%##+.-##..#%####%%=.=%%. .*%+.. . . . ...
|
||||
. ..++++++++++++...-++..... . .%%... -##..##=...=%#..*%*..=%#.. . .. ... . . . . .. . ...
|
||||
..-+++++++++++++..=++++... .....%#.. -##..#%-.. -##. .%%=.%%.. . . . . . ... .
|
||||
. .=++++++++++++++-+++++++.... . ...%%:...-##..#%-. .-%#. ..#%#%=.. . .. ... . . . .
|
||||
..=+++++++++++++++++++++++-. . ..=%%%+.-%#..##-. .-%#....-%%*.. . .. . .. .. ..
|
||||
.:+++++++++++=+++++++++++++. . ................ .......-%%... . .. . . .. .
|
||||
.++++++++++===+++++++++++++: . .................... . ...%%%#:........ . .. ..... ......... ....
|
||||
:+++++++++====+++++++++++++=.. ...-----------.....-+#*=:.....-------:.......:=*#+-.. ..--:.....--=.
|
||||
:++++++++======++++++++++++=.. ...#%%%%%%%%%#..-#%%###%%#=...#%####%%%=...+%%%###%%#...#%+.. ..#%%.
|
||||
.+++++++========+++++++++++- .. .#%%.. ..-%%+.. ..-%%+..#%*.. .*%%..*%%:. ..#%*..#%+... .#%%.
|
||||
.=++++++==========+++++++++: . .#%%.....#%#.... .*%#..#%*...-%%*..#%+. ... . ..##%#####%%%.
|
||||
..++++++===========+++++++-. . ...#%%. . .#%#. . .*%#..#%%%%%%#-. .#%+. . ....#%*-----#%%.
|
||||
...+++++===========++++++=. . . . .#%%... -%%+.....=%%+..#%*..+%%-. .*%%-.....#%*..%%+.. ..%%%.
|
||||
. ..-+++===========+++++.. . .. ..#%%. .:%%%###%%%=...#%*...+%%=...+%%####%%#...%%+.. ..%%%.
|
||||
. ...-++==========+++:.... ... . .===. ... ..-+++=.. ..-=-....-==: ..:=+++-.. ..==-... .===.
|
||||
....-+=======+-...... .. . . ... . . .. ... . . .... . . . . ..... . ... ..... .
|
||||
.... . ......:..... ... . .. . ... . . ... . . . ... . . . ... .. ..... . .
|
||||
@@ -1,9 +0,0 @@
|
||||
website:
|
||||
sidebar:
|
||||
contents:
|
||||
- section: layers
|
||||
contents:
|
||||
- layers/layers_dev.ipynb
|
||||
- section: setup
|
||||
contents:
|
||||
- setup/setup_dev.ipynb
|
||||
@@ -1,202 +0,0 @@
|
||||
# 🔥 Module: Tensor
|
||||
|
||||
Build the foundation of TinyTorch! This module implements the core Tensor class - the fundamental data structure that powers all neural networks and machine learning operations.
|
||||
|
||||
## 🎯 Learning Objectives
|
||||
|
||||
By the end of this module, you will:
|
||||
- ✅ Understand what tensors are and why they're essential for ML
|
||||
- ✅ Implement a complete Tensor class with core operations
|
||||
- ✅ Handle tensor shapes, data types, and memory management
|
||||
- ✅ Implement element-wise operations and reductions
|
||||
- ✅ Have a solid foundation for building neural networks
|
||||
|
||||
## 📋 Module Structure
|
||||
|
||||
```
|
||||
modules/tensor/
|
||||
├── README.md # 📖 This file - Module overview
|
||||
├── tensor_dev.ipynb # 📓 Main development notebook
|
||||
├── test_tensor.py # 🧪 Automated tests
|
||||
└── check_tensor.py # ✅ Manual verification (coming soon)
|
||||
```
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Step 1: Complete Prerequisites
|
||||
Make sure you've completed the setup module:
|
||||
```bash
|
||||
python bin/tito.py test --module setup # Should pass
|
||||
```
|
||||
|
||||
### Step 2: Open the Tensor Notebook
|
||||
```bash
|
||||
# Start from the tensor module directory
|
||||
cd modules/tensor/
|
||||
|
||||
# Open the development notebook
|
||||
jupyter lab tensor_dev.ipynb
|
||||
```
|
||||
|
||||
### Step 3: Work Through the Implementation
|
||||
The notebook guides you through building:
|
||||
1. **Basic Tensor class** - Constructor and properties
|
||||
2. **Shape management** - Understanding tensor dimensions
|
||||
3. **Arithmetic operations** - Addition, multiplication, etc.
|
||||
4. **Utility methods** - Reshape, transpose, sum, mean
|
||||
5. **Error handling** - Robust edge case management
|
||||
|
||||
### Step 4: Export and Test
|
||||
```bash
|
||||
# Export your tensor implementation
|
||||
python bin/tito.py sync
|
||||
|
||||
# Test your implementation
|
||||
python bin/tito.py test --module tensor
|
||||
```
|
||||
|
||||
## 📚 What You'll Implement
|
||||
|
||||
### Core Tensor Class
|
||||
You'll build a complete `Tensor` class that supports:
|
||||
|
||||
#### 1. Construction and Properties
|
||||
```python
|
||||
# Creating tensors
|
||||
a = Tensor([1, 2, 3]) # 1D tensor
|
||||
b = Tensor([[1, 2], [3, 4]]) # 2D tensor
|
||||
c = Tensor(5.0) # Scalar tensor
|
||||
|
||||
# Properties
|
||||
print(a.shape) # (3,)
|
||||
print(b.size) # 4
|
||||
print(c.dtype) # float32
|
||||
```
|
||||
|
||||
#### 2. Arithmetic Operations
|
||||
```python
|
||||
# Element-wise operations
|
||||
result = a + b # Addition
|
||||
result = a * 2 # Scalar multiplication
|
||||
result = a @ b # Matrix multiplication (bonus)
|
||||
```
|
||||
|
||||
#### 3. Utility Methods
|
||||
```python
|
||||
# Shape manipulation
|
||||
reshaped = b.reshape(1, 4) # Change shape
|
||||
transposed = b.transpose() # Swap dimensions
|
||||
|
||||
# Reductions
|
||||
total = a.sum() # Sum all elements
|
||||
mean_val = a.mean() # Average value
|
||||
max_val = a.max() # Maximum value
|
||||
```
|
||||
|
||||
### Technical Requirements
|
||||
Your Tensor class must:
|
||||
- Handle multiple data types (int, float)
|
||||
- Support N-dimensional arrays
|
||||
- Implement proper error checking
|
||||
- Work with NumPy arrays internally
|
||||
- Export to `tinytorch.core.tensor`
|
||||
|
||||
## 🧪 Testing Your Implementation
|
||||
|
||||
### Automated Tests
|
||||
```bash
|
||||
python bin/tito.py test --module tensor
|
||||
```
|
||||
|
||||
Tests verify:
|
||||
- ✅ Tensor creation (scalars, vectors, matrices)
|
||||
- ✅ Property access (shape, size, dtype)
|
||||
- ✅ Arithmetic operations (all combinations)
|
||||
- ✅ Utility methods (reshape, transpose, reductions)
|
||||
- ✅ Error handling (invalid operations)
|
||||
|
||||
### Interactive Testing
|
||||
```python
|
||||
# Test in the notebook or Python REPL
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
# Create and test tensors
|
||||
a = Tensor([1, 2, 3])
|
||||
b = Tensor([[1, 2], [3, 4]])
|
||||
print(a + 5) # Should work
|
||||
print(a.sum()) # Should return scalar
|
||||
```
|
||||
|
||||
## 🎯 Success Criteria
|
||||
|
||||
Your tensor module is complete when:
|
||||
|
||||
1. **All tests pass**: `python bin/tito.py test --module tensor`
|
||||
2. **Tensor imports correctly**: `from tinytorch.core.tensor import Tensor`
|
||||
3. **Basic operations work**: Can create tensors and do arithmetic
|
||||
4. **Properties work**: Shape, size, dtype return correct values
|
||||
5. **Utilities work**: Reshape, transpose, reductions function properly
|
||||
|
||||
## 💡 Implementation Tips
|
||||
|
||||
### Start with the Basics
|
||||
1. **Simple constructor** - Handle lists and NumPy arrays
|
||||
2. **Basic properties** - Shape, size, dtype
|
||||
3. **One operation** - Start with addition
|
||||
4. **Test frequently** - Verify each feature works
|
||||
|
||||
### Design Patterns
|
||||
```python
|
||||
class Tensor:
|
||||
def __init__(self, data, dtype=None):
|
||||
# Convert input to numpy array
|
||||
# Store shape, size, dtype
|
||||
|
||||
def __add__(self, other):
|
||||
# Handle tensor + tensor
|
||||
# Handle tensor + scalar
|
||||
# Return new Tensor
|
||||
|
||||
def sum(self, axis=None):
|
||||
# Reduce along specified axis
|
||||
# Return scalar or tensor
|
||||
```
|
||||
|
||||
### Common Challenges
|
||||
- **Shape compatibility** - Check dimensions for operations
|
||||
- **Data type handling** - Convert inputs consistently
|
||||
- **Memory efficiency** - Don't create unnecessary copies
|
||||
- **Error messages** - Provide helpful debugging info
|
||||
|
||||
## 🔧 Advanced Features (Optional)
|
||||
|
||||
If you finish early, try implementing:
|
||||
- **Broadcasting** - Operations on different-shaped tensors
|
||||
- **Slicing** - `tensor[1:3, :]` syntax
|
||||
- **In-place operations** - `tensor += other`
|
||||
- **Matrix multiplication** - `tensor @ other`
|
||||
|
||||
## 🚀 Next Steps
|
||||
|
||||
Once you complete the tensor module:
|
||||
|
||||
1. **Move to Autograd**: `cd modules/autograd/`
|
||||
2. **Build automatic differentiation**: Enable gradient computation
|
||||
3. **Combine with tensors**: Make tensors differentiable
|
||||
4. **Prepare for neural networks**: Ready for the MLP module
|
||||
|
||||
## 🔗 Why Tensors Matter
|
||||
|
||||
Tensors are the foundation of all ML systems:
|
||||
- **Neural networks** store weights and activations as tensors
|
||||
- **Training** computes gradients on tensors
|
||||
- **Data processing** represents batches as tensors
|
||||
- **GPU acceleration** operates on tensor primitives
|
||||
|
||||
Your tensor implementation will power everything else in TinyTorch!
|
||||
|
||||
## 🎉 Ready to Build?
|
||||
|
||||
The tensor module is where TinyTorch really begins. You're about to create the fundamental building block that will power neural networks, training loops, and production ML systems.
|
||||
|
||||
Take your time, test thoroughly, and enjoy building something that really works! 🔥
|
||||
@@ -1,390 +0,0 @@
|
||||
# ---
|
||||
# jupyter:
|
||||
# jupytext:
|
||||
# text_representation:
|
||||
# extension: .py
|
||||
# format_name: percent
|
||||
# format_version: '1.3'
|
||||
# jupytext_version: 1.17.1
|
||||
# ---
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
# Module 1: Tensor - Core Data Structure
|
||||
|
||||
Welcome to the Tensor module! This is where TinyTorch really begins. You'll implement the fundamental data structure that powers all ML systems.
|
||||
|
||||
## Learning Goals
|
||||
- Understand tensors as N-dimensional arrays with ML-specific operations
|
||||
- Implement a complete Tensor class with arithmetic operations
|
||||
- Handle shape management, data types, and memory layout
|
||||
- Build the foundation for neural networks and automatic differentiation
|
||||
|
||||
## Module → Package Structure
|
||||
**🎓 Teaching vs. 🔧 Building**:
|
||||
- **Learning side**: Work in `modules/tensor/tensor_dev.py`
|
||||
- **Building side**: Exports to `tinytorch/core/tensor.py`
|
||||
|
||||
This module builds the core data structure that all other TinyTorch components will use.
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| default_exp core.tensor
|
||||
|
||||
# Setup and imports
|
||||
import numpy as np
|
||||
import sys
|
||||
from typing import Union, List, Tuple, Optional, Any
|
||||
|
||||
print("🔥 TinyTorch Tensor Module")
|
||||
print(f"NumPy version: {np.__version__}")
|
||||
print(f"Python version: {sys.version_info.major}.{sys.version_info.minor}")
|
||||
print("Ready to build tensors!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1: What is a Tensor?
|
||||
|
||||
A **tensor** is an N-dimensional array with ML-specific operations. Think of it as:
|
||||
- **Scalar** (0D): A single number - `5.0`
|
||||
- **Vector** (1D): A list of numbers - `[1, 2, 3]`
|
||||
- **Matrix** (2D): A 2D array - `[[1, 2], [3, 4]]`
|
||||
- **Higher dimensions**: 3D, 4D, etc. for images, video, batches
|
||||
|
||||
**Why not just use NumPy?** We will use NumPy internally, but our Tensor class will add:
|
||||
- ML-specific operations (later: gradients, GPU support)
|
||||
- Consistent API for neural networks
|
||||
- Type safety and error checking
|
||||
- Integration with the rest of TinyTorch
|
||||
|
||||
Let's start building!
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
class Tensor:
|
||||
"""
|
||||
TinyTorch Tensor: N-dimensional array with ML operations.
|
||||
|
||||
The fundamental data structure for all TinyTorch operations.
|
||||
Wraps NumPy arrays with ML-specific functionality.
|
||||
|
||||
TODO: Implement the core Tensor class with data handling and properties.
|
||||
"""
|
||||
|
||||
def __init__(self, data: Union[int, float, List, np.ndarray], dtype: Optional[str] = None):
|
||||
"""
|
||||
Create a new tensor from data.
|
||||
|
||||
Args:
|
||||
data: Input data (scalar, list, or numpy array)
|
||||
dtype: Data type ('float32', 'int32', etc.). Defaults to auto-detect.
|
||||
|
||||
TODO: Implement tensor creation with proper type handling.
|
||||
"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
@property
|
||||
def data(self) -> np.ndarray:
|
||||
"""Access underlying numpy array."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
@property
|
||||
def shape(self) -> Tuple[int, ...]:
|
||||
"""Get tensor shape."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
"""Get total number of elements."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
@property
|
||||
def dtype(self) -> np.dtype:
|
||||
"""Get data type as numpy dtype."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __repr__(self) -> str:
|
||||
"""String representation."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
class Tensor:
|
||||
"""
|
||||
TinyTorch Tensor: N-dimensional array with ML operations.
|
||||
|
||||
The fundamental data structure for all TinyTorch operations.
|
||||
Wraps NumPy arrays with ML-specific functionality.
|
||||
"""
|
||||
|
||||
def __init__(self, data: Union[int, float, List, np.ndarray], dtype: Optional[str] = None):
|
||||
"""
|
||||
Create a new tensor from data.
|
||||
|
||||
Args:
|
||||
data: Input data (scalar, list, or numpy array)
|
||||
dtype: Data type ('float32', 'int32', etc.). Defaults to auto-detect.
|
||||
"""
|
||||
# Convert input to numpy array
|
||||
if isinstance(data, (int, float, np.number)):
|
||||
# Handle Python and NumPy scalars
|
||||
if dtype is None:
|
||||
# Auto-detect type: int for integers, float32 for floats
|
||||
if isinstance(data, int) or (isinstance(data, np.number) and np.issubdtype(type(data), np.integer)):
|
||||
dtype = 'int32'
|
||||
else:
|
||||
dtype = 'float32'
|
||||
self._data = np.array(data, dtype=dtype)
|
||||
elif isinstance(data, list):
|
||||
# Let NumPy auto-detect type, then convert if needed
|
||||
temp_array = np.array(data)
|
||||
if dtype is None:
|
||||
# Keep NumPy's auto-detected type, but prefer common ML types
|
||||
if np.issubdtype(temp_array.dtype, np.integer):
|
||||
dtype = 'int32'
|
||||
elif np.issubdtype(temp_array.dtype, np.floating):
|
||||
dtype = 'float32'
|
||||
else:
|
||||
dtype = temp_array.dtype
|
||||
self._data = temp_array.astype(dtype)
|
||||
elif isinstance(data, np.ndarray):
|
||||
self._data = data.astype(dtype or data.dtype)
|
||||
else:
|
||||
raise TypeError(f"Cannot create tensor from {type(data)}")
|
||||
|
||||
@property
|
||||
def data(self) -> np.ndarray:
|
||||
"""Access underlying numpy array."""
|
||||
return self._data
|
||||
|
||||
@property
|
||||
def shape(self) -> Tuple[int, ...]:
|
||||
"""Get tensor shape."""
|
||||
return self._data.shape
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
"""Get total number of elements."""
|
||||
return self._data.size
|
||||
|
||||
@property
|
||||
def dtype(self) -> np.dtype:
|
||||
"""Get data type as numpy dtype."""
|
||||
return self._data.dtype
|
||||
|
||||
def __repr__(self) -> str:
|
||||
"""String representation."""
|
||||
return f"Tensor({self._data.tolist()}, shape={self.shape}, dtype={self.dtype})"
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Tensor Class
|
||||
|
||||
Once you implement the Tensor class above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test the basic Tensor class
|
||||
try:
|
||||
print("=== Testing Tensor Creation ===")
|
||||
|
||||
# Scalar tensor
|
||||
scalar = Tensor(5.0)
|
||||
print(f"Scalar: {scalar}")
|
||||
|
||||
# Vector tensor
|
||||
vector = Tensor([1, 2, 3])
|
||||
print(f"Vector: {vector}")
|
||||
|
||||
# Matrix tensor
|
||||
matrix = Tensor([[1, 2], [3, 4]])
|
||||
print(f"Matrix: {matrix}")
|
||||
|
||||
print(f"\nProperties:")
|
||||
print(f"Matrix shape: {matrix.shape}")
|
||||
print(f"Matrix size: {matrix.size}")
|
||||
print(f"Matrix dtype: {matrix.dtype}")
|
||||
|
||||
except NotImplementedError as e:
|
||||
print(f"⚠️ {e}")
|
||||
print("Implement the Tensor class above first!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2: Arithmetic Operations
|
||||
|
||||
Now let's add the core arithmetic operations. These are essential for neural networks:
|
||||
- **Addition**: `tensor + other`
|
||||
- **Subtraction**: `tensor - other`
|
||||
- **Multiplication**: `tensor * other`
|
||||
- **Division**: `tensor / other`
|
||||
|
||||
Each operation should handle both **tensor + tensor** and **tensor + scalar** cases.
|
||||
"""
|
||||
|
||||
# %%
|
||||
#| export
|
||||
def _add_arithmetic_methods():
|
||||
"""
|
||||
Add arithmetic operations to Tensor class.
|
||||
|
||||
TODO: Implement arithmetic methods (__add__, __sub__, __mul__, __truediv__)
|
||||
and their reverse operations (__radd__, __rsub__, etc.)
|
||||
"""
|
||||
|
||||
def __add__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Addition: tensor + other"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __sub__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Subtraction: tensor - other"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __mul__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Multiplication: tensor * other"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def __truediv__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Division: tensor / other"""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# Add methods to Tensor class
|
||||
Tensor.__add__ = __add__
|
||||
Tensor.__sub__ = __sub__
|
||||
Tensor.__mul__ = __mul__
|
||||
Tensor.__truediv__ = __truediv__
|
||||
|
||||
# %%
|
||||
#| hide
|
||||
#| export
|
||||
def _add_arithmetic_methods():
|
||||
"""Add arithmetic operations to Tensor class."""
|
||||
|
||||
def __add__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Addition: tensor + other"""
|
||||
if isinstance(other, Tensor):
|
||||
return Tensor(self._data + other._data)
|
||||
else: # scalar
|
||||
return Tensor(self._data + other)
|
||||
|
||||
def __sub__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Subtraction: tensor - other"""
|
||||
if isinstance(other, Tensor):
|
||||
return Tensor(self._data - other._data)
|
||||
else: # scalar
|
||||
return Tensor(self._data - other)
|
||||
|
||||
def __mul__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Multiplication: tensor * other"""
|
||||
if isinstance(other, Tensor):
|
||||
return Tensor(self._data * other._data)
|
||||
else: # scalar
|
||||
return Tensor(self._data * other)
|
||||
|
||||
def __truediv__(self, other: Union['Tensor', int, float]) -> 'Tensor':
|
||||
"""Division: tensor / other"""
|
||||
if isinstance(other, Tensor):
|
||||
return Tensor(self._data / other._data)
|
||||
else: # scalar
|
||||
return Tensor(self._data / other)
|
||||
|
||||
def __radd__(self, other: Union[int, float]) -> 'Tensor':
|
||||
"""Reverse addition: scalar + tensor"""
|
||||
return Tensor(other + self._data)
|
||||
|
||||
def __rsub__(self, other: Union[int, float]) -> 'Tensor':
|
||||
"""Reverse subtraction: scalar - tensor"""
|
||||
return Tensor(other - self._data)
|
||||
|
||||
def __rmul__(self, other: Union[int, float]) -> 'Tensor':
|
||||
"""Reverse multiplication: scalar * tensor"""
|
||||
return Tensor(other * self._data)
|
||||
|
||||
def __rtruediv__(self, other: Union[int, float]) -> 'Tensor':
|
||||
"""Reverse division: scalar / tensor"""
|
||||
return Tensor(other / self._data)
|
||||
|
||||
# Add methods to Tensor class
|
||||
Tensor.__add__ = __add__
|
||||
Tensor.__sub__ = __sub__
|
||||
Tensor.__mul__ = __mul__
|
||||
Tensor.__truediv__ = __truediv__
|
||||
Tensor.__radd__ = __radd__
|
||||
Tensor.__rsub__ = __rsub__
|
||||
Tensor.__rmul__ = __rmul__
|
||||
Tensor.__rtruediv__ = __rtruediv__
|
||||
|
||||
# Call the function to add arithmetic methods
|
||||
_add_arithmetic_methods()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Arithmetic Operations
|
||||
|
||||
Once you implement the arithmetic methods above, run this cell to test them:
|
||||
"""
|
||||
|
||||
# %%
|
||||
# Test arithmetic operations
|
||||
try:
|
||||
print("=== Testing Arithmetic Operations ===")
|
||||
|
||||
a = Tensor([1, 2, 3])
|
||||
b = Tensor([4, 5, 6])
|
||||
|
||||
print(f"a = {a}")
|
||||
print(f"b = {b}")
|
||||
print()
|
||||
|
||||
# Tensor + Tensor
|
||||
print(f"a + b = {a + b}")
|
||||
print(f"a - b = {a - b}")
|
||||
print(f"a * b = {a * b}")
|
||||
print(f"a / b = {a / b}")
|
||||
print()
|
||||
|
||||
# Tensor + Scalar
|
||||
print(f"a + 10 = {a + 10}")
|
||||
print(f"a * 2 = {a * 2}")
|
||||
print()
|
||||
|
||||
# Scalar + Tensor (reverse operations)
|
||||
print(f"10 + a = {10 + a}")
|
||||
print(f"2 * a = {2 * a}")
|
||||
|
||||
except (NotImplementedError, AttributeError) as e:
|
||||
print(f"⚠️ {e}")
|
||||
print("Implement the arithmetic methods above first!")
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 3: Try the Export Process
|
||||
|
||||
Now let's export our tensor code! In your terminal, run:
|
||||
|
||||
```bash
|
||||
python bin/tito.py sync --module tensor
|
||||
```
|
||||
|
||||
This will export the code marked with `#| export` to `tinytorch/core/tensor.py`.
|
||||
|
||||
Then test it with:
|
||||
|
||||
```bash
|
||||
python bin/tito.py test --module tensor
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
🎉 **Congratulations!** You've built the foundation of TinyTorch - the Tensor class.
|
||||
|
||||
In the next modules, you'll add:
|
||||
- **Automatic differentiation** (gradients)
|
||||
- **Neural network layers**
|
||||
- **Optimizers and training loops**
|
||||
- **GPU acceleration**
|
||||
|
||||
Each builds on this tensor foundation!
|
||||
"""
|
||||
@@ -1,346 +0,0 @@
|
||||
"""
|
||||
Tests for TinyTorch Tensor module.
|
||||
|
||||
Tests the core tensor functionality including creation, arithmetic operations,
|
||||
utility methods, and edge cases.
|
||||
|
||||
These tests work with the current implementation and provide stretch goals
|
||||
for students to implement additional methods.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import pytest
|
||||
import numpy as np
|
||||
|
||||
# Add the parent directory to path to import tensor_dev
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Import from the module's development file
|
||||
# Note: This imports the instructor version with full implementation
|
||||
from tensor_dev import Tensor
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .numpy() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'numpy'):
|
||||
return tensor.numpy()
|
||||
else:
|
||||
return tensor.data
|
||||
|
||||
def safe_item(tensor):
|
||||
"""Get scalar value from tensor, using .item() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'item'):
|
||||
return tensor.item()
|
||||
else:
|
||||
return float(tensor.data)
|
||||
|
||||
class TestTensorCreation:
|
||||
"""Test tensor creation from different data types."""
|
||||
|
||||
def test_scalar_creation(self):
|
||||
"""Test creating tensors from scalars."""
|
||||
# Float scalar
|
||||
t1 = Tensor(5.0)
|
||||
assert t1.shape == ()
|
||||
assert t1.size == 1
|
||||
assert safe_item(t1) == 5.0
|
||||
|
||||
# Integer scalar
|
||||
t2 = Tensor(42)
|
||||
assert t2.shape == ()
|
||||
assert t2.size == 1
|
||||
assert safe_item(t2) == 42.0 # Should convert to float32
|
||||
|
||||
def test_vector_creation(self):
|
||||
"""Test creating 1D tensors."""
|
||||
t = Tensor([1, 2, 3, 4])
|
||||
assert t.shape == (4,)
|
||||
assert t.size == 4
|
||||
assert t.dtype == np.int32 # Integer list defaults to int32
|
||||
np.testing.assert_array_equal(safe_numpy(t), [1, 2, 3, 4])
|
||||
|
||||
def test_matrix_creation(self):
|
||||
"""Test creating 2D tensors."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
assert t.shape == (2, 2)
|
||||
assert t.size == 4
|
||||
expected = np.array([[1.0, 2.0], [3.0, 4.0]], dtype='float32')
|
||||
np.testing.assert_array_equal(safe_numpy(t), expected)
|
||||
|
||||
def test_numpy_array_creation(self):
|
||||
"""Test creating tensors from numpy arrays."""
|
||||
arr = np.array([1, 2, 3], dtype='int32')
|
||||
t = Tensor(arr)
|
||||
assert t.shape == (3,)
|
||||
assert t.dtype in ['int32', 'float32'] # May convert
|
||||
|
||||
def test_dtype_specification(self):
|
||||
"""Test explicit dtype specification."""
|
||||
t = Tensor([1, 2, 3], dtype='int32')
|
||||
assert t.dtype == np.int32
|
||||
|
||||
def test_invalid_data_type(self):
|
||||
"""Test error handling for invalid data types."""
|
||||
with pytest.raises(TypeError):
|
||||
Tensor("invalid")
|
||||
with pytest.raises(TypeError):
|
||||
Tensor({"dict": "invalid"})
|
||||
|
||||
class TestTensorProperties:
|
||||
"""Test tensor properties and methods."""
|
||||
|
||||
def test_shape_property(self):
|
||||
"""Test shape property for different dimensions."""
|
||||
assert Tensor(5).shape == ()
|
||||
assert Tensor([1, 2, 3]).shape == (3,)
|
||||
assert Tensor([[1, 2], [3, 4]]).shape == (2, 2)
|
||||
assert Tensor([[[1]]]).shape == (1, 1, 1)
|
||||
|
||||
def test_size_property(self):
|
||||
"""Test size property."""
|
||||
assert Tensor(5).size == 1
|
||||
assert Tensor([1, 2, 3]).size == 3
|
||||
assert Tensor([[1, 2], [3, 4]]).size == 4
|
||||
assert Tensor([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]).size == 8
|
||||
|
||||
def test_dtype_property(self):
|
||||
"""Test dtype property."""
|
||||
t1 = Tensor(5.0)
|
||||
assert t1.dtype == np.float32
|
||||
|
||||
t2 = Tensor([1, 2, 3], dtype='int32')
|
||||
assert t2.dtype == np.int32
|
||||
|
||||
def test_repr(self):
|
||||
"""Test string representation."""
|
||||
t = Tensor([1, 2, 3])
|
||||
repr_str = repr(t)
|
||||
assert 'Tensor' in repr_str
|
||||
assert 'shape=' in repr_str
|
||||
assert 'dtype=' in repr_str
|
||||
|
||||
class TestArithmeticOperations:
|
||||
"""Test tensor arithmetic operations."""
|
||||
|
||||
def test_tensor_addition(self):
|
||||
"""Test tensor + tensor addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
b = Tensor([4, 5, 6])
|
||||
result = a + b
|
||||
expected = [5.0, 7.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_addition(self):
|
||||
"""Test tensor + scalar addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = a + 10
|
||||
expected = [11.0, 12.0, 13.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_reverse_addition(self):
|
||||
"""Test scalar + tensor addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = 10 + a
|
||||
expected = [11.0, 12.0, 13.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_subtraction(self):
|
||||
"""Test tensor - tensor subtraction."""
|
||||
a = Tensor([5, 7, 9])
|
||||
b = Tensor([1, 2, 3])
|
||||
result = a - b
|
||||
expected = [4.0, 5.0, 6.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_subtraction(self):
|
||||
"""Test tensor - scalar subtraction."""
|
||||
a = Tensor([10, 20, 30])
|
||||
result = a - 5
|
||||
expected = [5.0, 15.0, 25.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_multiplication(self):
|
||||
"""Test tensor * tensor multiplication."""
|
||||
a = Tensor([2, 3, 4])
|
||||
b = Tensor([5, 6, 7])
|
||||
result = a * b
|
||||
expected = [10.0, 18.0, 28.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_multiplication(self):
|
||||
"""Test tensor * scalar multiplication."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = a * 3
|
||||
expected = [3.0, 6.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_reverse_multiplication(self):
|
||||
"""Test scalar * tensor multiplication."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = 3 * a
|
||||
expected = [3.0, 6.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_division(self):
|
||||
"""Test tensor / tensor division."""
|
||||
a = Tensor([6, 8, 10])
|
||||
b = Tensor([2, 4, 5])
|
||||
result = a / b
|
||||
expected = [3.0, 2.0, 2.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_division(self):
|
||||
"""Test tensor / scalar division."""
|
||||
a = Tensor([6, 8, 10])
|
||||
result = a / 2
|
||||
expected = [3.0, 4.0, 5.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
class TestUtilityMethods:
|
||||
"""Test tensor utility methods (stretch goals for students)."""
|
||||
|
||||
def test_reshape(self):
|
||||
"""Test tensor reshaping (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'reshape'):
|
||||
reshaped = t.reshape(4)
|
||||
assert reshaped.shape == (4,)
|
||||
expected = [1.0, 2.0, 3.0, 4.0]
|
||||
np.testing.assert_array_equal(safe_numpy(reshaped), expected)
|
||||
|
||||
# Reshape to 2D
|
||||
reshaped2 = t.reshape(1, 4)
|
||||
assert reshaped2.shape == (1, 4)
|
||||
else:
|
||||
pytest.skip("reshape method not implemented - stretch goal for students")
|
||||
|
||||
def test_transpose(self):
|
||||
"""Test tensor transpose (if implemented)."""
|
||||
t = Tensor([[1, 2, 3], [4, 5, 6]])
|
||||
if hasattr(t, 'transpose'):
|
||||
transposed = t.transpose()
|
||||
assert transposed.shape == (3, 2)
|
||||
expected = [[1.0, 4.0], [2.0, 5.0], [3.0, 6.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(transposed), expected)
|
||||
else:
|
||||
pytest.skip("transpose method not implemented - stretch goal for students")
|
||||
|
||||
def test_sum_all(self):
|
||||
"""Test summing all elements (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'sum'):
|
||||
result = t.sum()
|
||||
expected = 10.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("sum method not implemented - stretch goal for students")
|
||||
|
||||
def test_sum_axis(self):
|
||||
"""Test summing along specific axes (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'sum'):
|
||||
# Sum along axis 0 (columns)
|
||||
sum0 = t.sum(axis=0)
|
||||
expected0 = [4.0, 6.0]
|
||||
np.testing.assert_array_equal(safe_numpy(sum0), expected0)
|
||||
|
||||
# Sum along axis 1 (rows)
|
||||
sum1 = t.sum(axis=1)
|
||||
expected1 = [3.0, 7.0]
|
||||
np.testing.assert_array_equal(safe_numpy(sum1), expected1)
|
||||
else:
|
||||
pytest.skip("sum method not implemented - stretch goal for students")
|
||||
|
||||
def test_mean(self):
|
||||
"""Test mean calculation (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'mean'):
|
||||
result = t.mean()
|
||||
expected = 2.5
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("mean method not implemented - stretch goal for students")
|
||||
|
||||
def test_max(self):
|
||||
"""Test maximum value (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'max'):
|
||||
result = t.max()
|
||||
expected = 4.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("max method not implemented - stretch goal for students")
|
||||
|
||||
def test_min(self):
|
||||
"""Test minimum value (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'min'):
|
||||
result = t.min()
|
||||
expected = 1.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("min method not implemented - stretch goal for students")
|
||||
|
||||
def test_item_scalar(self):
|
||||
"""Test converting single-element tensor to scalar (if implemented)."""
|
||||
t = Tensor(42.0)
|
||||
if hasattr(t, 'item'):
|
||||
assert t.item() == 42.0
|
||||
else:
|
||||
pytest.skip("item method not implemented - stretch goal for students")
|
||||
|
||||
def test_item_error(self):
|
||||
"""Test item() error for multi-element tensors (if implemented)."""
|
||||
t = Tensor([1, 2, 3])
|
||||
if hasattr(t, 'item'):
|
||||
with pytest.raises(ValueError):
|
||||
t.item()
|
||||
else:
|
||||
pytest.skip("item method not implemented - stretch goal for students")
|
||||
|
||||
def test_numpy_conversion(self):
|
||||
"""Test converting tensor to numpy array (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'numpy'):
|
||||
arr = t.numpy()
|
||||
assert isinstance(arr, np.ndarray)
|
||||
expected = [[1.0, 2.0], [3.0, 4.0]]
|
||||
np.testing.assert_array_equal(arr, expected)
|
||||
else:
|
||||
pytest.skip("numpy method not implemented - stretch goal for students")
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and error handling."""
|
||||
|
||||
def test_empty_list(self):
|
||||
"""Test creating tensor from empty list."""
|
||||
t = Tensor([])
|
||||
assert t.shape == (0,)
|
||||
assert t.size == 0
|
||||
|
||||
def test_mixed_operations(self):
|
||||
"""Test combining different operations."""
|
||||
a = Tensor([[1, 2], [3, 4]])
|
||||
b = Tensor([[2, 2], [2, 2]])
|
||||
|
||||
# Complex expression
|
||||
result = (a + b) * 2 - 1
|
||||
expected = [[5.0, 7.0], [9.0, 11.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_chained_operations(self):
|
||||
"""Test chaining multiple operations (if methods implemented)."""
|
||||
t = Tensor([[1, 2, 3], [4, 5, 6]])
|
||||
if hasattr(t, 'sum') and hasattr(t, 'mean'):
|
||||
result = t.sum(axis=1).mean()
|
||||
expected = 10.5 # (6 + 15) / 2
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("Advanced methods not implemented - stretch goal for students")
|
||||
|
||||
def run_tensor_tests():
|
||||
"""Run all tensor tests."""
|
||||
pytest.main([__file__, "-v"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_tensor_tests()
|
||||
6
bin/tito
6
bin/tito
@@ -2,7 +2,7 @@
|
||||
"""
|
||||
TinyTorch CLI Wrapper
|
||||
|
||||
Backward compatibility wrapper that calls the comprehensive CLI structure.
|
||||
Backward compatibility wrapper that calls the new CLI structure.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -12,8 +12,8 @@ from pathlib import Path
|
||||
project_root = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
# Import and run the comprehensive CLI
|
||||
from bin.tito import main
|
||||
# Import and run the new CLI
|
||||
from tito.main import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
973
bin/tito.py
973
bin/tito.py
@@ -1,973 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
TinyTorch CLI (tito)
|
||||
|
||||
The main command-line interface for the TinyTorch ML system.
|
||||
Students use this CLI for testing, training, and project management.
|
||||
|
||||
Usage: python bin/tito.py [command] [options]
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
|
||||
# Rich imports for beautiful terminal output
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from rich.panel import Panel
|
||||
from rich.text import Text
|
||||
from rich.tree import Tree
|
||||
from rich.progress import Progress, SpinnerColumn, BarColumn, TextColumn
|
||||
|
||||
# Add this import at the top with other imports
|
||||
import subprocess
|
||||
import json
|
||||
import re
|
||||
|
||||
# Add the project root to Python path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
# Create Rich console instance
|
||||
console = Console()
|
||||
|
||||
def print_banner():
|
||||
"""Print the TinyTorch banner using Rich."""
|
||||
banner_text = Text("Tiny🔥Torch: Build ML Systems from Scratch", style="bold red")
|
||||
console.print(Panel(banner_text, style="bright_blue", padding=(1, 2)))
|
||||
|
||||
def check_setup_status():
|
||||
"""Check if setup project is complete."""
|
||||
try:
|
||||
from tinytorch.core.utils import hello_tinytorch
|
||||
return "✅ Implemented"
|
||||
except ImportError:
|
||||
return "❌ Not Implemented"
|
||||
|
||||
def check_tensor_status():
|
||||
"""Check if tensor project is complete."""
|
||||
try:
|
||||
from tinytorch.core.tensor import Tensor
|
||||
# Test actual functionality, not just import
|
||||
t1 = Tensor([1, 2, 3])
|
||||
t2 = Tensor([4, 5, 6])
|
||||
result = t1 + t2 # Should work if implemented
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_mlp_status():
|
||||
"""Check if MLP project is complete."""
|
||||
try:
|
||||
from tinytorch.core.modules import MLP
|
||||
# Test actual functionality
|
||||
mlp = MLP(input_size=10, hidden_size=5, output_size=2)
|
||||
from tinytorch.core.tensor import Tensor
|
||||
x = Tensor([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]])
|
||||
output = mlp(x) # Should work if implemented
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_cnn_status():
|
||||
"""Check if CNN project is complete."""
|
||||
try:
|
||||
from tinytorch.core.modules import Conv2d
|
||||
# Test actual functionality
|
||||
conv = Conv2d(in_channels=3, out_channels=16, kernel_size=3)
|
||||
from tinytorch.core.tensor import Tensor
|
||||
x = Tensor(np.random.randn(1, 3, 32, 32))
|
||||
output = conv(x) # Should work if implemented
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_data_status():
|
||||
"""Check if data project is complete."""
|
||||
try:
|
||||
from tinytorch.core.dataloader import DataLoader
|
||||
# Test actual functionality
|
||||
data = [(np.random.randn(3, 32, 32), 0) for _ in range(10)]
|
||||
loader = DataLoader(data, batch_size=2, shuffle=True)
|
||||
batch = next(iter(loader)) # Should work if implemented
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError, StopIteration):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_training_status():
|
||||
"""Check if training project is complete."""
|
||||
try:
|
||||
from tinytorch.core.optimizer import SGD
|
||||
from tinytorch.core.tensor import Tensor
|
||||
# Test actual functionality
|
||||
t = Tensor([1.0, 2.0, 3.0], requires_grad=True)
|
||||
optimizer = SGD([t], lr=0.01)
|
||||
t.backward() # Should work if autograd is implemented
|
||||
optimizer.step() # Should work if optimizer is implemented
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_profiling_status():
|
||||
"""Check if profiling project is complete."""
|
||||
try:
|
||||
from tinytorch.core.profiler import Profiler
|
||||
# Test basic functionality
|
||||
profiler = Profiler()
|
||||
profiler.start("test")
|
||||
profiler.end("test")
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_compression_status():
|
||||
"""Check if compression project is complete."""
|
||||
try:
|
||||
from tinytorch.core.compression import Pruner
|
||||
# Test basic functionality
|
||||
pruner = Pruner(sparsity=0.5)
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_kernels_status():
|
||||
"""Check if kernels project is complete."""
|
||||
try:
|
||||
from tinytorch.core.kernels import optimized_matmul
|
||||
# Test basic functionality
|
||||
a = np.random.randn(3, 3)
|
||||
b = np.random.randn(3, 3)
|
||||
result = optimized_matmul(a, b)
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_benchmarking_status():
|
||||
"""Check if benchmarking project is complete."""
|
||||
try:
|
||||
from tinytorch.core.benchmark import Benchmark
|
||||
# Test basic functionality
|
||||
benchmark = Benchmark()
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def check_mlops_status():
|
||||
"""Check if MLOps project is complete."""
|
||||
try:
|
||||
from tinytorch.core.mlops import ModelMonitor
|
||||
from tinytorch.core.tensor import Tensor
|
||||
# Test that the methods are actually implemented, not just placeholders
|
||||
monitor = ModelMonitor(model=None, baseline_metrics={})
|
||||
# Try to use a method that should be implemented
|
||||
test_inputs = Tensor([1.0, 2.0, 3.0])
|
||||
test_predictions = Tensor([0.5, 0.8, 0.2])
|
||||
monitor.log_prediction(test_inputs, test_predictions)
|
||||
return "✅ Implemented"
|
||||
except (ImportError, NotImplementedError, AttributeError, TypeError):
|
||||
return "⏳ Not Started"
|
||||
|
||||
def validate_environment():
|
||||
"""Validate environment setup before running commands."""
|
||||
issues = []
|
||||
|
||||
# Check virtual environment
|
||||
in_venv = (hasattr(sys, 'real_prefix') or
|
||||
(hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix))
|
||||
if not in_venv:
|
||||
issues.append("Virtual environment not activated. Run: source .venv/bin/activate")
|
||||
|
||||
# Check Python version
|
||||
if sys.version_info < (3, 8):
|
||||
issues.append(f"Python 3.8+ required, found {sys.version_info.major}.{sys.version_info.minor}")
|
||||
|
||||
# Check core dependencies
|
||||
try:
|
||||
import numpy
|
||||
import pytest
|
||||
except ImportError as e:
|
||||
issues.append(f"Missing dependency: {e.name}. Run: pip install -r requirements.txt")
|
||||
|
||||
# Check TinyTorch package structure
|
||||
tinytorch_path = Path(__file__).parent.parent / "tinytorch"
|
||||
if not tinytorch_path.exists():
|
||||
issues.append("TinyTorch package not found. Check project structure.")
|
||||
|
||||
if issues:
|
||||
issue_text = Text()
|
||||
issue_text.append("❌ Environment Issues Detected\n\n", style="bold red")
|
||||
for issue in issues:
|
||||
issue_text.append(f" • {issue}\n", style="yellow")
|
||||
issue_text.append("\nRun 'python3 bin/tito.py doctor' for detailed diagnosis", style="dim")
|
||||
|
||||
console.print(Panel(issue_text, title="Environment Problems", border_style="red"))
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def cmd_version(args):
|
||||
"""Show TinyTorch version."""
|
||||
version_text = Text()
|
||||
version_text.append("Tiny🔥Torch v0.1.0\n", style="bold red")
|
||||
version_text.append("Machine Learning Systems Course", style="cyan")
|
||||
console.print(Panel(version_text, title="Version Info", border_style="bright_blue"))
|
||||
|
||||
def cmd_info(args):
|
||||
"""Show system information and status."""
|
||||
print_banner()
|
||||
console.print()
|
||||
|
||||
# System Information Panel
|
||||
info_text = Text()
|
||||
info_text.append(f"Python: {sys.version.split()[0]}\n", style="cyan")
|
||||
info_text.append(f"Platform: {sys.platform}\n", style="cyan")
|
||||
info_text.append(f"Working Directory: {os.getcwd()}\n", style="cyan")
|
||||
|
||||
# Virtual environment check - use same robust detection as doctor
|
||||
venv_path = Path(".venv")
|
||||
venv_exists = venv_path.exists()
|
||||
in_venv = (
|
||||
# Method 1: Check VIRTUAL_ENV environment variable (most reliable for activation)
|
||||
os.environ.get('VIRTUAL_ENV') is not None or
|
||||
# Method 2: Check sys.prefix vs sys.base_prefix (works for running Python in venv)
|
||||
(hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix) or
|
||||
# Method 3: Check for sys.real_prefix (older Python versions)
|
||||
hasattr(sys, 'real_prefix')
|
||||
)
|
||||
|
||||
if venv_exists and in_venv:
|
||||
venv_style = "green"
|
||||
venv_icon = "✅"
|
||||
venv_status = "Ready & Active"
|
||||
elif venv_exists:
|
||||
venv_style = "yellow"
|
||||
venv_icon = "✅"
|
||||
venv_status = "Ready (Not Active)"
|
||||
else:
|
||||
venv_style = "red"
|
||||
venv_icon = "❌"
|
||||
venv_status = "Not Found"
|
||||
|
||||
info_text.append(f"Virtual Environment: {venv_icon} ", style=venv_style)
|
||||
info_text.append(venv_status, style=f"bold {venv_style}")
|
||||
|
||||
console.print(Panel(info_text, title="📋 System Information", border_style="bright_blue"))
|
||||
console.print()
|
||||
|
||||
# Course Navigation Panel
|
||||
nav_text = Text()
|
||||
nav_text.append("📖 Course Overview: ", style="dim")
|
||||
nav_text.append("README.md\n", style="cyan underline")
|
||||
nav_text.append("🎯 Detailed Guide: ", style="dim")
|
||||
nav_text.append("COURSE_GUIDE.md\n", style="cyan underline")
|
||||
nav_text.append("🚀 Start Here: ", style="dim")
|
||||
nav_text.append("modules/setup/README.md", style="cyan underline")
|
||||
|
||||
console.print(Panel(nav_text, title="📋 Course Navigation", border_style="bright_green"))
|
||||
console.print()
|
||||
|
||||
# Implementation status
|
||||
modules = [
|
||||
("Setup", "hello_tinytorch function", check_setup_status),
|
||||
("Tensor", "basic tensor operations", check_tensor_status),
|
||||
("MLP", "multi-layer perceptron (manual)", check_mlp_status),
|
||||
("CNN", "convolutional networks (basic)", check_cnn_status),
|
||||
("Data", "data loading pipeline", check_data_status),
|
||||
("Training", "autograd engine & optimization", check_training_status),
|
||||
("Profiling", "performance profiling", check_profiling_status),
|
||||
("Compression", "model compression", check_compression_status),
|
||||
("Kernels", "custom compute kernels", check_kernels_status),
|
||||
("Benchmarking", "performance benchmarking", check_benchmarking_status),
|
||||
("MLOps", "production monitoring", check_mlops_status),
|
||||
]
|
||||
|
||||
# Module Status Table
|
||||
status_table = Table(title="🚀 Module Implementation Status", show_header=True, header_style="bold blue")
|
||||
status_table.add_column("ID", style="dim", width=3, justify="center")
|
||||
status_table.add_column("Project", style="bold cyan", width=12)
|
||||
status_table.add_column("Status", width=18, justify="center")
|
||||
status_table.add_column("Description", style="dim", width=40)
|
||||
|
||||
for i, (name, desc, check_func) in enumerate(modules):
|
||||
status_text = check_func()
|
||||
if "✅" in status_text:
|
||||
status_style = "[green]✅ Implemented[/green]"
|
||||
elif "❌" in status_text:
|
||||
status_style = "[red]❌ Not Implemented[/red]"
|
||||
else:
|
||||
status_style = "[yellow]⏳ Not Started[/yellow]"
|
||||
|
||||
status_table.add_row(str(i), name, status_style, desc)
|
||||
|
||||
console.print(status_table)
|
||||
|
||||
if args.hello and check_setup_status() == "✅ Implemented":
|
||||
try:
|
||||
from tinytorch.core.utils import hello_tinytorch
|
||||
hello_text = Text(hello_tinytorch(), style="bold red")
|
||||
console.print()
|
||||
console.print(Panel(hello_text, style="bright_red", padding=(1, 2)))
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
if args.show_architecture:
|
||||
console.print()
|
||||
|
||||
# Create architecture tree
|
||||
arch_tree = Tree("🏗️ TinyTorch System Architecture", style="bold blue")
|
||||
|
||||
cli_branch = arch_tree.add("CLI Interface", style="cyan")
|
||||
cli_branch.add("bin/tito.py - Command line tools", style="dim")
|
||||
|
||||
training_branch = arch_tree.add("Training Orchestration", style="cyan")
|
||||
training_branch.add("trainer.py - Training loop management", style="dim")
|
||||
|
||||
core_branch = arch_tree.add("Core Components", style="cyan")
|
||||
model_sub = core_branch.add("Model Definition", style="yellow")
|
||||
model_sub.add("modules.py - Neural network layers", style="dim")
|
||||
|
||||
data_sub = core_branch.add("Data Pipeline", style="yellow")
|
||||
data_sub.add("dataloader.py - Efficient data loading", style="dim")
|
||||
|
||||
opt_sub = core_branch.add("Optimization", style="yellow")
|
||||
opt_sub.add("optimizer.py - SGD, Adam, etc.", style="dim")
|
||||
|
||||
autograd_branch = arch_tree.add("Automatic Differentiation Engine", style="cyan")
|
||||
autograd_branch.add("autograd.py - Gradient computation", style="dim")
|
||||
|
||||
tensor_branch = arch_tree.add("Tensor Operations & Storage", style="cyan")
|
||||
tensor_branch.add("tensor.py - Core tensor implementation", style="dim")
|
||||
|
||||
system_branch = arch_tree.add("System Tools", style="cyan")
|
||||
system_branch.add("profiler.py - Performance measurement", style="dim")
|
||||
system_branch.add("mlops.py - Production monitoring", style="dim")
|
||||
|
||||
console.print(Panel(arch_tree, title="🏗️ System Architecture", border_style="bright_blue"))
|
||||
|
||||
def cmd_test(args):
|
||||
"""Run tests for a specific module."""
|
||||
valid_modules = ["setup", "tensor", "activations", "layers", "cnn", "data", "training",
|
||||
"profiling", "compression", "kernels", "benchmarking", "mlops"]
|
||||
|
||||
if args.all:
|
||||
# Run all tests with progress bar
|
||||
import subprocess
|
||||
failed_modules = []
|
||||
|
||||
# Count existing test files in modules/{module}/tests/
|
||||
existing_tests = []
|
||||
for module in valid_modules:
|
||||
test_path = Path(f"modules/{module}/tests/test_{module}.py")
|
||||
if test_path.exists():
|
||||
existing_tests.append(module)
|
||||
|
||||
console.print(Panel(f"🧪 Running tests for {len(existing_tests)} modules",
|
||||
title="Test Suite", border_style="bright_cyan"))
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
console=console
|
||||
) as progress:
|
||||
|
||||
task = progress.add_task("Running tests...", total=len(existing_tests))
|
||||
|
||||
for module in existing_tests:
|
||||
progress.update(task, description=f"Testing {module}...")
|
||||
|
||||
test_file = f"modules/{module}/tests/test_{module}.py"
|
||||
result = subprocess.run([sys.executable, "-m", "pytest", test_file, "-v"],
|
||||
capture_output=True, text=True)
|
||||
|
||||
if result.returncode != 0:
|
||||
failed_modules.append(module)
|
||||
console.print(f"[red]❌ {module} tests failed[/red]")
|
||||
else:
|
||||
console.print(f"[green]✅ {module} tests passed[/green]")
|
||||
|
||||
progress.advance(task)
|
||||
|
||||
# Results summary
|
||||
if failed_modules:
|
||||
console.print(Panel(f"[red]❌ Failed modules: {', '.join(failed_modules)}[/red]",
|
||||
title="Test Results", border_style="red"))
|
||||
return 1
|
||||
else:
|
||||
console.print(Panel("[green]✅ All tests passed![/green]",
|
||||
title="Test Results", border_style="green"))
|
||||
return 0
|
||||
|
||||
elif args.module in valid_modules:
|
||||
# Run specific module tests
|
||||
import subprocess
|
||||
test_file = f"modules/{args.module}/tests/test_{args.module}.py"
|
||||
|
||||
console.print(Panel(f"🧪 Running tests for module: [bold cyan]{args.module}[/bold cyan]",
|
||||
title="Single Module Test", border_style="bright_cyan"))
|
||||
|
||||
if not Path(test_file).exists():
|
||||
console.print(Panel(f"[yellow]⏳ Test file not found: {test_file}\n"
|
||||
f"Module '{args.module}' may not be implemented yet.[/yellow]",
|
||||
title="Test Not Found", border_style="yellow"))
|
||||
return 1
|
||||
|
||||
console.print(f"[dim]Running: pytest {test_file} -v[/dim]")
|
||||
console.print()
|
||||
|
||||
result = subprocess.run([sys.executable, "-m", "pytest", test_file, "-v"],
|
||||
capture_output=False)
|
||||
|
||||
# Show result summary
|
||||
if result.returncode == 0:
|
||||
console.print(Panel(f"[green]✅ All tests passed for {args.module}![/green]",
|
||||
title="Test Results", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Some tests failed for {args.module}[/red]",
|
||||
title="Test Results", border_style="red"))
|
||||
|
||||
return result.returncode
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Unknown module: {args.module}[/red]\n"
|
||||
f"Valid modules: {', '.join(valid_modules)}",
|
||||
title="Invalid Module", border_style="red"))
|
||||
return 1
|
||||
|
||||
def cmd_submit(args):
|
||||
"""Submit module for grading."""
|
||||
submit_text = Text()
|
||||
submit_text.append(f"📤 Submitting module: {args.module}\n\n", style="bold cyan")
|
||||
submit_text.append("🚧 Submission system not yet implemented.\n\n", style="yellow")
|
||||
submit_text.append("For now, make sure all tests pass with:\n", style="dim")
|
||||
submit_text.append(f" python -m pytest modules/{args.module}/tests/test_{args.module}.py -v", style="bold white")
|
||||
|
||||
console.print(Panel(submit_text, title="Module Submission", border_style="bright_yellow"))
|
||||
|
||||
def cmd_status(args):
|
||||
"""Check module status."""
|
||||
status_text = Text()
|
||||
status_text.append(f"📊 Status for module: {args.module}\n\n", style="bold cyan")
|
||||
status_text.append("🚧 Status system not yet implemented.", style="yellow")
|
||||
|
||||
console.print(Panel(status_text, title="Module Status", border_style="bright_yellow"))
|
||||
|
||||
|
||||
|
||||
|
||||
def cmd_doctor(args):
|
||||
"""Run comprehensive environment diagnosis."""
|
||||
console.print(Panel("🔬 TinyTorch Environment Diagnosis",
|
||||
title="System Doctor", border_style="bright_magenta"))
|
||||
console.print()
|
||||
|
||||
# Environment checks table
|
||||
env_table = Table(title="Environment Check", show_header=True, header_style="bold blue")
|
||||
env_table.add_column("Component", style="cyan", width=20)
|
||||
env_table.add_column("Status", justify="left")
|
||||
env_table.add_column("Details", style="dim", width=30)
|
||||
|
||||
# Python environment
|
||||
env_table.add_row("Python", "[green]✅ OK[/green]", f"{sys.version.split()[0]} ({sys.platform})")
|
||||
|
||||
# Virtual environment - check if it exists and if we're using it
|
||||
venv_path = Path(".venv")
|
||||
venv_exists = venv_path.exists()
|
||||
in_venv = (
|
||||
# Method 1: Check VIRTUAL_ENV environment variable (most reliable for activation)
|
||||
os.environ.get('VIRTUAL_ENV') is not None or
|
||||
# Method 2: Check sys.prefix vs sys.base_prefix (works for running Python in venv)
|
||||
(hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix) or
|
||||
# Method 3: Check for sys.real_prefix (older Python versions)
|
||||
hasattr(sys, 'real_prefix')
|
||||
)
|
||||
|
||||
if venv_exists and in_venv:
|
||||
venv_status = "[green]✅ Ready & Active[/green]"
|
||||
elif venv_exists:
|
||||
venv_status = "[yellow]✅ Ready (Not Active)[/yellow]"
|
||||
else:
|
||||
venv_status = "[red]❌ Not Found[/red]"
|
||||
env_table.add_row("Virtual Environment", venv_status, ".venv")
|
||||
|
||||
# Dependencies
|
||||
dependencies = ['numpy', 'matplotlib', 'pytest', 'yaml', 'black', 'rich']
|
||||
for dep in dependencies:
|
||||
try:
|
||||
module = __import__(dep)
|
||||
version = getattr(module, '__version__', 'unknown')
|
||||
env_table.add_row(dep.title(), "[green]✅ OK[/green]", f"v{version}")
|
||||
except ImportError:
|
||||
env_table.add_row(dep.title(), "[red]❌ Missing[/red]", "Not installed")
|
||||
|
||||
console.print(env_table)
|
||||
console.print()
|
||||
|
||||
# Module structure table
|
||||
struct_table = Table(title="Module Structure", show_header=True, header_style="bold magenta")
|
||||
struct_table.add_column("Path", style="cyan", width=25)
|
||||
struct_table.add_column("Status", justify="left")
|
||||
struct_table.add_column("Type", style="dim", width=25)
|
||||
|
||||
required_paths = [
|
||||
('tinytorch/', 'Package directory'),
|
||||
('tinytorch/core/', 'Core module directory'),
|
||||
('modules/', 'Module directory'),
|
||||
('bin/tito.py', 'CLI script'),
|
||||
('requirements.txt', 'Dependencies file')
|
||||
]
|
||||
|
||||
for path, desc in required_paths:
|
||||
if Path(path).exists():
|
||||
struct_table.add_row(path, "[green]✅ Found[/green]", desc)
|
||||
else:
|
||||
struct_table.add_row(path, "[red]❌ Missing[/red]", desc)
|
||||
|
||||
console.print(struct_table)
|
||||
console.print()
|
||||
|
||||
# Module implementations
|
||||
console.print(Panel("📋 Implementation Status",
|
||||
title="Module Status", border_style="bright_blue"))
|
||||
cmd_info(argparse.Namespace(hello=False, show_architecture=False))
|
||||
|
||||
def cmd_jupyter(args):
|
||||
"""Start Jupyter notebook server."""
|
||||
import subprocess
|
||||
|
||||
console.print(Panel("📓 Jupyter Notebook Server",
|
||||
title="Interactive Development", border_style="bright_green"))
|
||||
|
||||
# Determine which Jupyter to start
|
||||
if args.lab:
|
||||
cmd = ["jupyter", "lab", "--port", str(args.port)]
|
||||
console.print(f"🚀 Starting JupyterLab on port {args.port}...")
|
||||
else:
|
||||
cmd = ["jupyter", "notebook", "--port", str(args.port)]
|
||||
console.print(f"🚀 Starting Jupyter Notebook on port {args.port}...")
|
||||
|
||||
console.print("💡 Open your browser to the URL shown above")
|
||||
console.print("📁 Navigate to your module's notebook directory")
|
||||
console.print("🔄 Press Ctrl+C to stop the server")
|
||||
|
||||
try:
|
||||
subprocess.run(cmd)
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n🛑 Jupyter server stopped")
|
||||
except FileNotFoundError:
|
||||
console.print(Panel("[red]❌ Jupyter not found. Install with: pip install jupyter[/red]",
|
||||
title="Error", border_style="red"))
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
def cmd_sync(args):
|
||||
"""Export notebook code to Python package using nbdev."""
|
||||
import subprocess
|
||||
|
||||
# Determine what to sync
|
||||
if hasattr(args, 'module') and args.module:
|
||||
module_path = f"modules/{args.module}"
|
||||
if not Path(module_path).exists():
|
||||
console.print(Panel(f"[red]❌ Module '{args.module}' not found at {module_path}[/red]",
|
||||
title="Module Not Found", border_style="red"))
|
||||
return 1
|
||||
|
||||
console.print(Panel(f"🔄 Synchronizing Module: {args.module}",
|
||||
title="nbdev Export", border_style="bright_cyan"))
|
||||
console.print(f"🔄 Exporting {args.module} notebook to tinytorch package...")
|
||||
|
||||
# Use nbdev_export with --path for specific module
|
||||
cmd = ["nbdev_export", "--path", module_path]
|
||||
else:
|
||||
console.print(Panel("🔄 Synchronizing All Notebooks to Package",
|
||||
title="nbdev Export", border_style="bright_cyan"))
|
||||
console.print("🔄 Exporting all notebook code to tinytorch package...")
|
||||
|
||||
# Use nbdev_export for all modules
|
||||
cmd = ["nbdev_export"]
|
||||
|
||||
try:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, cwd=Path.cwd())
|
||||
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Successfully exported notebook code to tinytorch package![/green]",
|
||||
title="Export Success", border_style="green"))
|
||||
|
||||
# Show what was exported
|
||||
exports_text = Text()
|
||||
exports_text.append("📦 Exported modules:\n", style="bold cyan")
|
||||
|
||||
# Check for exported files
|
||||
tinytorch_path = Path("tinytorch")
|
||||
if tinytorch_path.exists():
|
||||
for py_file in tinytorch_path.rglob("*.py"):
|
||||
if py_file.name != "__init__.py" and py_file.stat().st_size > 100: # Non-empty files
|
||||
rel_path = py_file.relative_to(tinytorch_path)
|
||||
exports_text.append(f" ✅ tinytorch/{rel_path}\n", style="green")
|
||||
|
||||
exports_text.append("\n💡 Next steps:\n", style="bold yellow")
|
||||
exports_text.append(" • Run: tito test --module setup\n", style="white")
|
||||
exports_text.append(" • Or: tito test --all\n", style="white")
|
||||
|
||||
console.print(Panel(exports_text, title="Export Summary", border_style="bright_green"))
|
||||
|
||||
else:
|
||||
error_msg = result.stderr.strip() if result.stderr else "Unknown error"
|
||||
console.print(Panel(f"[red]❌ Export failed:\n{error_msg}[/red]",
|
||||
title="Export Error", border_style="red"))
|
||||
|
||||
# Helpful error guidance
|
||||
help_text = Text()
|
||||
help_text.append("💡 Common issues:\n", style="bold yellow")
|
||||
help_text.append(" • Missing #| default_exp directive in notebook\n", style="white")
|
||||
help_text.append(" • Syntax errors in exported code\n", style="white")
|
||||
help_text.append(" • Missing settings.ini configuration\n", style="white")
|
||||
help_text.append("\n🔧 Run 'tito doctor' for detailed diagnosis", style="cyan")
|
||||
|
||||
console.print(Panel(help_text, title="Troubleshooting", border_style="yellow"))
|
||||
|
||||
return result.returncode
|
||||
|
||||
except FileNotFoundError:
|
||||
console.print(Panel("[red]❌ nbdev not found. Install with: pip install nbdev[/red]",
|
||||
title="Missing Dependency", border_style="red"))
|
||||
return 1
|
||||
|
||||
|
||||
def cmd_nbdev(args):
|
||||
"""Run nbdev commands for notebook development."""
|
||||
import subprocess
|
||||
|
||||
console.print(Panel("📓 nbdev Notebook Development",
|
||||
title="Notebook Tools", border_style="bright_cyan"))
|
||||
|
||||
if args.export:
|
||||
return cmd_sync(args) # Use the same logic as sync command
|
||||
|
||||
elif args.build_docs:
|
||||
console.print("📚 Building documentation from notebooks...")
|
||||
result = subprocess.run(["nbdev_docs"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Documentation built successfully![/green]",
|
||||
title="Docs Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Docs build failed: {result.stderr}[/red]",
|
||||
title="Docs Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
elif args.test:
|
||||
console.print("🧪 Running notebook tests...")
|
||||
result = subprocess.run(["nbdev_test"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ All notebook tests passed![/green]",
|
||||
title="Test Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Some tests failed: {result.stderr}[/red]",
|
||||
title="Test Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
elif args.clean:
|
||||
console.print("🧹 Cleaning notebook outputs...")
|
||||
result = subprocess.run(["nbdev_clean"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Cleaned successfully![/green]",
|
||||
title="Clean Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Clean failed: {result.stderr}[/red]",
|
||||
title="Clean Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
else:
|
||||
help_text = Text()
|
||||
help_text.append("📓 nbdev Commands:\n\n", style="bold cyan")
|
||||
help_text.append(" tito sync - Export notebooks to Python package\n", style="white")
|
||||
help_text.append(" tito nbdev --export - Same as sync\n", style="white")
|
||||
help_text.append(" tito nbdev --build-docs - Build documentation\n", style="white")
|
||||
help_text.append(" tito nbdev --test - Run notebook tests\n", style="white")
|
||||
help_text.append(" tito nbdev --clean - Clean notebook outputs\n\n", style="white")
|
||||
help_text.append("💡 Development workflow:\n", style="bold yellow")
|
||||
help_text.append(" 1. Work in notebooks/*.ipynb\n", style="dim")
|
||||
help_text.append(" 2. Test interactively in notebook\n", style="dim")
|
||||
help_text.append(" 3. Run: tito sync\n", style="dim")
|
||||
help_text.append(" 4. Run: tito test\n", style="dim")
|
||||
|
||||
console.print(Panel(help_text, title="nbdev Help", border_style="bright_cyan"))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_reset(args):
|
||||
"""Reset tinytorch package to clean state."""
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
console.print(Panel("🔄 Resetting TinyTorch Package",
|
||||
title="Package Reset", border_style="bright_yellow"))
|
||||
|
||||
tinytorch_path = Path("tinytorch")
|
||||
|
||||
if not tinytorch_path.exists():
|
||||
console.print(Panel("[yellow]⚠️ TinyTorch package directory not found. Nothing to reset.[/yellow]",
|
||||
title="Nothing to Reset", border_style="yellow"))
|
||||
return 0
|
||||
|
||||
# Ask for confirmation unless --force is used
|
||||
if not (hasattr(args, 'force') and args.force):
|
||||
console.print("\n[yellow]This will remove all exported Python files from the tinytorch package.[/yellow]")
|
||||
console.print("[yellow]Notebooks in modules/ will be preserved.[/yellow]\n")
|
||||
|
||||
try:
|
||||
response = input("Are you sure you want to reset? (y/N): ").strip().lower()
|
||||
if response not in ['y', 'yes']:
|
||||
console.print(Panel("[cyan]Reset cancelled.[/cyan]",
|
||||
title="Cancelled", border_style="cyan"))
|
||||
return 0
|
||||
except KeyboardInterrupt:
|
||||
console.print(Panel("[cyan]Reset cancelled.[/cyan]",
|
||||
title="Cancelled", border_style="cyan"))
|
||||
return 0
|
||||
|
||||
reset_text = Text()
|
||||
reset_text.append("🗑️ Removing generated files:\n", style="bold red")
|
||||
|
||||
# Remove generated Python files but keep __init__.py files and directory structure
|
||||
files_removed = 0
|
||||
for py_file in tinytorch_path.rglob("*.py"):
|
||||
if py_file.name != "__init__.py":
|
||||
# Check if it's an auto-generated file
|
||||
try:
|
||||
with open(py_file, 'r') as f:
|
||||
first_line = f.readline().strip()
|
||||
if "AUTOGENERATED" in first_line or "_modidx.py" in str(py_file):
|
||||
rel_path = py_file.relative_to(tinytorch_path)
|
||||
reset_text.append(f" 🗑️ tinytorch/{rel_path}\n", style="red")
|
||||
py_file.unlink()
|
||||
files_removed += 1
|
||||
except Exception:
|
||||
# If we can't read the file, skip it for safety
|
||||
pass
|
||||
|
||||
# Remove __pycache__ directories
|
||||
for pycache in tinytorch_path.rglob("__pycache__"):
|
||||
if pycache.is_dir():
|
||||
reset_text.append(f" 🗑️ {pycache}/\n", style="red")
|
||||
shutil.rmtree(pycache)
|
||||
|
||||
# Remove .pytest_cache if it exists
|
||||
pytest_cache = Path(".pytest_cache")
|
||||
if pytest_cache.exists():
|
||||
reset_text.append(f" 🗑️ .pytest_cache/\n", style="red")
|
||||
shutil.rmtree(pytest_cache)
|
||||
|
||||
if files_removed > 0:
|
||||
reset_text.append(f"\n✅ Reset complete! Removed {files_removed} generated files.\n", style="bold green")
|
||||
reset_text.append("\n💡 Next steps:\n", style="bold yellow")
|
||||
reset_text.append(" • Run: tito sync - Re-export notebooks\n", style="white")
|
||||
reset_text.append(" • Run: tito sync --module setup - Export specific module\n", style="white")
|
||||
reset_text.append(" • Run: tito test --all - Test everything\n", style="white")
|
||||
|
||||
console.print(Panel(reset_text, title="Reset Complete", border_style="green"))
|
||||
else:
|
||||
console.print(Panel("[yellow]No generated files found to remove.[/yellow]",
|
||||
title="Nothing to Reset", border_style="yellow"))
|
||||
|
||||
return 0
|
||||
|
||||
def cmd_notebooks(args):
|
||||
"""Build all notebooks from Python files in modules."""
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
|
||||
console.print(Panel("📓 Building Notebooks from Python Files",
|
||||
title="Notebook Generation", border_style="bright_cyan"))
|
||||
|
||||
modules_dir = Path("modules")
|
||||
if not modules_dir.exists():
|
||||
console.print(Panel("[red]❌ modules/ directory not found[/red]",
|
||||
title="Error", border_style="red"))
|
||||
return 1
|
||||
|
||||
# Find all *_dev.py files in modules
|
||||
dev_files = []
|
||||
for module_dir in modules_dir.iterdir():
|
||||
if module_dir.is_dir():
|
||||
dev_py = module_dir / f"{module_dir.name}_dev.py"
|
||||
if dev_py.exists():
|
||||
dev_files.append(dev_py)
|
||||
|
||||
if not dev_files:
|
||||
console.print(Panel("[yellow]⚠️ No *_dev.py files found in modules/[/yellow]",
|
||||
title="Nothing to Convert", border_style="yellow"))
|
||||
return 0
|
||||
|
||||
# Determine what to build
|
||||
if hasattr(args, 'module') and args.module:
|
||||
# Build specific module
|
||||
module_file = modules_dir / args.module / f"{args.module}_dev.py"
|
||||
if not module_file.exists():
|
||||
console.print(Panel(f"[red]❌ Module '{args.module}' not found or no {args.module}_dev.py file[/red]",
|
||||
title="Module Not Found", border_style="red"))
|
||||
return 1
|
||||
dev_files = [module_file]
|
||||
console.print(f"🔄 Building notebook for module: {args.module}")
|
||||
else:
|
||||
# Build all modules
|
||||
console.print(f"🔄 Building notebooks for {len(dev_files)} modules...")
|
||||
|
||||
# Convert each file using the separate tool
|
||||
success_count = 0
|
||||
error_count = 0
|
||||
|
||||
for dev_file in dev_files:
|
||||
try:
|
||||
# Use the separate py_to_notebook.py tool
|
||||
result = subprocess.run([
|
||||
sys.executable, "bin/py_to_notebook.py", str(dev_file)
|
||||
], capture_output=True, text=True)
|
||||
|
||||
module_name = dev_file.parent.name
|
||||
|
||||
if result.returncode == 0:
|
||||
success_count += 1
|
||||
# Extract success message from the tool output
|
||||
output_lines = result.stdout.strip().split('\n')
|
||||
success_msg = output_lines[-1] if output_lines else f"{dev_file.name} → {dev_file.with_suffix('.ipynb').name}"
|
||||
# Clean up the message to remove the ✅ emoji since we'll add our own
|
||||
clean_msg = success_msg.replace('✅ ', '').replace('Converted ', '')
|
||||
console.print(f" ✅ {module_name}: {clean_msg}")
|
||||
else:
|
||||
error_count += 1
|
||||
error_msg = result.stderr.strip() if result.stderr.strip() else "Conversion failed"
|
||||
console.print(f" ❌ {module_name}: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
error_count += 1
|
||||
module_name = dev_file.parent.name
|
||||
console.print(f" ❌ {module_name}: {str(e)}")
|
||||
|
||||
# Summary
|
||||
summary_text = Text()
|
||||
if success_count > 0:
|
||||
summary_text.append(f"✅ Successfully built {success_count} notebook(s)\n", style="bold green")
|
||||
if error_count > 0:
|
||||
summary_text.append(f"❌ Failed to build {error_count} notebook(s)\n", style="bold red")
|
||||
|
||||
if success_count > 0:
|
||||
summary_text.append("\n💡 Next steps:\n", style="bold yellow")
|
||||
summary_text.append(" • Open notebooks with: jupyter lab\n", style="white")
|
||||
summary_text.append(" • Work interactively in the notebooks\n", style="white")
|
||||
summary_text.append(" • Export code with: tito sync\n", style="white")
|
||||
summary_text.append(" • Run tests with: tito test\n", style="white")
|
||||
|
||||
console.print(Panel(summary_text, title="Notebook Generation Complete", border_style="green" if error_count == 0 else "yellow"))
|
||||
|
||||
return 0 if error_count == 0 else 1
|
||||
|
||||
|
||||
def main():
|
||||
"""Main CLI entry point."""
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="tito",
|
||||
description="TinyTorch CLI - Build ML systems from scratch"
|
||||
)
|
||||
parser.add_argument("--version", action="store_true", help="Show version")
|
||||
|
||||
subparsers = parser.add_subparsers(dest="command", help="Available commands")
|
||||
|
||||
# Info command
|
||||
info_parser = subparsers.add_parser("info", help="Show system information")
|
||||
info_parser.add_argument("--hello", action="store_true", help="Show hello message")
|
||||
info_parser.add_argument("--show-architecture", action="store_true", help="Show system architecture")
|
||||
|
||||
# Test command
|
||||
test_parser = subparsers.add_parser("test", help="Run module tests")
|
||||
test_parser.add_argument("--module", help="Module to test")
|
||||
test_parser.add_argument("--all", action="store_true", help="Run all module tests")
|
||||
|
||||
# Submit command
|
||||
submit_parser = subparsers.add_parser("submit", help="Submit module")
|
||||
submit_parser.add_argument("--module", required=True, help="Module to submit")
|
||||
|
||||
# Status command
|
||||
status_parser = subparsers.add_parser("status", help="Check module status")
|
||||
status_parser.add_argument("--module", required=True, help="Module to check")
|
||||
|
||||
# Doctor command
|
||||
doctor_parser = subparsers.add_parser("doctor", help="Run environment diagnosis")
|
||||
|
||||
# Sync command (primary nbdev export)
|
||||
sync_parser = subparsers.add_parser("sync", help="Export notebook code to Python package")
|
||||
sync_parser.add_argument("--module", help="Sync specific module (e.g., setup, tensor)")
|
||||
|
||||
# Reset command
|
||||
reset_parser = subparsers.add_parser("reset", help="Reset tinytorch package to clean state")
|
||||
reset_parser.add_argument("--force", action="store_true", help="Skip confirmation prompt")
|
||||
|
||||
# Notebooks command
|
||||
notebooks_parser = subparsers.add_parser("notebooks", help="Build notebooks from Python files")
|
||||
notebooks_parser.add_argument("--module", help="Build notebook for specific module")
|
||||
|
||||
# nbdev commands
|
||||
nbdev_parser = subparsers.add_parser("nbdev", help="nbdev notebook development commands")
|
||||
nbdev_parser.add_argument("--export", action="store_true", help="Export notebooks to Python package")
|
||||
nbdev_parser.add_argument("--build-docs", action="store_true", help="Build documentation from notebooks")
|
||||
nbdev_parser.add_argument("--test", action="store_true", help="Run notebook tests")
|
||||
nbdev_parser.add_argument("--clean", action="store_true", help="Clean notebook outputs")
|
||||
|
||||
# Jupyter command
|
||||
jupyter_parser = subparsers.add_parser("jupyter", help="Start Jupyter notebook server")
|
||||
jupyter_parser.add_argument("--notebook", action="store_true", help="Start classic notebook")
|
||||
jupyter_parser.add_argument("--lab", action="store_true", help="Start JupyterLab")
|
||||
jupyter_parser.add_argument("--port", type=int, default=8888, help="Port to run on (default: 8888)")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Handle version flag
|
||||
if args.version:
|
||||
cmd_version(args)
|
||||
return 0
|
||||
|
||||
# Environment validation (skip for doctor, info, and reset commands)
|
||||
if args.command not in ["doctor", "info", "reset"] and not validate_environment():
|
||||
return 1
|
||||
|
||||
# Validate test command arguments
|
||||
if args.command == "test":
|
||||
if not args.all and not args.module:
|
||||
error_text = Text()
|
||||
error_text.append("❌ Error: Must specify either --module or --all\n\n", style="bold red")
|
||||
error_text.append("Usage: python bin/tito.py test --module <name> | --all", style="cyan")
|
||||
console.print(Panel(error_text, title="Invalid Arguments", border_style="red"))
|
||||
return 1
|
||||
|
||||
# Handle commands
|
||||
if args.command == "info":
|
||||
cmd_info(args)
|
||||
elif args.command == "test":
|
||||
return cmd_test(args)
|
||||
elif args.command == "submit":
|
||||
cmd_submit(args)
|
||||
elif args.command == "status":
|
||||
cmd_status(args)
|
||||
elif args.command == "doctor":
|
||||
cmd_doctor(args)
|
||||
elif args.command == "sync":
|
||||
return cmd_sync(args)
|
||||
elif args.command == "reset":
|
||||
return cmd_reset(args)
|
||||
elif args.command == "notebooks":
|
||||
return cmd_notebooks(args)
|
||||
elif args.command == "nbdev":
|
||||
return cmd_nbdev(args)
|
||||
elif args.command == "jupyter":
|
||||
return cmd_jupyter(args)
|
||||
else:
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user