github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-05-02 09:51:34 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

PILOT: Implement standardized module README structure (Tensor module)

Browse Source 
New Standard Structure Applied:
✅ 📊 Module Info - Consistent difficulty, time, prerequisites
✅ 🎯 Learning Objectives - Clear, measurable outcomes
✅ 🧠 Build → Use → Understand - Pedagogical framework
✅ 📚 What You'll Build - Concrete code examples
✅ 🚀 Getting Started - Prerequisites check + workflow
✅ 🧪 Testing Your Implementation - Inline + module + manual tests
✅ 🎯 Key Concepts - Real-world connections + core ideas
✅ 🎉 Ready to Build? - Motivational ending + grid cards

Benefits for Students:
- Predictable navigation structure
- Clear learning outcomes upfront
- Concrete examples of what they'll build
- Multiple testing approaches for confidence
- Real-world context for motivation

Benefits for Instructors:
- Professional consistency across modules
- Clear pedagogical progression
- Easy to maintain and update
- Coherent course experience

Next: Review this pilot, then apply to remaining 13 modules
This commit is contained in:
Vijay Janapa Reddi
2025-07-16 01:31:00 -04:00
parent 647b5677b5
commit 9f8a5a8aa3
2 changed files with 191 additions and 319 deletions
Download Patch File Download Diff File Expand all files Collapse all files

254
modules/source/02_tensor/README.md
View File

@@ -11,195 +11,131 @@ Build the foundation of TinyTorch! This module implements the core Tensor class
## 🎯 Learning Objectives
By the end of this module, you will:
- ✅ Understand what tensors are and why they're essential for ML
- ✅ Implement a complete Tensor class with core operations
- ✅ Handle tensor shapes, data types, and memory management
- ✅ Implement element-wise operations and reductions
- ✅ Have a solid foundation for building neural networks
- Understand what tensors are and why they're essential for ML
- Implement a complete Tensor class with core operations
- Handle tensor shapes, data types, and memory management
- Implement element-wise operations and reductions
- Have a solid foundation for building neural networks
## 📋 Module Structure
## 🧠 Build → Use → Understand
1. **Build**: Complete Tensor class with arithmetic operations, shape management, and reductions
2. **Use**: Create tensors, perform operations, and validate with real data
3. **Understand**: How tensors serve as the foundation for all neural network computations
## 📚 What You'll Build
### Core Tensor Class
```python
# Creating tensors
x = Tensor([[1.0, 2.0], [3.0, 4.0]])
y = Tensor([[0.5, 1.5], [2.5, 3.5]])
# Properties
print(x.shape) # (2, 2)
print(x.size) # 4
print(x.dtype) # float64
# Element-wise operations
z = x + y # Addition
w = x * y # Multiplication
p = x ** 2 # Exponentiation
# Shape manipulation
reshaped = x.reshape(4, 1) # (4, 1)
transposed = x.T # (2, 2) transposed
# Reductions
total = x.sum() # Scalar sum
means = x.mean(axis=0) # Mean along axis
```
modules/tensor/
├── README.md # 📖 This file - Module overview
├── tensor_dev.ipynb # 📓 Main development notebook
├── test_tensor.py # 🧪 Automated tests
└── check_tensor.py # ✅ Manual verification (coming soon)
```
### Essential Operations
- **Arithmetic**: Addition, subtraction, multiplication, division, powers
- **Shape management**: Reshape, transpose, broadcasting rules
- **Reductions**: Sum, mean, min, max along any axis
- **Memory handling**: Efficient data storage and copying
## 🚀 Getting Started
### Step 1: Complete Prerequisites
Make sure you've completed the setup module:
### Prerequisites Check
```bash
python bin/tito.py test --module setup # Should pass
tito test --module setup # Should pass ✅
```
### Step 2: Open the Tensor Notebook
### Development Workflow
```bash
# Start from the tensor module directory
cd modules/tensor/
# Navigate to tensor module
cd modules/source/02_tensor
# Open the development notebook
jupyter lab tensor_dev.ipynb
# Open development file
jupyter notebook tensor_dev.ipynb
# OR edit directly: code tensor_dev.py
```
### Step 3: Work Through the Implementation
The notebook guides you through building:
### Step-by-Step Implementation
1. **Basic Tensor class** - Constructor and properties
2. **Shape management** - Understanding tensor dimensions
2. **Shape management** - Understanding tensor dimensions
3. **Arithmetic operations** - Addition, multiplication, etc.
4. **Utility methods** - Reshape, transpose, sum, mean
5. **Error handling** - Robust edge case management
### Step 4: Export and Test
```bash
# Export your tensor implementation
python bin/tito.py sync
# Test your implementation
python bin/tito.py test --module tensor
```
## 📚 What You'll Implement
### Core Tensor Class
You'll build a complete `Tensor` class that supports:
#### 1. Construction and Properties
```python
# Creating tensors
a = Tensor([1, 2, 3]) # 1D tensor
b = Tensor([[1, 2], [3, 4]]) # 2D tensor
c = Tensor(5.0) # Scalar tensor
# Properties
print(a.shape) # (3,)
print(b.size) # 4
print(c.dtype) # float32
```
#### 2. Arithmetic Operations
```python
# Element-wise operations
result = a + b # Addition
result = a * 2 # Scalar multiplication
result = a @ b # Matrix multiplication (bonus)
```
#### 3. Utility Methods
```python
# Shape manipulation
reshaped = b.reshape(1, 4) # Change shape
transposed = b.transpose() # Swap dimensions
# Reductions
total = a.sum() # Sum all elements
mean_val = a.mean() # Average value
max_val = a.max() # Maximum value
```
### Technical Requirements
Your Tensor class must:
- Handle multiple data types (int, float)
- Support N-dimensional arrays
- Implement proper error checking
- Work with NumPy arrays internally
- Export to `tinytorch.core.tensor`
## 🧪 Testing Your Implementation
### Automated Tests
```bash
python bin/tito.py test --module tensor
```
Tests verify:
- ✅ Tensor creation (scalars, vectors, matrices)
- ✅ Property access (shape, size, dtype)
- ✅ Arithmetic operations (all combinations)
- ✅ Utility methods (reshape, transpose, reductions)
- ✅ Error handling (invalid operations)
### Interactive Testing
### Inline Testing
```python
# Test in the notebook or Python REPL
x = Tensor([[1.0, 2.0], [3.0, 4.0]])
print(f"Shape: {x.shape}") # Should be (2, 2)
print(f"Sum: {x.sum()}") # Should be 10.0
```
### Module Tests
```bash
# Export your tensor implementation
tito export
# Test your implementation
tito test --module tensor
```
### Manual Verification
```python
# Create and test tensors
from tinytorch.core.tensor import Tensor
# Create and test tensors
a = Tensor([1, 2, 3])
b = Tensor([[1, 2], [3, 4]])
print(a + 5) # Should work
print(a.sum()) # Should return scalar
x = Tensor([1, 2, 3, 4, 5])
y = Tensor([2, 4, 6, 8, 10])
# Test operations
assert (x + y).data.tolist() == [3, 6, 9, 12, 15]
assert (x * 2).data.tolist() == [2, 4, 6, 8, 10]
print("✅ Basic operations working!")
```
## 🎯 Success Criteria
## 🎯 Key Concepts
Your tensor module is complete when:
### **Tensors as Universal Data Structures**
- **Scalars**: 0-dimensional tensors (single numbers)
- **Vectors**: 1-dimensional tensors (arrays)
- **Matrices**: 2-dimensional tensors (common in ML)
- **Higher dimensions**: Images (3D), video (4D), etc.
1. **All tests pass**: `python bin/tito.py test --module tensor`
2. **Tensor imports correctly**: `from tinytorch.core.tensor import Tensor`
3. **Basic operations work**: Can create tensors and do arithmetic
4. **Properties work**: Shape, size, dtype return correct values
5. **Utilities work**: Reshape, transpose, reductions function properly
### **Why Tensors Matter in ML**
- **Neural networks**: All computations operate on tensors
- **GPU acceleration**: operates on tensor primitives
- **Broadcasting**: Efficient operations across different shapes
- **Vectorization**: Process entire datasets simultaneously
## 💡 Implementation Tips
### **Real-World Connections**
- **PyTorch/TensorFlow**: Your implementation mirrors production frameworks
- **NumPy**: Foundation for scientific computing (we build similar abstractions)
- **Production systems**: Understanding tensors is essential for ML engineering
### Start with the Basics
1. **Simple constructor** - Handle lists and NumPy arrays
2. **Basic properties** - Shape, size, dtype
3. **One operation** - Start with addition
4. **Test frequently** - Verify each feature works
### Design Patterns
```python
class Tensor:
def __init__(self, data, dtype=None):
# Convert input to numpy array
# Store shape, size, dtype
def __add__(self, other):
# Handle tensor + tensor
# Handle tensor + scalar
# Return new Tensor
def sum(self, axis=None):
# Reduce along specified axis
# Return scalar or tensor
```
### Common Challenges
- **Shape compatibility** - Check dimensions for operations
- **Data type handling** - Convert inputs consistently
- **Memory efficiency** - Don't create unnecessary copies
- **Error messages** - Provide helpful debugging info
## 🔧 Advanced Features (Optional)
If you finish early, try implementing:
- **Broadcasting** - Operations on different-shaped tensors
- **Slicing** - `tensor[1:3, :]` syntax
- **In-place operations** - `tensor += other`
- **Matrix multiplication** - `tensor @ other`
## 🚀 Next Steps
Once you complete the tensor module:
1. **Move to Autograd**: `cd modules/autograd/`
2. **Build automatic differentiation**: Enable gradient computation
3. **Combine with tensors**: Make tensors differentiable
4. **Prepare for neural networks**: Ready for the MLP module
## 🔗 Why Tensors Matter
Tensors are the foundation of all ML systems:
- **Neural networks** store weights and activations as tensors
- **Training** computes gradients on tensors
- **Data processing** represents batches as tensors
- **GPU acceleration** operates on tensor primitives
Your tensor implementation will power everything else in TinyTorch!
### **Memory and Performance**
- **Data layout**: How tensors store data efficiently
- **Broadcasting**: Smart operations without data copying
- **View vs Copy**: Understanding memory management
## 🎉 Ready to Build?