mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-05-31 23:35:52 -05:00
Improve 5 C's format: Use integrated code-comment style
Replace verbose bullet format with code-comment approach that: - Integrates concepts directly with implementation preview - Shows exactly where each principle applies in actual code - Feels more natural and less academic - Maintains educational value while respecting student time - Bridges gap between understanding and coding The code-comment style helps students see the connection between concepts and implementation rather than treating them as separate academic content.
This commit is contained in:
@@ -327,47 +327,35 @@ Let's implement our tensor foundation!
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## 🧱 The 5 C's: Understanding Tensor Implementation
|
||||
### Building Your Tensor Implementation
|
||||
|
||||
### 1️⃣ CONCEPT: What is a Tensor?
|
||||
A **Tensor** is a generalized mathematical object that represents scalars (0D), vectors (1D), matrices (2D), and higher-dimensional arrays (3D+). In machine learning, tensors are the fundamental data structure that flows through neural networks, carrying both data and gradients.
|
||||
|
||||
### 2️⃣ CODE STRUCTURE: What We're Building
|
||||
```python
|
||||
# CONCEPT: Tensors are N-dimensional arrays that carry data through neural networks
|
||||
# Think NumPy arrays with ML superpowers - same math, more ML-focused capabilities
|
||||
|
||||
class Tensor:
|
||||
def __init__(self, data): # Create from data
|
||||
self._data = np.array(data) # Store as NumPy array
|
||||
def __init__(self, data):
|
||||
# CORE IMPLEMENTATION: Your tensor needs these fundamentals
|
||||
# - Store N-dimensional data efficiently
|
||||
# - Support math operations: +, *, .sum(), .reshape()
|
||||
# - Handle broadcasting (auto-shape matching)
|
||||
# - Work with multiple data types (float32, int64, etc.)
|
||||
pass
|
||||
|
||||
@property
|
||||
def data(self): # Access underlying data
|
||||
def __add__(self, other):
|
||||
# REAL-WORLD CONNECTION: This mirrors torch.Tensor.__add__()
|
||||
# Every + operation in PyTorch goes through similar logic
|
||||
pass
|
||||
|
||||
def __add__(self, other): # Enable: tensor + tensor
|
||||
def __mul__(self, other): # Enable: tensor * tensor
|
||||
# ... more operations
|
||||
def sum(self):
|
||||
# WHY THIS MATTERS: Sum operations are everywhere in ML
|
||||
# - Loss calculations (MSE, CrossEntropy)
|
||||
# - Gradient accumulation
|
||||
# - Batch processing statistics
|
||||
pass
|
||||
```
|
||||
|
||||
### 3️⃣ CONNECTIONS: Real-World Integration
|
||||
- **PyTorch equivalent**: `torch.Tensor` - same concept, optimized implementation
|
||||
- **TensorFlow equivalent**: `tf.Tensor` - distributed computing focus
|
||||
- **NumPy relationship**: We wrap `np.ndarray` with ML-specific operations
|
||||
- **Production usage**: Every neural network processes millions of tensors per second
|
||||
|
||||
### 4️⃣ CONSTRAINTS: Implementation Requirements
|
||||
- **Data storage**: Must efficiently wrap NumPy arrays
|
||||
- **Type safety**: Handle different data types (float32, int32, etc.)
|
||||
- **Broadcasting**: Automatic shape handling for operations
|
||||
- **Memory efficiency**: Copy data only when necessary
|
||||
- **Operator support**: Enable natural mathematical notation (+, -, *, /)
|
||||
|
||||
### 5️⃣ CONTEXT: Why This Matters in ML Systems
|
||||
Tensors are the **universal language** of machine learning:
|
||||
- **Neural networks**: All computations operate on tensors
|
||||
- **Automatic differentiation**: Gradients flow through tensor operations
|
||||
- **Hardware acceleration**: GPUs are optimized for tensor operations
|
||||
- **Distributed computing**: Tensors can be split across multiple devices
|
||||
- **Research foundation**: Every ML breakthrough builds on tensor mathematics
|
||||
|
||||
**Understanding tensors deeply means understanding the foundation of all modern AI.**
|
||||
*This is the same concept as `torch.Tensor` - you're building the foundation that powers production ML frameworks.*
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": false, "grade_id": "tensor-class", "locked": false, "schema_version": 3, "solution": true, "task": false}
|
||||
|
||||
Reference in New Issue
Block a user