mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-08-02 22:07:52 -05:00
Remove module-level tests directories, keep only main tests/ for exported package validation
- Remove all tests/ directories under modules/source/ - Keep main tests/ directory for testing exported functionality - Update status command to check tests in main tests/ directory - Update documentation to reflect new test structure - Reduce maintenance burden by eliminating duplicate test systems - Focus on inline NBGrader tests for development, main tests for package validation
This commit is contained in:
Vijay Janapa Reddi
@@ -0,0 +1,152 @@
|
||||
# TinyTorch Module Analysis Summary
|
||||
|
||||
## Key Findings
|
||||
|
||||
### ✅ **Excellent Foundation (setup_dev.py)**
|
||||
- **Perfect structure**: Follows explain → code → test → repeat pattern
|
||||
- **Rich scaffolding**: Every TODO has step-by-step guidance
|
||||
- **Immediate feedback**: Tests run after each concept
|
||||
- **Educational flow**: Concepts build logically with real-world connections
|
||||
|
||||
### ⚠️ **Structural Issues (Modules 01-07)**
|
||||
- **Content quality**: Excellent mathematical explanations and implementations
|
||||
- **Testing pattern**: All tests at end instead of progressive testing
|
||||
- **TODO scaffolding**: Generic `NotImplementedError` without guidance
|
||||
- **Student experience**: Large amounts of code before getting feedback
|
||||
|
||||
### ❌ **Missing Modules (08-13)**
|
||||
- **Empty directories**: 5 out of 13 modules are completely empty
|
||||
- **Critical gaps**: Optimizers, training, MLOps missing
|
||||
|
||||
## Immediate Action Items
|
||||
|
||||
### 1. **Fix Testing Pattern (High Priority)**
|
||||
Transform this poor pattern:
|
||||
```python
|
||||
# All implementations
|
||||
def concept_1(): pass
|
||||
def concept_2(): pass
|
||||
def concept_3(): pass
|
||||
|
||||
# All tests at end
|
||||
def test_everything(): pass
|
||||
```
|
||||
|
||||
To this excellent pattern:
|
||||
```python
|
||||
# Concept 1
|
||||
def concept_1(): pass
|
||||
def test_concept_1(): pass
|
||||
print("✅ Concept 1 tests passed!")
|
||||
|
||||
# Concept 2
|
||||
def concept_2(): pass
|
||||
def test_concept_2(): pass
|
||||
print("✅ Concept 2 tests passed!")
|
||||
```
|
||||
|
||||
### 2. **Enhance TODO Blocks (High Priority)**
|
||||
Replace generic todos:
|
||||
```python
|
||||
def add(self, other):
|
||||
"""Add two tensors."""
|
||||
raise NotImplementedError("Student implementation required")
|
||||
```
|
||||
|
||||
With rich scaffolding:
|
||||
```python
|
||||
def add(self, other):
|
||||
"""
|
||||
TODO: Implement tensor addition.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Get numpy data from both tensors
|
||||
2. Use numpy's + operator
|
||||
3. Create new Tensor with result
|
||||
4. Return the new tensor
|
||||
|
||||
EXAMPLE USAGE:
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[5, 6], [7, 8]])
|
||||
result = t1.add(t2) # [[6, 8], [10, 12]]
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- Use self._data + other._data
|
||||
- Wrap result in new Tensor
|
||||
- NumPy handles broadcasting
|
||||
"""
|
||||
```
|
||||
|
||||
### 3. **Module Priority for Fixes**
|
||||
1. **01_tensor** (Highest) - Foundation for everything
|
||||
2. **02_activations** (High) - Used in all networks
|
||||
3. **03_layers** (High) - Core building blocks
|
||||
4. **07_autograd** (High) - Enables training
|
||||
5. **04_networks** (Medium) - Compositions
|
||||
6. **05_cnn** (Medium) - Specialized operations
|
||||
7. **06_dataloader** (Medium) - Data handling
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Phase 1: Transform Existing Modules (Weeks 1-2)
|
||||
For each module (01-07):
|
||||
1. **Identify breakpoints**: Find natural concept boundaries
|
||||
2. **Reorganize structure**: Create Step 1, Step 2, etc. with explanations
|
||||
3. **Add immediate testing**: Test after each major concept
|
||||
4. **Enhance TODO blocks**: Add step-by-step guidance
|
||||
5. **Include success messages**: Clear progress indicators
|
||||
|
||||
### Phase 2: Create Missing Modules (Weeks 3-4)
|
||||
Using the improved structure:
|
||||
- **08_optimizers**: SGD, Adam, learning rate scheduling
|
||||
- **09_training**: Training loops, loss functions, metrics
|
||||
- **10_compression**: Pruning, quantization, knowledge distillation
|
||||
- **11_kernels**: Custom operations, CUDA kernels
|
||||
- **12_benchmarking**: Performance measurement, profiling
|
||||
- **13_mlops**: Model deployment, monitoring, versioning
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Student Experience
|
||||
- **Immediate feedback**: Results after each concept
|
||||
- **Clear guidance**: Step-by-step implementation instructions
|
||||
- **Progressive complexity**: Each step builds on previous success
|
||||
- **Debugging support**: Clear error messages and examples
|
||||
|
||||
### Educational Quality
|
||||
- **Consistent structure**: All modules follow same pattern
|
||||
- **Rich scaffolding**: Every function has detailed guidance
|
||||
- **Real-world connections**: Theory linked to practice
|
||||
- **Integration**: Modules work together seamlessly
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Week 1: Start with Tensor Module
|
||||
1. **Backup current**: Create `tensor_dev_backup.py`
|
||||
2. **Reorganize structure**: Break into progressive steps
|
||||
3. **Add immediate testing**: Test after each operation type
|
||||
4. **Test with students**: Validate improved experience
|
||||
|
||||
### Week 2: Apply to Activations & Layers
|
||||
1. **Apply same pattern**: Use tensor module as template
|
||||
2. **Focus on scaffolding**: Rich TODO blocks
|
||||
3. **Add visualizations**: Where helpful for understanding
|
||||
4. **Progressive testing**: After each activation/layer type
|
||||
|
||||
### Week 3-4: Complete Missing Modules
|
||||
1. **Use proven pattern**: Follow successful structure
|
||||
2. **Real-world focus**: Production-ready implementations
|
||||
3. **Integration testing**: Ensure modules work together
|
||||
4. **Documentation**: Clear learning outcomes
|
||||
|
||||
## Key Principle
|
||||
|
||||
**Always follow: Explain → Code → Test → Repeat**
|
||||
|
||||
This pattern maximizes student success through:
|
||||
- Immediate feedback prevents confusion
|
||||
- Rich scaffolding reduces frustration
|
||||
- Progressive complexity builds confidence
|
||||
- Clear connections show the bigger picture
|
||||
|
||||
The goal is to transform TinyTorch from reference material into a guided learning experience that creates deep understanding of ML systems.
|
||||
@@ -0,0 +1,592 @@
|
||||
great# Module Improvement Guide: From Poor to Excellent Structure
|
||||
|
||||
## Example: Transforming 01_tensor Module
|
||||
|
||||
This guide shows how to transform the tensor module from its current structure to follow the **explain → code → test → repeat** pattern exemplified by `setup_dev.py`.
|
||||
|
||||
## Current Problem Structure
|
||||
|
||||
```python
|
||||
# Current tensor_dev.py structure (POOR)
|
||||
# Lines 1-300: All explanations
|
||||
# Lines 300-700: All implementations
|
||||
# Lines 700-1536: All tests at the end
|
||||
|
||||
class Tensor:
|
||||
def __init__(self):
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def add(self):
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
def multiply(self):
|
||||
raise NotImplementedError("Student implementation required")
|
||||
|
||||
# Much later...
|
||||
def test_tensor_creation_comprehensive():
|
||||
# Tests everything at once
|
||||
pass
|
||||
|
||||
def test_tensor_arithmetic_comprehensive():
|
||||
# Tests everything at once
|
||||
pass
|
||||
```
|
||||
|
||||
## Improved Structure (EXCELLENT)
|
||||
|
||||
```python
|
||||
# Improved tensor_dev.py structure (EXCELLENT)
|
||||
# Following: Explain → Code → Test → Repeat
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1: What is a Tensor?
|
||||
|
||||
### Definition
|
||||
A **tensor** is an N-dimensional array with ML-specific operations.
|
||||
|
||||
### Why Tensors Matter
|
||||
- **Foundation**: Every ML framework uses tensors
|
||||
- **Efficiency**: Vectorized operations are faster
|
||||
- **Flexibility**: Same operations work on scalars, vectors, matrices
|
||||
|
||||
### Real-World Examples
|
||||
```python
|
||||
# Scalar (0D): A single number
|
||||
temperature = Tensor(25.0)
|
||||
|
||||
# Vector (1D): A list of numbers
|
||||
rgb_color = Tensor([255, 128, 0])
|
||||
|
||||
# Matrix (2D): Image pixels
|
||||
image = Tensor([[100, 150], [200, 250]])
|
||||
```
|
||||
|
||||
Let's build this step by step!
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1A: Tensor Creation
|
||||
|
||||
### The Foundation Operation
|
||||
Creating tensors is the first thing you'll do in any ML system. Our Tensor class needs to:
|
||||
1. Accept various input types (lists, numpy arrays, scalars)
|
||||
2. Store data efficiently
|
||||
3. Track shape and type information
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": false, "grade_id": "tensor-creation", "locked": false, "schema_version": 3, "solution": true, "task": false}
|
||||
#| export
|
||||
class Tensor:
|
||||
def __init__(self, data: Union[int, float, List, np.ndarray], dtype: Optional[str] = None):
|
||||
"""
|
||||
Create a tensor from various input types.
|
||||
|
||||
TODO: Implement tensor creation with proper data handling.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Convert input data to numpy array using np.array()
|
||||
2. Handle dtype conversion if specified
|
||||
3. Store the numpy array in self._data
|
||||
4. Validate that data is numeric (not strings, objects, etc.)
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
# From scalar
|
||||
t1 = Tensor(5.0)
|
||||
|
||||
# From list
|
||||
t2 = Tensor([1, 2, 3])
|
||||
|
||||
# From nested list (matrix)
|
||||
t3 = Tensor([[1, 2], [3, 4]])
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- Use np.array(data) to convert input
|
||||
- Check dtype parameter: if provided, use np.array(data, dtype=dtype)
|
||||
- Validate: ensure data is numeric (int, float, complex)
|
||||
- Store in self._data for internal use
|
||||
|
||||
LEARNING CONNECTIONS:
|
||||
- This is like torch.tensor() in PyTorch
|
||||
- Similar to tf.constant() in TensorFlow
|
||||
- Foundation for all other tensor operations
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
if dtype is not None:
|
||||
self._data = np.array(data, dtype=dtype)
|
||||
else:
|
||||
self._data = np.array(data)
|
||||
|
||||
# Validate numeric data
|
||||
if not np.issubdtype(self._data.dtype, np.number):
|
||||
raise ValueError(f"Tensor data must be numeric, got {self._data.dtype}")
|
||||
### END SOLUTION
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Tensor Creation
|
||||
|
||||
Once you implement the `__init__` method above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": true, "grade_id": "test-tensor-creation", "locked": true, "points": 10, "schema_version": 3, "solution": false, "task": false}
|
||||
def test_tensor_creation():
|
||||
"""Test tensor creation with various input types"""
|
||||
print("Testing tensor creation...")
|
||||
|
||||
# Test scalar creation
|
||||
t1 = Tensor(5.0)
|
||||
assert t1._data.shape == (), "Scalar tensor should have empty shape"
|
||||
assert t1._data.item() == 5.0, "Scalar value should be 5.0"
|
||||
|
||||
# Test list creation
|
||||
t2 = Tensor([1, 2, 3])
|
||||
assert t2._data.shape == (3,), "1D tensor should have shape (3,)"
|
||||
assert np.array_equal(t2._data, [1, 2, 3]), "1D tensor values should match"
|
||||
|
||||
# Test matrix creation
|
||||
t3 = Tensor([[1, 2], [3, 4]])
|
||||
assert t3._data.shape == (2, 2), "2D tensor should have shape (2, 2)"
|
||||
assert np.array_equal(t3._data, [[1, 2], [3, 4]]), "2D tensor values should match"
|
||||
|
||||
# Test dtype specification
|
||||
t4 = Tensor([1, 2, 3], dtype='float32')
|
||||
assert t4._data.dtype == np.float32, "Specified dtype should be respected"
|
||||
|
||||
print("✅ Tensor creation tests passed!")
|
||||
print(f"✅ Created tensors: scalar, vector, matrix")
|
||||
print(f"✅ Handled data types correctly")
|
||||
|
||||
# Run the test
|
||||
test_tensor_creation()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 1B: Tensor Properties
|
||||
|
||||
### Essential Information Access
|
||||
Every tensor needs to provide basic information about itself:
|
||||
- **Shape**: Dimensions of the tensor
|
||||
- **Size**: Total number of elements
|
||||
- **Data access**: Get the underlying data
|
||||
|
||||
### Why Properties Matter
|
||||
- **Debugging**: Quickly see tensor dimensions
|
||||
- **Validation**: Check compatibility for operations
|
||||
- **Integration**: Interface with other libraries
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": false, "grade_id": "tensor-properties", "locked": false, "schema_version": 3, "solution": true, "task": false}
|
||||
#| export
|
||||
@property
|
||||
def data(self) -> np.ndarray:
|
||||
"""
|
||||
Get the underlying numpy array data.
|
||||
|
||||
TODO: Implement data property access.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Return self._data directly
|
||||
2. This gives users access to the numpy array
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
print(t.data) # [[1 2]
|
||||
# [3 4]]
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- Simple property: just return self._data
|
||||
- No validation needed here
|
||||
- This is like tensor.numpy() in PyTorch
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
return self._data
|
||||
### END SOLUTION
|
||||
|
||||
@property
|
||||
def shape(self) -> Tuple[int, ...]:
|
||||
"""
|
||||
Get the shape (dimensions) of the tensor.
|
||||
|
||||
TODO: Implement shape property.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Return self._data.shape
|
||||
2. This gives the dimensions as a tuple
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
print(t.shape) # (2, 2)
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- NumPy arrays have a .shape attribute
|
||||
- Return self._data.shape
|
||||
- This is like tensor.shape in PyTorch
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
return self._data.shape
|
||||
### END SOLUTION
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
"""
|
||||
Get the total number of elements in the tensor.
|
||||
|
||||
TODO: Implement size property.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Return self._data.size
|
||||
2. This gives total elements across all dimensions
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
print(t.size) # 4 (2×2 = 4 elements)
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- NumPy arrays have a .size attribute
|
||||
- Return self._data.size
|
||||
- This is like tensor.numel() in PyTorch
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
return self._data.size
|
||||
### END SOLUTION
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Tensor Properties
|
||||
|
||||
Once you implement the properties above, run this cell to test them:
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": true, "grade_id": "test-tensor-properties", "locked": true, "points": 10, "schema_version": 3, "solution": false, "task": false}
|
||||
def test_tensor_properties():
|
||||
"""Test tensor properties: data, shape, size"""
|
||||
print("Testing tensor properties...")
|
||||
|
||||
# Test scalar properties
|
||||
t1 = Tensor(5.0)
|
||||
assert t1.shape == (), "Scalar shape should be empty tuple"
|
||||
assert t1.size == 1, "Scalar size should be 1"
|
||||
assert t1.data.item() == 5.0, "Scalar data should be accessible"
|
||||
|
||||
# Test vector properties
|
||||
t2 = Tensor([1, 2, 3, 4])
|
||||
assert t2.shape == (4,), "Vector shape should be (4,)"
|
||||
assert t2.size == 4, "Vector size should be 4"
|
||||
assert np.array_equal(t2.data, [1, 2, 3, 4]), "Vector data should match"
|
||||
|
||||
# Test matrix properties
|
||||
t3 = Tensor([[1, 2, 3], [4, 5, 6]])
|
||||
assert t3.shape == (2, 3), "Matrix shape should be (2, 3)"
|
||||
assert t3.size == 6, "Matrix size should be 6"
|
||||
assert np.array_equal(t3.data, [[1, 2, 3], [4, 5, 6]]), "Matrix data should match"
|
||||
|
||||
print("✅ Tensor properties tests passed!")
|
||||
print(f"✅ Shape, size, and data access working correctly")
|
||||
|
||||
# Run the test
|
||||
test_tensor_properties()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2: Tensor Arithmetic
|
||||
|
||||
### The Heart of ML: Mathematical Operations
|
||||
Now we implement the core mathematical operations that make ML possible:
|
||||
- **Addition**: Element-wise addition of tensors
|
||||
- **Multiplication**: Element-wise multiplication
|
||||
- **Subtraction**: Element-wise subtraction
|
||||
- **Division**: Element-wise division
|
||||
|
||||
### Why Arithmetic Matters
|
||||
- **Neural networks**: Every layer uses tensor arithmetic
|
||||
- **Optimization**: Gradient updates use arithmetic
|
||||
- **Data processing**: Normalization, scaling, transformations
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2A: Tensor Addition
|
||||
|
||||
### The Foundation Operation
|
||||
Addition is the most basic and important tensor operation:
|
||||
- **Element-wise**: Each element adds to corresponding element
|
||||
- **Broadcasting**: Smaller tensors can add to larger ones
|
||||
- **Commutative**: a + b = b + a
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": false, "grade_id": "tensor-addition", "locked": false, "schema_version": 3, "solution": true, "task": false}
|
||||
#| export
|
||||
def add(self, other: 'Tensor') -> 'Tensor':
|
||||
"""
|
||||
Add two tensors element-wise.
|
||||
|
||||
TODO: Implement tensor addition.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Get the numpy data from both tensors
|
||||
2. Use numpy's + operator for element-wise addition
|
||||
3. Create a new Tensor with the result
|
||||
4. Return the new tensor
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[5, 6], [7, 8]])
|
||||
result = t1.add(t2)
|
||||
print(result.data) # [[6, 8], [10, 12]]
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- Use self._data + other._data for numpy addition
|
||||
- Wrap result in new Tensor: return Tensor(result)
|
||||
- NumPy handles broadcasting automatically
|
||||
- This is like torch.add() in PyTorch
|
||||
|
||||
LEARNING CONNECTIONS:
|
||||
- This is used in every neural network layer
|
||||
- Gradient updates use addition: params = params + learning_rate * gradients
|
||||
- Data preprocessing: adding bias, normalization
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
result = self._data + other._data
|
||||
return Tensor(result)
|
||||
### END SOLUTION
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Tensor Addition
|
||||
|
||||
Once you implement the `add` method above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": true, "grade_id": "test-tensor-addition", "locked": true, "points": 15, "schema_version": 3, "solution": false, "task": false}
|
||||
def test_tensor_addition():
|
||||
"""Test tensor addition with various shapes"""
|
||||
print("Testing tensor addition...")
|
||||
|
||||
# Test same-shape addition
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[5, 6], [7, 8]])
|
||||
result = t1.add(t2)
|
||||
expected = np.array([[6, 8], [10, 12]])
|
||||
assert np.array_equal(result.data, expected), "Same-shape addition failed"
|
||||
|
||||
# Test scalar addition (broadcasting)
|
||||
t3 = Tensor([[1, 2], [3, 4]])
|
||||
t4 = Tensor(10)
|
||||
result = t3.add(t4)
|
||||
expected = np.array([[11, 12], [13, 14]])
|
||||
assert np.array_equal(result.data, expected), "Scalar addition failed"
|
||||
|
||||
# Test vector addition (broadcasting)
|
||||
t5 = Tensor([[1, 2], [3, 4]])
|
||||
t6 = Tensor([10, 20])
|
||||
result = t5.add(t6)
|
||||
expected = np.array([[11, 22], [13, 24]])
|
||||
assert np.array_equal(result.data, expected), "Vector addition failed"
|
||||
|
||||
print("✅ Tensor addition tests passed!")
|
||||
print(f"✅ Same-shape, scalar, and vector addition working")
|
||||
|
||||
# Run the test
|
||||
test_tensor_addition()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Step 2B: Tensor Multiplication
|
||||
|
||||
### Scaling and Element-wise Products
|
||||
Multiplication is crucial for scaling values and computing element-wise products:
|
||||
- **Element-wise**: Each element multiplies with corresponding element
|
||||
- **Broadcasting**: Works with different shapes
|
||||
- **Commutative**: a * b = b * a
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": false, "grade_id": "tensor-multiplication", "locked": false, "schema_version": 3, "solution": true, "task": false}
|
||||
#| export
|
||||
def multiply(self, other: 'Tensor') -> 'Tensor':
|
||||
"""
|
||||
Multiply two tensors element-wise.
|
||||
|
||||
TODO: Implement tensor multiplication.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
1. Get the numpy data from both tensors
|
||||
2. Use numpy's * operator for element-wise multiplication
|
||||
3. Create a new Tensor with the result
|
||||
4. Return the new tensor
|
||||
|
||||
EXAMPLE USAGE:
|
||||
```python
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[2, 3], [4, 5]])
|
||||
result = t1.multiply(t2)
|
||||
print(result.data) # [[2, 6], [12, 20]]
|
||||
```
|
||||
|
||||
IMPLEMENTATION HINTS:
|
||||
- Use self._data * other._data for numpy multiplication
|
||||
- Wrap result in new Tensor: return Tensor(result)
|
||||
- NumPy handles broadcasting automatically
|
||||
- This is like torch.mul() in PyTorch
|
||||
|
||||
LEARNING CONNECTIONS:
|
||||
- Used in activation functions: ReLU masks
|
||||
- Attention mechanisms: attention weights * values
|
||||
- Scaling: learning_rate * gradients
|
||||
"""
|
||||
### BEGIN SOLUTION
|
||||
result = self._data * other._data
|
||||
return Tensor(result)
|
||||
### END SOLUTION
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
### 🧪 Test Your Tensor Multiplication
|
||||
|
||||
Once you implement the `multiply` method above, run this cell to test it:
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": true, "grade_id": "test-tensor-multiplication", "locked": true, "points": 15, "schema_version": 3, "solution": false, "task": false}
|
||||
def test_tensor_multiplication():
|
||||
"""Test tensor multiplication with various shapes"""
|
||||
print("Testing tensor multiplication...")
|
||||
|
||||
# Test same-shape multiplication
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[2, 3], [4, 5]])
|
||||
result = t1.multiply(t2)
|
||||
expected = np.array([[2, 6], [12, 20]])
|
||||
assert np.array_equal(result.data, expected), "Same-shape multiplication failed"
|
||||
|
||||
# Test scalar multiplication (broadcasting)
|
||||
t3 = Tensor([[1, 2], [3, 4]])
|
||||
t4 = Tensor(2)
|
||||
result = t3.multiply(t4)
|
||||
expected = np.array([[2, 4], [6, 8]])
|
||||
assert np.array_equal(result.data, expected), "Scalar multiplication failed"
|
||||
|
||||
# Test vector multiplication (broadcasting)
|
||||
t5 = Tensor([[1, 2], [3, 4]])
|
||||
t6 = Tensor([2, 3])
|
||||
result = t5.multiply(t6)
|
||||
expected = np.array([[2, 6], [6, 12]])
|
||||
assert np.array_equal(result.data, expected), "Vector multiplication failed"
|
||||
|
||||
print("✅ Tensor multiplication tests passed!")
|
||||
print(f"✅ Same-shape, scalar, and vector multiplication working")
|
||||
|
||||
# Run the test
|
||||
test_tensor_multiplication()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## 🎯 Step 3: Integration Test
|
||||
|
||||
### Putting It All Together
|
||||
Now let's test that all our tensor operations work together in realistic scenarios:
|
||||
"""
|
||||
|
||||
# %% nbgrader={"grade": true, "grade_id": "test-tensor-integration", "locked": true, "points": 20, "schema_version": 3, "solution": false, "task": false}
|
||||
def test_tensor_integration():
|
||||
"""Test complete tensor functionality together"""
|
||||
print("Testing tensor integration...")
|
||||
|
||||
# Create test tensors
|
||||
t1 = Tensor([[1, 2], [3, 4]])
|
||||
t2 = Tensor([[2, 1], [1, 2]])
|
||||
scalar = Tensor(0.5)
|
||||
|
||||
# Test chained operations
|
||||
result = t1.add(t2).multiply(scalar)
|
||||
expected = np.array([[1.5, 1.5], [2.0, 3.0]])
|
||||
assert np.array_equal(result.data, expected), "Chained operations failed"
|
||||
|
||||
# Test properties after operations
|
||||
assert result.shape == (2, 2), "Result shape should be (2, 2)"
|
||||
assert result.size == 4, "Result size should be 4"
|
||||
|
||||
# Test with different shapes (broadcasting)
|
||||
t3 = Tensor([1, 2, 3])
|
||||
t4 = Tensor([[1], [2], [3]])
|
||||
result = t3.add(t4)
|
||||
assert result.shape == (3, 3), "Broadcasting result should be (3, 3)"
|
||||
|
||||
print("✅ Tensor integration tests passed!")
|
||||
print(f"✅ All tensor operations work together correctly")
|
||||
print(f"✅ Ready to build neural networks!")
|
||||
|
||||
# Run the integration test
|
||||
test_tensor_integration()
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## 🎯 Module Summary: Tensor Mastery Achieved!
|
||||
|
||||
Congratulations! You've successfully implemented the core Tensor class with:
|
||||
|
||||
### ✅ What You've Built
|
||||
- **Tensor Creation**: Handle various input types (scalars, lists, arrays)
|
||||
- **Properties**: Access shape, size, and data efficiently
|
||||
- **Arithmetic**: Add and multiply tensors with broadcasting support
|
||||
- **Integration**: Operations work together seamlessly
|
||||
|
||||
### ✅ Key Learning Outcomes
|
||||
- **Understanding**: Tensors as the foundation of ML systems
|
||||
- **Implementation**: Built tensor operations from scratch
|
||||
- **Testing**: Comprehensive validation at each step
|
||||
- **Integration**: Chained operations for complex computations
|
||||
|
||||
### ✅ Ready for Next Steps
|
||||
Your tensor implementation is now ready to power:
|
||||
- **Activations**: ReLU, Sigmoid, Tanh will operate on your tensors
|
||||
- **Layers**: Dense layers will use tensor arithmetic
|
||||
- **Networks**: Complete neural networks built on your foundation
|
||||
|
||||
**Next Module**: Activations - Adding nonlinearity to enable complex learning!
|
||||
"""
|
||||
```
|
||||
|
||||
## Key Improvements Demonstrated
|
||||
|
||||
### 1. **Progressive Structure**
|
||||
- Each concept is explained, implemented, and tested before moving on
|
||||
- Students get immediate feedback after each step
|
||||
- No overwhelming amount of code without validation
|
||||
|
||||
### 2. **Rich Scaffolding**
|
||||
- Every TODO has step-by-step implementation guidance
|
||||
- Example usage shows exactly what the function should do
|
||||
- Implementation hints provide specific technical guidance
|
||||
- Learning connections show how concepts fit together
|
||||
|
||||
### 3. **Immediate Testing**
|
||||
- Each function is tested immediately after implementation
|
||||
- Tests provide clear success messages and specific achievements
|
||||
- Integration tests show how concepts work together
|
||||
|
||||
### 4. **Educational Flow**
|
||||
- Concepts build logically from simple to complex
|
||||
- Real-world motivation before technical implementation
|
||||
- Visual examples and concrete cases before abstract theory
|
||||
|
||||
## Implementation Steps for Other Modules
|
||||
|
||||
1. **Identify natural breakpoints** in the current module
|
||||
2. **Reorganize** into Step 1, Step 2, etc. with explanations
|
||||
3. **Add rich TODO blocks** with step-by-step guidance
|
||||
4. **Insert immediate testing** after each major concept
|
||||
5. **Add success messages** and progress indicators
|
||||
6. **Include learning connections** between concepts
|
||||
|
||||
This transformation turns modules from reference material into guided learning experiences that maximize student success through immediate feedback and clear progression.
|
||||
@@ -283,7 +283,7 @@ class TestBasicMLPipeline:
|
||||
### Test Organization
|
||||
```
|
||||
modules/source/{module}/{module}_dev.py # Implementation + comprehensive inline tests
|
||||
tests/test_{module}.py # Module tests with mocks (for grading)
|
||||
tests/test_{module}.py # Package tests for exported functionality
|
||||
tests/integration/ # Cross-module tests with vetted solutions
|
||||
```
|
||||
|
||||
|
||||
@@ -77,7 +77,7 @@ Run the comprehensive test suite using pytest:
|
||||
tito test --module setup
|
||||
|
||||
# Or directly with pytest
|
||||
python -m pytest modules/setup/tests/test_setup.py -v
|
||||
python -m pytest tests/test_setup.py -v
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,337 +0,0 @@
|
||||
"""
|
||||
Test suite for the tensor module.
|
||||
This tests the student implementations to ensure they work correctly.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Import from the main package (rock solid foundation)
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .numpy() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'numpy'):
|
||||
return tensor.numpy()
|
||||
else:
|
||||
return tensor.data
|
||||
|
||||
def safe_item(tensor):
|
||||
"""Get scalar value from tensor, using .item() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'item'):
|
||||
return tensor.item()
|
||||
else:
|
||||
return float(tensor.data)
|
||||
|
||||
class TestTensorCreation:
|
||||
"""Test tensor creation from different data types."""
|
||||
|
||||
def test_scalar_creation(self):
|
||||
"""Test creating tensors from scalars."""
|
||||
# Float scalar
|
||||
t1 = Tensor(5.0)
|
||||
assert t1.shape == ()
|
||||
assert t1.size == 1
|
||||
assert safe_item(t1) == 5.0
|
||||
|
||||
# Integer scalar
|
||||
t2 = Tensor(42)
|
||||
assert t2.shape == ()
|
||||
assert t2.size == 1
|
||||
assert safe_item(t2) == 42.0 # Should convert to float32
|
||||
|
||||
def test_vector_creation(self):
|
||||
"""Test creating 1D tensors."""
|
||||
t = Tensor([1, 2, 3, 4])
|
||||
assert t.shape == (4,)
|
||||
assert t.size == 4
|
||||
assert t.dtype == np.int32 # Integer list defaults to int32
|
||||
np.testing.assert_array_equal(safe_numpy(t), [1, 2, 3, 4])
|
||||
|
||||
def test_matrix_creation(self):
|
||||
"""Test creating 2D tensors."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
assert t.shape == (2, 2)
|
||||
assert t.size == 4
|
||||
expected = np.array([[1.0, 2.0], [3.0, 4.0]], dtype='float32')
|
||||
np.testing.assert_array_equal(safe_numpy(t), expected)
|
||||
|
||||
def test_numpy_array_creation(self):
|
||||
"""Test creating tensors from numpy arrays."""
|
||||
arr = np.array([1, 2, 3], dtype='int32')
|
||||
t = Tensor(arr)
|
||||
assert t.shape == (3,)
|
||||
assert t.dtype in ['int32', 'float32'] # May convert
|
||||
|
||||
def test_dtype_specification(self):
|
||||
"""Test explicit dtype specification."""
|
||||
t = Tensor([1, 2, 3], dtype='int32')
|
||||
assert t.dtype == np.int32
|
||||
|
||||
def test_invalid_data_type(self):
|
||||
"""Test error handling for invalid data types."""
|
||||
with pytest.raises(TypeError):
|
||||
Tensor("invalid")
|
||||
with pytest.raises(TypeError):
|
||||
Tensor({"dict": "invalid"})
|
||||
|
||||
class TestTensorProperties:
|
||||
"""Test tensor properties and methods."""
|
||||
|
||||
def test_shape_property(self):
|
||||
"""Test shape property for different dimensions."""
|
||||
assert Tensor(5).shape == ()
|
||||
assert Tensor([1, 2, 3]).shape == (3,)
|
||||
assert Tensor([[1, 2], [3, 4]]).shape == (2, 2)
|
||||
assert Tensor([[[1]]]).shape == (1, 1, 1)
|
||||
|
||||
def test_size_property(self):
|
||||
"""Test size property."""
|
||||
assert Tensor(5).size == 1
|
||||
assert Tensor([1, 2, 3]).size == 3
|
||||
assert Tensor([[1, 2], [3, 4]]).size == 4
|
||||
assert Tensor([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]).size == 8
|
||||
|
||||
def test_dtype_property(self):
|
||||
"""Test dtype property."""
|
||||
t1 = Tensor(5.0)
|
||||
assert t1.dtype == np.float32
|
||||
|
||||
t2 = Tensor([1, 2, 3], dtype='int32')
|
||||
assert t2.dtype == np.int32
|
||||
|
||||
def test_repr(self):
|
||||
"""Test string representation."""
|
||||
t = Tensor([1, 2, 3])
|
||||
repr_str = repr(t)
|
||||
assert 'Tensor' in repr_str
|
||||
assert 'shape=' in repr_str
|
||||
assert 'dtype=' in repr_str
|
||||
|
||||
class TestArithmeticOperations:
|
||||
"""Test tensor arithmetic operations."""
|
||||
|
||||
def test_tensor_addition(self):
|
||||
"""Test tensor + tensor addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
b = Tensor([4, 5, 6])
|
||||
result = a + b
|
||||
expected = [5.0, 7.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_addition(self):
|
||||
"""Test tensor + scalar addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = a + 10
|
||||
expected = [11.0, 12.0, 13.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_reverse_addition(self):
|
||||
"""Test scalar + tensor addition."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = 10 + a
|
||||
expected = [11.0, 12.0, 13.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_subtraction(self):
|
||||
"""Test tensor - tensor subtraction."""
|
||||
a = Tensor([5, 7, 9])
|
||||
b = Tensor([1, 2, 3])
|
||||
result = a - b
|
||||
expected = [4.0, 5.0, 6.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_subtraction(self):
|
||||
"""Test tensor - scalar subtraction."""
|
||||
a = Tensor([10, 20, 30])
|
||||
result = a - 5
|
||||
expected = [5.0, 15.0, 25.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_multiplication(self):
|
||||
"""Test tensor * tensor multiplication."""
|
||||
a = Tensor([2, 3, 4])
|
||||
b = Tensor([5, 6, 7])
|
||||
result = a * b
|
||||
expected = [10.0, 18.0, 28.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_multiplication(self):
|
||||
"""Test tensor * scalar multiplication."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = a * 3
|
||||
expected = [3.0, 6.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_reverse_multiplication(self):
|
||||
"""Test scalar * tensor multiplication."""
|
||||
a = Tensor([1, 2, 3])
|
||||
result = 3 * a
|
||||
expected = [3.0, 6.0, 9.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_tensor_division(self):
|
||||
"""Test tensor / tensor division."""
|
||||
a = Tensor([6, 8, 10])
|
||||
b = Tensor([2, 4, 5])
|
||||
result = a / b
|
||||
expected = [3.0, 2.0, 2.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_scalar_division(self):
|
||||
"""Test tensor / scalar division."""
|
||||
a = Tensor([6, 8, 10])
|
||||
result = a / 2
|
||||
expected = [3.0, 4.0, 5.0]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
class TestUtilityMethods:
|
||||
"""Test tensor utility methods (stretch goals for students)."""
|
||||
|
||||
def test_reshape(self):
|
||||
"""Test tensor reshaping (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'reshape'):
|
||||
reshaped = t.reshape(4)
|
||||
assert reshaped.shape == (4,)
|
||||
expected = [1.0, 2.0, 3.0, 4.0]
|
||||
np.testing.assert_array_equal(safe_numpy(reshaped), expected)
|
||||
|
||||
# Reshape to 2D
|
||||
reshaped2 = t.reshape(1, 4)
|
||||
assert reshaped2.shape == (1, 4)
|
||||
else:
|
||||
pytest.skip("reshape method not implemented - stretch goal for students")
|
||||
|
||||
def test_transpose(self):
|
||||
"""Test tensor transpose (if implemented)."""
|
||||
t = Tensor([[1, 2, 3], [4, 5, 6]])
|
||||
if hasattr(t, 'transpose'):
|
||||
transposed = t.transpose()
|
||||
assert transposed.shape == (3, 2)
|
||||
expected = [[1.0, 4.0], [2.0, 5.0], [3.0, 6.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(transposed), expected)
|
||||
else:
|
||||
pytest.skip("transpose method not implemented - stretch goal for students")
|
||||
|
||||
def test_sum_all(self):
|
||||
"""Test summing all elements (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'sum'):
|
||||
result = t.sum()
|
||||
expected = 10.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("sum method not implemented - stretch goal for students")
|
||||
|
||||
def test_sum_axis(self):
|
||||
"""Test summing along specific axes (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'sum'):
|
||||
# Sum along axis 0 (columns)
|
||||
sum0 = t.sum(axis=0)
|
||||
expected0 = [4.0, 6.0]
|
||||
np.testing.assert_array_equal(safe_numpy(sum0), expected0)
|
||||
|
||||
# Sum along axis 1 (rows)
|
||||
sum1 = t.sum(axis=1)
|
||||
expected1 = [3.0, 7.0]
|
||||
np.testing.assert_array_equal(safe_numpy(sum1), expected1)
|
||||
else:
|
||||
pytest.skip("sum method not implemented - stretch goal for students")
|
||||
|
||||
def test_mean(self):
|
||||
"""Test mean calculation (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'mean'):
|
||||
result = t.mean()
|
||||
expected = 2.5
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("mean method not implemented - stretch goal for students")
|
||||
|
||||
def test_max(self):
|
||||
"""Test maximum value (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'max'):
|
||||
result = t.max()
|
||||
expected = 4.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("max method not implemented - stretch goal for students")
|
||||
|
||||
def test_min(self):
|
||||
"""Test minimum value (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'min'):
|
||||
result = t.min()
|
||||
expected = 1.0
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("min method not implemented - stretch goal for students")
|
||||
|
||||
def test_item_scalar(self):
|
||||
"""Test converting single-element tensor to scalar (if implemented)."""
|
||||
t = Tensor(42.0)
|
||||
if hasattr(t, 'item'):
|
||||
assert t.item() == 42.0
|
||||
else:
|
||||
pytest.skip("item method not implemented - stretch goal for students")
|
||||
|
||||
def test_item_error(self):
|
||||
"""Test item() error for multi-element tensors (if implemented)."""
|
||||
t = Tensor([1, 2, 3])
|
||||
if hasattr(t, 'item'):
|
||||
with pytest.raises(ValueError):
|
||||
t.item()
|
||||
else:
|
||||
pytest.skip("item method not implemented - stretch goal for students")
|
||||
|
||||
def test_numpy_conversion(self):
|
||||
"""Test converting tensor to numpy array (if implemented)."""
|
||||
t = Tensor([[1, 2], [3, 4]])
|
||||
if hasattr(t, 'numpy'):
|
||||
arr = t.numpy()
|
||||
assert isinstance(arr, np.ndarray)
|
||||
expected = [[1.0, 2.0], [3.0, 4.0]]
|
||||
np.testing.assert_array_equal(arr, expected)
|
||||
else:
|
||||
pytest.skip("numpy method not implemented - stretch goal for students")
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and error handling."""
|
||||
|
||||
def test_empty_list(self):
|
||||
"""Test creating tensor from empty list."""
|
||||
t = Tensor([])
|
||||
assert t.shape == (0,)
|
||||
assert t.size == 0
|
||||
|
||||
def test_mixed_operations(self):
|
||||
"""Test combining different operations."""
|
||||
a = Tensor([[1, 2], [3, 4]])
|
||||
b = Tensor([[2, 2], [2, 2]])
|
||||
|
||||
# Complex expression
|
||||
result = (a + b) * 2 - 1
|
||||
expected = [[5.0, 7.0], [9.0, 11.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(result), expected)
|
||||
|
||||
def test_chained_operations(self):
|
||||
"""Test chaining multiple operations (if methods implemented)."""
|
||||
t = Tensor([[1, 2, 3], [4, 5, 6]])
|
||||
if hasattr(t, 'sum') and hasattr(t, 'mean'):
|
||||
result = t.sum(axis=1).mean()
|
||||
expected = 10.5 # (6 + 15) / 2
|
||||
assert abs(safe_item(result) - expected) < 1e-6
|
||||
else:
|
||||
pytest.skip("Advanced methods not implemented - stretch goal for students")
|
||||
|
||||
def run_tensor_tests():
|
||||
"""Run all tensor tests."""
|
||||
pytest.main([__file__, "-v"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_tensor_tests()
|
||||
@@ -230,11 +230,11 @@ Once you implement the ReLU forward method above, run this cell to test it:
|
||||
def test_relu_activation():
|
||||
"""Test ReLU activation function"""
|
||||
print("Testing ReLU activation...")
|
||||
|
||||
# Create ReLU instance
|
||||
relu = ReLU()
|
||||
|
||||
# Test with mixed positive/negative values
|
||||
|
||||
# Create ReLU instance
|
||||
relu = ReLU()
|
||||
|
||||
# Test with mixed positive/negative values
|
||||
test_input = Tensor([[-2, -1, 0, 1, 2]])
|
||||
result = relu(test_input)
|
||||
expected = np.array([[0, 0, 0, 1, 2]])
|
||||
@@ -368,10 +368,10 @@ Once you implement the Sigmoid forward method above, run this cell to test it:
|
||||
def test_sigmoid_activation():
|
||||
"""Test Sigmoid activation function"""
|
||||
print("Testing Sigmoid activation...")
|
||||
|
||||
# Create Sigmoid instance
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
|
||||
# Create Sigmoid instance
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test with known values
|
||||
test_input = Tensor([[0]])
|
||||
result = sigmoid(test_input)
|
||||
@@ -514,10 +514,10 @@ Once you implement the Tanh forward method above, run this cell to test it:
|
||||
def test_tanh_activation():
|
||||
"""Test Tanh activation function"""
|
||||
print("Testing Tanh activation...")
|
||||
|
||||
# Create Tanh instance
|
||||
tanh = Tanh()
|
||||
|
||||
|
||||
# Create Tanh instance
|
||||
tanh = Tanh()
|
||||
|
||||
# Test with zero (should be 0)
|
||||
test_input = Tensor([[0]])
|
||||
result = tanh(test_input)
|
||||
@@ -676,10 +676,10 @@ Once you implement the Softmax forward method above, run this cell to test it:
|
||||
def test_softmax_activation():
|
||||
"""Test Softmax activation function"""
|
||||
print("Testing Softmax activation...")
|
||||
|
||||
# Create Softmax instance
|
||||
softmax = Softmax()
|
||||
|
||||
|
||||
# Create Softmax instance
|
||||
softmax = Softmax()
|
||||
|
||||
# Test with simple input
|
||||
test_input = Tensor([[1, 2, 3]])
|
||||
result = softmax(test_input)
|
||||
@@ -718,8 +718,8 @@ def test_softmax_activation():
|
||||
|
||||
large_sum = np.sum(large_result.data)
|
||||
assert abs(large_sum - 1.0) < 1e-6, "Large values should still sum to 1"
|
||||
|
||||
# Test shape preservation
|
||||
|
||||
# Test shape preservation
|
||||
assert batch_result.shape == batch_input.shape, "Softmax should preserve shape"
|
||||
|
||||
print("✅ Softmax activation tests passed!")
|
||||
@@ -751,9 +751,9 @@ def test_activations_integration():
|
||||
print("Testing activation functions integration...")
|
||||
|
||||
# Create instances of all activation functions
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
softmax = Softmax()
|
||||
|
||||
# Test data: simulating neural network layer outputs
|
||||
@@ -791,7 +791,7 @@ def test_activations_integration():
|
||||
# Test Softmax properties
|
||||
softmax_sum = np.sum(softmax_result.data)
|
||||
assert abs(softmax_sum - 1.0) < 1e-6, "Softmax outputs should sum to 1"
|
||||
|
||||
|
||||
# Test chaining activations (realistic neural network scenario)
|
||||
# Hidden layer with ReLU
|
||||
hidden_output = relu(test_data)
|
||||
@@ -815,8 +815,8 @@ def test_activations_integration():
|
||||
])
|
||||
|
||||
batch_softmax = softmax(batch_data)
|
||||
|
||||
# Each row should sum to 1
|
||||
|
||||
# Each row should sum to 1
|
||||
for i in range(batch_data.shape[0]):
|
||||
row_sum = np.sum(batch_softmax.data[i])
|
||||
assert abs(row_sum - 1.0) < 1e-6, f"Batch row {i} should sum to 1"
|
||||
|
||||
@@ -1,332 +0,0 @@
|
||||
"""
|
||||
Test suite for the activations module.
|
||||
This tests the student implementations to ensure they work correctly.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Import from the main package (rock solid foundation)
|
||||
from tinytorch.core.tensor import Tensor
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh, Softmax
|
||||
|
||||
|
||||
class TestReLU:
|
||||
"""Test the ReLU activation function."""
|
||||
|
||||
def test_relu_basic_functionality(self):
|
||||
"""Test basic ReLU behavior: max(0, x)"""
|
||||
relu = ReLU()
|
||||
|
||||
# Test mixed positive/negative values
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
y = relu(x)
|
||||
expected = np.array([[0.0, 0.0, 0.0, 1.0, 2.0]])
|
||||
|
||||
assert np.allclose(y.data, expected), f"Expected {expected}, got {y.data}"
|
||||
|
||||
def test_relu_all_positive(self):
|
||||
"""Test ReLU with all positive values (should be unchanged)"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[1.0, 2.5, 3.7, 10.0]])
|
||||
y = relu(x)
|
||||
|
||||
assert np.allclose(y.data, x.data), "ReLU should preserve positive values"
|
||||
|
||||
def test_relu_all_negative(self):
|
||||
"""Test ReLU with all negative values (should be zeros)"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[-1.0, -2.5, -3.7, -10.0]])
|
||||
y = relu(x)
|
||||
expected = np.zeros_like(x.data)
|
||||
|
||||
assert np.allclose(y.data, expected), "ReLU should zero out negative values"
|
||||
|
||||
def test_relu_zero_input(self):
|
||||
"""Test ReLU with zero input"""
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[0.0]])
|
||||
y = relu(x)
|
||||
|
||||
assert y.data[0, 0] == 0.0, "ReLU(0) should be 0"
|
||||
|
||||
def test_relu_shape_preservation(self):
|
||||
"""Test that ReLU preserves tensor shape"""
|
||||
relu = ReLU()
|
||||
|
||||
# Test different shapes
|
||||
shapes = [(1, 5), (2, 3), (4, 1), (3, 3)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = relu(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_relu_callable(self):
|
||||
"""Test that ReLU can be called directly"""
|
||||
relu = ReLU()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = relu(x)
|
||||
y2 = relu.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestSigmoid:
|
||||
"""Test the Sigmoid activation function."""
|
||||
|
||||
def test_sigmoid_basic_functionality(self):
|
||||
"""Test basic Sigmoid behavior"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test known values
|
||||
x = Tensor([[0.0]])
|
||||
y = sigmoid(x)
|
||||
assert abs(y.data[0, 0] - 0.5) < 1e-6, "Sigmoid(0) should be 0.5"
|
||||
|
||||
def test_sigmoid_range(self):
|
||||
"""Test that Sigmoid outputs are in (0, 1)"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test wide range of inputs
|
||||
x = Tensor([[-10.0, -5.0, -1.0, 0.0, 1.0, 5.0, 10.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
assert np.all(y.data > 0), "Sigmoid outputs should be > 0"
|
||||
assert np.all(y.data < 1), "Sigmoid outputs should be < 1"
|
||||
|
||||
def test_sigmoid_numerical_stability(self):
|
||||
"""Test Sigmoid with extreme values (numerical stability)"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Test extreme values that could cause overflow
|
||||
x = Tensor([[-100.0, -50.0, 50.0, 100.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Should not contain NaN or inf
|
||||
assert not np.any(np.isnan(y.data)), "Sigmoid should not produce NaN"
|
||||
assert not np.any(np.isinf(y.data)), "Sigmoid should not produce inf"
|
||||
|
||||
# Should be close to 0 for very negative, close to 1 for very positive
|
||||
assert y.data[0, 0] < 1e-10, "Sigmoid(-100) should be very close to 0"
|
||||
assert y.data[0, 1] < 1e-10, "Sigmoid(-50) should be very close to 0"
|
||||
assert y.data[0, 2] > 1 - 1e-10, "Sigmoid(50) should be very close to 1"
|
||||
assert y.data[0, 3] > 1 - 1e-10, "Sigmoid(100) should be very close to 1"
|
||||
|
||||
def test_sigmoid_monotonicity(self):
|
||||
"""Test that Sigmoid is monotonically increasing"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
x = Tensor([[-3.0, -1.0, 0.0, 1.0, 3.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Check that outputs are increasing
|
||||
for i in range(len(y.data[0]) - 1):
|
||||
assert y.data[0, i] < y.data[0, i + 1], "Sigmoid should be monotonically increasing"
|
||||
|
||||
def test_sigmoid_shape_preservation(self):
|
||||
"""Test that Sigmoid preserves tensor shape"""
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
shapes = [(1, 5), (2, 3), (4, 1)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = sigmoid(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_sigmoid_callable(self):
|
||||
"""Test that Sigmoid can be called directly"""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = sigmoid(x)
|
||||
y2 = sigmoid.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestTanh:
|
||||
"""Test the Tanh activation function."""
|
||||
|
||||
def test_tanh_basic_functionality(self):
|
||||
"""Test basic Tanh behavior"""
|
||||
tanh = Tanh()
|
||||
|
||||
# Test known values
|
||||
x = Tensor([[0.0]])
|
||||
y = tanh(x)
|
||||
assert abs(y.data[0, 0] - 0.0) < 1e-6, "Tanh(0) should be 0"
|
||||
|
||||
def test_tanh_range(self):
|
||||
"""Test that Tanh outputs are in [-1, 1]"""
|
||||
tanh = Tanh()
|
||||
|
||||
# Test wide range of inputs
|
||||
x = Tensor([[-10.0, -5.0, -1.0, 0.0, 1.0, 5.0, 10.0]])
|
||||
y = tanh(x)
|
||||
|
||||
assert np.all(y.data >= -1), "Tanh outputs should be >= -1"
|
||||
assert np.all(y.data <= 1), "Tanh outputs should be <= 1"
|
||||
|
||||
def test_tanh_symmetry(self):
|
||||
"""Test that Tanh is symmetric: tanh(-x) = -tanh(x)"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
x_neg = Tensor([[-1.0, -2.0, -3.0]])
|
||||
|
||||
y_pos = tanh(x)
|
||||
y_neg = tanh(x_neg)
|
||||
|
||||
assert np.allclose(y_neg.data, -y_pos.data), "Tanh should be symmetric"
|
||||
|
||||
def test_tanh_monotonicity(self):
|
||||
"""Test that Tanh is monotonically increasing"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-3.0, -1.0, 0.0, 1.0, 3.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Check that outputs are increasing
|
||||
for i in range(len(y.data[0]) - 1):
|
||||
assert y.data[0, i] < y.data[0, i + 1], "Tanh should be monotonically increasing"
|
||||
|
||||
def test_tanh_extreme_values(self):
|
||||
"""Test Tanh with extreme values"""
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-100.0, 100.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Should be close to -1 and 1 respectively
|
||||
assert abs(y.data[0, 0] - (-1.0)) < 1e-10, "Tanh(-100) should be very close to -1"
|
||||
assert abs(y.data[0, 1] - 1.0) < 1e-10, "Tanh(100) should be very close to 1"
|
||||
|
||||
def test_tanh_shape_preservation(self):
|
||||
"""Test that Tanh preserves tensor shape"""
|
||||
tanh = Tanh()
|
||||
|
||||
shapes = [(1, 5), (2, 3), (4, 1)]
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.randn(*shape))
|
||||
y = tanh(x)
|
||||
assert y.shape == x.shape, f"Shape mismatch: expected {x.shape}, got {y.shape}"
|
||||
|
||||
def test_tanh_callable(self):
|
||||
"""Test that Tanh can be called directly"""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = tanh(x)
|
||||
y2 = tanh.forward(x)
|
||||
|
||||
assert np.allclose(y1.data, y2.data), "Direct call should match forward method"
|
||||
|
||||
|
||||
class TestActivationComparison:
|
||||
"""Test interactions and comparisons between activation functions."""
|
||||
|
||||
def test_activation_consistency(self):
|
||||
"""Test that all activations work with the same input"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
|
||||
# All should process without error
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
# All should preserve shape
|
||||
assert y_relu.shape == x.shape
|
||||
assert y_sigmoid.shape == x.shape
|
||||
assert y_tanh.shape == x.shape
|
||||
|
||||
def test_activation_ranges(self):
|
||||
"""Test that activations have expected output ranges"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[-5.0, -2.0, 0.0, 2.0, 5.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
# ReLU: [0, inf)
|
||||
assert np.all(y_relu.data >= 0), "ReLU should be non-negative"
|
||||
|
||||
# Sigmoid: (0, 1)
|
||||
assert np.all(y_sigmoid.data > 0), "Sigmoid should be positive"
|
||||
assert np.all(y_sigmoid.data < 1), "Sigmoid should be less than 1"
|
||||
|
||||
# Tanh: (-1, 1)
|
||||
assert np.all(y_tanh.data > -1), "Tanh should be greater than -1"
|
||||
assert np.all(y_tanh.data < 1), "Tanh should be less than 1"
|
||||
|
||||
|
||||
# Integration tests with edge cases
|
||||
class TestActivationEdgeCases:
|
||||
"""Test edge cases and boundary conditions."""
|
||||
|
||||
def test_zero_tensor(self):
|
||||
"""Test all activations with zero tensor"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[0.0, 0.0, 0.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert np.allclose(y_relu.data, [0.0, 0.0, 0.0]), "ReLU(0) should be 0"
|
||||
assert np.allclose(y_sigmoid.data, [0.5, 0.5, 0.5]), "Sigmoid(0) should be 0.5"
|
||||
assert np.allclose(y_tanh.data, [0.0, 0.0, 0.0]), "Tanh(0) should be 0"
|
||||
|
||||
def test_single_element_tensor(self):
|
||||
"""Test all activations with single element tensor"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
x = Tensor([[1.0]])
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert y_relu.shape == (1, 1)
|
||||
assert y_sigmoid.shape == (1, 1)
|
||||
assert y_tanh.shape == (1, 1)
|
||||
|
||||
def test_large_tensor(self):
|
||||
"""Test activations with larger tensors"""
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
|
||||
# Create a 10x10 tensor
|
||||
x = Tensor(np.random.randn(10, 10))
|
||||
|
||||
y_relu = relu(x)
|
||||
y_sigmoid = sigmoid(x)
|
||||
y_tanh = tanh(x)
|
||||
|
||||
assert y_relu.shape == (10, 10)
|
||||
assert y_sigmoid.shape == (10, 10)
|
||||
assert y_tanh.shape == (10, 10)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run tests with pytest
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -46,8 +46,8 @@ except ImportError:
|
||||
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '01_tensor'))
|
||||
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '02_activations'))
|
||||
try:
|
||||
from tensor_dev import Tensor
|
||||
from activations_dev import ReLU, Sigmoid, Tanh, Softmax
|
||||
from tensor_dev import Tensor
|
||||
from activations_dev import ReLU, Sigmoid, Tanh, Softmax
|
||||
except ImportError:
|
||||
# If the local modules are not available, use relative imports
|
||||
from ..tensor.tensor_dev import Tensor
|
||||
@@ -188,7 +188,7 @@ def matmul_naive(A: np.ndarray, B: np.ndarray) -> np.ndarray:
|
||||
Naive matrix multiplication using explicit for-loops.
|
||||
|
||||
This helps you understand what matrix multiplication really does!
|
||||
|
||||
|
||||
TODO: Implement matrix multiplication using three nested for-loops.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
@@ -259,8 +259,8 @@ Once you implement the `matmul_naive` function above, run this cell to test it:
|
||||
def test_matrix_multiplication():
|
||||
"""Test matrix multiplication implementation"""
|
||||
print("Testing matrix multiplication...")
|
||||
|
||||
# Test simple 2x2 case
|
||||
|
||||
# Test simple 2x2 case
|
||||
A = np.array([[1, 2], [3, 4]], dtype=np.float32)
|
||||
B = np.array([[5, 6], [7, 8]], dtype=np.float32)
|
||||
|
||||
@@ -272,8 +272,8 @@ def test_matrix_multiplication():
|
||||
# Compare with NumPy
|
||||
numpy_result = A @ B
|
||||
assert np.allclose(result, numpy_result), f"Doesn't match NumPy: got {result}, expected {numpy_result}"
|
||||
|
||||
# Test different shapes
|
||||
|
||||
# Test different shapes
|
||||
A2 = np.array([[1, 2, 3]], dtype=np.float32) # 1x3
|
||||
B2 = np.array([[4], [5], [6]], dtype=np.float32) # 3x1
|
||||
result2 = matmul_naive(A2, B2)
|
||||
@@ -423,7 +423,7 @@ class Dense:
|
||||
else:
|
||||
self.bias = None
|
||||
### END SOLUTION
|
||||
|
||||
|
||||
def forward(self, x: Tensor) -> Tensor:
|
||||
"""
|
||||
Forward pass through the Dense layer.
|
||||
@@ -472,7 +472,7 @@ class Dense:
|
||||
|
||||
return Tensor(linear_output)
|
||||
### END SOLUTION
|
||||
|
||||
|
||||
def __call__(self, x: Tensor) -> Tensor:
|
||||
"""Make the layer callable: layer(x) instead of layer.forward(x)"""
|
||||
return self.forward(x)
|
||||
@@ -509,8 +509,8 @@ def test_dense_layer():
|
||||
batch_output = layer(batch_input)
|
||||
|
||||
assert batch_output.shape == (2, 2), f"Batch output shape should be (2, 2), got {batch_output.shape}"
|
||||
|
||||
# Test without bias
|
||||
|
||||
# Test without bias
|
||||
no_bias_layer = Dense(input_size=3, output_size=2, use_bias=False)
|
||||
assert no_bias_layer.bias is None, "Layer without bias should have None bias"
|
||||
|
||||
@@ -538,7 +538,7 @@ def test_dense_layer():
|
||||
scaled_output = layer(scaled_input)
|
||||
|
||||
# Due to bias, this won't be exactly 2*output, but the linear part should scale
|
||||
print("✅ Dense layer tests passed!")
|
||||
print("✅ Dense layer tests passed!")
|
||||
print(f"✅ Correct weight and bias initialization")
|
||||
print(f"✅ Forward pass produces correct shapes")
|
||||
print(f"✅ Batch processing works correctly")
|
||||
@@ -582,7 +582,7 @@ def test_layer_activation_integration():
|
||||
|
||||
# Create layer and activation functions
|
||||
layer = Dense(input_size=4, output_size=3)
|
||||
relu = ReLU()
|
||||
relu = ReLU()
|
||||
sigmoid = Sigmoid()
|
||||
tanh = Tanh()
|
||||
softmax = Softmax()
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,336 +0,0 @@
|
||||
"""
|
||||
Test suite for the layers module.
|
||||
This tests the student implementations to ensure they work correctly.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Import from the main package (rock solid foundation)
|
||||
from tinytorch.core.tensor import Tensor
|
||||
from tinytorch.core.layers import Dense
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .numpy() if available, otherwise .data"""
|
||||
if hasattr(tensor, 'numpy'):
|
||||
return tensor.numpy()
|
||||
else:
|
||||
return tensor.data
|
||||
|
||||
class TestDenseLayer:
|
||||
"""Test Dense (Linear) layer functionality."""
|
||||
|
||||
def test_dense_creation(self):
|
||||
"""Test creating Dense layers with different configurations."""
|
||||
# Basic dense layer
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
assert layer.input_size == 3
|
||||
assert layer.output_size == 2
|
||||
assert layer.use_bias == True
|
||||
assert layer.weights.shape == (3, 2)
|
||||
assert layer.bias.shape == (2,)
|
||||
|
||||
# Dense layer without bias
|
||||
layer_no_bias = Dense(input_size=4, output_size=3, use_bias=False)
|
||||
assert layer_no_bias.use_bias == False
|
||||
assert layer_no_bias.bias is None
|
||||
|
||||
def test_dense_forward_single(self):
|
||||
"""Test Dense layer forward pass with single input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Single input
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 2)
|
||||
assert isinstance(y, Tensor)
|
||||
|
||||
def test_dense_forward_batch(self):
|
||||
"""Test Dense layer forward pass with batch input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Batch input
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (2, 2)
|
||||
assert isinstance(y, Tensor)
|
||||
|
||||
def test_dense_no_bias(self):
|
||||
"""Test Dense layer without bias."""
|
||||
layer = Dense(input_size=2, output_size=1, use_bias=False)
|
||||
|
||||
x = Tensor([[1.0, 2.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 1)
|
||||
# Should be just matrix multiplication without bias
|
||||
expected = safe_numpy(x) @ safe_numpy(layer.weights)
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), expected)
|
||||
|
||||
def test_dense_callable(self):
|
||||
"""Test that Dense layer is callable."""
|
||||
layer = Dense(input_size=2, output_size=1)
|
||||
x = Tensor([[1.0, 2.0]])
|
||||
|
||||
# Both should work
|
||||
y1 = layer.forward(x)
|
||||
y2 = layer(x)
|
||||
|
||||
np.testing.assert_array_equal(safe_numpy(y1), safe_numpy(y2))
|
||||
|
||||
class TestActivationFunctions:
|
||||
"""Test activation function implementations."""
|
||||
|
||||
def test_relu_basic(self):
|
||||
"""Test ReLU activation function."""
|
||||
relu = ReLU()
|
||||
x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
|
||||
y = relu(x)
|
||||
|
||||
expected = [[0.0, 0.0, 0.0, 1.0, 2.0]]
|
||||
np.testing.assert_array_equal(safe_numpy(y), expected)
|
||||
|
||||
def test_relu_callable(self):
|
||||
"""Test that ReLU is callable."""
|
||||
relu = ReLU()
|
||||
x = Tensor([[1.0, -1.0]])
|
||||
|
||||
y1 = relu.forward(x)
|
||||
y2 = relu(x)
|
||||
|
||||
np.testing.assert_array_equal(safe_numpy(y1), safe_numpy(y2))
|
||||
|
||||
def test_sigmoid_basic(self):
|
||||
"""Test Sigmoid activation function."""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[0.0]]) # sigmoid(0) = 0.5
|
||||
y = sigmoid(x)
|
||||
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), [[0.5]])
|
||||
|
||||
def test_sigmoid_range(self):
|
||||
"""Test Sigmoid output range."""
|
||||
sigmoid = Sigmoid()
|
||||
x = Tensor([[-10.0, 0.0, 10.0]])
|
||||
y = sigmoid(x)
|
||||
|
||||
# Should be in range [0, 1] - use reasonable bounds
|
||||
assert np.all(safe_numpy(y) >= 0)
|
||||
assert np.all(safe_numpy(y) <= 1)
|
||||
# Check that extreme values are close to bounds
|
||||
assert safe_numpy(y)[0][0] < 0.01 # Very small for -10
|
||||
assert safe_numpy(y)[0][2] > 0.99 # Very large for 10
|
||||
|
||||
def test_tanh_basic(self):
|
||||
"""Test Tanh activation function."""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[0.0]]) # tanh(0) = 0
|
||||
y = tanh(x)
|
||||
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), [[0.0]])
|
||||
|
||||
def test_tanh_range(self):
|
||||
"""Test Tanh output range."""
|
||||
tanh = Tanh()
|
||||
x = Tensor([[-10.0, 0.0, 10.0]])
|
||||
y = tanh(x)
|
||||
|
||||
# Should be in range [-1, 1] - use reasonable bounds
|
||||
assert np.all(safe_numpy(y) >= -1)
|
||||
assert np.all(safe_numpy(y) <= 1)
|
||||
# Check that extreme values are close to bounds
|
||||
assert safe_numpy(y)[0][0] < -0.99 # Very negative for -10
|
||||
assert safe_numpy(y)[0][2] > 0.99 # Very positive for 10
|
||||
|
||||
class TestLayerComposition:
|
||||
"""Test composing layers into neural networks."""
|
||||
|
||||
def test_simple_network(self):
|
||||
"""Test a simple 2-layer network."""
|
||||
# 3 → 4 → 2 network
|
||||
layer1 = Dense(input_size=3, output_size=4)
|
||||
relu = ReLU()
|
||||
layer2 = Dense(input_size=4, output_size=2)
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
# Forward pass
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
h1 = layer1(x)
|
||||
h1_activated = relu(h1)
|
||||
h2 = layer2(h1_activated)
|
||||
output = sigmoid(h2)
|
||||
|
||||
assert h1.shape == (1, 4)
|
||||
assert h1_activated.shape == (1, 4)
|
||||
assert h2.shape == (1, 2)
|
||||
assert output.shape == (1, 2)
|
||||
|
||||
# Output should be in sigmoid range
|
||||
assert np.all(safe_numpy(output) >= 0)
|
||||
assert np.all(safe_numpy(output) <= 1)
|
||||
|
||||
def test_batch_network(self):
|
||||
"""Test network with batch processing."""
|
||||
layer1 = Dense(input_size=2, output_size=3)
|
||||
relu = ReLU()
|
||||
layer2 = Dense(input_size=3, output_size=1)
|
||||
|
||||
# Batch of 4 examples
|
||||
x = Tensor([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0], [7.0, 8.0]])
|
||||
|
||||
h1 = layer1(x)
|
||||
h1_activated = relu(h1)
|
||||
output = layer2(h1_activated)
|
||||
|
||||
assert output.shape == (4, 1)
|
||||
|
||||
def test_deep_network(self):
|
||||
"""Test deeper network composition."""
|
||||
# 5-layer network
|
||||
layers = [
|
||||
Dense(input_size=10, output_size=8),
|
||||
ReLU(),
|
||||
Dense(input_size=8, output_size=6),
|
||||
ReLU(),
|
||||
Dense(input_size=6, output_size=4),
|
||||
ReLU(),
|
||||
Dense(input_size=4, output_size=2),
|
||||
Sigmoid()
|
||||
]
|
||||
|
||||
x = Tensor([[1.0] * 10]) # 10 features
|
||||
|
||||
# Forward pass through all layers
|
||||
current = x
|
||||
for layer in layers:
|
||||
current = layer(current)
|
||||
|
||||
assert current.shape == (1, 2)
|
||||
# Final output should be in sigmoid range
|
||||
assert np.all(safe_numpy(current) >= 0)
|
||||
assert np.all(safe_numpy(current) <= 1)
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and error conditions."""
|
||||
|
||||
def test_zero_input(self):
|
||||
"""Test layers with zero input."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
relu = ReLU()
|
||||
|
||||
x = Tensor([[0.0, 0.0, 0.0]])
|
||||
y = layer(x)
|
||||
y_relu = relu(y)
|
||||
|
||||
assert y.shape == (1, 2)
|
||||
assert y_relu.shape == (1, 2)
|
||||
|
||||
def test_large_input(self):
|
||||
"""Test layers with large input values."""
|
||||
layer = Dense(input_size=2, output_size=1)
|
||||
sigmoid = Sigmoid()
|
||||
|
||||
x = Tensor([[1000.0, -1000.0]])
|
||||
y = layer(x)
|
||||
y_sigmoid = sigmoid(y)
|
||||
|
||||
# Should not overflow
|
||||
assert not np.any(np.isnan(safe_numpy(y_sigmoid)))
|
||||
assert not np.any(np.isinf(safe_numpy(y_sigmoid)))
|
||||
|
||||
def test_single_neuron(self):
|
||||
"""Test single neuron layers."""
|
||||
layer = Dense(input_size=1, output_size=1)
|
||||
x = Tensor([[5.0]])
|
||||
y = layer(x)
|
||||
|
||||
assert y.shape == (1, 1)
|
||||
|
||||
# Stretch goal tests (these will be skipped if methods don't exist)
|
||||
class TestStretchGoals:
|
||||
"""Stretch goal tests for advanced features."""
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Weight initialization methods")
|
||||
def test_weight_initialization_methods(self):
|
||||
"""Test different weight initialization strategies."""
|
||||
# Xavier initialization
|
||||
layer_xavier = Dense(input_size=100, output_size=50, init_method='xavier')
|
||||
weights_xavier = safe_numpy(layer_xavier.weights)
|
||||
|
||||
# He initialization
|
||||
layer_he = Dense(input_size=100, output_size=50, init_method='he')
|
||||
weights_he = safe_numpy(layer_he.weights)
|
||||
|
||||
# Check initialization ranges
|
||||
xavier_limit = np.sqrt(6.0 / (100 + 50))
|
||||
assert np.all(np.abs(weights_xavier) <= xavier_limit)
|
||||
|
||||
he_limit = np.sqrt(2.0 / 100)
|
||||
assert np.std(weights_he) <= he_limit * 1.5 # Some tolerance
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Layer parameter access")
|
||||
def test_layer_parameters(self):
|
||||
"""Test accessing and modifying layer parameters."""
|
||||
layer = Dense(input_size=3, output_size=2)
|
||||
|
||||
# Should be able to access parameters
|
||||
assert hasattr(layer, 'parameters')
|
||||
params = layer.parameters()
|
||||
assert len(params) == 2 # weights and bias
|
||||
|
||||
# Should be able to set parameters
|
||||
new_weights = Tensor(np.ones((3, 2)))
|
||||
layer.set_weights(new_weights)
|
||||
np.testing.assert_array_equal(safe_numpy(layer.weights), safe_numpy(new_weights))
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Additional activation functions")
|
||||
def test_additional_activations(self):
|
||||
"""Test additional activation functions."""
|
||||
# Leaky ReLU
|
||||
leaky_relu = LeakyReLU(alpha=0.1)
|
||||
x = Tensor([[-1.0, 0.0, 1.0]])
|
||||
y = leaky_relu(x)
|
||||
expected = [[-0.1, 0.0, 1.0]]
|
||||
np.testing.assert_array_almost_equal(safe_numpy(y), expected)
|
||||
|
||||
# Softmax
|
||||
softmax = Softmax()
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
y = softmax(x)
|
||||
# Should sum to 1
|
||||
assert np.allclose(np.sum(safe_numpy(y)), 1.0)
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Dropout layer")
|
||||
def test_dropout_layer(self):
|
||||
"""Test dropout layer implementation."""
|
||||
dropout = Dropout(p=0.5)
|
||||
x = Tensor([[1.0, 2.0, 3.0, 4.0]])
|
||||
|
||||
# Training mode
|
||||
dropout.train()
|
||||
y_train = dropout(x)
|
||||
|
||||
# Inference mode
|
||||
dropout.eval()
|
||||
y_eval = dropout(x)
|
||||
|
||||
# In eval mode, should be same as input
|
||||
np.testing.assert_array_equal(safe_numpy(y_eval), safe_numpy(x))
|
||||
|
||||
@pytest.mark.skip(reason="Stretch goal: Batch normalization")
|
||||
def test_batch_normalization(self):
|
||||
"""Test batch normalization layer."""
|
||||
bn = BatchNorm1d(num_features=3)
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
y = bn(x)
|
||||
|
||||
# Should normalize across batch dimension
|
||||
assert y.shape == x.shape
|
||||
# Mean should be close to 0, std close to 1
|
||||
assert np.allclose(np.mean(safe_numpy(y), axis=0), 0.0, atol=1e-6)
|
||||
assert np.allclose(np.std(safe_numpy(y), axis=0), 1.0, atol=1e-6)
|
||||
@@ -524,19 +524,19 @@ wide = create_mlp(10, [50], 1)
|
||||
- **Efficiency:** Balance between performance and computation
|
||||
|
||||
### Different Activation Functions
|
||||
```python
|
||||
```python
|
||||
# ReLU networks (most common)
|
||||
relu_net = create_mlp(10, [20], 1, activation=ReLU)
|
||||
|
||||
|
||||
# Tanh networks (centered around 0)
|
||||
tanh_net = create_mlp(10, [20], 1, activation=Tanh)
|
||||
|
||||
|
||||
# Multi-class classification
|
||||
classifier = create_mlp(10, [20], 3, output_activation=Softmax)
|
||||
```
|
||||
```
|
||||
|
||||
Let's test different architectures!
|
||||
"""
|
||||
"""
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
@@ -560,7 +560,7 @@ try:
|
||||
classifier = create_mlp(input_size=3, hidden_sizes=[4], output_size=3, output_activation=Softmax)
|
||||
|
||||
# Test with sample data
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
|
||||
# Test ReLU network
|
||||
y_relu = relu_net(x)
|
||||
@@ -575,9 +575,9 @@ try:
|
||||
# Test multi-class classifier
|
||||
y_multi = classifier(x)
|
||||
assert y_multi.shape == (1, 3), "Multi-class classifier should work"
|
||||
|
||||
# Check softmax properties
|
||||
assert abs(np.sum(y_multi.data) - 1.0) < 1e-6, "Softmax outputs should sum to 1"
|
||||
|
||||
# Check softmax properties
|
||||
assert abs(np.sum(y_multi.data) - 1.0) < 1e-6, "Softmax outputs should sum to 1"
|
||||
print("✅ Multi-class classifier with Softmax works correctly")
|
||||
|
||||
# Test different architectures
|
||||
@@ -595,7 +595,7 @@ try:
|
||||
|
||||
print("✅ All network architectures work correctly")
|
||||
|
||||
except Exception as e:
|
||||
except Exception as e:
|
||||
print(f"❌ Architecture test failed: {e}")
|
||||
raise
|
||||
|
||||
@@ -643,18 +643,18 @@ try:
|
||||
iris_classifier = create_mlp(input_size=4, hidden_sizes=[8, 6], output_size=3, output_activation=Softmax)
|
||||
|
||||
# Simulate iris features: [sepal_length, sepal_width, petal_length, petal_width]
|
||||
iris_samples = Tensor([
|
||||
iris_samples = Tensor([
|
||||
[5.1, 3.5, 1.4, 0.2], # Setosa
|
||||
[7.0, 3.2, 4.7, 1.4], # Versicolor
|
||||
[6.3, 3.3, 6.0, 2.5] # Virginica
|
||||
])
|
||||
|
||||
iris_predictions = iris_classifier(iris_samples)
|
||||
])
|
||||
|
||||
iris_predictions = iris_classifier(iris_samples)
|
||||
assert iris_predictions.shape == (3, 3), "Iris classifier should output 3 classes for 3 samples"
|
||||
|
||||
|
||||
# Check softmax properties
|
||||
row_sums = np.sum(iris_predictions.data, axis=1)
|
||||
assert np.allclose(row_sums, 1.0), "Each prediction should sum to 1"
|
||||
row_sums = np.sum(iris_predictions.data, axis=1)
|
||||
assert np.allclose(row_sums, 1.0), "Each prediction should sum to 1"
|
||||
print("✅ Multi-class classification works correctly")
|
||||
|
||||
# Test 2: Regression Task (Housing prices)
|
||||
@@ -691,38 +691,38 @@ try:
|
||||
# Test 4: Network Composition
|
||||
print("\n4. Network Composition Test:")
|
||||
# Create a feature extractor and classifier separately
|
||||
feature_extractor = Sequential([
|
||||
feature_extractor = Sequential([
|
||||
Dense(input_size=10, output_size=5),
|
||||
ReLU(),
|
||||
ReLU(),
|
||||
Dense(input_size=5, output_size=3),
|
||||
ReLU()
|
||||
])
|
||||
|
||||
classifier_head = Sequential([
|
||||
ReLU()
|
||||
])
|
||||
|
||||
classifier_head = Sequential([
|
||||
Dense(input_size=3, output_size=2),
|
||||
Softmax()
|
||||
])
|
||||
|
||||
Softmax()
|
||||
])
|
||||
|
||||
# Test composition
|
||||
raw_data = Tensor(np.random.randn(5, 10))
|
||||
features = feature_extractor(raw_data)
|
||||
final_predictions = classifier_head(features)
|
||||
features = feature_extractor(raw_data)
|
||||
final_predictions = classifier_head(features)
|
||||
|
||||
assert features.shape == (5, 3), "Feature extractor should output 3 features"
|
||||
assert final_predictions.shape == (5, 2), "Classifier should output 2 classes"
|
||||
|
||||
row_sums = np.sum(final_predictions.data, axis=1)
|
||||
|
||||
row_sums = np.sum(final_predictions.data, axis=1)
|
||||
assert np.allclose(row_sums, 1.0), "Composed network predictions should be valid"
|
||||
print("✅ Network composition works correctly")
|
||||
|
||||
|
||||
print("\n🎉 Integration test passed! Your networks work correctly for:")
|
||||
print(" • Multi-class classification (Iris flowers)")
|
||||
print(" • Regression tasks (housing prices)")
|
||||
print(" • Multi-class classification (Iris flowers)")
|
||||
print(" • Regression tasks (housing prices)")
|
||||
print(" • Deep learning architectures")
|
||||
print(" • Network composition and feature extraction")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Integration test failed: {e}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Integration test failed: {e}")
|
||||
raise
|
||||
|
||||
print("📈 Final Progress: Complete network architectures ready for real ML applications!")
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,453 +0,0 @@
|
||||
"""
|
||||
Tests for the Networks module.
|
||||
|
||||
Tests network composition, visualization, and practical applications.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add the project root to the path
|
||||
project_root = Path(__file__).parent.parent.parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
# Import the modules we're testing
|
||||
from tinytorch.core.tensor import Tensor
|
||||
from tinytorch.core.layers import Dense
|
||||
from tinytorch.core.activations import ReLU, Sigmoid, Tanh
|
||||
|
||||
# Import the networks module
|
||||
try:
|
||||
# Import from the exported package
|
||||
from tinytorch.core.networks import (
|
||||
Sequential,
|
||||
create_mlp
|
||||
)
|
||||
# These functions may not be implemented yet - use fallback
|
||||
try:
|
||||
from tinytorch.core.networks import (
|
||||
create_classification_network,
|
||||
create_regression_network,
|
||||
visualize_network_architecture,
|
||||
visualize_data_flow,
|
||||
compare_networks,
|
||||
analyze_network_behavior
|
||||
)
|
||||
except ImportError:
|
||||
# Create mock functions for missing functionality
|
||||
def create_classification_network(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return create_mlp(*args, **kwargs)
|
||||
|
||||
def create_regression_network(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return create_mlp(*args, **kwargs)
|
||||
|
||||
def visualize_network_architecture(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return "Network visualization placeholder"
|
||||
|
||||
def visualize_data_flow(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return "Data flow visualization placeholder"
|
||||
|
||||
def compare_networks(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return "Network comparison placeholder"
|
||||
|
||||
def analyze_network_behavior(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return "Network behavior analysis placeholder"
|
||||
|
||||
except ImportError:
|
||||
# Fallback for when module isn't exported yet
|
||||
sys.path.append(str(project_root / "modules" / "source" / "04_networks"))
|
||||
from networks_dev import (
|
||||
Sequential,
|
||||
create_mlp,
|
||||
create_classification_network,
|
||||
create_regression_network,
|
||||
visualize_network_architecture,
|
||||
visualize_data_flow,
|
||||
compare_networks,
|
||||
analyze_network_behavior
|
||||
)
|
||||
|
||||
|
||||
class TestSequentialNetwork:
|
||||
"""Test the Sequential network class."""
|
||||
|
||||
def test_sequential_initialization(self):
|
||||
"""Test Sequential network initialization."""
|
||||
layers = [Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()]
|
||||
network = Sequential(layers)
|
||||
|
||||
assert len(network.layers) == 4
|
||||
assert isinstance(network.layers[0], Dense)
|
||||
assert isinstance(network.layers[1], ReLU)
|
||||
assert isinstance(network.layers[2], Dense)
|
||||
assert isinstance(network.layers[3], Sigmoid)
|
||||
|
||||
def test_sequential_forward_pass(self):
|
||||
"""Test Sequential network forward pass."""
|
||||
network = Sequential([
|
||||
Dense(3, 4),
|
||||
ReLU(),
|
||||
Dense(4, 2),
|
||||
Sigmoid()
|
||||
])
|
||||
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
output = network(x)
|
||||
|
||||
assert output.shape == (2, 2)
|
||||
assert isinstance(output, Tensor)
|
||||
# Sigmoid output should be between 0 and 1
|
||||
assert np.all(output.data >= 0) and np.all(output.data <= 1)
|
||||
|
||||
def test_sequential_callable(self):
|
||||
"""Test that Sequential network is callable."""
|
||||
network = Sequential([Dense(2, 3), ReLU()])
|
||||
x = Tensor([[1.0, 2.0]])
|
||||
|
||||
# Test both forward() and __call__()
|
||||
output1 = network.forward(x)
|
||||
output2 = network(x)
|
||||
|
||||
assert np.allclose(output1.data, output2.data)
|
||||
|
||||
def test_empty_sequential(self):
|
||||
"""Test Sequential network with no layers."""
|
||||
network = Sequential([])
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
|
||||
# Should return input unchanged
|
||||
output = network(x)
|
||||
assert np.allclose(output.data, x.data)
|
||||
|
||||
|
||||
class TestMLPCreation:
|
||||
"""Test MLP creation functions."""
|
||||
|
||||
def test_create_mlp_basic(self):
|
||||
"""Test basic MLP creation."""
|
||||
mlp = create_mlp(input_size=3, hidden_sizes=[4], output_size=2)
|
||||
|
||||
assert len(mlp.layers) == 4 # Dense + ReLU + Dense + Sigmoid
|
||||
assert isinstance(mlp.layers[0], Dense)
|
||||
assert mlp.layers[0].input_size == 3
|
||||
assert mlp.layers[0].output_size == 4
|
||||
assert isinstance(mlp.layers[1], ReLU)
|
||||
assert isinstance(mlp.layers[2], Dense)
|
||||
assert mlp.layers[2].input_size == 4
|
||||
assert mlp.layers[2].output_size == 2
|
||||
assert isinstance(mlp.layers[3], Sigmoid)
|
||||
|
||||
def test_create_mlp_multiple_hidden(self):
|
||||
"""Test MLP creation with multiple hidden layers."""
|
||||
mlp = create_mlp(input_size=10, hidden_sizes=[16, 8, 4], output_size=3)
|
||||
|
||||
assert len(mlp.layers) == 8 # 3 Dense + 3 ReLU + 1 Dense + 1 Sigmoid
|
||||
|
||||
# Check Dense layers
|
||||
dense_layers = [layer for layer in mlp.layers if isinstance(layer, Dense)]
|
||||
assert len(dense_layers) == 4
|
||||
|
||||
assert dense_layers[0].input_size == 10
|
||||
assert dense_layers[0].output_size == 16
|
||||
assert dense_layers[1].input_size == 16
|
||||
assert dense_layers[1].output_size == 8
|
||||
assert dense_layers[2].input_size == 8
|
||||
assert dense_layers[2].output_size == 4
|
||||
assert dense_layers[3].input_size == 4
|
||||
assert dense_layers[3].output_size == 3
|
||||
|
||||
def test_create_mlp_no_hidden(self):
|
||||
"""Test MLP creation with no hidden layers."""
|
||||
mlp = create_mlp(input_size=5, hidden_sizes=[], output_size=2)
|
||||
|
||||
assert len(mlp.layers) == 2 # Dense + Sigmoid
|
||||
assert isinstance(mlp.layers[0], Dense)
|
||||
assert mlp.layers[0].input_size == 5
|
||||
assert mlp.layers[0].output_size == 2
|
||||
assert isinstance(mlp.layers[1], Sigmoid)
|
||||
|
||||
def test_create_mlp_custom_activation(self):
|
||||
"""Test MLP creation with custom activation functions."""
|
||||
mlp = create_mlp(
|
||||
input_size=3,
|
||||
hidden_sizes=[4],
|
||||
output_size=2,
|
||||
activation=Tanh,
|
||||
output_activation=Tanh
|
||||
)
|
||||
|
||||
assert len(mlp.layers) == 4
|
||||
assert isinstance(mlp.layers[1], Tanh) # Hidden activation
|
||||
assert isinstance(mlp.layers[3], Tanh) # Output activation
|
||||
|
||||
|
||||
class TestSpecializedNetworks:
|
||||
"""Test specialized network creation functions."""
|
||||
|
||||
def test_create_classification_network(self):
|
||||
"""Test classification network creation."""
|
||||
classifier = create_classification_network(
|
||||
input_size=100,
|
||||
num_classes=5,
|
||||
hidden_sizes=[32, 16]
|
||||
)
|
||||
|
||||
assert len(classifier.layers) == 6 # Dense(100→32) + ReLU + Dense(32→16) + ReLU + Dense(16→5) + Softmax
|
||||
|
||||
# Check output layer
|
||||
dense_layers = [layer for layer in classifier.layers if isinstance(layer, Dense)]
|
||||
assert dense_layers[-1].output_size == 5
|
||||
# Should use Softmax for multi-class classification
|
||||
from tinytorch.core.activations import Softmax
|
||||
assert isinstance(classifier.layers[-1], Softmax)
|
||||
|
||||
def test_create_classification_network_default(self):
|
||||
"""Test classification network with default hidden sizes."""
|
||||
classifier = create_classification_network(input_size=50, num_classes=3)
|
||||
|
||||
# Should use default hidden size of input_size // 2
|
||||
expected_hidden = 50 // 2
|
||||
dense_layers = [layer for layer in classifier.layers if isinstance(layer, Dense)]
|
||||
assert dense_layers[0].output_size == expected_hidden
|
||||
assert dense_layers[1].output_size == 3
|
||||
|
||||
def test_create_regression_network(self):
|
||||
"""Test regression network creation."""
|
||||
regressor = create_regression_network(
|
||||
input_size=13,
|
||||
output_size=1,
|
||||
hidden_sizes=[8, 4]
|
||||
)
|
||||
|
||||
assert len(regressor.layers) == 6 # Dense(13→8) + ReLU + Dense(8→4) + ReLU + Dense(4→1) + Tanh
|
||||
|
||||
# Check output layer
|
||||
dense_layers = [layer for layer in regressor.layers if isinstance(layer, Dense)]
|
||||
assert dense_layers[-1].output_size == 1
|
||||
assert isinstance(regressor.layers[-1], Tanh)
|
||||
|
||||
def test_create_regression_network_default(self):
|
||||
"""Test regression network with default parameters."""
|
||||
regressor = create_regression_network(input_size=20)
|
||||
|
||||
# Should use default output_size=1 and hidden_size=input_size//2
|
||||
expected_hidden = 20 // 2
|
||||
dense_layers = [layer for layer in regressor.layers if isinstance(layer, Dense)]
|
||||
assert dense_layers[0].output_size == expected_hidden
|
||||
assert dense_layers[1].output_size == 1
|
||||
|
||||
|
||||
class TestNetworkBehavior:
|
||||
"""Test network behavior and functionality."""
|
||||
|
||||
def test_network_shape_transformations(self):
|
||||
"""Test that networks properly transform tensor shapes."""
|
||||
network = Sequential([
|
||||
Dense(3, 4),
|
||||
ReLU(),
|
||||
Dense(4, 2),
|
||||
Sigmoid()
|
||||
])
|
||||
|
||||
x = Tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
|
||||
output = network(x)
|
||||
|
||||
assert x.shape == (2, 3)
|
||||
assert output.shape == (2, 2)
|
||||
|
||||
def test_network_activations(self):
|
||||
"""Test that activation functions are properly applied."""
|
||||
network = Sequential([
|
||||
Dense(2, 3),
|
||||
ReLU(),
|
||||
Dense(3, 1),
|
||||
Sigmoid()
|
||||
])
|
||||
|
||||
x = Tensor([[-1.0, 1.0]])
|
||||
output = network(x)
|
||||
|
||||
# ReLU should zero out negative values
|
||||
# Sigmoid should output values between 0 and 1
|
||||
assert np.all(output.data >= 0) and np.all(output.data <= 1)
|
||||
|
||||
def test_network_parameter_count(self):
|
||||
"""Test that networks have the expected number of parameters."""
|
||||
network = Sequential([
|
||||
Dense(3, 4), # 3*4 + 4 = 16 parameters
|
||||
ReLU(),
|
||||
Dense(4, 2), # 4*2 + 2 = 10 parameters
|
||||
Sigmoid()
|
||||
])
|
||||
|
||||
# Count parameters (weights + biases)
|
||||
total_params = 0
|
||||
for layer in network.layers:
|
||||
if hasattr(layer, 'weights'):
|
||||
total_params += layer.weights.data.size
|
||||
if hasattr(layer, 'bias') and layer.bias is not None:
|
||||
total_params += layer.bias.data.size
|
||||
|
||||
assert total_params == 26 # 16 + 10
|
||||
|
||||
|
||||
class TestVisualizationFunctions:
|
||||
"""Test visualization functions (basic functionality, not visual output)."""
|
||||
|
||||
def test_visualize_network_architecture_exists(self):
|
||||
"""Test that visualization function exists and is callable."""
|
||||
network = Sequential([Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()])
|
||||
|
||||
# Should not raise an error
|
||||
try:
|
||||
visualize_network_architecture(network, "Test Network")
|
||||
except Exception as e:
|
||||
pytest.fail(f"visualize_network_architecture raised {e}")
|
||||
|
||||
def test_visualize_data_flow_exists(self):
|
||||
"""Test that data flow visualization function exists and is callable."""
|
||||
network = Sequential([Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()])
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
|
||||
# Should not raise an error
|
||||
try:
|
||||
visualize_data_flow(network, x, "Test Data Flow")
|
||||
except Exception as e:
|
||||
pytest.fail(f"visualize_data_flow raised {e}")
|
||||
|
||||
def test_compare_networks_exists(self):
|
||||
"""Test that network comparison function exists and is callable."""
|
||||
network1 = Sequential([Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()])
|
||||
network2 = Sequential([Dense(3, 8), ReLU(), Dense(8, 2), Sigmoid()])
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
|
||||
# Should not raise an error
|
||||
try:
|
||||
compare_networks([network1, network2], ["Small", "Large"], x, "Test Comparison")
|
||||
except Exception as e:
|
||||
pytest.fail(f"compare_networks raised {e}")
|
||||
|
||||
def test_analyze_network_behavior_exists(self):
|
||||
"""Test that behavior analysis function exists and is callable."""
|
||||
network = Sequential([Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()])
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
|
||||
# Should not raise an error
|
||||
try:
|
||||
analyze_network_behavior(network, x, "Test Behavior")
|
||||
except Exception as e:
|
||||
pytest.fail(f"analyze_network_behavior raised {e}")
|
||||
|
||||
|
||||
class TestPracticalApplications:
|
||||
"""Test practical network applications."""
|
||||
|
||||
def test_digit_classification_network(self):
|
||||
"""Test creating a network for digit classification."""
|
||||
classifier = create_classification_network(
|
||||
input_size=784, # 28x28 image
|
||||
num_classes=10, # 10 digits
|
||||
hidden_sizes=[128, 64]
|
||||
)
|
||||
|
||||
# Test with fake image data
|
||||
fake_image = Tensor(np.random.randn(1, 784).astype(np.float32))
|
||||
output = classifier(fake_image)
|
||||
|
||||
assert output.shape == (1, 10)
|
||||
assert np.all(output.data >= 0) and np.all(output.data <= 1)
|
||||
# Should sum to approximately 1 (probability distribution)
|
||||
assert np.abs(np.sum(output.data) - 1.0) < 0.1
|
||||
|
||||
def test_sentiment_analysis_network(self):
|
||||
"""Test creating a network for sentiment analysis."""
|
||||
classifier = create_classification_network(
|
||||
input_size=100, # 100-dimensional embeddings
|
||||
num_classes=2, # Positive/Negative
|
||||
hidden_sizes=[32, 16]
|
||||
)
|
||||
|
||||
# Test with fake text embeddings
|
||||
fake_embeddings = Tensor(np.random.randn(1, 100).astype(np.float32))
|
||||
output = classifier(fake_embeddings)
|
||||
|
||||
assert output.shape == (1, 2)
|
||||
assert np.all(output.data >= 0) and np.all(output.data <= 1)
|
||||
|
||||
def test_house_price_prediction_network(self):
|
||||
"""Test creating a network for house price prediction."""
|
||||
regressor = create_regression_network(
|
||||
input_size=13, # 13 house features
|
||||
output_size=1, # 1 price prediction
|
||||
hidden_sizes=[8, 4]
|
||||
)
|
||||
|
||||
# Test with fake house features
|
||||
fake_features = Tensor(np.random.randn(1, 13).astype(np.float32))
|
||||
output = regressor(fake_features)
|
||||
|
||||
assert output.shape == (1, 1)
|
||||
# Tanh output should be between -1 and 1
|
||||
assert np.all(output.data >= -1) and np.all(output.data <= 1)
|
||||
|
||||
|
||||
class TestNetworkIntegration:
|
||||
"""Test integration with other modules."""
|
||||
|
||||
def test_network_with_tensor_operations(self):
|
||||
"""Test that networks work with tensor operations."""
|
||||
network = Sequential([Dense(3, 4), ReLU(), Dense(4, 2), Sigmoid()])
|
||||
|
||||
# Create input using tensor operations
|
||||
x1 = Tensor([[1.0, 2.0, 3.0]])
|
||||
x2 = Tensor([[4.0, 5.0, 6.0]])
|
||||
x_combined = Tensor(np.vstack([x1.data, x2.data]))
|
||||
|
||||
output = network(x_combined)
|
||||
assert output.shape == (2, 2)
|
||||
|
||||
def test_network_with_activations_module(self):
|
||||
"""Test that networks properly use activations from the activations module."""
|
||||
# This test ensures we're using the activations from the activations module
|
||||
# rather than re-implementing them
|
||||
network = Sequential([
|
||||
Dense(2, 3),
|
||||
ReLU(), # From activations module
|
||||
Dense(3, 1),
|
||||
Sigmoid() # From activations module
|
||||
])
|
||||
|
||||
x = Tensor([[-1.0, 1.0]])
|
||||
output = network(x)
|
||||
|
||||
# Test that activations work correctly
|
||||
assert np.all(output.data >= 0) and np.all(output.data <= 1)
|
||||
|
||||
def test_network_with_layers_module(self):
|
||||
"""Test that networks properly use layers from the layers module."""
|
||||
# This test ensures we're using the Dense layers from the layers module
|
||||
network = Sequential([
|
||||
Dense(3, 4), # From layers module
|
||||
ReLU(),
|
||||
Dense(4, 2), # From layers module
|
||||
Sigmoid()
|
||||
])
|
||||
|
||||
x = Tensor([[1.0, 2.0, 3.0]])
|
||||
output = network(x)
|
||||
|
||||
# Test that layers work correctly
|
||||
assert output.shape == (1, 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run the tests
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -607,50 +607,50 @@ try:
|
||||
print("\n1. Simple CNN Pipeline Test:")
|
||||
|
||||
# Create pipeline: Conv2D → ReLU → Flatten → Dense
|
||||
conv = Conv2D(kernel_size=(2, 2))
|
||||
relu = ReLU()
|
||||
dense = Dense(input_size=4, output_size=3)
|
||||
|
||||
# Input image
|
||||
image = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
|
||||
|
||||
# Forward pass
|
||||
conv = Conv2D(kernel_size=(2, 2))
|
||||
relu = ReLU()
|
||||
dense = Dense(input_size=4, output_size=3)
|
||||
|
||||
# Input image
|
||||
image = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
|
||||
|
||||
# Forward pass
|
||||
features = conv(image) # (3,3) → (2,2)
|
||||
activated = relu(features) # (2,2) → (2,2)
|
||||
flattened = flatten(activated) # (2,2) → (1,4)
|
||||
output = dense(flattened) # (1,4) → (1,3)
|
||||
|
||||
assert features.shape == (2, 2), f"Conv output shape wrong: {features.shape}"
|
||||
assert activated.shape == (2, 2), f"ReLU output shape wrong: {activated.shape}"
|
||||
assert flattened.shape == (1, 4), f"Flatten output shape wrong: {flattened.shape}"
|
||||
assert output.shape == (1, 3), f"Dense output shape wrong: {output.shape}"
|
||||
|
||||
|
||||
assert features.shape == (2, 2), f"Conv output shape wrong: {features.shape}"
|
||||
assert activated.shape == (2, 2), f"ReLU output shape wrong: {activated.shape}"
|
||||
assert flattened.shape == (1, 4), f"Flatten output shape wrong: {flattened.shape}"
|
||||
assert output.shape == (1, 3), f"Dense output shape wrong: {output.shape}"
|
||||
|
||||
print("✅ Simple CNN pipeline works correctly")
|
||||
|
||||
# Test 2: Multi-layer CNN
|
||||
print("\n2. Multi-layer CNN Test:")
|
||||
|
||||
# Create deeper pipeline: Conv2D → ReLU → Conv2D → ReLU → Flatten → Dense
|
||||
conv1 = Conv2D(kernel_size=(2, 2))
|
||||
relu1 = ReLU()
|
||||
conv2 = Conv2D(kernel_size=(2, 2))
|
||||
relu2 = ReLU()
|
||||
conv1 = Conv2D(kernel_size=(2, 2))
|
||||
relu1 = ReLU()
|
||||
conv2 = Conv2D(kernel_size=(2, 2))
|
||||
relu2 = ReLU()
|
||||
dense_multi = Dense(input_size=9, output_size=2)
|
||||
|
||||
# Larger input for multi-layer processing
|
||||
|
||||
# Larger input for multi-layer processing
|
||||
large_image = Tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]])
|
||||
|
||||
# Forward pass
|
||||
|
||||
# Forward pass
|
||||
h1 = conv1(large_image) # (5,5) → (4,4)
|
||||
h2 = relu1(h1) # (4,4) → (4,4)
|
||||
h3 = conv2(h2) # (4,4) → (3,3)
|
||||
h4 = relu2(h3) # (3,3) → (3,3)
|
||||
h5 = flatten(h4) # (3,3) → (1,9)
|
||||
output_multi = dense_multi(h5) # (1,9) → (1,2)
|
||||
|
||||
assert h1.shape == (4, 4), f"Conv1 output wrong: {h1.shape}"
|
||||
assert h3.shape == (3, 3), f"Conv2 output wrong: {h3.shape}"
|
||||
assert h5.shape == (1, 9), f"Flatten output wrong: {h5.shape}"
|
||||
|
||||
assert h1.shape == (4, 4), f"Conv1 output wrong: {h1.shape}"
|
||||
assert h3.shape == (3, 3), f"Conv2 output wrong: {h3.shape}"
|
||||
assert h5.shape == (1, 9), f"Flatten output wrong: {h5.shape}"
|
||||
assert output_multi.shape == (1, 2), f"Final output wrong: {output_multi.shape}"
|
||||
|
||||
print("✅ Multi-layer CNN works correctly")
|
||||
@@ -667,22 +667,22 @@ try:
|
||||
[0, 1, 1, 0, 0, 1, 1, 0],
|
||||
[0, 0, 1, 1, 1, 1, 0, 0],
|
||||
[1, 1, 0, 0, 0, 0, 1, 1]])
|
||||
|
||||
# CNN for digit classification
|
||||
|
||||
# CNN for digit classification
|
||||
feature_extractor = Conv2D(kernel_size=(3, 3)) # (8,8) → (6,6)
|
||||
activation = ReLU()
|
||||
classifier = Dense(input_size=36, output_size=10) # 10 digit classes
|
||||
|
||||
# Forward pass
|
||||
features = feature_extractor(digit_image)
|
||||
activated_features = activation(features)
|
||||
activation = ReLU()
|
||||
classifier = Dense(input_size=36, output_size=10) # 10 digit classes
|
||||
|
||||
# Forward pass
|
||||
features = feature_extractor(digit_image)
|
||||
activated_features = activation(features)
|
||||
feature_vector = flatten(activated_features)
|
||||
digit_scores = classifier(feature_vector)
|
||||
|
||||
assert features.shape == (6, 6), f"Feature extraction shape wrong: {features.shape}"
|
||||
assert feature_vector.shape == (1, 36), f"Feature vector shape wrong: {feature_vector.shape}"
|
||||
assert digit_scores.shape == (1, 10), f"Digit scores shape wrong: {digit_scores.shape}"
|
||||
|
||||
digit_scores = classifier(feature_vector)
|
||||
|
||||
assert features.shape == (6, 6), f"Feature extraction shape wrong: {features.shape}"
|
||||
assert feature_vector.shape == (1, 36), f"Feature vector shape wrong: {feature_vector.shape}"
|
||||
assert digit_scores.shape == (1, 10), f"Digit scores shape wrong: {digit_scores.shape}"
|
||||
|
||||
print("✅ Image classification scenario works correctly")
|
||||
|
||||
# Test 4: Feature Extraction and Composition
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,368 +0,0 @@
|
||||
"""
|
||||
Test suite for the CNN module.
|
||||
This tests the CNN implementations to ensure they work correctly.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add the CNN module to the path
|
||||
sys.path.append(str(Path(__file__).parent.parent))
|
||||
|
||||
try:
|
||||
# Import from the exported package
|
||||
from tinytorch.core.cnn import conv2d_naive, Conv2D, flatten
|
||||
except ImportError:
|
||||
# Fallback for when module isn't exported yet
|
||||
from cnn_dev import conv2d_naive, Conv2D, flatten
|
||||
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .data attribute"""
|
||||
return tensor.data
|
||||
|
||||
|
||||
class TestConv2DNaive:
|
||||
"""Test the naive convolution implementation."""
|
||||
|
||||
def test_conv2d_naive_small(self):
|
||||
"""Test basic convolution with small matrices."""
|
||||
input = np.array([
|
||||
[1, 2, 3],
|
||||
[4, 5, 6],
|
||||
[7, 8, 9]
|
||||
], dtype=np.float32)
|
||||
kernel = np.array([
|
||||
[1, 0],
|
||||
[0, -1]
|
||||
], dtype=np.float32)
|
||||
expected = np.array([
|
||||
[1*1+2*0+4*0+5*(-1), 2*1+3*0+5*0+6*(-1)],
|
||||
[4*1+5*0+7*0+8*(-1), 5*1+6*0+8*0+9*(-1)]
|
||||
], dtype=np.float32)
|
||||
output = conv2d_naive(input, kernel)
|
||||
assert np.allclose(output, expected), f"conv2d_naive output incorrect!\nExpected:\n{expected}\nGot:\n{output}"
|
||||
|
||||
def test_conv2d_naive_edge_detection(self):
|
||||
"""Test convolution with edge detection kernel."""
|
||||
input = np.array([
|
||||
[0, 0, 0, 0, 0],
|
||||
[0, 1, 1, 1, 0],
|
||||
[0, 1, 1, 1, 0],
|
||||
[0, 1, 1, 1, 0],
|
||||
[0, 0, 0, 0, 0]
|
||||
], dtype=np.float32)
|
||||
|
||||
# Vertical edge detection kernel
|
||||
kernel = np.array([
|
||||
[-1, 0, 1],
|
||||
[-2, 0, 2],
|
||||
[-1, 0, 1]
|
||||
], dtype=np.float32)
|
||||
|
||||
output = conv2d_naive(input, kernel)
|
||||
assert output.shape == (3, 3), f"Expected shape (3, 3), got {output.shape}"
|
||||
|
||||
# Should detect vertical edges
|
||||
assert np.abs(output[1, 0]) > 0, "Should detect left edge"
|
||||
assert np.abs(output[1, 2]) > 0, "Should detect right edge"
|
||||
assert np.abs(output[1, 1]) < 1, "Should be small in center"
|
||||
|
||||
def test_conv2d_naive_identity_kernel(self):
|
||||
"""Test convolution with identity kernel."""
|
||||
input = np.array([
|
||||
[1, 2, 3],
|
||||
[4, 5, 6],
|
||||
[7, 8, 9]
|
||||
], dtype=np.float32)
|
||||
|
||||
# Identity kernel
|
||||
kernel = np.array([
|
||||
[0, 0, 0],
|
||||
[0, 1, 0],
|
||||
[0, 0, 0]
|
||||
], dtype=np.float32)
|
||||
|
||||
output = conv2d_naive(input, kernel)
|
||||
expected = np.array([[5]], dtype=np.float32) # Only center value
|
||||
assert np.allclose(output, expected), f"Identity kernel failed: got {output}, expected {expected}"
|
||||
|
||||
def test_conv2d_naive_different_sizes(self):
|
||||
"""Test convolution with different input and kernel sizes."""
|
||||
# 4x4 input, 2x2 kernel
|
||||
input = np.array([
|
||||
[1, 2, 3, 4],
|
||||
[5, 6, 7, 8],
|
||||
[9, 10, 11, 12],
|
||||
[13, 14, 15, 16]
|
||||
], dtype=np.float32)
|
||||
|
||||
kernel = np.array([
|
||||
[1, 1],
|
||||
[1, 1]
|
||||
], dtype=np.float32)
|
||||
|
||||
output = conv2d_naive(input, kernel)
|
||||
assert output.shape == (3, 3), f"Expected shape (3, 3), got {output.shape}"
|
||||
|
||||
# Check first element: 1+2+5+6 = 14
|
||||
assert np.isclose(output[0, 0], 14), f"First element should be 14, got {output[0, 0]}"
|
||||
|
||||
def test_conv2d_naive_single_pixel(self):
|
||||
"""Test convolution with single pixel input."""
|
||||
input = np.array([[5]], dtype=np.float32)
|
||||
kernel = np.array([[2]], dtype=np.float32)
|
||||
|
||||
output = conv2d_naive(input, kernel)
|
||||
expected = np.array([[10]], dtype=np.float32)
|
||||
assert np.allclose(output, expected), f"Single pixel convolution failed: got {output}, expected {expected}"
|
||||
|
||||
|
||||
class TestConv2DLayer:
|
||||
"""Test the Conv2D layer implementation."""
|
||||
|
||||
def test_conv2d_layer_creation(self):
|
||||
"""Test Conv2D layer creation."""
|
||||
conv = Conv2D((3, 3))
|
||||
assert conv.kernel_size == (3, 3), f"Kernel size should be (3, 3), got {conv.kernel_size}"
|
||||
assert conv.kernel.shape == (3, 3), f"Kernel shape should be (3, 3), got {conv.kernel.shape}"
|
||||
|
||||
def test_conv2d_layer_forward_pass(self):
|
||||
"""Test Conv2D layer forward pass."""
|
||||
conv = Conv2D((2, 2))
|
||||
x = Tensor(np.ones((4, 4), dtype=np.float32))
|
||||
|
||||
output = conv(x)
|
||||
assert output.shape == (3, 3), f"Expected output shape (3, 3), got {output.shape}"
|
||||
assert hasattr(output, 'data'), "Output should be a Tensor with data attribute"
|
||||
|
||||
def test_conv2d_layer_different_sizes(self):
|
||||
"""Test Conv2D layer with different input sizes."""
|
||||
conv = Conv2D((2, 2))
|
||||
|
||||
# Test with 3x3 input
|
||||
x1 = Tensor(np.ones((3, 3), dtype=np.float32))
|
||||
out1 = conv(x1)
|
||||
assert out1.shape == (2, 2), f"3x3 input should give (2, 2) output, got {out1.shape}"
|
||||
|
||||
# Test with 5x5 input
|
||||
x2 = Tensor(np.ones((5, 5), dtype=np.float32))
|
||||
out2 = conv(x2)
|
||||
assert out2.shape == (4, 4), f"5x5 input should give (4, 4) output, got {out2.shape}"
|
||||
|
||||
def test_conv2d_layer_kernel_initialization(self):
|
||||
"""Test that Conv2D layer initializes kernel properly."""
|
||||
conv = Conv2D((3, 3))
|
||||
|
||||
# Kernel should not be all zeros
|
||||
assert not np.allclose(conv.kernel, 0), "Kernel should not be all zeros"
|
||||
|
||||
# Kernel should be reasonable size (not too large)
|
||||
assert np.abs(conv.kernel).max() < 10, "Kernel values should be reasonable"
|
||||
|
||||
def test_conv2d_layer_reproducibility(self):
|
||||
"""Test that Conv2D layer gives consistent results."""
|
||||
conv = Conv2D((2, 2))
|
||||
x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=np.float32))
|
||||
|
||||
# Multiple forward passes should give same result
|
||||
out1 = conv(x)
|
||||
out2 = conv(x)
|
||||
|
||||
assert np.allclose(safe_numpy(out1), safe_numpy(out2)), "Conv2D should be deterministic"
|
||||
|
||||
|
||||
class TestFlattenFunction:
|
||||
"""Test the flatten function implementation."""
|
||||
|
||||
def test_flatten_2d_matrix(self):
|
||||
"""Test flattening a 2D matrix."""
|
||||
x = Tensor(np.array([[1, 2], [3, 4]], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
expected = np.array([[1, 2, 3, 4]], dtype=np.float32)
|
||||
assert np.array_equal(safe_numpy(flattened), expected), f"Flatten failed: got {safe_numpy(flattened)}, expected {expected}"
|
||||
assert flattened.shape == (1, 4), f"Expected shape (1, 4), got {flattened.shape}"
|
||||
|
||||
def test_flatten_3d_tensor(self):
|
||||
"""Test flattening a 3D tensor."""
|
||||
x = Tensor(np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
expected = np.array([[1, 2, 3, 4, 5, 6, 7, 8]], dtype=np.float32)
|
||||
assert np.array_equal(safe_numpy(flattened), expected), f"3D flatten failed: got {safe_numpy(flattened)}, expected {expected}"
|
||||
assert flattened.shape == (1, 8), f"Expected shape (1, 8), got {flattened.shape}"
|
||||
|
||||
def test_flatten_1d_tensor(self):
|
||||
"""Test flattening a 1D tensor."""
|
||||
x = Tensor(np.array([1, 2, 3, 4], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
expected = np.array([[1, 2, 3, 4]], dtype=np.float32)
|
||||
assert np.array_equal(safe_numpy(flattened), expected), f"1D flatten failed: got {safe_numpy(flattened)}, expected {expected}"
|
||||
assert flattened.shape == (1, 4), f"Expected shape (1, 4), got {flattened.shape}"
|
||||
|
||||
def test_flatten_single_element(self):
|
||||
"""Test flattening a single element tensor."""
|
||||
x = Tensor(np.array([[[[5]]]], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
expected = np.array([[5]], dtype=np.float32)
|
||||
assert np.array_equal(safe_numpy(flattened), expected), f"Single element flatten failed: got {safe_numpy(flattened)}, expected {expected}"
|
||||
assert flattened.shape == (1, 1), f"Expected shape (1, 1), got {flattened.shape}"
|
||||
|
||||
def test_flatten_preserves_data_type(self):
|
||||
"""Test that flatten preserves data type."""
|
||||
x = Tensor(np.array([[1, 2], [3, 4]], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
assert safe_numpy(flattened).dtype == np.float32, f"Data type should be preserved: got {safe_numpy(flattened).dtype}"
|
||||
|
||||
|
||||
class TestCNNIntegration:
|
||||
"""Test integration between CNN components."""
|
||||
|
||||
def test_conv_then_flatten(self):
|
||||
"""Test convolution followed by flatten (typical CNN pattern)."""
|
||||
# Create a simple input
|
||||
x = Tensor(np.array([
|
||||
[1, 2, 3, 4],
|
||||
[5, 6, 7, 8],
|
||||
[9, 10, 11, 12],
|
||||
[13, 14, 15, 16]
|
||||
], dtype=np.float32))
|
||||
|
||||
# Apply convolution
|
||||
conv = Conv2D((2, 2))
|
||||
conv_out = conv(x)
|
||||
assert conv_out.shape == (3, 3), f"Conv output should be (3, 3), got {conv_out.shape}"
|
||||
|
||||
# Apply flatten
|
||||
flat_out = flatten(conv_out)
|
||||
assert flat_out.shape == (1, 9), f"Flatten output should be (1, 9), got {flat_out.shape}"
|
||||
|
||||
# Check that data is preserved
|
||||
assert safe_numpy(flat_out).size == 9, "Should have 9 elements after flatten"
|
||||
|
||||
def test_multiple_conv_layers(self):
|
||||
"""Test multiple convolution layers (deeper CNN)."""
|
||||
x = Tensor(np.ones((5, 5), dtype=np.float32))
|
||||
|
||||
# First conv layer
|
||||
conv1 = Conv2D((2, 2))
|
||||
out1 = conv1(x)
|
||||
assert out1.shape == (4, 4), f"First conv should give (4, 4), got {out1.shape}"
|
||||
|
||||
# Second conv layer
|
||||
conv2 = Conv2D((2, 2))
|
||||
out2 = conv2(out1)
|
||||
assert out2.shape == (3, 3), f"Second conv should give (3, 3), got {out2.shape}"
|
||||
|
||||
# Final flatten
|
||||
final = flatten(out2)
|
||||
assert final.shape == (1, 9), f"Final flatten should give (1, 9), got {final.shape}"
|
||||
|
||||
def test_conv_output_range(self):
|
||||
"""Test that convolution outputs are in reasonable range."""
|
||||
# Create input with known range
|
||||
x = Tensor(np.random.rand(4, 4).astype(np.float32)) # Values 0-1
|
||||
|
||||
conv = Conv2D((2, 2))
|
||||
output = conv(x)
|
||||
|
||||
# Output should be finite
|
||||
assert np.all(np.isfinite(safe_numpy(output))), "Conv output should be finite"
|
||||
|
||||
# Output should not be extremely large
|
||||
assert np.abs(safe_numpy(output)).max() < 100, "Conv output should not be extremely large"
|
||||
|
||||
|
||||
class TestCNNEdgeCases:
|
||||
"""Test edge cases and error conditions."""
|
||||
|
||||
def test_conv2d_naive_minimum_size(self):
|
||||
"""Test convolution with minimum possible sizes."""
|
||||
# 1x1 input, 1x1 kernel
|
||||
input = np.array([[1]], dtype=np.float32)
|
||||
kernel = np.array([[2]], dtype=np.float32)
|
||||
|
||||
output = conv2d_naive(input, kernel)
|
||||
expected = np.array([[2]], dtype=np.float32)
|
||||
assert np.allclose(output, expected), f"Minimum size convolution failed: got {output}, expected {expected}"
|
||||
|
||||
def test_conv2d_layer_minimum_size(self):
|
||||
"""Test Conv2D layer with minimum input size."""
|
||||
conv = Conv2D((1, 1))
|
||||
x = Tensor(np.array([[5]], dtype=np.float32))
|
||||
|
||||
output = conv(x)
|
||||
assert output.shape == (1, 1), f"Minimum size layer should give (1, 1), got {output.shape}"
|
||||
|
||||
def test_flatten_empty_handling(self):
|
||||
"""Test flatten with various edge cases."""
|
||||
# Very small tensor
|
||||
x = Tensor(np.array([1], dtype=np.float32))
|
||||
flattened = flatten(x)
|
||||
assert flattened.shape == (1, 1), f"Single element should give (1, 1), got {flattened.shape}"
|
||||
|
||||
def test_conv_with_zeros(self):
|
||||
"""Test convolution with zero inputs."""
|
||||
# All zeros input
|
||||
x = Tensor(np.zeros((3, 3), dtype=np.float32))
|
||||
conv = Conv2D((2, 2))
|
||||
output = conv(x)
|
||||
|
||||
# Should not crash and should produce valid output
|
||||
assert output.shape == (2, 2), f"Zero input should give (2, 2), got {output.shape}"
|
||||
assert np.all(np.isfinite(safe_numpy(output))), "Zero input should produce finite output"
|
||||
|
||||
def test_conv_with_negative_values(self):
|
||||
"""Test convolution with negative inputs."""
|
||||
x = Tensor(np.array([[-1, -2], [-3, -4]], dtype=np.float32))
|
||||
conv = Conv2D((2, 2))
|
||||
output = conv(x)
|
||||
|
||||
# Should handle negative values properly
|
||||
assert output.shape == (1, 1), f"Negative input should give (1, 1), got {output.shape}"
|
||||
assert np.all(np.isfinite(safe_numpy(output))), "Negative input should produce finite output"
|
||||
|
||||
|
||||
class TestCNNPerformance:
|
||||
"""Test performance characteristics of CNN operations."""
|
||||
|
||||
def test_conv_reasonable_speed(self):
|
||||
"""Test that convolution completes in reasonable time."""
|
||||
import time
|
||||
|
||||
# Medium-sized input
|
||||
x = Tensor(np.random.rand(10, 10).astype(np.float32))
|
||||
conv = Conv2D((3, 3))
|
||||
|
||||
start_time = time.time()
|
||||
output = conv(x)
|
||||
end_time = time.time()
|
||||
|
||||
# Should complete quickly (less than 1 second)
|
||||
assert end_time - start_time < 1.0, "Convolution should complete quickly"
|
||||
assert output.shape == (8, 8), f"Expected (8, 8), got {output.shape}"
|
||||
|
||||
def test_flatten_preserves_size(self):
|
||||
"""Test that flatten preserves total number of elements."""
|
||||
shapes = [(2, 3), (4, 4), (1, 10), (5, 2, 3)]
|
||||
|
||||
for shape in shapes:
|
||||
x = Tensor(np.random.rand(*shape).astype(np.float32))
|
||||
flattened = flatten(x)
|
||||
|
||||
original_size = np.prod(shape)
|
||||
flattened_size = flattened.shape[1] # Second dimension since flatten returns (1, N)
|
||||
|
||||
assert original_size == flattened_size, f"Size mismatch for shape {shape}: {original_size} != {flattened_size}"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run the tests
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -753,22 +753,22 @@ try:
|
||||
dataset = SimpleDataset(size=20, num_features=5, num_classes=4)
|
||||
|
||||
print(f"Dataset created: size={len(dataset)}, features={dataset.num_features}, classes={dataset.get_num_classes()}")
|
||||
|
||||
# Test basic properties
|
||||
|
||||
# Test basic properties
|
||||
assert len(dataset) == 20, f"Dataset length should be 20, got {len(dataset)}"
|
||||
assert dataset.get_num_classes() == 4, f"Should have 4 classes, got {dataset.get_num_classes()}"
|
||||
print("✅ SimpleDataset basic properties work correctly")
|
||||
|
||||
|
||||
# Test sample access
|
||||
data, label = dataset[0]
|
||||
assert isinstance(data, Tensor), "Data should be a Tensor"
|
||||
assert isinstance(label, Tensor), "Label should be a Tensor"
|
||||
data, label = dataset[0]
|
||||
assert isinstance(data, Tensor), "Data should be a Tensor"
|
||||
assert isinstance(label, Tensor), "Label should be a Tensor"
|
||||
assert data.shape == (5,), f"Data shape should be (5,), got {data.shape}"
|
||||
assert label.shape == (), f"Label shape should be (), got {label.shape}"
|
||||
print("✅ SimpleDataset sample access works correctly")
|
||||
|
||||
|
||||
# Test sample shape
|
||||
sample_shape = dataset.get_sample_shape()
|
||||
sample_shape = dataset.get_sample_shape()
|
||||
assert sample_shape == (5,), f"Sample shape should be (5,), got {sample_shape}"
|
||||
print("✅ SimpleDataset get_sample_shape works correctly")
|
||||
|
||||
@@ -787,7 +787,7 @@ try:
|
||||
assert np.array_equal(label1.data, label2.data), "Labels should be deterministic"
|
||||
print("✅ SimpleDataset data is deterministic")
|
||||
|
||||
except Exception as e:
|
||||
except Exception as e:
|
||||
print(f"❌ SimpleDataset test failed: {e}")
|
||||
raise
|
||||
|
||||
@@ -861,9 +861,9 @@ try:
|
||||
# Verify batch properties
|
||||
assert batch_data.shape[1] == 8, f"Features should be 8, got {batch_data.shape[1]}"
|
||||
assert len(batch_labels.shape) == 1, f"Labels should be 1D, got shape {batch_labels.shape}"
|
||||
assert isinstance(batch_data, Tensor), "Batch data should be Tensor"
|
||||
assert isinstance(batch_labels, Tensor), "Batch labels should be Tensor"
|
||||
|
||||
assert isinstance(batch_data, Tensor), "Batch data should be Tensor"
|
||||
assert isinstance(batch_labels, Tensor), "Batch labels should be Tensor"
|
||||
|
||||
assert epoch_samples == 100, f"Should process 100 samples, got {epoch_samples}"
|
||||
expected_batches = (100 + 16 - 1) // 16
|
||||
assert epoch_batches == expected_batches, f"Should have {expected_batches} batches, got {epoch_batches}"
|
||||
@@ -943,11 +943,11 @@ try:
|
||||
dataset = SimpleDataset(size=60, num_features=6, num_classes=3)
|
||||
loader = DataLoader(dataset, batch_size=20, shuffle=True)
|
||||
|
||||
for epoch in range(3):
|
||||
epoch_samples = 0
|
||||
for epoch in range(3):
|
||||
epoch_samples = 0
|
||||
for batch_data, batch_labels in loader:
|
||||
epoch_samples += batch_data.shape[0]
|
||||
|
||||
epoch_samples += batch_data.shape[0]
|
||||
|
||||
# Verify shapes remain consistent across epochs
|
||||
assert batch_data.shape[1] == 6, f"Features should be 6 in epoch {epoch}"
|
||||
assert len(batch_labels.shape) == 1, f"Labels should be 1D in epoch {epoch}"
|
||||
@@ -963,7 +963,7 @@ try:
|
||||
print(" • Memory-efficient processing")
|
||||
print(" • Multi-epoch training scenarios")
|
||||
|
||||
except Exception as e:
|
||||
except Exception as e:
|
||||
print(f"❌ Integration test failed: {e}")
|
||||
raise
|
||||
|
||||
@@ -1038,7 +1038,7 @@ Congratulations! You've successfully implemented the core components of data loa
|
||||
for epoch in range(num_epochs):
|
||||
for batch_data, batch_labels in loader:
|
||||
# Train model
|
||||
pass
|
||||
pass
|
||||
```
|
||||
4. **Explore advanced topics**: Data augmentation, distributed loading, streaming datasets!
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,80 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate small test data for data module testing.
|
||||
|
||||
This creates a small mock dataset that mimics CIFAR-10 structure but is tiny
|
||||
and doesn't require downloading anything.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import pickle
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
def generate_test_cifar10_data():
|
||||
"""Generate small test data that mimics CIFAR-10 structure."""
|
||||
|
||||
# CIFAR-10 class names
|
||||
class_names = [
|
||||
'airplane', 'automobile', 'bird', 'cat', 'deer',
|
||||
'dog', 'frog', 'horse', 'ship', 'truck'
|
||||
]
|
||||
|
||||
# Create small test dataset
|
||||
train_size = 50 # Small training set
|
||||
test_size = 20 # Small test set
|
||||
|
||||
# Generate random image data (3x32x32, values 0-255)
|
||||
train_data = np.random.randint(0, 256, size=(train_size, 3, 32, 32), dtype=np.uint8)
|
||||
train_labels = np.random.randint(0, 10, size=(train_size,), dtype=np.uint8)
|
||||
|
||||
test_data = np.random.randint(0, 256, size=(test_size, 3, 32, 32), dtype=np.uint8)
|
||||
test_labels = np.random.randint(0, 10, size=(test_size,), dtype=np.uint8)
|
||||
|
||||
# Create the data directory
|
||||
data_dir = Path(__file__).parent / "test_data"
|
||||
data_dir.mkdir(exist_ok=True)
|
||||
|
||||
# Save training data (mimics CIFAR-10 format)
|
||||
train_dict = {
|
||||
b'data': train_data.reshape(train_size, -1), # Flatten to (N, 3072)
|
||||
b'labels': train_labels.tolist(),
|
||||
b'batch_label': b'training batch 1 of 1',
|
||||
b'filenames': [f'train_image_{i}.png'.encode() for i in range(train_size)]
|
||||
}
|
||||
|
||||
with open(data_dir / "data_batch_1", "wb") as f:
|
||||
pickle.dump(train_dict, f)
|
||||
|
||||
# Save test data
|
||||
test_dict = {
|
||||
b'data': test_data.reshape(test_size, -1), # Flatten to (N, 3072)
|
||||
b'labels': test_labels.tolist(),
|
||||
b'batch_label': b'testing batch 1 of 1',
|
||||
b'filenames': [f'test_image_{i}.png'.encode() for i in range(test_size)]
|
||||
}
|
||||
|
||||
with open(data_dir / "test_batch", "wb") as f:
|
||||
pickle.dump(test_dict, f)
|
||||
|
||||
# Save metadata
|
||||
meta_dict = {
|
||||
b'label_names': [name.encode() for name in class_names],
|
||||
b'num_cases_per_batch': [train_size],
|
||||
b'num_vis': 3072 # 32*32*3
|
||||
}
|
||||
|
||||
with open(data_dir / "batches.meta", "wb") as f:
|
||||
pickle.dump(meta_dict, f)
|
||||
|
||||
print(f"✅ Generated test data:")
|
||||
print(f" - Training samples: {train_size}")
|
||||
print(f" - Test samples: {test_size}")
|
||||
print(f" - Image shape: (3, 32, 32)")
|
||||
print(f" - Classes: {len(class_names)}")
|
||||
print(f" - Saved to: {data_dir}")
|
||||
|
||||
return data_dir
|
||||
|
||||
if __name__ == "__main__":
|
||||
generate_test_cifar10_data()
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,460 +0,0 @@
|
||||
"""
|
||||
Test suite for the dataloader module.
|
||||
This tests the student implementations to ensure they work correctly.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
import os
|
||||
import tempfile
|
||||
import shutil
|
||||
import pickle
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
# Import from the main package (rock solid foundation)
|
||||
try:
|
||||
from tinytorch.core.dataloader import Dataset, DataLoader, SimpleDataset
|
||||
# These may not be implemented yet - use fallback
|
||||
try:
|
||||
from tinytorch.core.dataloader import CIFAR10Dataset, Normalizer, create_data_pipeline
|
||||
except ImportError:
|
||||
# Create mock classes for missing functionality
|
||||
class CIFAR10Dataset:
|
||||
"""Mock implementation for testing"""
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
def __len__(self):
|
||||
return 100
|
||||
def __getitem__(self, idx):
|
||||
return ([0.5] * 32 * 32 * 3, 1)
|
||||
|
||||
class Normalizer:
|
||||
"""Mock implementation for testing"""
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
def __call__(self, x):
|
||||
return x
|
||||
|
||||
def create_data_pipeline(*args, **kwargs):
|
||||
"""Mock implementation for testing"""
|
||||
return SimpleDataset([([0.5] * 10, 1)] * 100)
|
||||
|
||||
except ImportError:
|
||||
# Fallback for when module isn't exported yet
|
||||
project_root = Path(__file__).parent.parent.parent
|
||||
sys.path.append(str(project_root / "modules" / "source" / "06_dataloader"))
|
||||
from dataloader_dev import Dataset, DataLoader, CIFAR10Dataset, Normalizer, create_data_pipeline
|
||||
|
||||
from tinytorch.core.tensor import Tensor
|
||||
|
||||
def safe_numpy(tensor):
|
||||
"""Get numpy array from tensor, using .data attribute"""
|
||||
return tensor.data
|
||||
|
||||
def safe_item(tensor):
|
||||
"""Get scalar value from tensor"""
|
||||
return float(tensor.data)
|
||||
|
||||
class TestCIFAR10Dataset(Dataset):
|
||||
"""Test dataset that uses local test data instead of downloading CIFAR-10."""
|
||||
|
||||
def __init__(self, root_dir: str, train: bool = True, download: bool = True):
|
||||
"""Initialize with local test data."""
|
||||
self.root_dir = root_dir
|
||||
self.train = train
|
||||
self.download = download
|
||||
|
||||
# Use local test data
|
||||
test_data_dir = Path(__file__).parent / "test_data"
|
||||
if not test_data_dir.exists():
|
||||
raise FileNotFoundError(f"Test data not found at {test_data_dir}")
|
||||
|
||||
self._load_test_data(test_data_dir)
|
||||
|
||||
def _load_test_data(self, data_dir):
|
||||
"""Load the small test dataset."""
|
||||
# Load metadata
|
||||
with open(data_dir / "batches.meta", "rb") as f:
|
||||
meta_dict = pickle.load(f)
|
||||
|
||||
self.class_names = [name.decode() for name in meta_dict[b'label_names']]
|
||||
|
||||
# Load training or test data
|
||||
if self.train:
|
||||
with open(data_dir / "data_batch_1", "rb") as f:
|
||||
data_dict = pickle.load(f)
|
||||
else:
|
||||
with open(data_dir / "test_batch", "rb") as f:
|
||||
data_dict = pickle.load(f)
|
||||
|
||||
# Reshape data from (N, 3072) to (N, 3, 32, 32)
|
||||
self.data = data_dict[b'data'].reshape(-1, 3, 32, 32)
|
||||
self.labels = data_dict[b'labels']
|
||||
|
||||
def __getitem__(self, index: int):
|
||||
"""Get a single sample and label."""
|
||||
image = self.data[index]
|
||||
label = self.labels[index]
|
||||
|
||||
return Tensor(image.astype(np.float32)), Tensor(np.array(label))
|
||||
|
||||
def __len__(self) -> int:
|
||||
"""Get the total number of samples."""
|
||||
return len(self.data)
|
||||
|
||||
def get_num_classes(self) -> int:
|
||||
"""Get the number of classes."""
|
||||
return len(self.class_names)
|
||||
|
||||
class TestDatasetInterface:
|
||||
"""Test the base Dataset class interface (abstract class behavior)."""
|
||||
|
||||
def test_dataset_is_abstract(self):
|
||||
"""Test that Dataset base class is abstract."""
|
||||
dataset = Dataset()
|
||||
|
||||
# Should raise NotImplementedError for abstract methods
|
||||
with pytest.raises(NotImplementedError):
|
||||
dataset[0]
|
||||
|
||||
with pytest.raises(NotImplementedError):
|
||||
len(dataset)
|
||||
|
||||
with pytest.raises(NotImplementedError):
|
||||
dataset.get_num_classes()
|
||||
|
||||
def test_concrete_dataset_implementation(self):
|
||||
"""Test that concrete datasets work properly."""
|
||||
class TestDataset(Dataset):
|
||||
def __init__(self, size=10):
|
||||
self.size = size
|
||||
self.data = [np.random.randn(3, 32, 32) for _ in range(size)]
|
||||
self.labels = [i % 3 for i in range(size)]
|
||||
|
||||
def __getitem__(self, index):
|
||||
return Tensor(self.data[index]), Tensor(np.array(self.labels[index]))
|
||||
|
||||
def __len__(self):
|
||||
return self.size
|
||||
|
||||
def get_num_classes(self):
|
||||
return 3
|
||||
|
||||
dataset = TestDataset(5)
|
||||
|
||||
# Test basic functionality
|
||||
assert len(dataset) == 5
|
||||
assert dataset.get_num_classes() == 3
|
||||
|
||||
# Test indexing
|
||||
sample, label = dataset[0]
|
||||
assert sample.shape == (3, 32, 32)
|
||||
assert label.shape == ()
|
||||
|
||||
# Test get_sample_shape
|
||||
assert dataset.get_sample_shape() == (3, 32, 32)
|
||||
|
||||
class TestLocalCIFAR10Dataset:
|
||||
"""Test CIFAR-10 dataset with local test data."""
|
||||
|
||||
def test_cifar10_train_set_load(self):
|
||||
"""Test loading training set from local test data."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Use local test data
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=True, download=True)
|
||||
|
||||
# Verify basic properties
|
||||
assert len(dataset) == 50 # Our test training set size
|
||||
assert dataset.get_num_classes() == 10
|
||||
|
||||
# Test sample access
|
||||
image, label = dataset[0]
|
||||
assert image.shape == (3, 32, 32) # CIFAR-10 image shape
|
||||
assert 0 <= safe_item(label) < 10 # Valid class label
|
||||
|
||||
# Test class names
|
||||
assert len(dataset.class_names) == 10
|
||||
assert 'airplane' in dataset.class_names
|
||||
assert 'truck' in dataset.class_names
|
||||
|
||||
def test_cifar10_test_set_load(self):
|
||||
"""Test loading test set from local test data."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Use local test data
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=False, download=True)
|
||||
|
||||
# Verify test set properties
|
||||
assert len(dataset) == 20 # Our test test set size
|
||||
assert dataset.get_num_classes() == 10
|
||||
|
||||
# Test sample access
|
||||
image, label = dataset[0]
|
||||
assert image.shape == (3, 32, 32)
|
||||
assert 0 <= safe_item(label) < 10
|
||||
|
||||
def test_cifar10_data_types(self):
|
||||
"""Test that test data has correct types and ranges."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=True, download=True)
|
||||
|
||||
# Test first few samples
|
||||
for i in range(5):
|
||||
image, label = dataset[i]
|
||||
|
||||
# Check data types
|
||||
assert isinstance(image, Tensor)
|
||||
assert isinstance(label, Tensor)
|
||||
|
||||
# Check value ranges (our test data uses 0-255 range)
|
||||
assert 0 <= safe_numpy(image).min() <= 255
|
||||
assert 0 <= safe_numpy(image).max() <= 255
|
||||
|
||||
# Check label is valid class
|
||||
assert 0 <= safe_item(label) < 10
|
||||
|
||||
class TestDataLoader:
|
||||
"""Test DataLoader with local test data."""
|
||||
|
||||
def setup_method(self):
|
||||
"""Set up local test dataset for DataLoader tests."""
|
||||
self.temp_dir = tempfile.mkdtemp()
|
||||
# Use local test data
|
||||
self.dataset = TestCIFAR10Dataset(self.temp_dir, train=True, download=True)
|
||||
|
||||
def teardown_method(self):
|
||||
"""Clean up temporary directory."""
|
||||
shutil.rmtree(self.temp_dir, ignore_errors=True)
|
||||
|
||||
def test_dataloader_creation(self):
|
||||
"""Test DataLoader creation with local test data."""
|
||||
# Test with default parameters
|
||||
loader = DataLoader(self.dataset, batch_size=16)
|
||||
assert len(loader) == 4 # 50 samples / 16 batch_size = 4 batches (rounded up)
|
||||
|
||||
# Test with custom batch size
|
||||
loader = DataLoader(self.dataset, batch_size=10)
|
||||
assert len(loader) == 5 # 50 samples / 10 batch_size = 5 batches
|
||||
|
||||
def test_dataloader_iteration_test_data(self):
|
||||
"""Test DataLoader iteration with local test data."""
|
||||
loader = DataLoader(self.dataset, batch_size=8, shuffle=True)
|
||||
|
||||
batch_count = 0
|
||||
total_samples = 0
|
||||
|
||||
for batch_data, batch_labels in loader:
|
||||
batch_count += 1
|
||||
batch_size = batch_data.shape[0]
|
||||
total_samples += batch_size
|
||||
|
||||
# Check batch shapes
|
||||
assert batch_data.shape[1:] == (3, 32, 32) # CIFAR-10 image shape
|
||||
assert batch_labels.shape == (batch_size,)
|
||||
|
||||
# Check data types
|
||||
assert isinstance(batch_data, Tensor)
|
||||
assert isinstance(batch_labels, Tensor)
|
||||
|
||||
# Check test data properties
|
||||
assert 0 <= safe_numpy(batch_data).min() <= 255
|
||||
assert 0 <= safe_numpy(batch_data).max() <= 255
|
||||
assert 0 <= safe_numpy(batch_labels).min() < 10
|
||||
assert 0 <= safe_numpy(batch_labels).max() < 10
|
||||
|
||||
# Check batch size
|
||||
assert batch_size <= 8
|
||||
|
||||
if batch_count >= 3: # Test first few batches
|
||||
break
|
||||
|
||||
assert batch_count > 0
|
||||
assert total_samples <= len(self.dataset)
|
||||
|
||||
def test_dataloader_shuffling_test_data(self):
|
||||
"""Test that shuffling works with test data."""
|
||||
loader1 = DataLoader(self.dataset, batch_size=10, shuffle=True)
|
||||
loader2 = DataLoader(self.dataset, batch_size=10, shuffle=True)
|
||||
|
||||
# Get first batch from each loader
|
||||
batch1_data, batch1_labels = next(iter(loader1))
|
||||
batch2_data, batch2_labels = next(iter(loader2))
|
||||
|
||||
# With shuffling, batches should likely be different
|
||||
# (This test might occasionally fail due to randomness, but very unlikely)
|
||||
different = not np.array_equal(safe_numpy(batch1_labels), safe_numpy(batch2_labels))
|
||||
# Note: We don't assert this because random shuffling might occasionally produce same order
|
||||
|
||||
def test_dataloader_no_shuffle_test_data(self):
|
||||
"""Test DataLoader without shuffling uses test data in order."""
|
||||
loader = DataLoader(self.dataset, batch_size=10, shuffle=False)
|
||||
|
||||
# Get first batch
|
||||
batch_data, batch_labels = next(iter(loader))
|
||||
|
||||
# Without shuffling, should get first 10 samples in order
|
||||
expected_samples = [self.dataset[i] for i in range(10)]
|
||||
expected_labels = [safe_item(sample[1]) for sample in expected_samples]
|
||||
|
||||
np.testing.assert_array_equal(safe_numpy(batch_labels), expected_labels)
|
||||
|
||||
class TestNormalizer:
|
||||
"""Test Normalizer with local test data."""
|
||||
|
||||
def setup_method(self):
|
||||
"""Set up local test data for normalization tests."""
|
||||
self.temp_dir = tempfile.mkdtemp()
|
||||
dataset = TestCIFAR10Dataset(self.temp_dir, train=True, download=True)
|
||||
|
||||
# Get first 20 samples for testing
|
||||
self.test_data = []
|
||||
for i in range(20):
|
||||
image, _ = dataset[i]
|
||||
self.test_data.append(image)
|
||||
|
||||
def teardown_method(self):
|
||||
"""Clean up temporary directory."""
|
||||
shutil.rmtree(self.temp_dir, ignore_errors=True)
|
||||
|
||||
def test_normalizer_fit_test_data(self):
|
||||
"""Test Normalizer fit with local test data."""
|
||||
normalizer = Normalizer()
|
||||
normalizer.fit(self.test_data)
|
||||
|
||||
# Check computed statistics
|
||||
assert normalizer.mean is not None
|
||||
assert normalizer.std is not None
|
||||
|
||||
# Our test data has pixel values 0-255, so mean should be reasonable
|
||||
assert 0 <= normalizer.mean <= 255
|
||||
assert normalizer.std > 0 # Should have some variation
|
||||
|
||||
def test_normalizer_transform_test_data(self):
|
||||
"""Test Normalizer transform with local test data."""
|
||||
normalizer = Normalizer()
|
||||
normalizer.fit(self.test_data)
|
||||
|
||||
# Transform single sample
|
||||
sample = self.test_data[0]
|
||||
normalized = normalizer.transform(sample)
|
||||
|
||||
# Check that normalization changes the values
|
||||
assert not np.allclose(safe_numpy(sample), safe_numpy(normalized))
|
||||
|
||||
# Check that normalized data has different statistics
|
||||
original_mean = np.mean(safe_numpy(sample))
|
||||
normalized_mean = np.mean(safe_numpy(normalized))
|
||||
assert abs(normalized_mean) < abs(original_mean) # Should be closer to 0
|
||||
|
||||
def test_normalizer_transform_batch_test_data(self):
|
||||
"""Test Normalizer with batch of test data."""
|
||||
normalizer = Normalizer()
|
||||
normalizer.fit(self.test_data)
|
||||
|
||||
# Transform batch
|
||||
batch = self.test_data[:5]
|
||||
normalized_batch = normalizer.transform(batch)
|
||||
|
||||
# Check that we get same number of samples
|
||||
assert len(normalized_batch) == len(batch)
|
||||
|
||||
# Check that each sample is normalized
|
||||
for original, normalized in zip(batch, normalized_batch):
|
||||
assert not np.allclose(safe_numpy(original), safe_numpy(normalized))
|
||||
|
||||
class TestDataPipeline:
|
||||
"""Test complete data pipeline with local test data."""
|
||||
|
||||
def test_create_data_pipeline_test_data(self):
|
||||
"""Test creating data pipeline with local test data."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Copy test data to temp directory
|
||||
test_data_dir = Path(__file__).parent / "test_data"
|
||||
import shutil
|
||||
shutil.copytree(test_data_dir, temp_dir + "/test_data")
|
||||
|
||||
# Create pipeline (this would normally download CIFAR-10)
|
||||
# For testing, we'll create a simple pipeline manually
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=True, download=True)
|
||||
dataloader = DataLoader(dataset, batch_size=8, shuffle=True)
|
||||
|
||||
# Test pipeline components
|
||||
assert len(dataset) == 50 # Our test training set
|
||||
assert len(dataloader) == 7 # 50 samples / 8 batch_size = 7 batches
|
||||
|
||||
# Test that we can iterate through the pipeline
|
||||
batch_count = 0
|
||||
for batch_data, batch_labels in dataloader:
|
||||
batch_count += 1
|
||||
assert batch_data.shape[1:] == (3, 32, 32)
|
||||
assert batch_labels.shape[0] <= 8
|
||||
|
||||
if batch_count >= 3: # Test first few batches
|
||||
break
|
||||
|
||||
assert batch_count > 0
|
||||
|
||||
def test_pipeline_normalization_test_data(self):
|
||||
"""Test pipeline with normalization using local test data."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=True, download=True)
|
||||
|
||||
# Get some samples for normalization
|
||||
samples = [dataset[i][0] for i in range(10)]
|
||||
|
||||
# Create and fit normalizer
|
||||
normalizer = Normalizer()
|
||||
normalizer.fit(samples)
|
||||
|
||||
# Test that normalization works
|
||||
normalized = normalizer.transform(samples[0])
|
||||
assert not np.allclose(safe_numpy(samples[0]), safe_numpy(normalized))
|
||||
|
||||
# Test with dataloader
|
||||
dataloader = DataLoader(dataset, batch_size=5, shuffle=False)
|
||||
batch_data, batch_labels = next(iter(dataloader))
|
||||
|
||||
# Normalize batch
|
||||
normalized_batch = []
|
||||
for i in range(batch_data.shape[0]):
|
||||
sample = Tensor(batch_data.data[i])
|
||||
normalized_sample = normalizer.transform(sample)
|
||||
normalized_batch.append(normalized_sample.data)
|
||||
|
||||
normalized_batch = Tensor(np.stack(normalized_batch))
|
||||
|
||||
# Check that batch normalization works
|
||||
assert normalized_batch.shape == batch_data.shape
|
||||
assert not np.allclose(safe_numpy(batch_data), safe_numpy(normalized_batch))
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases with local test data."""
|
||||
|
||||
def test_small_batch_size_test_data(self):
|
||||
"""Test with very small batch size using local test data."""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Create small dataset
|
||||
dataset = TestCIFAR10Dataset(temp_dir, train=True, download=True)
|
||||
|
||||
# Use batch size of 1
|
||||
loader = DataLoader(dataset, batch_size=1, shuffle=False)
|
||||
|
||||
# Test first few batches
|
||||
batch_count = 0
|
||||
for batch_data, batch_labels in loader:
|
||||
assert batch_data.shape == (1, 3, 32, 32)
|
||||
assert batch_labels.shape == (1,)
|
||||
|
||||
batch_count += 1
|
||||
if batch_count >= 5:
|
||||
break
|
||||
|
||||
assert batch_count == 5
|
||||
|
||||
def run_data_tests():
|
||||
"""Run all data tests."""
|
||||
pytest.main([__file__, "-v"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_data_tests()
|
||||
@@ -38,7 +38,7 @@ from collections import defaultdict
|
||||
|
||||
# Import our existing components
|
||||
try:
|
||||
from tinytorch.core.tensor import Tensor
|
||||
from tinytorch.core.tensor import Tensor
|
||||
except ImportError:
|
||||
# For development, import from local modules
|
||||
import os
|
||||
@@ -123,7 +123,7 @@ Let's build the engine that powers modern AI!
|
||||
### What is a Variable?
|
||||
A **Variable** wraps a Tensor and tracks:
|
||||
- **Data**: The actual values (forward pass)
|
||||
- **Gradient**: The computed gradients (backward pass)
|
||||
- **Gradient**: The computed gradients (backward pass)
|
||||
- **Computation history**: How this Variable was created
|
||||
- **Backward function**: How to compute gradients
|
||||
|
||||
@@ -167,7 +167,7 @@ class Variable:
|
||||
requires_grad: bool = True, grad_fn: Optional[Callable] = None):
|
||||
"""
|
||||
Create a Variable with gradient tracking.
|
||||
|
||||
|
||||
TODO: Implement Variable initialization with gradient tracking.
|
||||
|
||||
STEP-BY-STEP IMPLEMENTATION:
|
||||
@@ -275,33 +275,33 @@ class Variable:
|
||||
if self.requires_grad:
|
||||
if self.grad is None:
|
||||
self.grad = gradient
|
||||
else:
|
||||
else:
|
||||
# Accumulate gradients
|
||||
self.grad = Variable(self.grad.data.data + gradient.data.data)
|
||||
|
||||
if self.grad_fn is not None:
|
||||
self.grad_fn(gradient)
|
||||
if self.grad_fn is not None:
|
||||
self.grad_fn(gradient)
|
||||
### END SOLUTION
|
||||
|
||||
|
||||
def zero_grad(self) -> None:
|
||||
"""Reset gradients to zero."""
|
||||
self.grad = None
|
||||
|
||||
|
||||
def __add__(self, other: Union['Variable', float, int]) -> 'Variable':
|
||||
"""Addition operator: self + other"""
|
||||
return add(self, other)
|
||||
|
||||
|
||||
def __mul__(self, other: Union['Variable', float, int]) -> 'Variable':
|
||||
"""Multiplication operator: self * other"""
|
||||
return multiply(self, other)
|
||||
|
||||
|
||||
def __sub__(self, other: Union['Variable', float, int]) -> 'Variable':
|
||||
"""Subtraction operator: self - other"""
|
||||
return subtract(self, other)
|
||||
|
||||
|
||||
def __truediv__(self, other: Union['Variable', float, int]) -> 'Variable':
|
||||
"""Division operator: self / other"""
|
||||
return divide(self, other)
|
||||
return divide(self, other)
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
@@ -817,12 +817,12 @@ Let's see how autograd enables neural network training:
|
||||
4. **Parameter update**: Update weights using gradients
|
||||
|
||||
### Example: Simple Linear Regression
|
||||
```python
|
||||
```python
|
||||
# Model: y = wx + b
|
||||
w = Variable(0.5, requires_grad=True)
|
||||
b = Variable(0.1, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
# Forward pass
|
||||
prediction = w * x + b
|
||||
|
||||
# Loss: mean squared error
|
||||
@@ -870,7 +870,7 @@ def test_neural_network_training():
|
||||
x = Variable(x_val, requires_grad=False)
|
||||
target = Variable(y_val, requires_grad=False)
|
||||
|
||||
# Forward pass
|
||||
# Forward pass
|
||||
prediction = add(multiply(w, x), b) # wx + b
|
||||
|
||||
# Loss: squared error
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,698 +0,0 @@
|
||||
"""
|
||||
Test suite for the autograd module.
|
||||
This tests the autograd implementations using mock classes to avoid cross-module dependencies.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import numpy as np
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add the module path for testing
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
|
||||
|
||||
# Import the autograd module directly
|
||||
from autograd_dev import Variable, add, multiply, subtract, divide, relu_with_grad, sigmoid_with_grad
|
||||
|
||||
|
||||
class MockTensor:
|
||||
"""Mock Tensor class for testing autograd without dependencies."""
|
||||
|
||||
def __init__(self, data):
|
||||
if isinstance(data, (int, float)):
|
||||
self._data = np.array(data, dtype=np.float32)
|
||||
elif isinstance(data, list):
|
||||
self._data = np.array(data, dtype=np.float32)
|
||||
elif isinstance(data, np.ndarray):
|
||||
self._data = data.astype(np.float32)
|
||||
else:
|
||||
self._data = np.array(data, dtype=np.float32)
|
||||
|
||||
@property
|
||||
def data(self):
|
||||
return self._data
|
||||
|
||||
@property
|
||||
def shape(self):
|
||||
return self._data.shape
|
||||
|
||||
@property
|
||||
def size(self):
|
||||
return self._data.size
|
||||
|
||||
def __add__(self, other):
|
||||
if isinstance(other, MockTensor):
|
||||
return MockTensor(self._data + other._data)
|
||||
else:
|
||||
return MockTensor(self._data + other)
|
||||
|
||||
def __mul__(self, other):
|
||||
if isinstance(other, MockTensor):
|
||||
return MockTensor(self._data * other._data)
|
||||
else:
|
||||
return MockTensor(self._data * other)
|
||||
|
||||
def __sub__(self, other):
|
||||
if isinstance(other, MockTensor):
|
||||
return MockTensor(self._data - other._data)
|
||||
else:
|
||||
return MockTensor(self._data - other)
|
||||
|
||||
def __truediv__(self, other):
|
||||
if isinstance(other, MockTensor):
|
||||
return MockTensor(self._data / other._data)
|
||||
else:
|
||||
return MockTensor(self._data / other)
|
||||
|
||||
def item(self):
|
||||
return self._data.item()
|
||||
|
||||
|
||||
class TestVariableCreation:
|
||||
"""Test Variable creation and basic properties."""
|
||||
|
||||
def test_variable_from_scalar(self):
|
||||
"""Test creating Variable from scalar values."""
|
||||
# Float scalar
|
||||
v1 = Variable(5.0)
|
||||
assert v1.shape == ()
|
||||
assert v1.size == 1
|
||||
assert v1.requires_grad == True
|
||||
assert v1.is_leaf == True
|
||||
assert v1.grad is None
|
||||
|
||||
# Integer scalar
|
||||
v2 = Variable(42)
|
||||
assert v2.shape == ()
|
||||
assert v2.size == 1
|
||||
assert abs(v2.data.data.item() - 42.0) < 1e-6
|
||||
|
||||
def test_variable_from_list(self):
|
||||
"""Test creating Variable from list."""
|
||||
v = Variable([1.0, 2.0, 3.0])
|
||||
assert v.shape == (3,)
|
||||
assert v.size == 3
|
||||
assert v.requires_grad == True
|
||||
assert v.is_leaf == True
|
||||
np.testing.assert_array_almost_equal(v.data.data, [1.0, 2.0, 3.0])
|
||||
|
||||
def test_variable_from_numpy(self):
|
||||
"""Test creating Variable from numpy array."""
|
||||
arr = np.array([[1.0, 2.0], [3.0, 4.0]])
|
||||
v = Variable(arr)
|
||||
assert v.shape == (2, 2)
|
||||
assert v.size == 4
|
||||
np.testing.assert_array_almost_equal(v.data.data, arr)
|
||||
|
||||
def test_variable_requires_grad_flag(self):
|
||||
"""Test requires_grad flag functionality."""
|
||||
v1 = Variable(5.0, requires_grad=True)
|
||||
assert v1.requires_grad == True
|
||||
|
||||
v2 = Variable(5.0, requires_grad=False)
|
||||
assert v2.requires_grad == False
|
||||
|
||||
def test_variable_with_grad_fn(self):
|
||||
"""Test Variable with gradient function (non-leaf)."""
|
||||
def dummy_grad_fn(grad):
|
||||
pass
|
||||
|
||||
v = Variable(5.0, requires_grad=True, grad_fn=dummy_grad_fn)
|
||||
assert v.requires_grad == True
|
||||
assert v.is_leaf == False
|
||||
assert v.grad_fn == dummy_grad_fn
|
||||
|
||||
def test_variable_repr(self):
|
||||
"""Test string representation of Variable."""
|
||||
v = Variable(5.0)
|
||||
repr_str = repr(v)
|
||||
assert 'Variable' in repr_str
|
||||
assert 'requires_grad' in repr_str
|
||||
|
||||
|
||||
class TestBasicOperations:
|
||||
"""Test basic arithmetic operations with gradient tracking."""
|
||||
|
||||
def test_addition_operation(self):
|
||||
"""Test addition operation and gradients."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
y = Variable(3.0, requires_grad=True)
|
||||
z = add(x, y)
|
||||
|
||||
# Test forward pass
|
||||
assert abs(z.data.data.item() - 5.0) < 1e-6
|
||||
assert z.requires_grad == True
|
||||
assert z.is_leaf == False
|
||||
|
||||
# Test backward pass
|
||||
z.backward()
|
||||
assert abs(x.grad.data.data.item() - 1.0) < 1e-6
|
||||
assert abs(y.grad.data.data.item() - 1.0) < 1e-6
|
||||
|
||||
def test_multiplication_operation(self):
|
||||
"""Test multiplication operation and gradients."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
y = Variable(3.0, requires_grad=True)
|
||||
z = multiply(x, y)
|
||||
|
||||
# Test forward pass
|
||||
assert abs(z.data.data.item() - 6.0) < 1e-6
|
||||
assert z.requires_grad == True
|
||||
assert z.is_leaf == False
|
||||
|
||||
# Test backward pass
|
||||
z.backward()
|
||||
assert abs(x.grad.data.data.item() - 3.0) < 1e-6 # dy/dx = y = 3
|
||||
assert abs(y.grad.data.data.item() - 2.0) < 1e-6 # dy/dy = x = 2
|
||||
|
||||
def test_subtraction_operation(self):
|
||||
"""Test subtraction operation and gradients."""
|
||||
x = Variable(5.0, requires_grad=True)
|
||||
y = Variable(3.0, requires_grad=True)
|
||||
z = subtract(x, y)
|
||||
|
||||
# Test forward pass
|
||||
assert abs(z.data.data.item() - 2.0) < 1e-6
|
||||
assert z.requires_grad == True
|
||||
assert z.is_leaf == False
|
||||
|
||||
# Test backward pass
|
||||
z.backward()
|
||||
assert abs(x.grad.data.data.item() - 1.0) < 1e-6 # dz/dx = 1
|
||||
assert abs(y.grad.data.data.item() - (-1.0)) < 1e-6 # dz/dy = -1
|
||||
|
||||
def test_division_operation(self):
|
||||
"""Test division operation and gradients."""
|
||||
x = Variable(6.0, requires_grad=True)
|
||||
y = Variable(2.0, requires_grad=True)
|
||||
z = divide(x, y)
|
||||
|
||||
# Test forward pass
|
||||
assert abs(z.data.data.item() - 3.0) < 1e-6
|
||||
assert z.requires_grad == True
|
||||
assert z.is_leaf == False
|
||||
|
||||
# Test backward pass
|
||||
z.backward()
|
||||
assert abs(x.grad.data.data.item() - 0.5) < 1e-6 # dz/dx = 1/y = 1/2
|
||||
assert abs(y.grad.data.data.item() - (-1.5)) < 1e-6 # dz/dy = -x/y² = -6/4
|
||||
|
||||
def test_operations_with_constants(self):
|
||||
"""Test operations with constant values."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Addition with constant
|
||||
z1 = add(x, 3.0)
|
||||
assert abs(z1.data.data.item() - 5.0) < 1e-6
|
||||
z1.backward()
|
||||
assert abs(x.grad.data.data.item() - 1.0) < 1e-6
|
||||
|
||||
# Reset gradient
|
||||
x.zero_grad()
|
||||
|
||||
# Multiplication with constant
|
||||
z2 = multiply(x, 4.0)
|
||||
assert abs(z2.data.data.item() - 8.0) < 1e-6
|
||||
z2.backward()
|
||||
assert abs(x.grad.data.data.item() - 4.0) < 1e-6
|
||||
|
||||
def test_no_grad_propagation(self):
|
||||
"""Test that gradients don't propagate when requires_grad=False."""
|
||||
x = Variable(2.0, requires_grad=False)
|
||||
y = Variable(3.0, requires_grad=True)
|
||||
z = add(x, y)
|
||||
|
||||
z.backward()
|
||||
assert x.grad is None # No gradient for x
|
||||
assert abs(y.grad.data.data.item() - 1.0) < 1e-6
|
||||
|
||||
|
||||
class TestChainRule:
|
||||
"""Test chain rule implementation with complex expressions."""
|
||||
|
||||
def test_simple_chain_rule(self):
|
||||
"""Test f(x, y) = (x + y) * (x - y) = x² - y²."""
|
||||
x = Variable(3.0, requires_grad=True)
|
||||
y = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
sum_xy = add(x, y)
|
||||
diff_xy = subtract(x, y)
|
||||
result = multiply(sum_xy, diff_xy)
|
||||
|
||||
# Check forward pass
|
||||
assert abs(result.data.data.item() - 5.0) < 1e-6 # (3+2)*(3-2) = 5
|
||||
|
||||
# Backward pass
|
||||
result.backward()
|
||||
|
||||
# Check gradients: df/dx = 2x = 6, df/dy = -2y = -4
|
||||
assert abs(x.grad.data.data.item() - 6.0) < 1e-6
|
||||
assert abs(y.grad.data.data.item() - (-4.0)) < 1e-6
|
||||
|
||||
def test_cubic_function(self):
|
||||
"""Test f(x) = x³ using x * x * x."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
x_squared = multiply(x, x)
|
||||
x_cubed = multiply(x_squared, x)
|
||||
|
||||
# Check forward pass
|
||||
assert abs(x_cubed.data.data.item() - 8.0) < 1e-6 # 2³ = 8
|
||||
|
||||
# Backward pass
|
||||
x_cubed.backward()
|
||||
|
||||
# Check gradient: df/dx = 3x² = 12
|
||||
assert abs(x.grad.data.data.item() - 12.0) < 1e-6
|
||||
|
||||
def test_complex_expression(self):
|
||||
"""Test f(x, y) = (x * y) + (x / y)."""
|
||||
x = Variable(4.0, requires_grad=True)
|
||||
y = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
product = multiply(x, y)
|
||||
quotient = divide(x, y)
|
||||
result = add(product, quotient)
|
||||
|
||||
# Check forward pass: (4*2) + (4/2) = 8 + 2 = 10
|
||||
assert abs(result.data.data.item() - 10.0) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
result.backward()
|
||||
|
||||
# Check gradients: df/dx = y + 1/y = 2 + 0.5 = 2.5
|
||||
# df/dy = x - x/y² = 4 - 4/4 = 3
|
||||
assert abs(x.grad.data.data.item() - 2.5) < 1e-6
|
||||
assert abs(y.grad.data.data.item() - 3.0) < 1e-6
|
||||
|
||||
def test_gradient_accumulation(self):
|
||||
"""Test that gradients accumulate correctly."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# First computation
|
||||
y1 = multiply(x, 3.0)
|
||||
y1.backward()
|
||||
first_grad = x.grad.data.data.item()
|
||||
|
||||
# Second computation (should accumulate)
|
||||
y2 = multiply(x, 4.0)
|
||||
y2.backward()
|
||||
second_grad = x.grad.data.data.item()
|
||||
|
||||
# Gradient should accumulate: 3 + 4 = 7
|
||||
assert abs(second_grad - 7.0) < 1e-6
|
||||
|
||||
def test_zero_grad_functionality(self):
|
||||
"""Test zero_grad functionality."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
y = multiply(x, 3.0)
|
||||
y.backward()
|
||||
|
||||
# Check gradient exists
|
||||
assert x.grad is not None
|
||||
assert abs(x.grad.data.data.item() - 3.0) < 1e-6
|
||||
|
||||
# Zero the gradient
|
||||
x.zero_grad()
|
||||
assert abs(x.grad.data.data.item() - 0.0) < 1e-6
|
||||
|
||||
|
||||
class TestActivationGradients:
|
||||
"""Test activation functions with gradient computation."""
|
||||
|
||||
def test_relu_activation(self):
|
||||
"""Test ReLU activation and its gradient."""
|
||||
# Test positive input
|
||||
x1 = Variable(2.0, requires_grad=True)
|
||||
y1 = relu_with_grad(x1)
|
||||
|
||||
assert abs(y1.data.data.item() - 2.0) < 1e-6 # ReLU(2) = 2
|
||||
y1.backward()
|
||||
assert abs(x1.grad.data.data.item() - 1.0) < 1e-6 # gradient = 1 for x > 0
|
||||
|
||||
# Test negative input
|
||||
x2 = Variable(-1.0, requires_grad=True)
|
||||
y2 = relu_with_grad(x2)
|
||||
|
||||
assert abs(y2.data.data.item() - 0.0) < 1e-6 # ReLU(-1) = 0
|
||||
y2.backward()
|
||||
assert abs(x2.grad.data.data.item() - 0.0) < 1e-6 # gradient = 0 for x < 0
|
||||
|
||||
# Test zero input
|
||||
x3 = Variable(0.0, requires_grad=True)
|
||||
y3 = relu_with_grad(x3)
|
||||
|
||||
assert abs(y3.data.data.item() - 0.0) < 1e-6 # ReLU(0) = 0
|
||||
y3.backward()
|
||||
assert abs(x3.grad.data.data.item() - 0.0) < 1e-6 # gradient = 0 for x = 0
|
||||
|
||||
def test_sigmoid_activation(self):
|
||||
"""Test Sigmoid activation and its gradient."""
|
||||
# Test zero input
|
||||
x1 = Variable(0.0, requires_grad=True)
|
||||
y1 = sigmoid_with_grad(x1)
|
||||
|
||||
assert abs(y1.data.data.item() - 0.5) < 1e-6 # sigmoid(0) = 0.5
|
||||
y1.backward()
|
||||
assert abs(x1.grad.data.data.item() - 0.25) < 1e-6 # gradient = 0.5 * 0.5 = 0.25
|
||||
|
||||
# Test positive input
|
||||
x2 = Variable(2.0, requires_grad=True)
|
||||
y2 = sigmoid_with_grad(x2)
|
||||
|
||||
expected_sigmoid = 1.0 / (1.0 + np.exp(-2.0))
|
||||
assert abs(y2.data.data.item() - expected_sigmoid) < 1e-6
|
||||
|
||||
y2.backward()
|
||||
expected_grad = expected_sigmoid * (1.0 - expected_sigmoid)
|
||||
assert abs(x2.grad.data.data.item() - expected_grad) < 1e-6
|
||||
|
||||
# Test negative input
|
||||
x3 = Variable(-1.0, requires_grad=True)
|
||||
y3 = sigmoid_with_grad(x3)
|
||||
|
||||
expected_sigmoid = 1.0 / (1.0 + np.exp(1.0))
|
||||
assert abs(y3.data.data.item() - expected_sigmoid) < 1e-6
|
||||
|
||||
y3.backward()
|
||||
expected_grad = expected_sigmoid * (1.0 - expected_sigmoid)
|
||||
assert abs(x3.grad.data.data.item() - expected_grad) < 1e-6
|
||||
|
||||
def test_activation_chaining(self):
|
||||
"""Test chaining activation functions."""
|
||||
x = Variable(1.0, requires_grad=True)
|
||||
|
||||
# Chain: x -> ReLU -> Sigmoid
|
||||
relu_out = relu_with_grad(x)
|
||||
sigmoid_out = sigmoid_with_grad(relu_out)
|
||||
|
||||
# Forward pass
|
||||
expected_relu = 1.0 # ReLU(1) = 1
|
||||
expected_sigmoid = 1.0 / (1.0 + np.exp(-1.0)) # sigmoid(1)
|
||||
|
||||
assert abs(relu_out.data.data.item() - expected_relu) < 1e-6
|
||||
assert abs(sigmoid_out.data.data.item() - expected_sigmoid) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
sigmoid_out.backward()
|
||||
|
||||
# Check that gradient flows through both activations
|
||||
assert x.grad is not None
|
||||
assert abs(x.grad.data.data.item()) > 1e-6 # Should have non-zero gradient
|
||||
|
||||
|
||||
class TestNeuralNetworkScenarios:
|
||||
"""Test autograd in realistic neural network scenarios."""
|
||||
|
||||
def test_simple_linear_layer(self):
|
||||
"""Test simple linear transformation: y = Wx + b."""
|
||||
# Input
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Parameters
|
||||
w = Variable(0.5, requires_grad=True)
|
||||
b = Variable(0.1, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
linear_out = add(multiply(x, w), b) # y = x*w + b = 2*0.5 + 0.1 = 1.1
|
||||
|
||||
assert abs(linear_out.data.data.item() - 1.1) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
linear_out.backward()
|
||||
|
||||
# Check gradients
|
||||
assert abs(x.grad.data.data.item() - 0.5) < 1e-6 # dy/dx = w = 0.5
|
||||
assert abs(w.grad.data.data.item() - 2.0) < 1e-6 # dy/dw = x = 2.0
|
||||
assert abs(b.grad.data.data.item() - 1.0) < 1e-6 # dy/db = 1 = 1.0
|
||||
|
||||
def test_two_layer_network(self):
|
||||
"""Test two-layer neural network."""
|
||||
# Input
|
||||
x = Variable(1.0, requires_grad=True)
|
||||
|
||||
# Layer 1 parameters
|
||||
w1 = Variable(2.0, requires_grad=True)
|
||||
b1 = Variable(0.5, requires_grad=True)
|
||||
|
||||
# Layer 2 parameters
|
||||
w2 = Variable(1.5, requires_grad=True)
|
||||
b2 = Variable(0.2, requires_grad=True)
|
||||
|
||||
# Forward pass
|
||||
# Layer 1: h = x*w1 + b1 = 1*2 + 0.5 = 2.5
|
||||
h = add(multiply(x, w1), b1)
|
||||
# ReLU activation
|
||||
h_relu = relu_with_grad(h) # ReLU(2.5) = 2.5
|
||||
# Layer 2: y = h*w2 + b2 = 2.5*1.5 + 0.2 = 3.95
|
||||
y = add(multiply(h_relu, w2), b2)
|
||||
|
||||
assert abs(y.data.data.item() - 3.95) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
y.backward()
|
||||
|
||||
# Check that all parameters have gradients
|
||||
assert x.grad is not None
|
||||
assert w1.grad is not None
|
||||
assert b1.grad is not None
|
||||
assert w2.grad is not None
|
||||
assert b2.grad is not None
|
||||
|
||||
# Check specific gradient values
|
||||
assert abs(b2.grad.data.data.item() - 1.0) < 1e-6 # dy/db2 = 1
|
||||
assert abs(w2.grad.data.data.item() - 2.5) < 1e-6 # dy/dw2 = h_relu = 2.5
|
||||
assert abs(b1.grad.data.data.item() - 1.5) < 1e-6 # dy/db1 = w2 = 1.5
|
||||
assert abs(w1.grad.data.data.item() - 1.5) < 1e-6 # dy/dw1 = x * w2 = 1 * 1.5
|
||||
assert abs(x.grad.data.data.item() - 3.0) < 1e-6 # dy/dx = w1 * w2 = 2 * 1.5
|
||||
|
||||
def test_loss_computation(self):
|
||||
"""Test loss computation with gradients."""
|
||||
# Prediction and target
|
||||
pred = Variable(3.0, requires_grad=True)
|
||||
target = Variable(2.0, requires_grad=False)
|
||||
|
||||
# Mean squared error: loss = (pred - target)²
|
||||
diff = subtract(pred, target) # 3 - 2 = 1
|
||||
loss = multiply(diff, diff) # 1² = 1
|
||||
|
||||
assert abs(loss.data.data.item() - 1.0) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
loss.backward()
|
||||
|
||||
# Check gradient: d_loss/d_pred = 2 * (pred - target) = 2 * 1 = 2
|
||||
assert abs(pred.grad.data.data.item() - 2.0) < 1e-6
|
||||
assert target.grad is None # No gradient for target
|
||||
|
||||
def test_batch_processing_simulation(self):
|
||||
"""Test simulation of batch processing."""
|
||||
# Simulate batch of 3 samples
|
||||
x1 = Variable(1.0, requires_grad=True)
|
||||
x2 = Variable(2.0, requires_grad=True)
|
||||
x3 = Variable(3.0, requires_grad=True)
|
||||
|
||||
# Shared parameters
|
||||
w = Variable(0.5, requires_grad=True)
|
||||
b = Variable(0.1, requires_grad=True)
|
||||
|
||||
# Forward pass for each sample
|
||||
y1 = add(multiply(x1, w), b) # 1*0.5 + 0.1 = 0.6
|
||||
y2 = add(multiply(x2, w), b) # 2*0.5 + 0.1 = 1.1
|
||||
y3 = add(multiply(x3, w), b) # 3*0.5 + 0.1 = 1.6
|
||||
|
||||
# Compute batch loss (sum of individual losses)
|
||||
loss1 = multiply(y1, y1) # 0.6² = 0.36
|
||||
loss2 = multiply(y2, y2) # 1.1² = 1.21
|
||||
loss3 = multiply(y3, y3) # 1.6² = 2.56
|
||||
|
||||
batch_loss = add(add(loss1, loss2), loss3) # 0.36 + 1.21 + 2.56 = 4.13
|
||||
|
||||
assert abs(batch_loss.data.data.item() - 4.13) < 1e-6
|
||||
|
||||
# Backward pass
|
||||
batch_loss.backward()
|
||||
|
||||
# Check that gradients accumulated for shared parameters
|
||||
assert w.grad is not None
|
||||
assert b.grad is not None
|
||||
|
||||
# w gradient should be sum of individual contributions
|
||||
# dL/dw = 2*y1*x1 + 2*y2*x2 + 2*y3*x3 = 2*(0.6*1 + 1.1*2 + 1.6*3) = 2*7.6 = 15.2
|
||||
expected_w_grad = 2 * (0.6*1 + 1.1*2 + 1.6*3)
|
||||
assert abs(w.grad.data.data.item() - expected_w_grad) < 1e-6
|
||||
|
||||
# b gradient should be sum of individual contributions
|
||||
# dL/db = 2*y1 + 2*y2 + 2*y3 = 2*(0.6 + 1.1 + 1.6) = 2*3.3 = 6.6
|
||||
expected_b_grad = 2 * (0.6 + 1.1 + 1.6)
|
||||
assert abs(b.grad.data.data.item() - expected_b_grad) < 1e-6
|
||||
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and error conditions."""
|
||||
|
||||
def test_zero_division_handling(self):
|
||||
"""Test division by zero handling."""
|
||||
x = Variable(1.0, requires_grad=True)
|
||||
y = Variable(0.0, requires_grad=True)
|
||||
|
||||
# This should not crash but may produce inf/nan
|
||||
z = divide(x, y)
|
||||
|
||||
# Check that the operation completes
|
||||
assert z.data.data.item() == np.inf or np.isnan(z.data.data.item())
|
||||
|
||||
def test_large_gradient_values(self):
|
||||
"""Test handling of large gradient values."""
|
||||
x = Variable(100.0, requires_grad=True)
|
||||
y = Variable(100.0, requires_grad=True)
|
||||
|
||||
# Large multiplication
|
||||
z = multiply(x, y) # 100 * 100 = 10000
|
||||
z.backward()
|
||||
|
||||
# Gradients should be large but finite
|
||||
assert np.isfinite(x.grad.data.data.item())
|
||||
assert np.isfinite(y.grad.data.data.item())
|
||||
assert abs(x.grad.data.data.item() - 100.0) < 1e-6
|
||||
assert abs(y.grad.data.data.item() - 100.0) < 1e-6
|
||||
|
||||
def test_very_small_values(self):
|
||||
"""Test handling of very small values."""
|
||||
x = Variable(1e-10, requires_grad=True)
|
||||
y = Variable(2e-10, requires_grad=True)
|
||||
|
||||
z = add(x, y)
|
||||
z.backward()
|
||||
|
||||
# Gradients should still be computed correctly
|
||||
assert abs(x.grad.data.data.item() - 1.0) < 1e-6
|
||||
assert abs(y.grad.data.data.item() - 1.0) < 1e-6
|
||||
|
||||
def test_mixed_requires_grad(self):
|
||||
"""Test operations with mixed requires_grad settings."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
y = Variable(3.0, requires_grad=False)
|
||||
|
||||
z = multiply(x, y)
|
||||
|
||||
# Result should require gradients
|
||||
assert z.requires_grad == True
|
||||
|
||||
z.backward()
|
||||
|
||||
# Only x should have gradients
|
||||
assert x.grad is not None
|
||||
assert y.grad is None
|
||||
assert abs(x.grad.data.data.item() - 3.0) < 1e-6
|
||||
|
||||
|
||||
# Integration tests that combine multiple concepts
|
||||
class TestIntegration:
|
||||
"""Integration tests combining multiple autograd concepts."""
|
||||
|
||||
def test_complete_training_step(self):
|
||||
"""Test a complete training step simulation."""
|
||||
# Model parameters
|
||||
w1 = Variable(0.1, requires_grad=True)
|
||||
b1 = Variable(0.0, requires_grad=True)
|
||||
w2 = Variable(0.2, requires_grad=True)
|
||||
b2 = Variable(0.0, requires_grad=True)
|
||||
|
||||
# Training data
|
||||
x = Variable(1.5, requires_grad=False)
|
||||
target = Variable(2.0, requires_grad=False)
|
||||
|
||||
# Forward pass
|
||||
h1 = add(multiply(x, w1), b1) # Linear layer 1
|
||||
h1_relu = relu_with_grad(h1) # ReLU activation
|
||||
output = add(multiply(h1_relu, w2), b2) # Linear layer 2
|
||||
|
||||
# Loss computation (MSE)
|
||||
diff = subtract(output, target)
|
||||
loss = multiply(diff, diff)
|
||||
|
||||
# Backward pass
|
||||
loss.backward()
|
||||
|
||||
# Check that all parameters have gradients
|
||||
assert w1.grad is not None
|
||||
assert b1.grad is not None
|
||||
assert w2.grad is not None
|
||||
assert b2.grad is not None
|
||||
|
||||
# Simulate parameter update (gradient descent)
|
||||
learning_rate = 0.01
|
||||
|
||||
# Save old parameter values
|
||||
old_w1 = w1.data.data.item()
|
||||
old_b1 = b1.data.data.item()
|
||||
old_w2 = w2.data.data.item()
|
||||
old_b2 = b2.data.data.item()
|
||||
|
||||
# Update parameters: param = param - lr * grad
|
||||
w1.data._data -= learning_rate * w1.grad.data.data
|
||||
b1.data._data -= learning_rate * b1.grad.data.data
|
||||
w2.data._data -= learning_rate * w2.grad.data.data
|
||||
b2.data._data -= learning_rate * b2.grad.data.data
|
||||
|
||||
# Check that parameters actually changed
|
||||
assert abs(w1.data.data.item() - old_w1) > 1e-6
|
||||
assert abs(b1.data.data.item() - old_b1) > 1e-6
|
||||
assert abs(w2.data.data.item() - old_w2) > 1e-6
|
||||
assert abs(b2.data.data.item() - old_b2) > 1e-6
|
||||
|
||||
def test_multi_output_gradients(self):
|
||||
"""Test gradients when multiple outputs depend on same input."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Create multiple outputs from same input
|
||||
y1 = multiply(x, 3.0) # y1 = 3x
|
||||
y2 = multiply(x, x) # y2 = x²
|
||||
|
||||
# Combine outputs
|
||||
combined = add(y1, y2) # combined = 3x + x²
|
||||
|
||||
combined.backward()
|
||||
|
||||
# Gradient should be sum of individual contributions
|
||||
# d(combined)/dx = d(3x)/dx + d(x²)/dx = 3 + 2x = 3 + 2*2 = 7
|
||||
assert abs(x.grad.data.data.item() - 7.0) < 1e-6
|
||||
|
||||
def test_gradient_flow_through_complex_network(self):
|
||||
"""Test gradient flow through a more complex network."""
|
||||
# Input
|
||||
x = Variable(1.0, requires_grad=True)
|
||||
|
||||
# Create a diamond-shaped computation graph
|
||||
# x
|
||||
# / \
|
||||
# a b
|
||||
# \ /
|
||||
# c
|
||||
|
||||
a = multiply(x, 2.0) # a = 2x
|
||||
b = add(x, 1.0) # b = x + 1
|
||||
c = multiply(a, b) # c = a * b = 2x * (x + 1) = 2x² + 2x
|
||||
|
||||
# Expected: c = 2x² + 2x, so dc/dx = 4x + 2 = 4*1 + 2 = 6
|
||||
c.backward()
|
||||
|
||||
assert abs(x.grad.data.data.item() - 6.0) < 1e-6
|
||||
|
||||
def test_nested_function_composition(self):
|
||||
"""Test deeply nested function composition."""
|
||||
x = Variable(2.0, requires_grad=True)
|
||||
|
||||
# Create nested composition: f(g(h(x)))
|
||||
h = multiply(x, 2.0) # h(x) = 2x
|
||||
g = add(h, 1.0) # g(h(x)) = 2x + 1
|
||||
f = multiply(g, g) # f(g(h(x))) = (2x + 1)²
|
||||
|
||||
# Expected: f = (2x + 1)², so df/dx = 2(2x + 1) * 2 = 4(2x + 1) = 4(2*2 + 1) = 20
|
||||
f.backward()
|
||||
|
||||
assert abs(x.grad.data.data.item() - 20.0) < 1e-6
|
||||
@@ -0,0 +1 @@
|
||||
|
||||
@@ -84,7 +84,6 @@ addopts = [
|
||||
]
|
||||
testpaths = [
|
||||
"tests",
|
||||
"modules/*/tests",
|
||||
]
|
||||
python_files = ["test_*.py"]
|
||||
python_classes = ["Test*"]
|
||||
|
||||
+19
-6
@@ -52,9 +52,9 @@ class StatusCommand(BaseCommand):
|
||||
console = self.console
|
||||
|
||||
# Scan modules directory
|
||||
modules_dir = Path("modules")
|
||||
modules_dir = Path("modules/source")
|
||||
if not modules_dir.exists():
|
||||
console.print(Panel("[red]❌ modules/ directory not found[/red]",
|
||||
console.print(Panel("[red]❌ modules/source/ directory not found[/red]",
|
||||
title="Error", border_style="red"))
|
||||
return 1
|
||||
|
||||
@@ -150,14 +150,21 @@ class StatusCommand(BaseCommand):
|
||||
|
||||
# Check for required files
|
||||
dev_file = module_dir / f"{module_name}_dev.py"
|
||||
tests_dir = module_dir / "tests"
|
||||
test_file = tests_dir / f"test_{module_name}.py"
|
||||
readme_file = module_dir / "README.md"
|
||||
metadata_file = module_dir / "module.yaml"
|
||||
|
||||
# Check for tests in main tests directory
|
||||
# Extract short name from module directory name (e.g., "01_tensor" -> "tensor")
|
||||
if module_name.startswith(tuple(f"{i:02d}_" for i in range(100))):
|
||||
short_name = module_name[3:] # Remove "00_" prefix
|
||||
else:
|
||||
short_name = module_name
|
||||
|
||||
main_test_file = Path("tests") / f"test_{short_name}.py"
|
||||
|
||||
status = {
|
||||
'dev_file': dev_file.exists(),
|
||||
'tests': test_file.exists(),
|
||||
'tests': main_test_file.exists(),
|
||||
'readme': readme_file.exists(),
|
||||
'metadata_file': metadata_file.exists(),
|
||||
}
|
||||
@@ -187,7 +194,13 @@ class StatusCommand(BaseCommand):
|
||||
return 'in_progress'
|
||||
|
||||
# If tests exist, run them to determine status
|
||||
test_file = f"modules/{module_name}/tests/test_{module_name}.py"
|
||||
# Extract short name from module directory name (e.g., "01_tensor" -> "tensor")
|
||||
if module_name.startswith(tuple(f"{i:02d}_" for i in range(100))):
|
||||
short_name = module_name[3:] # Remove "00_" prefix
|
||||
else:
|
||||
short_name = module_name
|
||||
|
||||
test_file = f"tests/test_{short_name}.py"
|
||||
try:
|
||||
# Run pytest quietly to check if tests pass
|
||||
result = subprocess.run(
|
||||
|
||||
Reference in New Issue
Block a user