github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-03-12 01:23:34 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

feat: Merge testing patterns into comprehensive module development guide

Browse Source 
🔄 CONSOLIDATION: Merged separate testing patterns into module development best practices

�� COMPREHENSIVE GUIDE NOW INCLUDES:
- Complete testing architecture (inline → module → integration → system)
- pytest requirements for all testing levels (ALWAYS use pytest)
- Real data requirements for integration/system tests
- Stubbed dependencies for module-level isolation
- Detailed examples for each testing tier
- Anti-patterns covering testing mistakes
- Quality standards including testing requirements
- Complete development workflow with testing integration

🧪 TESTING ARCHITECTURE CLARIFIED:
- Inline unit tests: Immediate feedback in development code
- Module tests: pytest with stubbed dependencies (isolation)
- Integration tests: pytest with real modules and real data
- System tests: pytest with complete workflows and real datasets

📚 EDUCATIONAL BENEFITS:
- Students learn proper testing architecture
- Immediate feedback through inline tests
- Professional pytest usage throughout
- Real-world testing patterns and practices
- Clear separation of concerns across testing levels

This creates one comprehensive reference document for both module development and testing practices, eliminating duplication and ensuring consistency.
This commit is contained in:
Vijay Janapa Reddi
2025-07-13 15:44:11 -04:00
parent 2292aab5e3
commit 052ce00393
5 changed files with 221 additions and 441 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.cursor/rules/development-workflow.md
View File

@@ -9,7 +9,7 @@
1. **Plan**: Define what changes are needed and why
2. **Reason**: Think through the approach and potential issues
3. **Test**: Write tests to verify success before implementing
4. **Execute**: Implement changes in the branch
4. **Execute**: Implement changes in a new Git branch
5. **Verify**: Run all tests and ensure everything works
6. **Merge**: Only merge when fully tested and verified

97
.cursor/rules/development-workflow.mdc
View File

@@ -1,97 +0,0 @@
---
description: Development workflow
---
# TinyTorch Development Workflow
## CLI Tool: `tito`
The main development tool is the `tito` CLI. Common commands:
### Module Development
```bash
# Sync module: Python source → notebook → package export
tito sync --module {module_name}
# Test specific module (uses pytest internally)
tito test --module {module_name}
# Get project info and status
tito info
```
### Examples
```bash
# Work with setup module
tito sync --module setup
tito test --module setup
# Work with tensor module
tito sync --module tensor
tito test --module tensor
```
## Development Cycle
1. **Edit**: Modify `modules/{module}/{module}_dev.py`
- Use `#| export` to mark code for package export
- Use `#| hide` for instructor solutions
- Follow NBDev educational pattern
2. **Sync**: `tito sync --module {module}`
- Converts Python source to Jupyter notebook
- Exports marked code to `tinytorch/` package
- Updates package structure
3. **Test**: `tito test --module {module}`
- Runs module-specific pytest tests from `modules/{module}/tests/`
- Uses pytest framework for all test execution
- Validates exported package code
- Ensures everything works end-to-end
4. **Commit**: Regular commits at good stages
- See [Git Workflow Guidelines](mdc:.cursor/rules/git-workflow.mdc) for commit strategies
- Commit when reaching good milestones
- Clean up experimental files before committing
## Testing Requirements
- **All tests must use pytest** - No manual testing or other frameworks
- Test files must be named `test_{module}.py` and located in `modules/{module}/tests/`
- Tests should use pytest classes, fixtures, and assertions
- CLI tool uses pytest internally: `subprocess.run([sys.executable, "-m", "pytest", test_file, "-v"])`
## File Relationships
```
modules/{module}/{module}_dev.py → modules/{module}/{module}_dev.ipynb
↓
tinytorch/core/{component}.py
↑
modules/{module}/tests/test_{module}.py (pytest)
```
## Module Structure
Each module should be self-contained with:
- `{module}_dev.py` - Main development file (Python source)
- `{module}_dev.ipynb` - Generated notebook (auto-created)
- `README.md` - Module documentation
- `tests/` - Test directory
- `test_{module}.py` - Module tests (pytest format)
Students run and pass all pytest tests locally within the module before using nbdev to export code to the tinytorch package and build/test the overall package.
## Git Workflow
For commit strategies and Git best practices, see [Git Workflow Guidelines](mdc:.cursor/rules/git-workflow.mdc).
Key principles:
- **Incremental commits** for easy reverts
- **Test before committing** to avoid broken commits
- **Use feature branches** for larger changes
- **Descriptive commit messages** that explain what changed
- **Use feature branches** for larger changes
- **Descriptive commit messages** that explain what changed

260
.cursor/rules/module-development-best-practices.mdc
View File

@@ -1,5 +1,5 @@
# TinyTorch Module Development Best Practices
# TinyTorch Module Development & Testing Best Practices
## Core Principles
@@ -12,11 +12,18 @@
### Educational Excellence: Build → Use → Analyze/Test
- **"Build → Use → Analyze"**: Follow this cycle religiously with specific third-stage verbs
- **Test after each feature**: Unit tests immediately after implementation, not at the end
- **Test after each feature**: Inline unit tests immediately after implementation, not at the end
- **Progressive complexity**: Easy → Medium → Hard with clear difficulty indicators
- **Comprehensive guidance**: TODO sections with approach, examples, hints, systems thinking
- **Real-world connections**: Connect every concept to production ML engineering
### Testing Architecture (ALWAYS USE PYTEST)
- **ALWAYS USE PYTEST**: TinyTorch uses pytest as the standard testing framework for all tests
- **ALWAYS USE REAL DATA**: Tests must use actual datasets (CIFAR-10, ImageNet), not mock/synthetic data
- **Four-tier testing**: Inline → Module → Integration → System with proper separation
- **Immediate feedback**: Inline tests provide confidence after each feature
- **Isolation**: Module tests use stubbed dependencies to test logic in isolation
## Module Structure (Based on Our Best Modules)
### Ideal Module Layout
@@ -25,17 +32,16 @@ modules/source/XX_module_name/
├── module_name_dev.py # Main development file (Jupytext format)
├── module_name_dev.ipynb # Generated notebook (auto-created)
├── tests/
│ └── test_module_name.py # Unit tests ONLY for this module
│ └── test_module_name.py # Module-level tests with stubbed dependencies
├── README.md # Module overview and usage
└── module.yaml # Module metadata
```
### Test Organization (Critical)
- **Unit tests**: Inline in the notebook/Python code - immediate tests after each feature
- **Module-level tests**: `modules/source/XX_module/tests/` - test module in isolation with stubbed/fake data
- **Integration tests**: `tests/` (main directory) - real cross-module testing with actual dependencies
- **Test after each feature**: Write inline unit tests immediately after implementing each component
- **Module tests use fake data**: Module-level tests should stub out other modules with fake inputs/outputs
### Testing File Structure
- **Module tests**: `modules/source/XX_module/tests/test_module.py` - pytest with stubbed dependencies
- **Integration tests**: `tests/integration/` - pytest with real cross-module testing
- **System tests**: `tests/system/` - pytest with full end-to-end workflows
- **Package tests**: `tinytorch/tests/` - pytest for exported package functionality
## Development Workflow: Test-Driven Feature Development
@@ -50,8 +56,8 @@ class ComponentName:
# Step 2: Use the feature (immediate inline unit test)
component = ComponentName()
result = component.method()
print(f"✅ Component working: {result}")
assert result.shape == expected_shape # Inline unit test
print(f"✅ Component working: {result}")
# Step 3: Analyze/Test the feature (more comprehensive inline testing)
def test_component_method():
@@ -60,7 +66,7 @@ def test_component_method():
result = component.method()
assert result.shape == expected_shape
assert np.allclose(result.data, expected_data)
print("✅ Component unit test passed")
print("✅ Component inline unit test passed")
# Run the test immediately
test_component_method()
@@ -85,7 +91,14 @@ Our best modules follow specific third-stage verbs:
- **Questions**: "How can we make this faster?", "What about memory usage?"
- **Focus**: Production-ready systems engineering
## Testing Architecture (Corrected)
## Testing Architecture (Comprehensive)
### Test Organization (Critical)
- **Inline unit tests**: In notebook/Python code - immediate tests after each feature
- **Module-level tests**: `modules/source/XX_module/tests/` - pytest with stubbed dependencies
- **Integration tests**: `tests/integration/` - pytest with real cross-module testing
- **System tests**: `tests/system/` - pytest with full end-to-end workflows
- **Package tests**: `tinytorch/tests/` - pytest for exported package functionality
### 1. Inline Unit Tests (Immediate After Each Feature)
```python
@@ -112,16 +125,22 @@ assert np.all(y.data >= 0), "ReLU output should be non-negative"
print("✅ Sequential network unit test passed")
```
### 2. Module-Level Tests (Isolated with Stubbed Data)
### 2. Module-Level Tests (Isolated with Stubbed Data - PYTEST REQUIRED)
```python
# modules/source/04_networks/tests/test_networks.py
"""
Module-level tests for Networks module.
Tests the module in isolation using stubbed/fake data from other modules.
ALWAYS USE PYTEST - No exceptions, no manual testing.
"""
import pytest
import numpy as np
import sys
import os
# Add parent directory to path for module imports
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
# Create fake/stubbed versions of dependencies
class FakeTensor:
@@ -139,7 +158,7 @@ class FakeLayer:
# Return fake output with expected shape
return FakeTensor(np.random.randn(*self.output_shape))
# Import the actual module being tested
# Import from the module's development file
from networks_dev import Sequential
class TestSequentialIsolated:
@@ -168,14 +187,20 @@ class TestSequentialIsolated:
assert network.layers == layers
assert len(network.layers) == 2
def test_edge_cases(self):
"""Test edge cases and error conditions."""
with pytest.raises(ValueError):
Sequential([]) # Empty network should raise error
```
### 3. Integration Tests (Real Cross-Module Testing)
### 3. Integration Tests (Real Cross-Module Testing - PYTEST + REAL DATA)
```python
# tests/integration/test_ml_pipeline.py
"""
Integration tests using real implementations from all modules.
Tests how modules actually work together in realistic scenarios.
ALWAYS USE REAL DATA - No mocks, no synthetic data.
"""
import pytest
@@ -206,6 +231,103 @@ class TestRealMLPipeline:
assert output.shape == (1, 2)
assert isinstance(output, Tensor)
assert np.all(output.data >= 0) # ReLU ensures non-negative
def test_cifar10_integration(self):
"""Test with actual CIFAR-10 data."""
from tinytorch.core.dataloader import CIFAR10Dataset
# Real dataset - no mocks allowed
dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True)
assert len(dataset) == 50000 # Actual CIFAR-10 size
image, label = dataset[0]
assert image.shape == (3, 32, 32) # Real image dimensions
assert 0 <= label <= 9 # Real class labels
# Test with real network
network = Sequential([
Dense(input_size=3072, output_size=128), # Flattened 32x32x3
ReLU(),
Dense(input_size=128, output_size=10)
])
# Real forward pass with real data
flattened_image = image.reshape(1, -1)
output = network(flattened_image)
assert output.shape == (1, 10)
```
### 4. System Tests (Full End-to-End - PYTEST + REAL WORKFLOWS)
```python
# tests/system/test_complete_ml_system.py
"""
System tests for complete ML workflows.
Tests entire pipelines from data loading to model training.
ALWAYS USE REAL DATA and REAL WORKFLOWS.
"""
import pytest
from tinytorch.core.dataloader import DataLoader, CIFAR10Dataset
from tinytorch.core.networks import Sequential
from tinytorch.core.layers import Dense
from tinytorch.core.activations import ReLU
from tinytorch.core.optimizers import SGD # When available
class TestCompleteMLSystem:
"""Test complete ML system end-to-end."""
def test_full_training_pipeline(self):
"""Test complete training pipeline with real data."""
# Real data loading
dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True)
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
# Real network
network = Sequential([
Dense(input_size=3072, output_size=128),
ReLU(),
Dense(input_size=128, output_size=10)
])
# Test complete training step with real data
for batch_data, batch_labels in dataloader:
# Real forward pass
output = network(batch_data)
assert output.shape == (32, 10)
# Real loss computation (when available)
# loss = criterion(output, batch_labels)
# assert loss.item() > 0
break # Test one batch for system validation
```
### pytest Features to Use (REQUIRED)
- **Test classes** for organizing related tests
- **pytest fixtures** for setup/teardown
- **Parametrized tests** for testing multiple inputs
- **pytest.raises()** for exception testing
- **assert statements** with descriptive messages
- **Real data only** - No mocks, no synthetic data
### Running Tests (ALWAYS THROUGH PYTEST)
```bash
# Test specific module (recommended)
python bin/tito.py test --module networks
# Test all modules
python bin/tito.py test --all
# Run specific test file directly with pytest
python -m pytest modules/source/04_networks/tests/test_networks.py -v
# Run integration tests
python -m pytest tests/integration/ -v
# Run system tests
python -m pytest tests/system/ -v
```
## Student Implementation Structure
@@ -423,6 +545,27 @@ def test_sequential_with_fake_layers():
network = Sequential([FakeDense(), FakeReLU()])
```
### ❌ Don't Use Mock/Synthetic Data in Integration/System Tests
```python
# BAD: Mock/synthetic data in integration tests
class MockDataset:
def __init__(self, size):
self.data = np.random.randn(size, 784) # Fake data
def test_mock_dataset():
dataset = MockDataset(size=100) # Meaningless test
assert len(dataset) == 100
# GOOD: Real data in integration/system tests
def test_cifar10_dataset():
"""Test with actual CIFAR-10 data."""
dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True)
assert len(dataset) == 50000 # Actual CIFAR-10 size
image, label = dataset[0]
assert image.shape == (3, 32, 32) # Real image dimensions
assert 0 <= label <= 9 # Real class labels
```
### ❌ Don't Mix Testing Levels
```python
# BAD: Integration testing in module-level tests
@@ -442,38 +585,64 @@ def test_networks_with_real_dataloader():
# Test real cross-module integration
```
### ❌ Don't Use Mock Data
### ❌ Don't Skip pytest or Use Manual Testing
```python
# BAD: Synthetic/mock data
class MockDataset:
def __init__(self, size):
self.data = np.random.randn(size, 784) # Fake data
# BAD: Manual testing or custom test runners
def manual_test():
print("Testing component...")
component = Component()
if component.works():
print("PASS")
else:
print("FAIL")
# GOOD: Real data
class CIFAR10Dataset:
def __init__(self, root, train=True, download=True):
self._download_if_needed() # Real CIFAR-10 data
# BAD: Using unittest instead of pytest
import unittest
class TestComponent(unittest.TestCase):
def test_component(self):
self.assertTrue(component.works())
# GOOD: Always use pytest
import pytest
class TestComponent:
"""Test component functionality."""
def test_component_works(self):
"""Test that component works correctly."""
component = Component()
assert component.works()
def test_component_edge_cases(self):
"""Test edge cases and error conditions."""
with pytest.raises(ValueError):
Component(invalid_input)
```
## Quality Standards
### Before Release Checklist
- [ ] Uses real data, not synthetic/mock data
- [ ] Uses real data, not synthetic/mock data (for integration/system tests)
- [ ] ALWAYS uses pytest for all testing - no exceptions
- [ ] Includes progress feedback for long operations
- [ ] Visual feedback functions (development only, not exported)
- [ ] Inline unit tests after each feature implementation
- [ ] Module-level tests use stubbed/fake dependencies for isolation
- [ ] Integration tests use real cross-module implementations
- [ ] Integration tests use real cross-module implementations with real data
- [ ] System tests use complete workflows with real datasets
- [ ] Clear separation: inline → module → integration → system testing
- [ ] Follows "Build → Use → Analyze/Test" progression
- [ ] TODO guidance includes systems thinking
- [ ] Clean separation between development and exports
- [ ] All test files follow pytest structure and patterns
### Student Experience Requirements
- [ ] Clear learning progression with immediate inline feedback
- [ ] Inline unit tests provide confidence after each feature
- [ ] Module tests demonstrate isolation and stubbing concepts
- [ ] Integration tests show real-world module interactions
- [ ] Integration tests show real-world module interactions with real data
- [ ] System tests demonstrate complete ML workflows
- [ ] pytest provides consistent testing experience across all levels
- [ ] Real-world relevance and connections
- [ ] Smooth transition to next modules
- [ ] Test-driven development workflow
@@ -500,7 +669,7 @@ class CIFAR10Dataset:
---
**Remember**: We're teaching ML systems engineering with proper testing architecture. Inline unit tests provide immediate feedback after each feature, module-level tests use stubbed dependencies for isolation, integration tests use real cross-module implementations, and system tests validate complete workflows. Follow the "Build → Use → Analyze/Test" cycle with proper testing separation.
**Remember**: We're teaching ML systems engineering with comprehensive testing architecture. Inline unit tests provide immediate feedback, module-level tests use stubbed dependencies for isolation, integration tests use real cross-module implementations with real data, and system tests validate complete workflows. ALWAYS use pytest for all testing levels. Follow the "Build → Use → Analyze/Test" cycle with proper testing separation.
## Development Workflow Summary
@@ -510,8 +679,9 @@ class CIFAR10Dataset:
```bash
cd modules/source/XX_module_name/
# Work in module_name_dev.py (Jupytext format)
# Module tests go in tests/test_module_name.py (with stubs)
# Integration tests go in tests/integration/ (with real modules)
# Module tests go in tests/test_module_name.py (pytest with stubs)
# Integration tests go in tests/integration/ (pytest with real modules)
# System tests go in tests/system/ (pytest with real workflows)
```
2. **Feature Development Phase** (Repeat for each component)
@@ -542,7 +712,7 @@ class CIFAR10Dataset:
3. **Module Completion Phase**
```bash
# Run module-level tests (with stubbed dependencies)
# Run module-level tests (pytest with stubbed dependencies)
python -m pytest modules/source/XX_module_name/tests/test_module_name.py -v
# Export to package
@@ -551,10 +721,10 @@ class CIFAR10Dataset:
4. **Integration Testing Phase**
```bash
# Run integration tests (with real cross-module dependencies)
# Run integration tests (pytest with real cross-module dependencies)
python -m pytest tests/integration/ -v
# Run system tests (full end-to-end)
# Run system tests (pytest with full end-to-end workflows)
python -m pytest tests/system/ -v
```
@@ -562,15 +732,25 @@ class CIFAR10Dataset:
- **Morning**: Review previous day's inline tests, ensure all passing
- **Feature work**: Build → Use → Inline Test for each component
- **Module work**: Create stubbed module-level tests for isolation
- **Integration work**: Test real cross-module interactions
- **End of day**: Run all test levels, commit working features
- **Module work**: Create pytest-based module tests with stubbed dependencies
- **Integration work**: Create pytest-based integration tests with real modules and real data
- **System work**: Create pytest-based system tests with complete workflows
- **End of day**: Run all test levels with pytest, commit working features
### Quality Gates
- **Feature Level**: Inline unit test must pass immediately
- **Module Level**: Stubbed isolation tests must pass before export
- **Integration Level**: Real cross-module tests must pass before merge
- **System Level**: Full end-to-end tests must pass before release
- **Module Level**: pytest tests with stubbed dependencies must pass before export
- **Integration Level**: pytest tests with real cross-module dependencies must pass before merge
- **System Level**: pytest tests with full end-to-end workflows must pass before release
This workflow ensures students understand proper testing architecture while building confidence incrementally through immediate feedback.
### Testing Principles Summary
1. **ALWAYS use pytest** - No exceptions, no manual testing, no other frameworks
2. **Real data for integration/system** - No mocks in integration or system tests
3. **Stubbed dependencies for module tests** - Isolation through fake/stubbed components
4. **Immediate inline feedback** - Test every feature as you build it
5. **Four-tier architecture** - Inline → Module → Integration → System
6. **Proper separation** - Each level has distinct purpose and data requirements
This comprehensive workflow ensures students understand professional testing architecture while building confidence incrementally through immediate feedback and systematic validation at every level.

187
.cursor/rules/nbdev-educational-pattern.mdc
View File

@@ -1,187 +0,0 @@
# NBDev Educational Pattern for TinyTorch Modules
## File Format
Use Jupytext percent format with NBDev directives:
```python
# ---
# jupyter:
# jupytext:
# text_representation:
# extension: .py
# format_name: percent
# format_version: '1.3'
# jupytext_version: 1.17.1
# ---
# %% [markdown]
"""
# Module: {Title}
## Learning Objectives
- ✅ Build {core_concept} from scratch
- ✅ Use it with real data ({dataset_name})
- ✅ Understand {key_insight}
- ✅ Connect to production ML systems
"""
# %%
#| default_exp core.component_name
# Setup and imports
import required_libraries
import matplotlib.pyplot as plt # For visual feedback
# %% [markdown]
"""
## Step 1: Concept Explanation
Educational content explaining the concept...
"""
# %%
#| export
class ComponentName:
"""
Component description.
TODO: Student implementation instructions.
"""
def method(self):
raise NotImplementedError("Student implementation required")
# %%
#| hide
#| export
class ComponentName:
"""Complete implementation (hidden from students)."""
def method(self):
# Actual implementation
pass
# %% [markdown]
"""
## 🧪 Test Your Implementation
"""
# %%
# Test with real data
try:
result = ComponentName(real_data_example)
print(f"✅ Success: {result}")
except NotImplementedError:
print("⚠️ Implement the class above first!")
```
## Key NBDev Directives
- `#| default_exp core.module` - Sets export destination in tinytorch package
- `#| export` - Marks code for export to package
- `#| hide` - Hides cell from students (instructor solution)
- `# %% [markdown]` - Markdown cells for explanations
- `# %%` - Code cells
## Educational Structure Requirements
### 1. Conceptual Foundation
Each module must start with clear conceptual explanations:
```markdown
## What is [Concept]?
**Definition**: Clear, simple definition with examples
**Why it matters**: Real-world motivation and ML context
**How it works**: Intuitive explanation before math
**Connection**: How it builds on previous modules
```
### 2. Guided Implementation
Provide comprehensive TODO guidance:
```python
def method(self):
"""
TODO: Step-by-step implementation guide
APPROACH:
1. First step with specific guidance
2. Second step with specific guidance
3. Third step with specific guidance
EXAMPLE:
Input: actual_data_example
Expected: concrete_expected_output
HINTS:
- Helpful guidance without giving code
- Systems thinking consideration
- Real-world connection
"""
raise NotImplementedError("Student implementation required")
```
### 3. Development vs Export Separation
- **Rich feedback in development**: Visual confirmation, progress bars
- **Clean exports to package**: No matplotlib dependencies in exported code
- **Use conditional display**: `if not _should_show_plots(): return`
## Module Structure Template
```python
# ---
# jupyter:
# jupytext:
# text_representation:
# extension: .py
# format_name: percent
# format_version: '1.3'
# jupytext_version: 1.17.1
# ---
# %% [markdown]
"""
# Module: {Title}
## Learning Objectives
- ✅ Build {core_concept} from scratch
- ✅ Use it with real data ({dataset_name})
- ✅ Understand {key_insight}
- ✅ Connect to production ML systems
"""
# %%
#| default_exp core.{module}
import numpy as np
import matplotlib.pyplot as plt
from typing import Union, List, Optional
# %%
#| export
class MainClass:
"""Student-facing implementation with comprehensive TODO."""
def __init__(self, params):
# Comprehensive TODO guidance here
raise NotImplementedError("Student implementation required")
# %%
#| hide
#| export
class MainClass:
"""Complete implementation (hidden from students)."""
def __init__(self, params):
# Actual working implementation
pass
```
## Educational Principles
1. **Build → Use → Understand → Repeat**: Each module follows this cycle
2. **Concrete Before Abstract**: Start with examples, then generalize
3. **Scaffolded Learning**: Provide hints and guidance, not just empty functions
4. **Standalone Modules**: Each module should be understandable independently
5. **Real-world Context**: Connect to practical ML applications
## Naming Convention
Module files should be named `{module_name}_dev.py` to indicate development notebooks.

116
.cursor/rules/testing-patterns.mdc
View File

@@ -1,116 +0,0 @@
# Testing Patterns for TinyTorch
## Core Requirements
**ALWAYS USE PYTEST** - TinyTorch uses pytest as the standard testing framework for all tests. Never use manual testing, unittest, or custom test runners.
**ALWAYS USE REAL DATA** - Tests must use actual datasets (CIFAR-10, ImageNet), not mock/synthetic data.
## Test File Structure
### Module Tests (`modules/{module}/tests/test_{module}.py`)
- Test educational module implementations
- Validate both student TODOs and complete implementations
- Import from parent module's development file
- Use pytest classes and assertions
### Package Tests (`tinytorch/tests/`)
- Test exported package functionality
- Integration tests across components
- Production-level validation
## Test Structure Pattern (REQUIRED)
```python
import pytest
import sys
import os
# Add parent directory to path for module imports
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
# Import from the module's development file
from {module}_dev import ComponentName
class TestComponentName:
"""Test suite for ComponentName functionality."""
def test_basic_functionality(self):
"""Test basic component operations."""
# Arrange
component = ComponentName()
# Act
result = component.method()
# Assert
assert result == expected_value
def test_edge_cases(self):
"""Test edge cases and error conditions."""
with pytest.raises(ValueError):
ComponentName(invalid_input)
def test_integration(self):
"""Test integration with other components."""
# Test how this component works with others
pass
```
## Real Data Testing Examples
### ✅ GOOD: Actual Datasets
```python
def test_cifar10_dataset():
"""Test with actual CIFAR-10 data."""
dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True)
assert len(dataset) == 50000 # Actual CIFAR-10 size
image, label = dataset[0]
assert image.shape == (3, 32, 32) # Real image dimensions
assert 0 <= label <= 9 # Real class labels
```
### ❌ BAD: Mock/Synthetic Data
```python
# DON'T DO THIS - Violates real data principle
def test_mock_dataset():
dataset = MockDataset(size=100) # Fake data
assert len(dataset) == 100 # Meaningless test
```
## pytest Features to Use
- **Test classes** for organizing related tests
- **pytest fixtures** for setup/teardown
- **Parametrized tests** for testing multiple inputs
- **pytest.raises()** for exception testing
- **assert statements** with descriptive messages
## Running Tests
```bash
# Test specific module (recommended)
python bin/tito.py test --module tensor
# Test all modules
python bin/tito.py test --all
# Run specific test file directly
python -m pytest modules/tensor/tests/test_tensor.py -v
```
## Key Principles
- **Always use pytest** - No exceptions, no manual testing
- **Always use real data** - No mocks, no synthetic data
- Tests should pass with both TODO stubs and complete implementations
- Educational tests should guide student learning with real examples
- Package tests ensure production readiness
- Use descriptive test names that explain what's being tested
- Import from module development files, not the package (for module tests)
- Group related tests in test classes
This ensures all tests run through pytest with consistent output and reporting, using real data throughout the testing process.