From 052ce00393b607aea4a27cae082de984dcaf5963 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Sun, 13 Jul 2025 15:44:11 -0400 Subject: [PATCH] feat: Merge testing patterns into comprehensive module development guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ๐Ÿ”„ CONSOLIDATION: Merged separate testing patterns into module development best practices ๏ฟฝ๏ฟฝ COMPREHENSIVE GUIDE NOW INCLUDES: - Complete testing architecture (inline โ†’ module โ†’ integration โ†’ system) - pytest requirements for all testing levels (ALWAYS use pytest) - Real data requirements for integration/system tests - Stubbed dependencies for module-level isolation - Detailed examples for each testing tier - Anti-patterns covering testing mistakes - Quality standards including testing requirements - Complete development workflow with testing integration ๐Ÿงช TESTING ARCHITECTURE CLARIFIED: - Inline unit tests: Immediate feedback in development code - Module tests: pytest with stubbed dependencies (isolation) - Integration tests: pytest with real modules and real data - System tests: pytest with complete workflows and real datasets ๐Ÿ“š EDUCATIONAL BENEFITS: - Students learn proper testing architecture - Immediate feedback through inline tests - Professional pytest usage throughout - Real-world testing patterns and practices - Clear separation of concerns across testing levels This creates one comprehensive reference document for both module development and testing practices, eliminating duplication and ensuring consistency. --- .cursor/rules/development-workflow.md | 2 +- .cursor/rules/development-workflow.mdc | 97 ------- .../module-development-best-practices.mdc | 260 +++++++++++++++--- .cursor/rules/nbdev-educational-pattern.mdc | 187 ------------- .cursor/rules/testing-patterns.mdc | 116 -------- 5 files changed, 221 insertions(+), 441 deletions(-) delete mode 100644 .cursor/rules/development-workflow.mdc delete mode 100644 .cursor/rules/nbdev-educational-pattern.mdc delete mode 100644 .cursor/rules/testing-patterns.mdc diff --git a/.cursor/rules/development-workflow.md b/.cursor/rules/development-workflow.md index 8eee5b3e..f00daf9c 100644 --- a/.cursor/rules/development-workflow.md +++ b/.cursor/rules/development-workflow.md @@ -9,7 +9,7 @@ 1. **Plan**: Define what changes are needed and why 2. **Reason**: Think through the approach and potential issues 3. **Test**: Write tests to verify success before implementing -4. **Execute**: Implement changes in the branch +4. **Execute**: Implement changes in a new Git branch 5. **Verify**: Run all tests and ensure everything works 6. **Merge**: Only merge when fully tested and verified diff --git a/.cursor/rules/development-workflow.mdc b/.cursor/rules/development-workflow.mdc deleted file mode 100644 index ae6382d4..00000000 --- a/.cursor/rules/development-workflow.mdc +++ /dev/null @@ -1,97 +0,0 @@ ---- -description: Development workflow ---- - -# TinyTorch Development Workflow - -## CLI Tool: `tito` - -The main development tool is the `tito` CLI. Common commands: - -### Module Development -```bash -# Sync module: Python source โ†’ notebook โ†’ package export -tito sync --module {module_name} - -# Test specific module (uses pytest internally) -tito test --module {module_name} - -# Get project info and status -tito info -``` - -### Examples -```bash -# Work with setup module -tito sync --module setup -tito test --module setup - -# Work with tensor module -tito sync --module tensor -tito test --module tensor -``` - -## Development Cycle - -1. **Edit**: Modify `modules/{module}/{module}_dev.py` - - Use `#| export` to mark code for package export - - Use `#| hide` for instructor solutions - - Follow NBDev educational pattern - -2. **Sync**: `tito sync --module {module}` - - Converts Python source to Jupyter notebook - - Exports marked code to `tinytorch/` package - - Updates package structure - -3. **Test**: `tito test --module {module}` - - Runs module-specific pytest tests from `modules/{module}/tests/` - - Uses pytest framework for all test execution - - Validates exported package code - - Ensures everything works end-to-end - -4. **Commit**: Regular commits at good stages - - See [Git Workflow Guidelines](mdc:.cursor/rules/git-workflow.mdc) for commit strategies - - Commit when reaching good milestones - - Clean up experimental files before committing - -## Testing Requirements - -- **All tests must use pytest** - No manual testing or other frameworks -- Test files must be named `test_{module}.py` and located in `modules/{module}/tests/` -- Tests should use pytest classes, fixtures, and assertions -- CLI tool uses pytest internally: `subprocess.run([sys.executable, "-m", "pytest", test_file, "-v"])` - -## File Relationships - -``` -modules/{module}/{module}_dev.py โ†’ modules/{module}/{module}_dev.ipynb - โ†“ - tinytorch/core/{component}.py - โ†‘ - modules/{module}/tests/test_{module}.py (pytest) -``` - -## Module Structure - -Each module should be self-contained with: -- `{module}_dev.py` - Main development file (Python source) -- `{module}_dev.ipynb` - Generated notebook (auto-created) -- `README.md` - Module documentation -- `tests/` - Test directory - - `test_{module}.py` - Module tests (pytest format) - -Students run and pass all pytest tests locally within the module before using nbdev to export code to the tinytorch package and build/test the overall package. - -## Git Workflow - -For commit strategies and Git best practices, see [Git Workflow Guidelines](mdc:.cursor/rules/git-workflow.mdc). - -Key principles: -- **Incremental commits** for easy reverts -- **Test before committing** to avoid broken commits -- **Use feature branches** for larger changes -- **Descriptive commit messages** that explain what changed - -- **Use feature branches** for larger changes -- **Descriptive commit messages** that explain what changed - \ No newline at end of file diff --git a/.cursor/rules/module-development-best-practices.mdc b/.cursor/rules/module-development-best-practices.mdc index 4e28e058..ce1e158f 100644 --- a/.cursor/rules/module-development-best-practices.mdc +++ b/.cursor/rules/module-development-best-practices.mdc @@ -1,5 +1,5 @@ -# TinyTorch Module Development Best Practices +# TinyTorch Module Development & Testing Best Practices ## Core Principles @@ -12,11 +12,18 @@ ### Educational Excellence: Build โ†’ Use โ†’ Analyze/Test - **"Build โ†’ Use โ†’ Analyze"**: Follow this cycle religiously with specific third-stage verbs -- **Test after each feature**: Unit tests immediately after implementation, not at the end +- **Test after each feature**: Inline unit tests immediately after implementation, not at the end - **Progressive complexity**: Easy โ†’ Medium โ†’ Hard with clear difficulty indicators - **Comprehensive guidance**: TODO sections with approach, examples, hints, systems thinking - **Real-world connections**: Connect every concept to production ML engineering +### Testing Architecture (ALWAYS USE PYTEST) +- **ALWAYS USE PYTEST**: TinyTorch uses pytest as the standard testing framework for all tests +- **ALWAYS USE REAL DATA**: Tests must use actual datasets (CIFAR-10, ImageNet), not mock/synthetic data +- **Four-tier testing**: Inline โ†’ Module โ†’ Integration โ†’ System with proper separation +- **Immediate feedback**: Inline tests provide confidence after each feature +- **Isolation**: Module tests use stubbed dependencies to test logic in isolation + ## Module Structure (Based on Our Best Modules) ### Ideal Module Layout @@ -25,17 +32,16 @@ modules/source/XX_module_name/ โ”œโ”€โ”€ module_name_dev.py # Main development file (Jupytext format) โ”œโ”€โ”€ module_name_dev.ipynb # Generated notebook (auto-created) โ”œโ”€โ”€ tests/ -โ”‚ โ””โ”€โ”€ test_module_name.py # Unit tests ONLY for this module +โ”‚ โ””โ”€โ”€ test_module_name.py # Module-level tests with stubbed dependencies โ”œโ”€โ”€ README.md # Module overview and usage โ””โ”€โ”€ module.yaml # Module metadata ``` -### Test Organization (Critical) -- **Unit tests**: Inline in the notebook/Python code - immediate tests after each feature -- **Module-level tests**: `modules/source/XX_module/tests/` - test module in isolation with stubbed/fake data -- **Integration tests**: `tests/` (main directory) - real cross-module testing with actual dependencies -- **Test after each feature**: Write inline unit tests immediately after implementing each component -- **Module tests use fake data**: Module-level tests should stub out other modules with fake inputs/outputs +### Testing File Structure +- **Module tests**: `modules/source/XX_module/tests/test_module.py` - pytest with stubbed dependencies +- **Integration tests**: `tests/integration/` - pytest with real cross-module testing +- **System tests**: `tests/system/` - pytest with full end-to-end workflows +- **Package tests**: `tinytorch/tests/` - pytest for exported package functionality ## Development Workflow: Test-Driven Feature Development @@ -50,8 +56,8 @@ class ComponentName: # Step 2: Use the feature (immediate inline unit test) component = ComponentName() result = component.method() -print(f"โœ… Component working: {result}") assert result.shape == expected_shape # Inline unit test +print(f"โœ… Component working: {result}") # Step 3: Analyze/Test the feature (more comprehensive inline testing) def test_component_method(): @@ -60,7 +66,7 @@ def test_component_method(): result = component.method() assert result.shape == expected_shape assert np.allclose(result.data, expected_data) - print("โœ… Component unit test passed") + print("โœ… Component inline unit test passed") # Run the test immediately test_component_method() @@ -85,7 +91,14 @@ Our best modules follow specific third-stage verbs: - **Questions**: "How can we make this faster?", "What about memory usage?" - **Focus**: Production-ready systems engineering -## Testing Architecture (Corrected) +## Testing Architecture (Comprehensive) + +### Test Organization (Critical) +- **Inline unit tests**: In notebook/Python code - immediate tests after each feature +- **Module-level tests**: `modules/source/XX_module/tests/` - pytest with stubbed dependencies +- **Integration tests**: `tests/integration/` - pytest with real cross-module testing +- **System tests**: `tests/system/` - pytest with full end-to-end workflows +- **Package tests**: `tinytorch/tests/` - pytest for exported package functionality ### 1. Inline Unit Tests (Immediate After Each Feature) ```python @@ -112,16 +125,22 @@ assert np.all(y.data >= 0), "ReLU output should be non-negative" print("โœ… Sequential network unit test passed") ``` -### 2. Module-Level Tests (Isolated with Stubbed Data) +### 2. Module-Level Tests (Isolated with Stubbed Data - PYTEST REQUIRED) ```python # modules/source/04_networks/tests/test_networks.py """ Module-level tests for Networks module. Tests the module in isolation using stubbed/fake data from other modules. +ALWAYS USE PYTEST - No exceptions, no manual testing. """ import pytest import numpy as np +import sys +import os + +# Add parent directory to path for module imports +sys.path.insert(0, os.path.dirname(os.path.dirname(__file__))) # Create fake/stubbed versions of dependencies class FakeTensor: @@ -139,7 +158,7 @@ class FakeLayer: # Return fake output with expected shape return FakeTensor(np.random.randn(*self.output_shape)) -# Import the actual module being tested +# Import from the module's development file from networks_dev import Sequential class TestSequentialIsolated: @@ -168,14 +187,20 @@ class TestSequentialIsolated: assert network.layers == layers assert len(network.layers) == 2 + + def test_edge_cases(self): + """Test edge cases and error conditions.""" + with pytest.raises(ValueError): + Sequential([]) # Empty network should raise error ``` -### 3. Integration Tests (Real Cross-Module Testing) +### 3. Integration Tests (Real Cross-Module Testing - PYTEST + REAL DATA) ```python # tests/integration/test_ml_pipeline.py """ Integration tests using real implementations from all modules. Tests how modules actually work together in realistic scenarios. +ALWAYS USE REAL DATA - No mocks, no synthetic data. """ import pytest @@ -206,6 +231,103 @@ class TestRealMLPipeline: assert output.shape == (1, 2) assert isinstance(output, Tensor) assert np.all(output.data >= 0) # ReLU ensures non-negative + + def test_cifar10_integration(self): + """Test with actual CIFAR-10 data.""" + from tinytorch.core.dataloader import CIFAR10Dataset + + # Real dataset - no mocks allowed + dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True) + assert len(dataset) == 50000 # Actual CIFAR-10 size + + image, label = dataset[0] + assert image.shape == (3, 32, 32) # Real image dimensions + assert 0 <= label <= 9 # Real class labels + + # Test with real network + network = Sequential([ + Dense(input_size=3072, output_size=128), # Flattened 32x32x3 + ReLU(), + Dense(input_size=128, output_size=10) + ]) + + # Real forward pass with real data + flattened_image = image.reshape(1, -1) + output = network(flattened_image) + assert output.shape == (1, 10) +``` + +### 4. System Tests (Full End-to-End - PYTEST + REAL WORKFLOWS) +```python +# tests/system/test_complete_ml_system.py +""" +System tests for complete ML workflows. +Tests entire pipelines from data loading to model training. +ALWAYS USE REAL DATA and REAL WORKFLOWS. +""" + +import pytest +from tinytorch.core.dataloader import DataLoader, CIFAR10Dataset +from tinytorch.core.networks import Sequential +from tinytorch.core.layers import Dense +from tinytorch.core.activations import ReLU +from tinytorch.core.optimizers import SGD # When available + +class TestCompleteMLSystem: + """Test complete ML system end-to-end.""" + + def test_full_training_pipeline(self): + """Test complete training pipeline with real data.""" + # Real data loading + dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True) + dataloader = DataLoader(dataset, batch_size=32, shuffle=True) + + # Real network + network = Sequential([ + Dense(input_size=3072, output_size=128), + ReLU(), + Dense(input_size=128, output_size=10) + ]) + + # Test complete training step with real data + for batch_data, batch_labels in dataloader: + # Real forward pass + output = network(batch_data) + assert output.shape == (32, 10) + + # Real loss computation (when available) + # loss = criterion(output, batch_labels) + # assert loss.item() > 0 + + break # Test one batch for system validation +``` + +### pytest Features to Use (REQUIRED) + +- **Test classes** for organizing related tests +- **pytest fixtures** for setup/teardown +- **Parametrized tests** for testing multiple inputs +- **pytest.raises()** for exception testing +- **assert statements** with descriptive messages +- **Real data only** - No mocks, no synthetic data + +### Running Tests (ALWAYS THROUGH PYTEST) + +```bash +# Test specific module (recommended) +python bin/tito.py test --module networks + +# Test all modules +python bin/tito.py test --all + +# Run specific test file directly with pytest +python -m pytest modules/source/04_networks/tests/test_networks.py -v + +# Run integration tests +python -m pytest tests/integration/ -v + +# Run system tests +python -m pytest tests/system/ -v ``` ## Student Implementation Structure @@ -423,6 +545,27 @@ def test_sequential_with_fake_layers(): network = Sequential([FakeDense(), FakeReLU()]) ``` +### โŒ Don't Use Mock/Synthetic Data in Integration/System Tests +```python +# BAD: Mock/synthetic data in integration tests +class MockDataset: + def __init__(self, size): + self.data = np.random.randn(size, 784) # Fake data + +def test_mock_dataset(): + dataset = MockDataset(size=100) # Meaningless test + assert len(dataset) == 100 + +# GOOD: Real data in integration/system tests +def test_cifar10_dataset(): + """Test with actual CIFAR-10 data.""" + dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True) + assert len(dataset) == 50000 # Actual CIFAR-10 size + image, label = dataset[0] + assert image.shape == (3, 32, 32) # Real image dimensions + assert 0 <= label <= 9 # Real class labels +``` + ### โŒ Don't Mix Testing Levels ```python # BAD: Integration testing in module-level tests @@ -442,38 +585,64 @@ def test_networks_with_real_dataloader(): # Test real cross-module integration ``` -### โŒ Don't Use Mock Data +### โŒ Don't Skip pytest or Use Manual Testing ```python -# BAD: Synthetic/mock data -class MockDataset: - def __init__(self, size): - self.data = np.random.randn(size, 784) # Fake data +# BAD: Manual testing or custom test runners +def manual_test(): + print("Testing component...") + component = Component() + if component.works(): + print("PASS") + else: + print("FAIL") -# GOOD: Real data -class CIFAR10Dataset: - def __init__(self, root, train=True, download=True): - self._download_if_needed() # Real CIFAR-10 data +# BAD: Using unittest instead of pytest +import unittest +class TestComponent(unittest.TestCase): + def test_component(self): + self.assertTrue(component.works()) + +# GOOD: Always use pytest +import pytest + +class TestComponent: + """Test component functionality.""" + + def test_component_works(self): + """Test that component works correctly.""" + component = Component() + assert component.works() + + def test_component_edge_cases(self): + """Test edge cases and error conditions.""" + with pytest.raises(ValueError): + Component(invalid_input) ``` ## Quality Standards ### Before Release Checklist -- [ ] Uses real data, not synthetic/mock data +- [ ] Uses real data, not synthetic/mock data (for integration/system tests) +- [ ] ALWAYS uses pytest for all testing - no exceptions - [ ] Includes progress feedback for long operations - [ ] Visual feedback functions (development only, not exported) - [ ] Inline unit tests after each feature implementation - [ ] Module-level tests use stubbed/fake dependencies for isolation -- [ ] Integration tests use real cross-module implementations +- [ ] Integration tests use real cross-module implementations with real data +- [ ] System tests use complete workflows with real datasets - [ ] Clear separation: inline โ†’ module โ†’ integration โ†’ system testing - [ ] Follows "Build โ†’ Use โ†’ Analyze/Test" progression - [ ] TODO guidance includes systems thinking - [ ] Clean separation between development and exports +- [ ] All test files follow pytest structure and patterns ### Student Experience Requirements - [ ] Clear learning progression with immediate inline feedback - [ ] Inline unit tests provide confidence after each feature - [ ] Module tests demonstrate isolation and stubbing concepts -- [ ] Integration tests show real-world module interactions +- [ ] Integration tests show real-world module interactions with real data +- [ ] System tests demonstrate complete ML workflows +- [ ] pytest provides consistent testing experience across all levels - [ ] Real-world relevance and connections - [ ] Smooth transition to next modules - [ ] Test-driven development workflow @@ -500,7 +669,7 @@ class CIFAR10Dataset: --- -**Remember**: We're teaching ML systems engineering with proper testing architecture. Inline unit tests provide immediate feedback after each feature, module-level tests use stubbed dependencies for isolation, integration tests use real cross-module implementations, and system tests validate complete workflows. Follow the "Build โ†’ Use โ†’ Analyze/Test" cycle with proper testing separation. +**Remember**: We're teaching ML systems engineering with comprehensive testing architecture. Inline unit tests provide immediate feedback, module-level tests use stubbed dependencies for isolation, integration tests use real cross-module implementations with real data, and system tests validate complete workflows. ALWAYS use pytest for all testing levels. Follow the "Build โ†’ Use โ†’ Analyze/Test" cycle with proper testing separation. ## Development Workflow Summary @@ -510,8 +679,9 @@ class CIFAR10Dataset: ```bash cd modules/source/XX_module_name/ # Work in module_name_dev.py (Jupytext format) - # Module tests go in tests/test_module_name.py (with stubs) - # Integration tests go in tests/integration/ (with real modules) + # Module tests go in tests/test_module_name.py (pytest with stubs) + # Integration tests go in tests/integration/ (pytest with real modules) + # System tests go in tests/system/ (pytest with real workflows) ``` 2. **Feature Development Phase** (Repeat for each component) @@ -542,7 +712,7 @@ class CIFAR10Dataset: 3. **Module Completion Phase** ```bash - # Run module-level tests (with stubbed dependencies) + # Run module-level tests (pytest with stubbed dependencies) python -m pytest modules/source/XX_module_name/tests/test_module_name.py -v # Export to package @@ -551,10 +721,10 @@ class CIFAR10Dataset: 4. **Integration Testing Phase** ```bash - # Run integration tests (with real cross-module dependencies) + # Run integration tests (pytest with real cross-module dependencies) python -m pytest tests/integration/ -v - # Run system tests (full end-to-end) + # Run system tests (pytest with full end-to-end workflows) python -m pytest tests/system/ -v ``` @@ -562,15 +732,25 @@ class CIFAR10Dataset: - **Morning**: Review previous day's inline tests, ensure all passing - **Feature work**: Build โ†’ Use โ†’ Inline Test for each component -- **Module work**: Create stubbed module-level tests for isolation -- **Integration work**: Test real cross-module interactions -- **End of day**: Run all test levels, commit working features +- **Module work**: Create pytest-based module tests with stubbed dependencies +- **Integration work**: Create pytest-based integration tests with real modules and real data +- **System work**: Create pytest-based system tests with complete workflows +- **End of day**: Run all test levels with pytest, commit working features ### Quality Gates - **Feature Level**: Inline unit test must pass immediately -- **Module Level**: Stubbed isolation tests must pass before export -- **Integration Level**: Real cross-module tests must pass before merge -- **System Level**: Full end-to-end tests must pass before release +- **Module Level**: pytest tests with stubbed dependencies must pass before export +- **Integration Level**: pytest tests with real cross-module dependencies must pass before merge +- **System Level**: pytest tests with full end-to-end workflows must pass before release -This workflow ensures students understand proper testing architecture while building confidence incrementally through immediate feedback. +### Testing Principles Summary + +1. **ALWAYS use pytest** - No exceptions, no manual testing, no other frameworks +2. **Real data for integration/system** - No mocks in integration or system tests +3. **Stubbed dependencies for module tests** - Isolation through fake/stubbed components +4. **Immediate inline feedback** - Test every feature as you build it +5. **Four-tier architecture** - Inline โ†’ Module โ†’ Integration โ†’ System +6. **Proper separation** - Each level has distinct purpose and data requirements + +This comprehensive workflow ensures students understand professional testing architecture while building confidence incrementally through immediate feedback and systematic validation at every level. diff --git a/.cursor/rules/nbdev-educational-pattern.mdc b/.cursor/rules/nbdev-educational-pattern.mdc deleted file mode 100644 index f81300cb..00000000 --- a/.cursor/rules/nbdev-educational-pattern.mdc +++ /dev/null @@ -1,187 +0,0 @@ - - -# NBDev Educational Pattern for TinyTorch Modules - -## File Format -Use Jupytext percent format with NBDev directives: - -```python -# --- -# jupyter: -# jupytext: -# text_representation: -# extension: .py -# format_name: percent -# format_version: '1.3' -# jupytext_version: 1.17.1 -# --- - -# %% [markdown] -""" -# Module: {Title} - -## Learning Objectives -- โœ… Build {core_concept} from scratch -- โœ… Use it with real data ({dataset_name}) -- โœ… Understand {key_insight} -- โœ… Connect to production ML systems -""" - -# %% -#| default_exp core.component_name - -# Setup and imports -import required_libraries -import matplotlib.pyplot as plt # For visual feedback - -# %% [markdown] -""" -## Step 1: Concept Explanation -Educational content explaining the concept... -""" - -# %% -#| export -class ComponentName: - """ - Component description. - - TODO: Student implementation instructions. - """ - def method(self): - raise NotImplementedError("Student implementation required") - -# %% -#| hide -#| export -class ComponentName: - """Complete implementation (hidden from students).""" - def method(self): - # Actual implementation - pass - -# %% [markdown] -""" -## ๐Ÿงช Test Your Implementation -""" - -# %% -# Test with real data -try: - result = ComponentName(real_data_example) - print(f"โœ… Success: {result}") -except NotImplementedError: - print("โš ๏ธ Implement the class above first!") -``` - -## Key NBDev Directives - -- `#| default_exp core.module` - Sets export destination in tinytorch package -- `#| export` - Marks code for export to package -- `#| hide` - Hides cell from students (instructor solution) -- `# %% [markdown]` - Markdown cells for explanations -- `# %%` - Code cells - -## Educational Structure Requirements - -### 1. Conceptual Foundation -Each module must start with clear conceptual explanations: - -```markdown -## What is [Concept]? - -**Definition**: Clear, simple definition with examples -**Why it matters**: Real-world motivation and ML context -**How it works**: Intuitive explanation before math -**Connection**: How it builds on previous modules -``` - -### 2. Guided Implementation -Provide comprehensive TODO guidance: - -```python -def method(self): - """ - TODO: Step-by-step implementation guide - - APPROACH: - 1. First step with specific guidance - 2. Second step with specific guidance - 3. Third step with specific guidance - - EXAMPLE: - Input: actual_data_example - Expected: concrete_expected_output - - HINTS: - - Helpful guidance without giving code - - Systems thinking consideration - - Real-world connection - """ - raise NotImplementedError("Student implementation required") -``` - -### 3. Development vs Export Separation -- **Rich feedback in development**: Visual confirmation, progress bars -- **Clean exports to package**: No matplotlib dependencies in exported code -- **Use conditional display**: `if not _should_show_plots(): return` - -## Module Structure Template - -```python -# --- -# jupyter: -# jupytext: -# text_representation: -# extension: .py -# format_name: percent -# format_version: '1.3' -# jupytext_version: 1.17.1 -# --- - -# %% [markdown] -""" -# Module: {Title} - -## Learning Objectives -- โœ… Build {core_concept} from scratch -- โœ… Use it with real data ({dataset_name}) -- โœ… Understand {key_insight} -- โœ… Connect to production ML systems -""" - -# %% -#| default_exp core.{module} -import numpy as np -import matplotlib.pyplot as plt -from typing import Union, List, Optional - -# %% -#| export -class MainClass: - """Student-facing implementation with comprehensive TODO.""" - def __init__(self, params): - # Comprehensive TODO guidance here - raise NotImplementedError("Student implementation required") - -# %% -#| hide -#| export -class MainClass: - """Complete implementation (hidden from students).""" - def __init__(self, params): - # Actual working implementation - pass -``` - -## Educational Principles - -1. **Build โ†’ Use โ†’ Understand โ†’ Repeat**: Each module follows this cycle -2. **Concrete Before Abstract**: Start with examples, then generalize -3. **Scaffolded Learning**: Provide hints and guidance, not just empty functions -4. **Standalone Modules**: Each module should be understandable independently -5. **Real-world Context**: Connect to practical ML applications - -## Naming Convention - -Module files should be named `{module_name}_dev.py` to indicate development notebooks. diff --git a/.cursor/rules/testing-patterns.mdc b/.cursor/rules/testing-patterns.mdc deleted file mode 100644 index 09935788..00000000 --- a/.cursor/rules/testing-patterns.mdc +++ /dev/null @@ -1,116 +0,0 @@ - -# Testing Patterns for TinyTorch - -## Core Requirements - -**ALWAYS USE PYTEST** - TinyTorch uses pytest as the standard testing framework for all tests. Never use manual testing, unittest, or custom test runners. - -**ALWAYS USE REAL DATA** - Tests must use actual datasets (CIFAR-10, ImageNet), not mock/synthetic data. - -## Test File Structure - -### Module Tests (`modules/{module}/tests/test_{module}.py`) -- Test educational module implementations -- Validate both student TODOs and complete implementations -- Import from parent module's development file -- Use pytest classes and assertions - -### Package Tests (`tinytorch/tests/`) -- Test exported package functionality -- Integration tests across components -- Production-level validation - -## Test Structure Pattern (REQUIRED) - -```python -import pytest -import sys -import os - -# Add parent directory to path for module imports -sys.path.insert(0, os.path.dirname(os.path.dirname(__file__))) - -# Import from the module's development file -from {module}_dev import ComponentName - -class TestComponentName: - """Test suite for ComponentName functionality.""" - - def test_basic_functionality(self): - """Test basic component operations.""" - # Arrange - component = ComponentName() - - # Act - result = component.method() - - # Assert - assert result == expected_value - - def test_edge_cases(self): - """Test edge cases and error conditions.""" - with pytest.raises(ValueError): - ComponentName(invalid_input) - - def test_integration(self): - """Test integration with other components.""" - # Test how this component works with others - pass -``` - -## Real Data Testing Examples - -### โœ… GOOD: Actual Datasets -```python -def test_cifar10_dataset(): - """Test with actual CIFAR-10 data.""" - dataset = CIFAR10Dataset('data/cifar10/', train=True, download=True) - assert len(dataset) == 50000 # Actual CIFAR-10 size - image, label = dataset[0] - assert image.shape == (3, 32, 32) # Real image dimensions - assert 0 <= label <= 9 # Real class labels -``` - -### โŒ BAD: Mock/Synthetic Data -```python -# DON'T DO THIS - Violates real data principle -def test_mock_dataset(): - dataset = MockDataset(size=100) # Fake data - assert len(dataset) == 100 # Meaningless test -``` - -## pytest Features to Use - -- **Test classes** for organizing related tests -- **pytest fixtures** for setup/teardown -- **Parametrized tests** for testing multiple inputs -- **pytest.raises()** for exception testing -- **assert statements** with descriptive messages - -## Running Tests - -```bash -# Test specific module (recommended) -python bin/tito.py test --module tensor - -# Test all modules -python bin/tito.py test --all - -# Run specific test file directly -python -m pytest modules/tensor/tests/test_tensor.py -v -``` - -## Key Principles - -- **Always use pytest** - No exceptions, no manual testing -- **Always use real data** - No mocks, no synthetic data -- Tests should pass with both TODO stubs and complete implementations -- Educational tests should guide student learning with real examples -- Package tests ensure production readiness -- Use descriptive test names that explain what's being tested -- Import from module development files, not the package (for module tests) -- Group related tests in test classes - - -This ensures all tests run through pytest with consistent output and reporting, using real data throughout the testing process. - \ No newline at end of file