mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-12 02:53:34 -05:00
Add documentation standards and development setup
- Create .claude directory with team structure and guidelines - Add MODULE_DEVELOPMENT_GUIDELINES.md for educational patterns - Add EDUCATIONAL_PATTERN_TEMPLATE.md for consistent module structure - Add GIT_WORKFLOW_STANDARDS.md for branch management - Create setup-dev.sh for automated environment setup - Add notebook workflow documentation - Add CI/CD workflow for notebook testing This commit establishes consistent development standards and documentation for the TinyTorch educational ML framework development.
This commit is contained in:
212
.claude/GIT_WORKFLOW_STANDARDS.md
Normal file
212
.claude/GIT_WORKFLOW_STANDARDS.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# TinyTorch Git Workflow Standards
|
||||
|
||||
## 🎯 Core Principle: Always Work on Feature Branches
|
||||
|
||||
**NEVER work directly on main or dev branches.** Every piece of work, no matter how small, starts with creating a new branch.
|
||||
|
||||
## 📋 Branch Creation Standards
|
||||
|
||||
### When to Create a New Branch
|
||||
- **Before ANY module enhancement**
|
||||
- **Before ANY bug fix**
|
||||
- **Before ANY documentation update**
|
||||
- **Before ANY infrastructure change**
|
||||
- **Before ANY agent-driven development**
|
||||
|
||||
### Branch Naming Convention
|
||||
```bash
|
||||
# Format: <type>/<description>
|
||||
# Examples:
|
||||
feature/enhance-module-04-layers
|
||||
fix/tensor-import-error
|
||||
docs/update-educational-patterns
|
||||
infra/notebook-conversion-workflow
|
||||
agent/quality-assurance-validation
|
||||
```
|
||||
|
||||
### Branch Types
|
||||
- `feature/` - New features or module enhancements
|
||||
- `fix/` - Bug fixes
|
||||
- `docs/` - Documentation updates
|
||||
- `infra/` - Infrastructure, CI/CD, build system changes
|
||||
- `agent/` - Work initiated by specific agents
|
||||
- `test/` - Testing improvements
|
||||
- `refactor/` - Code refactoring
|
||||
|
||||
## 🔄 Standard Workflow
|
||||
|
||||
### 1. Start Work: Create Branch
|
||||
```bash
|
||||
# Always start from dev (or main if no dev)
|
||||
git checkout dev
|
||||
git pull origin dev
|
||||
|
||||
# Create and switch to new branch
|
||||
git checkout -b feature/enhance-module-04-layers
|
||||
```
|
||||
|
||||
### 2. During Work: Regular Commits
|
||||
```bash
|
||||
# Stage changes
|
||||
git add .
|
||||
|
||||
# Commit with meaningful message
|
||||
git commit -m "Add comprehensive educational scaffolding to layers module
|
||||
|
||||
- Enhanced conceptual foundation with 150+ lines
|
||||
- Added visual diagrams and mathematical explanations
|
||||
- Implemented TODO/APPROACH/EXAMPLE/HINTS pattern
|
||||
- Created integration tests with tensor and activations"
|
||||
```
|
||||
|
||||
### 3. Complete Work: Push and PR
|
||||
```bash
|
||||
# Push branch to remote
|
||||
git push -u origin feature/enhance-module-04-layers
|
||||
|
||||
# Create Pull Request via GitHub CLI
|
||||
gh pr create --title "Enhance Module 04 (Layers) with educational patterns" \
|
||||
--body "Description of changes..."
|
||||
```
|
||||
|
||||
## 📝 Commit Message Standards
|
||||
|
||||
### Format
|
||||
```
|
||||
<type>: <subject>
|
||||
|
||||
<body>
|
||||
|
||||
<footer>
|
||||
```
|
||||
|
||||
### Example
|
||||
```
|
||||
feature: Enhance Module 04 (Layers) with comprehensive educational scaffolding
|
||||
|
||||
- Added 150+ lines of conceptual foundation
|
||||
- Implemented visual diagrams for matrix multiplication
|
||||
- Enhanced Dense layer with production connections
|
||||
- Created integration tests with previous modules
|
||||
- Added checkpoint sections for student validation
|
||||
|
||||
This completes the foundation layer sequence (modules 01-05) with
|
||||
consistent educational patterns.
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.ai/code)
|
||||
Co-Authored-By: Claude <noreply@anthropic.com>
|
||||
```
|
||||
|
||||
## 🚫 Never Do This
|
||||
|
||||
### ❌ Bad Practices
|
||||
```bash
|
||||
# NEVER commit directly to main
|
||||
git checkout main
|
||||
git add .
|
||||
git commit -m "quick fix" # NO!
|
||||
|
||||
# NEVER work without a branch
|
||||
vim module.py
|
||||
git add .
|
||||
git commit -m "changes" # NO!
|
||||
|
||||
# NEVER use vague commit messages
|
||||
git commit -m "updates" # NO!
|
||||
git commit -m "fixes" # NO!
|
||||
```
|
||||
|
||||
### ✅ Good Practices
|
||||
```bash
|
||||
# ALWAYS create a branch first
|
||||
git checkout -b fix/tensor-shape-error
|
||||
|
||||
# ALWAYS use descriptive commits
|
||||
git commit -m "Fix tensor shape preservation in arithmetic operations"
|
||||
|
||||
# ALWAYS push to feature branch
|
||||
git push -u origin fix/tensor-shape-error
|
||||
```
|
||||
|
||||
## 🔀 Merging Standards
|
||||
|
||||
### Before Merging
|
||||
- [ ] All tests pass
|
||||
- [ ] Code reviewed (if team environment)
|
||||
- [ ] Documentation updated
|
||||
- [ ] Validation workflow passes
|
||||
- [ ] No conflicts with target branch
|
||||
|
||||
### Merge Process
|
||||
```bash
|
||||
# Update your branch with latest dev
|
||||
git checkout dev
|
||||
git pull origin dev
|
||||
git checkout feature/your-branch
|
||||
git merge dev
|
||||
|
||||
# Resolve any conflicts
|
||||
# Test everything works
|
||||
# Then merge via PR
|
||||
```
|
||||
|
||||
## 🏷️ Protected Branches
|
||||
|
||||
These branches should have protection rules:
|
||||
- `main` - Production ready code only
|
||||
- `dev` - Development integration branch
|
||||
- No direct pushes allowed
|
||||
- Require PR reviews
|
||||
- Require status checks to pass
|
||||
|
||||
## 📊 Branch Lifecycle
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[Create Branch] --> B[Develop]
|
||||
B --> C[Commit Changes]
|
||||
C --> D[Push to Remote]
|
||||
D --> E[Create PR]
|
||||
E --> F[Review/Test]
|
||||
F --> G[Merge to Dev]
|
||||
G --> H[Delete Branch]
|
||||
```
|
||||
|
||||
## 🎯 Agent-Specific Standards
|
||||
|
||||
When agents create branches:
|
||||
- **Education Architect**: `agent/education-design-module-XX`
|
||||
- **Module Developer**: `feature/enhance-module-XX-name`
|
||||
- **Quality Assurance**: `test/validation-module-XX`
|
||||
- **Documentation Publisher**: `docs/update-module-XX-docs`
|
||||
- **DevOps Engineer**: `infra/component-description`
|
||||
|
||||
## 🔧 Git Configuration
|
||||
|
||||
### Recommended Setup
|
||||
```bash
|
||||
# Set up useful aliases
|
||||
git config --global alias.co checkout
|
||||
git config --global alias.br branch
|
||||
git config --global alias.ci commit
|
||||
git config --global alias.st status
|
||||
|
||||
# Set default branch name
|
||||
git config --global init.defaultBranch main
|
||||
|
||||
# Set pull strategy
|
||||
git config --global pull.rebase false
|
||||
```
|
||||
|
||||
## 📝 Codified Rules
|
||||
|
||||
1. **ALWAYS create a new branch before starting ANY work**
|
||||
2. **NEVER commit directly to main or dev branches**
|
||||
3. **ALWAYS use descriptive branch names following conventions**
|
||||
4. **ALWAYS write meaningful commit messages**
|
||||
5. **ALWAYS test before pushing**
|
||||
6. **ALWAYS create a PR for code review**
|
||||
7. **ALWAYS delete branches after merging**
|
||||
8. **ALWAYS pull latest changes before creating new branches**
|
||||
|
||||
This workflow ensures code quality, enables collaboration, maintains history, and prevents conflicts in the TinyTorch project.
|
||||
170
.github/workflows/test-notebooks.yml
vendored
Normal file
170
.github/workflows/test-notebooks.yml
vendored
Normal file
@@ -0,0 +1,170 @@
|
||||
name: Test Notebook Conversion
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, dev ]
|
||||
paths:
|
||||
- 'modules/source/**/*.py'
|
||||
- 'tito/**/*.py'
|
||||
- 'requirements.txt'
|
||||
- '.github/workflows/test-notebooks.yml'
|
||||
pull_request:
|
||||
branches: [ main, dev ]
|
||||
paths:
|
||||
- 'modules/source/**/*.py'
|
||||
- 'tito/**/*.py'
|
||||
- 'requirements.txt'
|
||||
- '.github/workflows/test-notebooks.yml'
|
||||
|
||||
jobs:
|
||||
test-notebook-conversion:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
|
||||
- name: Create virtual environment
|
||||
run: |
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install --upgrade pip
|
||||
|
||||
- name: Install essential dependencies
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
pip install setuptools wheel
|
||||
pip install rich>=13.0.0
|
||||
pip install jupytext>=1.14.0
|
||||
pip install pytest>=7.0.0
|
||||
pip install numpy>=1.21.0
|
||||
|
||||
- name: Install remaining dependencies (allow failures)
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt || echo "Some dependencies failed - continuing with essential packages"
|
||||
|
||||
- name: Install TinyTorch in development mode
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
pip install -e . || echo "Development install failed - continuing"
|
||||
|
||||
- name: Run environment diagnosis
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
python -m tito.main system doctor || echo "Doctor check completed with issues"
|
||||
|
||||
- name: Test notebook conversion (dry run)
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
python -m tito.main module notebooks --dry-run
|
||||
|
||||
- name: Test conversion of specific modules
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
# Test tensor module
|
||||
if [ -f "modules/source/02_tensor/tensor_dev.py" ]; then
|
||||
python -m tito.main module notebooks --module 02_tensor
|
||||
[ -f "modules/source/02_tensor/tensor_dev.ipynb" ] && echo "✓ Tensor notebook created"
|
||||
fi
|
||||
|
||||
# Test activations module
|
||||
if [ -f "modules/source/03_activations/activations_dev.py" ]; then
|
||||
python -m tito.main module notebooks --module 03_activations
|
||||
[ -f "modules/source/03_activations/activations_dev.ipynb" ] && echo "✓ Activations notebook created"
|
||||
fi
|
||||
|
||||
- name: Validate notebook structure
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
# Install notebook validation tools
|
||||
pip install nbformat || echo "Could not install nbformat"
|
||||
|
||||
# Check generated notebooks have valid structure
|
||||
for notebook in modules/source/*/*.ipynb; do
|
||||
if [ -f "$notebook" ]; then
|
||||
echo "Validating $notebook"
|
||||
python -c "
|
||||
import json
|
||||
try:
|
||||
with open('$notebook') as f:
|
||||
nb = json.load(f)
|
||||
assert 'cells' in nb, 'No cells found'
|
||||
assert len(nb['cells']) > 0, 'Empty notebook'
|
||||
print('✓ $notebook is valid')
|
||||
except Exception as e:
|
||||
print('✗ $notebook validation failed:', e)
|
||||
exit(1)
|
||||
"
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Test batch conversion
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
# Clean up previous notebooks
|
||||
find modules/source -name "*.ipynb" -delete
|
||||
|
||||
# Test converting all modules at once
|
||||
python -m tito.main module notebooks
|
||||
|
||||
# Check that notebooks were created
|
||||
notebook_count=$(find modules/source -name "*.ipynb" | wc -l)
|
||||
echo "Created $notebook_count notebooks"
|
||||
[ "$notebook_count" -gt 0 ] && echo "✓ Batch conversion successful"
|
||||
|
||||
- name: Archive generated notebooks
|
||||
uses: actions/upload-artifact@v3
|
||||
if: always()
|
||||
with:
|
||||
name: generated-notebooks-python-${{ matrix.python-version }}
|
||||
path: modules/source/**/*.ipynb
|
||||
retention-days: 7
|
||||
|
||||
test-student-workflow:
|
||||
runs-on: ubuntu-latest
|
||||
needs: test-notebook-conversion
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.11
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Test automated setup script
|
||||
run: |
|
||||
chmod +x setup-dev.sh
|
||||
./setup-dev.sh
|
||||
|
||||
- name: Test student workflow
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
# Simulate student workflow
|
||||
echo "Testing complete student workflow..."
|
||||
|
||||
# Convert a module to notebook
|
||||
python -m tito.main module notebooks --module 03_activations
|
||||
|
||||
# Verify notebook exists and is valid
|
||||
[ -f "modules/source/03_activations/activations_dev.ipynb" ] && echo "✓ Student can create notebooks"
|
||||
|
||||
# Test TITO commands work
|
||||
python -m tito.main --help > /dev/null && echo "✓ TITO CLI accessible"
|
||||
python -m tito.main system doctor > /dev/null && echo "✓ Environment diagnosis works"
|
||||
326
NOTEBOOK_WORKFLOW.md
Normal file
326
NOTEBOOK_WORKFLOW.md
Normal file
@@ -0,0 +1,326 @@
|
||||
# TinyTorch Notebook Conversion Workflow
|
||||
|
||||
This document provides comprehensive instructions for converting TinyTorch Python modules to Jupyter notebooks using the TITO CLI.
|
||||
|
||||
## Overview
|
||||
|
||||
The TinyTorch project uses a sophisticated workflow where educational content is developed in Python files with special cell markers (`%%`) and then converted to interactive Jupyter notebooks using Jupytext. This approach provides:
|
||||
|
||||
- **Version control friendly**: Python files are easier to track in git
|
||||
- **Cell-based development**: Use `%%` markers to define notebook cells
|
||||
- **Automatic conversion**: TITO CLI handles the conversion seamlessly
|
||||
- **Student-friendly**: Students get interactive notebooks for learning
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Environment Setup
|
||||
|
||||
**Automated Setup (Recommended):**
|
||||
```bash
|
||||
# Clone and navigate to repository
|
||||
git clone https://github.com/your-org/TinyTorch.git
|
||||
cd TinyTorch
|
||||
|
||||
# Run automated setup script
|
||||
chmod +x setup-dev.sh
|
||||
./setup-dev.sh
|
||||
```
|
||||
|
||||
**Manual Setup:**
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
|
||||
# Install essential dependencies
|
||||
pip install rich>=13.0.0 jupytext>=1.14.0 pytest>=7.0.0 numpy>=1.21.0
|
||||
|
||||
# Install full requirements (optional)
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Install TinyTorch in development mode
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
### 2. Environment Verification
|
||||
|
||||
```bash
|
||||
# Activate virtual environment
|
||||
source .venv/bin/activate
|
||||
|
||||
# Run environment diagnosis
|
||||
python -m tito.main system doctor
|
||||
```
|
||||
|
||||
This should show:
|
||||
- ✅ Python 3.8+ detected
|
||||
- ✅ Virtual environment active
|
||||
- ✅ Essential dependencies installed
|
||||
|
||||
### 3. Convert Modules to Notebooks
|
||||
|
||||
**Convert Single Module:**
|
||||
```bash
|
||||
python -m tito.main module notebooks --module 03_activations
|
||||
```
|
||||
|
||||
**Convert All Modules:**
|
||||
```bash
|
||||
python -m tito.main module notebooks
|
||||
```
|
||||
|
||||
**Dry Run (Preview):**
|
||||
```bash
|
||||
python -m tito.main module notebooks --dry-run
|
||||
```
|
||||
|
||||
## Detailed Workflow
|
||||
|
||||
### Python Module Structure
|
||||
|
||||
Python modules are located in `modules/source/` with the following structure:
|
||||
|
||||
```
|
||||
modules/source/
|
||||
├── 01_setup/
|
||||
│ ├── setup_dev.py # Python module with %% cell markers
|
||||
│ ├── module.yaml # Module metadata
|
||||
│ └── README.md # Module documentation
|
||||
├── 02_tensor/
|
||||
│ ├── tensor_dev.py # Enhanced with educational content
|
||||
│ └── ...
|
||||
├── 03_activations/
|
||||
│ ├── activations_dev.py # Enhanced with educational content
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
### Cell Markers in Python Files
|
||||
|
||||
Use `%%` to define notebook cells in Python files:
|
||||
|
||||
```python
|
||||
# %% [markdown]
|
||||
"""
|
||||
# Activations - Nonlinearity in Neural Networks
|
||||
|
||||
Welcome to the Activations module!
|
||||
"""
|
||||
|
||||
# %%
|
||||
import numpy as np
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
## Understanding ReLU Activation
|
||||
"""
|
||||
|
||||
# %%
|
||||
def relu(x):
|
||||
"""ReLU activation function."""
|
||||
return np.maximum(0, x)
|
||||
```
|
||||
|
||||
### TITO CLI Commands
|
||||
|
||||
The TITO CLI provides comprehensive module management:
|
||||
|
||||
```bash
|
||||
# Get help
|
||||
python -m tito.main --help
|
||||
python -m tito.main module --help
|
||||
python -m tito.main module notebooks --help
|
||||
|
||||
# System commands
|
||||
python -m tito.main system info # System information
|
||||
python -m tito.main system doctor # Environment diagnosis
|
||||
|
||||
# Module management
|
||||
python -m tito.main module status # Module status
|
||||
python -m tito.main module notebooks # Convert to notebooks
|
||||
python -m tito.main module test # Run tests
|
||||
python -m tito.main module export # Export to package
|
||||
|
||||
# Notebook conversion
|
||||
python -m tito.main module notebooks --module 02_tensor # Single module
|
||||
python -m tito.main module notebooks --dry-run # Preview only
|
||||
python -m tito.main module notebooks --force # Force rebuild
|
||||
```
|
||||
|
||||
## Working with Generated Notebooks
|
||||
|
||||
### Opening Notebooks
|
||||
|
||||
```bash
|
||||
# Start Jupyter Lab
|
||||
jupyter lab
|
||||
|
||||
# Navigate to modules/source/03_activations/
|
||||
# Open activations_dev.ipynb
|
||||
```
|
||||
|
||||
### Student Workflow
|
||||
|
||||
1. **Get the repository:**
|
||||
```bash
|
||||
git clone https://github.com/your-org/TinyTorch.git
|
||||
cd TinyTorch
|
||||
```
|
||||
|
||||
2. **Setup environment:**
|
||||
```bash
|
||||
./setup-dev.sh
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
3. **Convert modules to notebooks:**
|
||||
```bash
|
||||
python -m tito.main module notebooks
|
||||
```
|
||||
|
||||
4. **Work with notebooks:**
|
||||
```bash
|
||||
jupyter lab
|
||||
```
|
||||
|
||||
### Developer Workflow
|
||||
|
||||
1. **Edit Python modules** in `modules/source/*/` directories
|
||||
2. **Test conversion:** `python -m tito.main module notebooks --dry-run`
|
||||
3. **Convert to notebooks:** `python -m tito.main module notebooks`
|
||||
4. **Export to package:** `python -m tito.main module export`
|
||||
5. **Run tests:** `python -m tito.main module test`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue: `ModuleNotFoundError: No module named 'rich'`**
|
||||
```bash
|
||||
# Solution: Install essential dependencies
|
||||
pip install rich>=13.0.0 jupytext>=1.14.0 pytest>=7.0.0
|
||||
```
|
||||
|
||||
**Issue: `jupytext not found`**
|
||||
```bash
|
||||
# Solution: Install jupytext
|
||||
pip install jupytext>=1.14.0
|
||||
```
|
||||
|
||||
**Issue: `Virtual environment not activated`**
|
||||
```bash
|
||||
# Solution: Activate virtual environment
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**Issue: `No *_dev.py files found`**
|
||||
```bash
|
||||
# Check you're in the right directory
|
||||
pwd # Should be /path/to/TinyTorch
|
||||
|
||||
# Check modules exist
|
||||
ls modules/source/
|
||||
```
|
||||
|
||||
**Issue: Architecture mismatch with NumPy**
|
||||
```bash
|
||||
# Create fresh virtual environment
|
||||
rm -rf .venv
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install --upgrade pip
|
||||
pip install numpy>=1.21.0,<2.0.0
|
||||
```
|
||||
|
||||
### Environment Diagnosis
|
||||
|
||||
Always start troubleshooting with the doctor command:
|
||||
|
||||
```bash
|
||||
python -m tito.main system doctor
|
||||
```
|
||||
|
||||
This will show:
|
||||
- Python version and virtual environment status
|
||||
- Installed dependencies and versions
|
||||
- Module structure validation
|
||||
- Actionable recommendations
|
||||
|
||||
### Getting Help
|
||||
|
||||
```bash
|
||||
# CLI help
|
||||
python -m tito.main --help
|
||||
|
||||
# Specific command help
|
||||
python -m tito.main module notebooks --help
|
||||
|
||||
# Environment diagnosis
|
||||
python -m tito.main system doctor
|
||||
|
||||
# Module status
|
||||
python -m tito.main module status
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
The repository includes automated testing of the notebook conversion workflow:
|
||||
|
||||
- **On every push**: Tests notebook conversion across Python 3.8-3.12
|
||||
- **Validates**: Generated notebooks have correct structure
|
||||
- **Archives**: Generated notebooks as artifacts
|
||||
- **Student workflow**: Tests the complete student setup process
|
||||
|
||||
See `.github/workflows/test-notebooks.yml` for implementation details.
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom Module Conversion
|
||||
|
||||
```bash
|
||||
# Convert specific modules only
|
||||
python -m tito.main module notebooks --module 02_tensor --module 03_activations
|
||||
|
||||
# Force rebuild existing notebooks
|
||||
python -m tito.main module notebooks --force
|
||||
|
||||
# Preview what would be converted
|
||||
python -m tito.main module notebooks --dry-run
|
||||
```
|
||||
|
||||
### Batch Operations
|
||||
|
||||
```bash
|
||||
# Convert all modules and export to package
|
||||
python -m tito.main module notebooks && python -m tito.main module export --all
|
||||
|
||||
# Test after conversion
|
||||
python -m tito.main module notebooks && python -m tito.main module test --all
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
After successful conversion, you'll find:
|
||||
|
||||
```
|
||||
modules/source/03_activations/
|
||||
├── activations_dev.py # Original Python module
|
||||
├── activations_dev.ipynb # Generated notebook ← NEW!
|
||||
├── module.yaml # Module metadata
|
||||
└── README.md # Module documentation
|
||||
```
|
||||
|
||||
The generated `.ipynb` files are fully functional Jupyter notebooks that can be opened in Jupyter Lab, VS Code, or any notebook environment.
|
||||
|
||||
## Summary
|
||||
|
||||
The TinyTorch notebook conversion workflow provides a robust, automated way to transform educational Python modules into interactive notebooks. The TITO CLI handles all the complexity, providing students with a simple, reliable workflow to get started with interactive learning.
|
||||
|
||||
**Key Benefits:**
|
||||
- **One-command setup**: `./setup-dev.sh`
|
||||
- **Reliable conversion**: TITO CLI with comprehensive error handling
|
||||
- **Student-friendly**: Clear error messages and helpful guidance
|
||||
- **Developer-friendly**: Automated CI/CD testing across Python versions
|
||||
- **Production-ready**: Used in real educational environments
|
||||
|
||||
For questions or issues, run `python -m tito.main system doctor` for comprehensive environment diagnosis.
|
||||
84
QUICK_START_NOTEBOOKS.md
Normal file
84
QUICK_START_NOTEBOOKS.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# TinyTorch: Quick Start for Interactive Notebooks
|
||||
|
||||
Get up and running with TinyTorch interactive notebooks in under 5 minutes!
|
||||
|
||||
## 🚀 One-Command Setup
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/TinyTorch.git
|
||||
cd TinyTorch
|
||||
|
||||
# Run automated setup
|
||||
chmod +x setup-dev.sh
|
||||
./setup-dev.sh
|
||||
|
||||
# Activate environment
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
## 📓 Convert Modules to Notebooks
|
||||
|
||||
```bash
|
||||
# Convert all modules to interactive notebooks
|
||||
python -m tito.main module notebooks
|
||||
|
||||
# Or convert specific modules
|
||||
python -m tito.main module notebooks --module 03_activations
|
||||
```
|
||||
|
||||
## 🎯 Start Learning
|
||||
|
||||
```bash
|
||||
# Open Jupyter Lab
|
||||
jupyter lab
|
||||
|
||||
# Navigate to modules/source/ and open any .ipynb file
|
||||
# Try: 03_activations/activations_dev.ipynb
|
||||
```
|
||||
|
||||
## ✅ Verify Everything Works
|
||||
|
||||
```bash
|
||||
# Run environment check
|
||||
python -m tito.main system doctor
|
||||
|
||||
# Should show:
|
||||
# ✅ Python 3.8+
|
||||
# ✅ Virtual Environment Active
|
||||
# ✅ Essential Dependencies Installed
|
||||
```
|
||||
|
||||
## 🆘 Need Help?
|
||||
|
||||
- **Full Documentation**: [NOTEBOOK_WORKFLOW.md](NOTEBOOK_WORKFLOW.md)
|
||||
- **Environment Issues**: `python -m tito.main system doctor`
|
||||
- **Command Help**: `python -m tito.main --help`
|
||||
|
||||
## 📋 Available Modules
|
||||
|
||||
After conversion, you'll have interactive notebooks for:
|
||||
|
||||
- **01_setup**: Environment setup and course introduction
|
||||
- **02_tensor**: Tensor operations and broadcasting
|
||||
- **03_activations**: Neural network activation functions
|
||||
- **04_layers**: Building neural network layers
|
||||
- **05_dense**: Fully connected networks
|
||||
- **06_spatial**: Convolutional neural networks
|
||||
- **07_attention**: Attention mechanisms and transformers
|
||||
- **08_dataloader**: Data loading and preprocessing
|
||||
- **09_autograd**: Automatic differentiation
|
||||
- **10_optimizers**: Gradient descent and optimization
|
||||
- **11_training**: Training loops and validation
|
||||
- **12_compression**: Model compression techniques
|
||||
- **13_kernels**: Custom CUDA kernels
|
||||
- **14_benchmarking**: Performance measurement
|
||||
- **15_mlops**: Production deployment
|
||||
|
||||
Each module includes:
|
||||
- 📚 Educational content with explanations
|
||||
- 💻 Interactive code cells
|
||||
- 🧪 Comprehensive tests via NBGrader
|
||||
- 🎯 Hands-on exercises and experiments
|
||||
|
||||
Happy learning! 🔥
|
||||
141
setup-dev.sh
Executable file
141
setup-dev.sh
Executable file
@@ -0,0 +1,141 @@
|
||||
#!/bin/bash
|
||||
# TinyTorch Development Environment Setup Script
|
||||
# This script sets up a complete development environment for TinyTorch
|
||||
|
||||
set -e # Exit on any error
|
||||
|
||||
echo "🔧 Setting up TinyTorch development environment..."
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
print_status() {
|
||||
echo -e "${BLUE}==>${NC} $1"
|
||||
}
|
||||
|
||||
print_success() {
|
||||
echo -e "${GREEN}✓${NC} $1"
|
||||
}
|
||||
|
||||
print_warning() {
|
||||
echo -e "${YELLOW}⚠${NC} $1"
|
||||
}
|
||||
|
||||
print_error() {
|
||||
echo -e "${RED}✗${NC} $1"
|
||||
}
|
||||
|
||||
# Function to check if command exists
|
||||
command_exists() {
|
||||
command -v "$1" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# Check Python version
|
||||
print_status "Checking Python installation..."
|
||||
if ! command_exists python3; then
|
||||
print_error "Python 3 is not installed. Please install Python 3.8+ first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python_version=$(python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
|
||||
required_version="3.8"
|
||||
|
||||
if ! python3 -c "import sys; exit(0 if sys.version_info >= (3, 8) else 1)"; then
|
||||
print_error "Python 3.8+ required, found Python $python_version"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
print_success "Python $python_version found"
|
||||
|
||||
# Create virtual environment if it doesn't exist
|
||||
print_status "Setting up virtual environment..."
|
||||
if [ ! -d ".venv" ]; then
|
||||
python3 -m venv .venv
|
||||
print_success "Virtual environment created"
|
||||
else
|
||||
print_warning "Virtual environment already exists"
|
||||
fi
|
||||
|
||||
# Activate virtual environment
|
||||
print_status "Activating virtual environment..."
|
||||
source .venv/bin/activate
|
||||
print_success "Virtual environment activated"
|
||||
|
||||
# Upgrade pip
|
||||
print_status "Upgrading pip..."
|
||||
pip install --upgrade pip
|
||||
|
||||
# Install requirements with error handling
|
||||
print_status "Installing dependencies..."
|
||||
|
||||
# Install core dependencies first (these are most likely to have issues)
|
||||
print_status "Installing core dependencies..."
|
||||
pip install setuptools>=70.0.0 wheel>=0.42.0
|
||||
|
||||
# Install numerical dependencies
|
||||
print_status "Installing numerical computing dependencies..."
|
||||
pip install "numpy>=1.21.0,<2.0.0"
|
||||
|
||||
# Install TITO CLI dependencies
|
||||
print_status "Installing TITO CLI dependencies..."
|
||||
pip install rich>=13.0.0 jupytext>=1.14.0
|
||||
|
||||
# Install testing dependencies
|
||||
print_status "Installing testing dependencies..."
|
||||
pip install pytest>=7.0.0
|
||||
|
||||
# Install remaining dependencies
|
||||
print_status "Installing remaining dependencies..."
|
||||
pip install -r requirements.txt || {
|
||||
print_warning "Some dependencies failed to install. Continuing with essential packages..."
|
||||
}
|
||||
|
||||
# Install the package in development mode
|
||||
print_status "Installing TinyTorch in development mode..."
|
||||
pip install -e . || {
|
||||
print_warning "Failed to install in development mode. You may need to run this manually."
|
||||
}
|
||||
|
||||
# Verify TITO CLI works
|
||||
print_status "Verifying TITO CLI installation..."
|
||||
if python -m tito.main --help > /dev/null 2>&1; then
|
||||
print_success "TITO CLI is working correctly"
|
||||
else
|
||||
print_error "TITO CLI verification failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run doctor command to check environment
|
||||
print_status "Running environment diagnostics..."
|
||||
python -m tito.main system doctor || {
|
||||
print_warning "Some environment issues detected. See output above."
|
||||
}
|
||||
|
||||
# Create helpful aliases
|
||||
print_status "Setting up helpful aliases..."
|
||||
echo "# TinyTorch Development Aliases" > .env
|
||||
echo "alias tito='python -m tito.main'" >> .env
|
||||
echo "alias nb='python -m tito.main module notebooks'" >> .env
|
||||
echo "alias export-all='python -m tito.main export --all'" >> .env
|
||||
|
||||
# Success message
|
||||
echo ""
|
||||
echo "🎉 Setup complete!"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Activate the environment: source .venv/bin/activate"
|
||||
echo " 2. Test TITO CLI: python -m tito.main"
|
||||
echo " 3. Convert modules to notebooks: python -m tito.main module notebooks"
|
||||
echo " 4. Open notebooks: jupyter lab"
|
||||
echo ""
|
||||
echo "Quick commands:"
|
||||
echo " • Convert single module: python -m tito.main module notebooks --module 03_activations"
|
||||
echo " • Convert all modules: python -m tito.main module notebooks"
|
||||
echo " • Export modules: python -m tito.main export --all"
|
||||
echo " • Run tests: python -m tito.main module test"
|
||||
echo ""
|
||||
print_success "TinyTorch development environment is ready!"
|
||||
Reference in New Issue
Block a user