mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-11 22:43:34 -05:00
Refactors to use .venv for virtual environment
Updates the project to use `.venv` as the standard virtual environment directory. This change: - Updates `.gitignore` to ignore `.venv/`. - Modifies the activation script to create and activate `.venv`. - Adjusts the `tito.py` script to check for `.venv`'s existence and activation. - Updates documentation and setup scripts to reflect the new virtual environment naming convention. This change streamlines environment management and aligns with common Python practices.
This commit is contained in:
2
.gitignore
vendored
2
.gitignore
vendored
@@ -129,7 +129,7 @@ env.bak/
|
||||
venv.bak/
|
||||
|
||||
# TinyTorch specific
|
||||
tinytorch-env/
|
||||
.venv/
|
||||
*.log
|
||||
logs/
|
||||
checkpoints/
|
||||
|
||||
@@ -33,7 +33,7 @@ Track your progress through the course:
|
||||
### Daily Workflow
|
||||
```bash
|
||||
cd TinyTorch
|
||||
source tinytorch-env/bin/activate # Always activate first!
|
||||
source .venv/bin/activate # Always activate first!
|
||||
python3 bin/tito.py info # Check system status
|
||||
```
|
||||
|
||||
|
||||
65
README.md
65
README.md
@@ -1,9 +1,6 @@
|
||||
# Tiny🔥Torch
|
||||
**_Build a Machine Learning System from Scratch._**
|
||||
# Tiny🔥Torch: Build a Machine Learning System from Scratch
|
||||
|
||||
TinyTorch is a pedagogical project designed to accompany the [*Machine Learning Systems*](https://mlsysbook.ai) textbook.
|
||||
|
||||
Inspired by OS and compiler courses where students build entire systems from first principles, TinyTorch is designed to guide students through building a complete ML training and inference runtime system, from autograd to data pipelines, optimizers to profilers. You’ll implement **everything, entirely from scratch.** The intent is to train the next generation of AI systems engineers.
|
||||
TinyTorch is a pedagogical project designed to accompany the [*Machine Learning Systems*](https://mlsysbook.ai) textbook. Inspired by OS and compiler courses where students build entire systems from first principles, TinyTorch guides you through building a complete ML training and inference runtime — from autograd to data pipelines, optimizers to profilers — **entirely from scratch**.
|
||||
|
||||
This is not a PyTorch tutorial. In TinyTorch, you'll **write the components that frameworks like PyTorch are built on.**
|
||||
|
||||
@@ -13,7 +10,7 @@ This is not a PyTorch tutorial. In TinyTorch, you'll **write the components that
|
||||
|
||||
By the end of this project, you'll have implemented a fully functional ML system capable of:
|
||||
|
||||
- **Training neural networks** (MLPs, CNNs) on real datasets (MNIST, CIFAR-10)
|
||||
- **Training neural networks** (MLPs, CNNs) on real datasets 10)
|
||||
- **Automatic differentiation** with a custom autograd engine
|
||||
- **Memory-efficient data loading** with custom DataLoader implementations
|
||||
- **Multiple optimization algorithms** (SGD, Adam, RMSprop)
|
||||
@@ -152,18 +149,18 @@ TinyTorch/
|
||||
│ ├── mnist.py
|
||||
│ ├── cifar10.py
|
||||
│ └── transforms.py
|
||||
├── projects/ # Component-specific projects
|
||||
│ ├── setup/ # Environment setup & onboarding
|
||||
│ ├── tensor/ # Basic tensor implementation
|
||||
│ ├── mlp/ # Multi-layer perceptron (manual backprop)
|
||||
│ ├── cnn/ # Convolutional neural networks (basic)
|
||||
│ ├── data/ # Data pipeline & loading
|
||||
│ ├── training/ # Autograd engine & training optimization
|
||||
│ ├── profiling/ # Performance profiling tools
|
||||
│ ├── compression/ # Model compression techniques
|
||||
│ ├── kernels/ # Custom compute kernels
|
||||
│ ├── benchmarking/ # Performance benchmarking
|
||||
│ └── mlops/ # MLOps & production monitoring
|
||||
├── modules/ # 🧩 System Modules
|
||||
│ ├── 01_setup/ # Environment setup & onboarding
|
||||
│ ├── 02_tensor/ # Basic tensor implementation
|
||||
│ ├── 03_mlp/ # Multi-layer perceptron (manual backprop)
|
||||
│ ├── 04_cnn/ # Convolutional neural networks (basic)
|
||||
│ ├── 05_data/ # Data pipeline & loading
|
||||
│ ├── 06_training/ # Autograd engine & training optimization
|
||||
│ ├── 07_profiling/ # Performance profiling tools
|
||||
│ ├── 08_compression/ # Model compression techniques
|
||||
│ ├── 09_kernels/ # Custom compute kernels
|
||||
│ ├── 10_benchmarking/ # Performance benchmarking
|
||||
│ └── 11_mlops/ # MLOps & production monitoring
|
||||
├── docs/ # Course documentation
|
||||
│ ├── tutorials/ # Step-by-step tutorials
|
||||
│ ├── api/ # API documentation
|
||||
@@ -197,24 +194,24 @@ TinyTorch/
|
||||
|
||||
## 🎯 Course Navigation & Getting Started
|
||||
|
||||
**New to TinyTorch?** Start here: [`projects/setup/README.md`](projects/setup/README.md)
|
||||
**New to TinyTorch?** Start here: [`modules/01_setup/README.md`](modules/01_setup/README.md)
|
||||
|
||||
### 📋 Project Sequence
|
||||
Each project builds on the previous ones. Click the links to jump to specific instructions:
|
||||
### 📋 Module Sequence
|
||||
Each module builds on the previous ones. Click the links to jump to specific instructions:
|
||||
|
||||
| Order | Project | Status | Description | Instructions |
|
||||
|-------|---------|--------|-------------|--------------|
|
||||
| 0 | **Setup** | 🚀 **START HERE** | Environment & CLI setup | [`projects/setup/README.md`](projects/setup/README.md) |
|
||||
| 1 | **Tensor** | ⏳ Coming Next | Basic tensor operations | [`projects/tensor/README.md`](projects/tensor/README.md) |
|
||||
| 2 | **MLP** | ⏳ Future | Multi-layer perceptron (manual backprop) | [`projects/mlp/README.md`](projects/mlp/README.md) |
|
||||
| 3 | **CNN** | ⏳ Future | Convolutional networks (basic) | [`projects/cnn/README.md`](projects/cnn/README.md) |
|
||||
| 4 | **Data** | ⏳ Future | Data loading pipeline | [`projects/data/README.md`](projects/data/README.md) |
|
||||
| 5 | **Training** | ⏳ Future | Autograd engine & optimization | [`projects/training/README.md`](projects/training/README.md) |
|
||||
| 6 | **Profiling** | ⏳ Future | Performance profiling | [`projects/profiling/README.md`](projects/profiling/README.md) |
|
||||
| 7 | **Compression** | ⏳ Future | Model compression | [`projects/compression/README.md`](projects/compression/README.md) |
|
||||
| 8 | **Kernels** | ⏳ Future | Custom compute kernels | [`projects/kernels/README.md`](projects/kernels/README.md) |
|
||||
| 9 | **Benchmarking** | ⏳ Future | Performance benchmarking | [`projects/benchmarking/README.md`](projects/benchmarking/README.md) |
|
||||
| 10 | **MLOps** | ⏳ Future | Production monitoring | [`projects/mlops/README.md`](projects/mlops/README.md) |
|
||||
| Order | Module | Status | Description | Instructions |
|
||||
|-------|--------|--------|-------------|--------------|
|
||||
| 0 | **Setup** | 🚀 **START HERE** | Environment & CLI setup | [`modules/01_setup/README.md`](modules/01_setup/README.md) |
|
||||
| 1 | **Tensor** | ⏳ Coming Next | Basic tensor operations | [`modules/02_tensor/README.md`](modules/02_tensor/README.md) |
|
||||
| 2 | **MLP** | ⏳ Future | Multi-layer perceptron (manual backprop) | [`modules/03_mlp/README.md`](modules/03_mlp/README.md) |
|
||||
| 3 | **CNN** | ⏳ Future | Convolutional networks (basic) | [`modules/04_cnn/README.md`](modules/04_cnn/README.md) |
|
||||
| 4 | **Data** | ⏳ Future | Data loading pipeline | [`modules/05_data/README.md`](modules/05_data/README.md) |
|
||||
| 5 | **Training** | ⏳ Future | Autograd engine & optimization | [`modules/06_training/README.md`](modules/06_training/README.md) |
|
||||
| 6 | **Profiling** | ⏳ Future | Performance profiling | [`modules/07_profiling/README.md`](modules/07_profiling/README.md) |
|
||||
| 7 | **Compression** | ⏳ Future | Model compression | [`modules/08_compression/README.md`](modules/08_compression/README.md) |
|
||||
| 8 | **Kernels** | ⏳ Future | Custom compute kernels | [`modules/09_kernels/README.md`](modules/09_kernels/README.md) |
|
||||
| 9 | **Benchmarking** | ⏳ Future | Performance benchmarking | [`modules/10_benchmarking/README.md`](modules/10_benchmarking/README.md) |
|
||||
| 10 | **MLOps** | ⏳ Future | Production monitoring | [`modules/11_mlops/README.md`](modules/11_mlops/README.md) |
|
||||
|
||||
### 🚀 Quick Start Guide
|
||||
**First time?** Follow this exact sequence:
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
# TinyTorch Environment Activation & Setup
|
||||
|
||||
# Check if virtual environment exists, create if not
|
||||
if [ ! -d "tinytorch-env" ]; then
|
||||
if [ ! -d ".venv" ]; then
|
||||
echo "🆕 First time setup - creating environment..."
|
||||
python3 -m venv tinytorch-env || {
|
||||
python3 -m venv .venv || {
|
||||
echo "❌ Failed to create virtual environment"
|
||||
exit 1
|
||||
}
|
||||
echo "📦 Installing dependencies..."
|
||||
tinytorch-env/bin/pip install -r requirements.txt || {
|
||||
.venv/bin/pip install -r requirements.txt || {
|
||||
echo "❌ Failed to install dependencies"
|
||||
exit 1
|
||||
}
|
||||
@@ -17,7 +17,7 @@ if [ ! -d "tinytorch-env" ]; then
|
||||
fi
|
||||
|
||||
echo "🔥 Activating TinyTorch environment..."
|
||||
source tinytorch-env/bin/activate
|
||||
source .venv/bin/activate
|
||||
|
||||
# Create tito alias for convenience
|
||||
alias tito="python3 bin/tito.py"
|
||||
@@ -27,3 +27,4 @@ echo "💡 Quick commands:"
|
||||
echo " tito info - Check system status"
|
||||
echo " tito test - Run tests"
|
||||
echo " tito doctor - Diagnose issues"
|
||||
echo " jupyter notebook - Start Jupyter for interactive development"
|
||||
|
||||
222
bin/tito.py
222
bin/tito.py
@@ -172,7 +172,7 @@ def validate_environment():
|
||||
in_venv = (hasattr(sys, 'real_prefix') or
|
||||
(hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix))
|
||||
if not in_venv:
|
||||
issues.append("Virtual environment not activated. Run: source tinytorch-env/bin/activate")
|
||||
issues.append("Virtual environment not activated. Run: source .venv/bin/activate")
|
||||
|
||||
# Check Python version
|
||||
if sys.version_info < (3, 8):
|
||||
@@ -221,7 +221,7 @@ def cmd_info(args):
|
||||
info_text.append(f"Working Directory: {os.getcwd()}\n", style="cyan")
|
||||
|
||||
# Virtual environment check - use same robust detection as doctor
|
||||
venv_path = Path("tinytorch-env")
|
||||
venv_path = Path(".venv")
|
||||
venv_exists = venv_path.exists()
|
||||
in_venv = (
|
||||
# Method 1: Check VIRTUAL_ENV environment variable (most reliable for activation)
|
||||
@@ -264,7 +264,7 @@ def cmd_info(args):
|
||||
console.print()
|
||||
|
||||
# Implementation status
|
||||
projects = [
|
||||
modules = [
|
||||
("Setup", "hello_tinytorch function", check_setup_status),
|
||||
("Tensor", "basic tensor operations", check_tensor_status),
|
||||
("MLP", "multi-layer perceptron (manual)", check_mlp_status),
|
||||
@@ -278,14 +278,14 @@ def cmd_info(args):
|
||||
("MLOps", "production monitoring", check_mlops_status),
|
||||
]
|
||||
|
||||
# Project Status Table
|
||||
status_table = Table(title="🚀 Project Implementation Status", show_header=True, header_style="bold blue")
|
||||
# Module Status Table
|
||||
status_table = Table(title="🚀 Module Implementation Status", show_header=True, header_style="bold blue")
|
||||
status_table.add_column("ID", style="dim", width=3, justify="center")
|
||||
status_table.add_column("Project", style="bold cyan", width=12)
|
||||
status_table.add_column("Status", width=18, justify="center")
|
||||
status_table.add_column("Description", style="dim", width=40)
|
||||
|
||||
for i, (name, desc, check_func) in enumerate(projects):
|
||||
for i, (name, desc, check_func) in enumerate(modules):
|
||||
status_text = check_func()
|
||||
if "✅" in status_text:
|
||||
status_style = "[green]✅ Implemented[/green]"
|
||||
@@ -342,19 +342,19 @@ def cmd_info(args):
|
||||
console.print(Panel(arch_tree, title="🏗️ System Architecture", border_style="bright_blue"))
|
||||
|
||||
def cmd_test(args):
|
||||
"""Run tests for a specific project."""
|
||||
valid_projects = ["setup", "tensor", "mlp", "cnn", "data", "training",
|
||||
"profiling", "compression", "kernels", "benchmarking", "mlops"]
|
||||
"""Run tests for a specific module."""
|
||||
valid_modules = ["setup", "tensor", "mlp", "cnn", "data", "training",
|
||||
"profiling", "compression", "kernels", "benchmarking", "mlops"]
|
||||
|
||||
if args.all:
|
||||
# Run all tests with progress bar
|
||||
import subprocess
|
||||
failed_projects = []
|
||||
failed_modules = []
|
||||
|
||||
# Count existing test files
|
||||
existing_tests = [p for p in valid_projects if Path(f"projects/{p}/test_{p}.py").exists()]
|
||||
existing_tests = [p for p in valid_modules if Path(f"modules/{p}/test_{p}.py").exists()]
|
||||
|
||||
console.print(Panel(f"🧪 Running tests for {len(existing_tests)} projects",
|
||||
console.print(Panel(f"🧪 Running tests for {len(existing_tests)} modules",
|
||||
title="Test Suite", border_style="bright_cyan"))
|
||||
|
||||
with Progress(
|
||||
@@ -367,24 +367,24 @@ def cmd_test(args):
|
||||
|
||||
task = progress.add_task("Running tests...", total=len(existing_tests))
|
||||
|
||||
for project in existing_tests:
|
||||
progress.update(task, description=f"Testing {project}...")
|
||||
for module in existing_tests:
|
||||
progress.update(task, description=f"Testing {module}...")
|
||||
|
||||
test_file = f"projects/{project}/test_{project}.py"
|
||||
test_file = f"modules/{module}/test_{module}.py"
|
||||
result = subprocess.run([sys.executable, "-m", "pytest", test_file, "-v"],
|
||||
capture_output=True, text=True)
|
||||
|
||||
if result.returncode != 0:
|
||||
failed_projects.append(project)
|
||||
console.print(f"[red]❌ {project} tests failed[/red]")
|
||||
failed_modules.append(module)
|
||||
console.print(f"[red]❌ {module} tests failed[/red]")
|
||||
else:
|
||||
console.print(f"[green]✅ {project} tests passed[/green]")
|
||||
console.print(f"[green]✅ {module} tests passed[/green]")
|
||||
|
||||
progress.advance(task)
|
||||
|
||||
# Results summary
|
||||
if failed_projects:
|
||||
console.print(Panel(f"[red]❌ Failed projects: {', '.join(failed_projects)}[/red]",
|
||||
if failed_modules:
|
||||
console.print(Panel(f"[red]❌ Failed modules: {', '.join(failed_modules)}[/red]",
|
||||
title="Test Results", border_style="red"))
|
||||
return 1
|
||||
else:
|
||||
@@ -392,17 +392,17 @@ def cmd_test(args):
|
||||
title="Test Results", border_style="green"))
|
||||
return 0
|
||||
|
||||
elif args.project in valid_projects:
|
||||
# Run specific project tests
|
||||
elif args.module in valid_modules:
|
||||
# Run specific module tests
|
||||
import subprocess
|
||||
test_file = f"projects/{args.project}/test_{args.project}.py"
|
||||
test_file = f"modules/{args.module}/test_{args.module}.py"
|
||||
|
||||
console.print(Panel(f"🧪 Running tests for project: [bold cyan]{args.project}[/bold cyan]",
|
||||
title="Single Project Test", border_style="bright_cyan"))
|
||||
console.print(Panel(f"🧪 Running tests for module: [bold cyan]{args.module}[/bold cyan]",
|
||||
title="Single Module Test", border_style="bright_cyan"))
|
||||
|
||||
if not Path(test_file).exists():
|
||||
console.print(Panel(f"[yellow]⏳ Test file not found: {test_file}\n"
|
||||
f"Project '{args.project}' may not be implemented yet.[/yellow]",
|
||||
f"Module '{args.module}' may not be implemented yet.[/yellow]",
|
||||
title="Test Not Found", border_style="yellow"))
|
||||
return 1
|
||||
|
||||
@@ -414,36 +414,36 @@ def cmd_test(args):
|
||||
|
||||
# Show result summary
|
||||
if result.returncode == 0:
|
||||
console.print(Panel(f"[green]✅ All tests passed for {args.project}![/green]",
|
||||
console.print(Panel(f"[green]✅ All tests passed for {args.module}![/green]",
|
||||
title="Test Results", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Some tests failed for {args.project}[/red]",
|
||||
console.print(Panel(f"[red]❌ Some tests failed for {args.module}[/red]",
|
||||
title="Test Results", border_style="red"))
|
||||
|
||||
return result.returncode
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Unknown project: {args.project}[/red]\n"
|
||||
f"Valid projects: {', '.join(valid_projects)}",
|
||||
title="Invalid Project", border_style="red"))
|
||||
console.print(Panel(f"[red]❌ Unknown module: {args.module}[/red]\n"
|
||||
f"Valid modules: {', '.join(valid_modules)}",
|
||||
title="Invalid Module", border_style="red"))
|
||||
return 1
|
||||
|
||||
def cmd_submit(args):
|
||||
"""Submit project for grading."""
|
||||
"""Submit module for grading."""
|
||||
submit_text = Text()
|
||||
submit_text.append(f"📤 Submitting project: {args.project}\n\n", style="bold cyan")
|
||||
submit_text.append(f"📤 Submitting module: {args.module}\n\n", style="bold cyan")
|
||||
submit_text.append("🚧 Submission system not yet implemented.\n\n", style="yellow")
|
||||
submit_text.append("For now, make sure all tests pass with:\n", style="dim")
|
||||
submit_text.append(f" python -m pytest projects/{args.project}/test_*.py -v", style="bold white")
|
||||
submit_text.append(f" python -m pytest modules/{args.module}/test_*.py -v", style="bold white")
|
||||
|
||||
console.print(Panel(submit_text, title="Project Submission", border_style="bright_yellow"))
|
||||
console.print(Panel(submit_text, title="Module Submission", border_style="bright_yellow"))
|
||||
|
||||
def cmd_status(args):
|
||||
"""Check project status."""
|
||||
"""Check module status."""
|
||||
status_text = Text()
|
||||
status_text.append(f"📊 Status for project: {args.project}\n\n", style="bold cyan")
|
||||
status_text.append(f"📊 Status for module: {args.module}\n\n", style="bold cyan")
|
||||
status_text.append("🚧 Status system not yet implemented.", style="yellow")
|
||||
|
||||
console.print(Panel(status_text, title="Project Status", border_style="bright_yellow"))
|
||||
console.print(Panel(status_text, title="Module Status", border_style="bright_yellow"))
|
||||
|
||||
|
||||
|
||||
@@ -481,7 +481,7 @@ def cmd_doctor(args):
|
||||
venv_status = "[yellow]✅ Ready (Not Active)[/yellow]"
|
||||
else:
|
||||
venv_status = "[red]❌ Not Found[/red]"
|
||||
env_table.add_row("Virtual Environment", venv_status, "tinytorch-env")
|
||||
env_table.add_row("Virtual Environment", venv_status, ".venv")
|
||||
|
||||
# Dependencies
|
||||
dependencies = ['numpy', 'matplotlib', 'pytest', 'yaml', 'black', 'rich']
|
||||
@@ -496,8 +496,8 @@ def cmd_doctor(args):
|
||||
console.print(env_table)
|
||||
console.print()
|
||||
|
||||
# Project structure table
|
||||
struct_table = Table(title="Project Structure", show_header=True, header_style="bold magenta")
|
||||
# Module structure table
|
||||
struct_table = Table(title="Module Structure", show_header=True, header_style="bold magenta")
|
||||
struct_table.add_column("Path", style="cyan", width=25)
|
||||
struct_table.add_column("Status", justify="left")
|
||||
struct_table.add_column("Type", style="dim", width=25)
|
||||
@@ -505,7 +505,7 @@ def cmd_doctor(args):
|
||||
required_paths = [
|
||||
('tinytorch/', 'Package directory'),
|
||||
('tinytorch/core/', 'Core module directory'),
|
||||
('projects/', 'Project directory'),
|
||||
('modules/', 'Module directory'),
|
||||
('bin/tito.py', 'CLI script'),
|
||||
('requirements.txt', 'Dependencies file')
|
||||
]
|
||||
@@ -519,11 +519,108 @@ def cmd_doctor(args):
|
||||
console.print(struct_table)
|
||||
console.print()
|
||||
|
||||
# Project implementations
|
||||
# Module implementations
|
||||
console.print(Panel("📋 Implementation Status",
|
||||
title="Project Status", border_style="bright_blue"))
|
||||
title="Module Status", border_style="bright_blue"))
|
||||
cmd_info(argparse.Namespace(hello=False, show_architecture=False))
|
||||
|
||||
def cmd_jupyter(args):
|
||||
"""Start Jupyter notebook server."""
|
||||
import subprocess
|
||||
|
||||
console.print(Panel("📓 Jupyter Notebook Server",
|
||||
title="Interactive Development", border_style="bright_green"))
|
||||
|
||||
# Determine which Jupyter to start
|
||||
if args.lab:
|
||||
cmd = ["jupyter", "lab", "--port", str(args.port)]
|
||||
console.print(f"🚀 Starting JupyterLab on port {args.port}...")
|
||||
else:
|
||||
cmd = ["jupyter", "notebook", "--port", str(args.port)]
|
||||
console.print(f"🚀 Starting Jupyter Notebook on port {args.port}...")
|
||||
|
||||
console.print("💡 Open your browser to the URL shown above")
|
||||
console.print("📁 Navigate to your module's notebook directory")
|
||||
console.print("🔄 Press Ctrl+C to stop the server")
|
||||
|
||||
try:
|
||||
subprocess.run(cmd)
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n🛑 Jupyter server stopped")
|
||||
except FileNotFoundError:
|
||||
console.print(Panel("[red]❌ Jupyter not found. Install with: pip install jupyter[/red]",
|
||||
title="Error", border_style="red"))
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
def cmd_nbdev(args):
|
||||
"""Run nbdev commands for notebook development."""
|
||||
import subprocess
|
||||
|
||||
console.print(Panel("📓 nbdev Notebook Development",
|
||||
title="Notebook Tools", border_style="bright_cyan"))
|
||||
|
||||
if args.build_lib:
|
||||
console.print("🔨 Building library from notebooks...")
|
||||
result = subprocess.run(["nbdev_build_lib"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Library built successfully![/green]",
|
||||
title="Build Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Build failed: {result.stderr}[/red]",
|
||||
title="Build Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
elif args.build_docs:
|
||||
console.print("📚 Building documentation from notebooks...")
|
||||
result = subprocess.run(["nbdev_build_docs"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Documentation built successfully![/green]",
|
||||
title="Docs Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Docs build failed: {result.stderr}[/red]",
|
||||
title="Docs Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
elif args.test:
|
||||
console.print("🧪 Running notebook tests...")
|
||||
result = subprocess.run(["nbdev_test"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ All notebook tests passed![/green]",
|
||||
title="Test Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Some tests failed: {result.stderr}[/red]",
|
||||
title="Test Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
elif args.clean:
|
||||
console.print("🧹 Cleaning notebook build artifacts...")
|
||||
result = subprocess.run(["nbdev_clean_nbs"], capture_output=True, text=True)
|
||||
if result.returncode == 0:
|
||||
console.print(Panel("[green]✅ Cleaned successfully![/green]",
|
||||
title="Clean Success", border_style="green"))
|
||||
else:
|
||||
console.print(Panel(f"[red]❌ Clean failed: {result.stderr}[/red]",
|
||||
title="Clean Error", border_style="red"))
|
||||
return result.returncode
|
||||
|
||||
else:
|
||||
help_text = Text()
|
||||
help_text.append("📓 nbdev Commands:\n\n", style="bold cyan")
|
||||
help_text.append(" tito nbdev --build-lib - Build library from notebooks\n", style="white")
|
||||
help_text.append(" tito nbdev --build-docs - Build documentation\n", style="white")
|
||||
help_text.append(" tito nbdev --test - Run notebook tests\n", style="white")
|
||||
help_text.append(" tito nbdev --clean - Clean build artifacts\n\n", style="white")
|
||||
help_text.append("💡 Development workflow:\n", style="bold yellow")
|
||||
help_text.append(" 1. Work in modules/*/notebook/*_dev.ipynb\n", style="dim")
|
||||
help_text.append(" 2. Test interactively\n", style="dim")
|
||||
help_text.append(" 3. Run: tito nbdev --build-lib\n", style="dim")
|
||||
help_text.append(" 4. Test compiled package\n", style="dim")
|
||||
|
||||
console.print(Panel(help_text, title="nbdev Help", border_style="bright_cyan"))
|
||||
return 0
|
||||
|
||||
def main():
|
||||
"""Main CLI entry point."""
|
||||
parser = argparse.ArgumentParser(
|
||||
@@ -542,21 +639,34 @@ def main():
|
||||
info_parser.add_argument("--show-architecture", action="store_true", help="Show system architecture")
|
||||
|
||||
# Test command
|
||||
test_parser = subparsers.add_parser("test", help="Run project tests")
|
||||
test_parser.add_argument("--project", help="Project to test")
|
||||
test_parser.add_argument("--all", action="store_true", help="Run all project tests")
|
||||
test_parser = subparsers.add_parser("test", help="Run module tests")
|
||||
test_parser.add_argument("--module", help="Module to test")
|
||||
test_parser.add_argument("--all", action="store_true", help="Run all module tests")
|
||||
|
||||
# Submit command
|
||||
submit_parser = subparsers.add_parser("submit", help="Submit project")
|
||||
submit_parser.add_argument("--project", required=True, help="Project to submit")
|
||||
submit_parser = subparsers.add_parser("submit", help="Submit module")
|
||||
submit_parser.add_argument("--module", required=True, help="Module to submit")
|
||||
|
||||
# Status command
|
||||
status_parser = subparsers.add_parser("status", help="Check project status")
|
||||
status_parser.add_argument("--project", required=True, help="Project to check")
|
||||
status_parser = subparsers.add_parser("status", help="Check module status")
|
||||
status_parser.add_argument("--module", required=True, help="Module to check")
|
||||
|
||||
# Doctor command
|
||||
doctor_parser = subparsers.add_parser("doctor", help="Run environment diagnosis")
|
||||
|
||||
# nbdev commands
|
||||
nbdev_parser = subparsers.add_parser("nbdev", help="nbdev notebook development commands")
|
||||
nbdev_parser.add_argument("--build-lib", action="store_true", help="Build library from notebooks")
|
||||
nbdev_parser.add_argument("--build-docs", action="store_true", help="Build documentation from notebooks")
|
||||
nbdev_parser.add_argument("--test", action="store_true", help="Run notebook tests")
|
||||
nbdev_parser.add_argument("--clean", action="store_true", help="Clean notebook build artifacts")
|
||||
|
||||
# Jupyter command
|
||||
jupyter_parser = subparsers.add_parser("jupyter", help="Start Jupyter notebook server")
|
||||
jupyter_parser.add_argument("--notebook", action="store_true", help="Start classic notebook")
|
||||
jupyter_parser.add_argument("--lab", action="store_true", help="Start JupyterLab")
|
||||
jupyter_parser.add_argument("--port", type=int, default=8888, help="Port to run on (default: 8888)")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Handle version flag
|
||||
@@ -570,10 +680,10 @@ def main():
|
||||
|
||||
# Validate test command arguments
|
||||
if args.command == "test":
|
||||
if not args.all and not args.project:
|
||||
if not args.all and not args.module:
|
||||
error_text = Text()
|
||||
error_text.append("❌ Error: Must specify either --project or --all\n\n", style="bold red")
|
||||
error_text.append("Usage: python bin/tito.py test --project <name> | --all", style="cyan")
|
||||
error_text.append("❌ Error: Must specify either --module or --all\n\n", style="bold red")
|
||||
error_text.append("Usage: python bin/tito.py test --module <name> | --all", style="cyan")
|
||||
console.print(Panel(error_text, title="Invalid Arguments", border_style="red"))
|
||||
return 1
|
||||
|
||||
@@ -588,6 +698,10 @@ def main():
|
||||
cmd_status(args)
|
||||
elif args.command == "doctor":
|
||||
cmd_doctor(args)
|
||||
elif args.command == "nbdev":
|
||||
return cmd_nbdev(args)
|
||||
elif args.command == "jupyter":
|
||||
return cmd_jupyter(args)
|
||||
else:
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
@@ -17,8 +17,8 @@ python3 projects/setup/create_env.py
|
||||
|
||||
**Step 3**: Activate environment and verify
|
||||
```bash
|
||||
source tinytorch-env/bin/activate # macOS/Linux
|
||||
# OR: tinytorch-env\Scripts\activate # Windows
|
||||
source .venv/bin/activate # macOS/Linux
|
||||
# OR: .venv\Scripts\activate # Windows
|
||||
|
||||
python3 projects/setup/check_setup.py
|
||||
```
|
||||
@@ -31,8 +31,8 @@ python3 projects/setup/check_setup.py
|
||||
|
||||
**Step 1**: Create virtual environment
|
||||
```bash
|
||||
python3 -m venv tinytorch-env
|
||||
source tinytorch-env/bin/activate
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**Step 2**: Install dependencies
|
||||
@@ -53,7 +53,7 @@ python3 projects/setup/check_setup.py
|
||||
**Start working** (run this every time):
|
||||
```bash
|
||||
cd TinyTorch
|
||||
source tinytorch-env/bin/activate # Always activate first!
|
||||
source .venv/bin/activate # Always activate first!
|
||||
```
|
||||
|
||||
**Check status**:
|
||||
@@ -72,7 +72,7 @@ python3 -m pytest projects/setup/test_setup.py -v
|
||||
|
||||
**"ModuleNotFoundError"**: You forgot to activate your virtual environment
|
||||
```bash
|
||||
source tinytorch-env/bin/activate
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**"Command not found"**: Make sure you're in the TinyTorch directory
|
||||
@@ -83,7 +83,7 @@ ls # Should see bin/, projects/, requirements.txt
|
||||
|
||||
**Dependencies missing**: Reinstall in the virtual environment
|
||||
```bash
|
||||
source tinytorch-env/bin/activate
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
@@ -45,7 +45,7 @@ def check_virtual_environment():
|
||||
return True
|
||||
else:
|
||||
print("⚠️ Virtual environment: Not detected")
|
||||
print(" Recommendation: Use 'source tinytorch-env/bin/activate'")
|
||||
print(" Recommendation: Use 'source .venv/bin/activate'")
|
||||
print(" (This is strongly recommended for consistency)")
|
||||
return True # Don't fail, just warn
|
||||
|
||||
@@ -45,7 +45,7 @@ def check_python_version():
|
||||
|
||||
def create_virtual_environment():
|
||||
"""Create the TinyTorch virtual environment."""
|
||||
env_path = Path("tinytorch-env")
|
||||
env_path = Path(".venv")
|
||||
|
||||
if env_path.exists():
|
||||
print(f"⚠️ Virtual environment already exists at {env_path}")
|
||||
@@ -58,7 +58,7 @@ def create_virtual_environment():
|
||||
return True
|
||||
|
||||
# Create virtual environment
|
||||
result = run_command([sys.executable, "-m", "venv", "tinytorch-env"])
|
||||
result = run_command([sys.executable, "-m", "venv", ".venv"])
|
||||
if result is None:
|
||||
print("❌ Failed to create virtual environment")
|
||||
return False
|
||||
@@ -69,9 +69,9 @@ def create_virtual_environment():
|
||||
def get_venv_python():
|
||||
"""Get the path to Python in the virtual environment."""
|
||||
if sys.platform == "win32":
|
||||
return Path("tinytorch-env/Scripts/python.exe")
|
||||
return Path(".venv/Scripts/python.exe")
|
||||
else:
|
||||
return Path("tinytorch-env/bin/python")
|
||||
return Path(".venv/bin/python")
|
||||
|
||||
def install_dependencies():
|
||||
"""Install required dependencies in the virtual environment."""
|
||||
@@ -151,9 +151,9 @@ def print_next_steps():
|
||||
print("=" * 60)
|
||||
|
||||
if sys.platform == "win32":
|
||||
activate_cmd = "tinytorch-env\\Scripts\\activate"
|
||||
activate_cmd = ".venv\\Scripts\\activate"
|
||||
else:
|
||||
activate_cmd = "source tinytorch-env/bin/activate"
|
||||
activate_cmd = "source .venv/bin/activate"
|
||||
|
||||
print(f"""
|
||||
Next steps:
|
||||
33
nbdev.yaml
Normal file
33
nbdev.yaml
Normal file
@@ -0,0 +1,33 @@
|
||||
# nbdev configuration for TinyTorch
|
||||
# This file tells nbdev how to compile notebooks into the tinytorch package
|
||||
|
||||
# Package settings
|
||||
lib_name: tinytorch
|
||||
lib_path: tinytorch
|
||||
doc_path: docs
|
||||
nbs_path: modules
|
||||
recursive: true
|
||||
|
||||
# Documentation settings
|
||||
doc_host: https://tinytorch.github.io
|
||||
doc_baseurl: /TinyTorch/
|
||||
git_url: https://github.com/tinytorch/TinyTorch
|
||||
lib_https: https://github.com/tinytorch/TinyTorch/blob/main/
|
||||
baseurl: https://tinytorch.github.io
|
||||
|
||||
# Build settings
|
||||
custom_sidebar: false
|
||||
use_relative_doc_path: true
|
||||
use_github: true
|
||||
github_username: tinytorch
|
||||
github_repo: TinyTorch
|
||||
|
||||
# Notebook settings
|
||||
tst_flags: notest
|
||||
version: 0.1.0
|
||||
title: TinyTorch
|
||||
description: Build ML systems from scratch
|
||||
author: TinyTorch Students
|
||||
author_email: tinytorch@example.com
|
||||
copyright: 2024 TinyTorch
|
||||
license: MIT
|
||||
@@ -25,5 +25,14 @@ black>=23.0.0
|
||||
mypy>=1.0.0
|
||||
flake8>=6.0.0
|
||||
|
||||
# nbdev for notebook-based development
|
||||
nbdev>=2.4.0
|
||||
|
||||
# Jupyter for interactive development
|
||||
jupyter>=1.0.0
|
||||
jupyterlab>=4.0.0
|
||||
ipykernel>=6.0.0
|
||||
ipywidgets>=8.0.0
|
||||
|
||||
# Type checking support
|
||||
typing-extensions>=4.0.0
|
||||
Reference in New Issue
Block a user