Restructure: Separate developer source (src/) from learner notebooks (modules/)

Major directory restructure to support both developer and learner workflows:

Structure Changes:
- NEW: src/ directory for Python source files (version controlled)
  - Files renamed: tensor.py → 01_tensor.py (matches directory naming)
  - All 20 modules moved from modules/ to src/
- CHANGED: modules/ now holds generated notebooks (gitignored)
  - Generated from src/*.py using jupytext
  - Learners work in notebooks, developers work in Python source
- UNCHANGED: tinytorch/ package (still auto-generated from notebooks)

Workflow: src/*.py → modules/*.ipynb → tinytorch/*.py

Command Updates:
- Updated export command to read from src/ and generate to modules/
- Export flow: discovers modules in src/, converts to notebooks in modules/, exports to tinytorch/
- All 20 modules tested and working

Configuration:
- Updated .gitignore to ignore modules/ directory
- Updated README.md with new three-layer architecture explanation
- Updated export.py source mappings and paths

Benefits:
- Clean separation: developers edit Python, learners use notebooks
- Better version control: only Python source committed, notebooks generated
- Flexible learning: can work in notebooks OR Python source
- Maintains backward compatibility: tinytorch package unchanged

Tested:
- Single module export: tito export 01_tensor 
- All modules export: tito export --all 
- Package imports: from tinytorch.core.tensor import Tensor 
- 20/20 modules successfully converted and exported
This commit is contained in:
Vijay Janapa Reddi
2025-11-25 00:02:21 -05:00
parent 68131e6be0
commit d3a126235c
79 changed files with 2233 additions and 11146 deletions

View File

@@ -5,33 +5,35 @@
# ║ This file is AUTOMATICALLY GENERATED from source modules. ║
# ║ ANY CHANGES MADE HERE WILL BE LOST when modules are re-exported! ║
# ║ ║
# ║ ✅ TO EDIT: modules/XX_loader/loader.py ║
# ║ ✅ TO EDIT: src/XX_loader/XX_loader.py
# ║ ✅ TO EXPORT: Run 'tito module complete <module_name>' ║
# ║ ║
# ║ 🛡️ STUDENT PROTECTION: This file contains optimized implementations. ║
# ║ Editing it directly may break module functionality and training. ║
# ║ ║
# ║ 🎓 LEARNING TIP: Work in modules/ - that's where real development
# ║ happens! The tinytorch/ directory is just the compiled output.
# ║ 🎓 LEARNING TIP: Work in src/ (developers) or modules/ (learners)
# ║ The tinytorch/ directory is generated code - edit source files instead!
# ╚═══════════════════════════════════════════════════════════════════════════════╝
# %% auto 0
__all__ = ['Dataset', 'TensorDataset', 'DataLoader']
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 0
# %% ../../modules/08_dataloader/08_dataloader.ipynb 0
#| default_exp data.loader
#| export
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 2
# %% ../../modules/08_dataloader/08_dataloader.ipynb 2
# Essential imports for data loading
import numpy as np
import random
import time
import sys
from typing import Iterator, Tuple, List, Optional, Union
from abc import ABC, abstractmethod
# Import real Tensor class from tinytorch package
from ..core.tensor import Tensor
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 4
# %% ../../modules/08_dataloader/08_dataloader.ipynb 4
class Dataset(ABC):
"""
Abstract base class for all datasets.
@@ -84,7 +86,7 @@ class Dataset(ABC):
pass
### END SOLUTION
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 7
# %% ../../modules/08_dataloader/08_dataloader.ipynb 7
class TensorDataset(Dataset):
"""
Dataset wrapping tensors for supervised learning.
@@ -161,7 +163,7 @@ class TensorDataset(Dataset):
return tuple(Tensor(tensor.data[idx]) for tensor in self.tensors)
### END SOLUTION
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 10
# %% ../../modules/08_dataloader/08_dataloader.ipynb 10
class DataLoader:
"""
Data loader with batching and shuffling support.