mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-11 19:13:51 -05:00
Restructure: Separate developer source (src/) from learner notebooks (modules/)
Major directory restructure to support both developer and learner workflows: Structure Changes: - NEW: src/ directory for Python source files (version controlled) - Files renamed: tensor.py → 01_tensor.py (matches directory naming) - All 20 modules moved from modules/ to src/ - CHANGED: modules/ now holds generated notebooks (gitignored) - Generated from src/*.py using jupytext - Learners work in notebooks, developers work in Python source - UNCHANGED: tinytorch/ package (still auto-generated from notebooks) Workflow: src/*.py → modules/*.ipynb → tinytorch/*.py Command Updates: - Updated export command to read from src/ and generate to modules/ - Export flow: discovers modules in src/, converts to notebooks in modules/, exports to tinytorch/ - All 20 modules tested and working Configuration: - Updated .gitignore to ignore modules/ directory - Updated README.md with new three-layer architecture explanation - Updated export.py source mappings and paths Benefits: - Clean separation: developers edit Python, learners use notebooks - Better version control: only Python source committed, notebooks generated - Flexible learning: can work in notebooks OR Python source - Maintains backward compatibility: tinytorch package unchanged Tested: - Single module export: tito export 01_tensor ✅ - All modules export: tito export --all ✅ - Package imports: from tinytorch.core.tensor import Tensor ✅ - 20/20 modules successfully converted and exported
This commit is contained in:
18
tinytorch/data/loader.py
generated
18
tinytorch/data/loader.py
generated
@@ -5,33 +5,35 @@
|
||||
# ║ This file is AUTOMATICALLY GENERATED from source modules. ║
|
||||
# ║ ANY CHANGES MADE HERE WILL BE LOST when modules are re-exported! ║
|
||||
# ║ ║
|
||||
# ║ ✅ TO EDIT: modules/XX_loader/loader.py ║
|
||||
# ║ ✅ TO EDIT: src/XX_loader/XX_loader.py ║
|
||||
# ║ ✅ TO EXPORT: Run 'tito module complete <module_name>' ║
|
||||
# ║ ║
|
||||
# ║ 🛡️ STUDENT PROTECTION: This file contains optimized implementations. ║
|
||||
# ║ Editing it directly may break module functionality and training. ║
|
||||
# ║ ║
|
||||
# ║ 🎓 LEARNING TIP: Work in modules/ - that's where real development ║
|
||||
# ║ happens! The tinytorch/ directory is just the compiled output. ║
|
||||
# ║ 🎓 LEARNING TIP: Work in src/ (developers) or modules/ (learners) ║
|
||||
# ║ The tinytorch/ directory is generated code - edit source files instead! ║
|
||||
# ╚═══════════════════════════════════════════════════════════════════════════════╝
|
||||
# %% auto 0
|
||||
__all__ = ['Dataset', 'TensorDataset', 'DataLoader']
|
||||
|
||||
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 0
|
||||
# %% ../../modules/08_dataloader/08_dataloader.ipynb 0
|
||||
#| default_exp data.loader
|
||||
#| export
|
||||
|
||||
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 2
|
||||
# %% ../../modules/08_dataloader/08_dataloader.ipynb 2
|
||||
# Essential imports for data loading
|
||||
import numpy as np
|
||||
import random
|
||||
import time
|
||||
import sys
|
||||
from typing import Iterator, Tuple, List, Optional, Union
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
# Import real Tensor class from tinytorch package
|
||||
from ..core.tensor import Tensor
|
||||
|
||||
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 4
|
||||
# %% ../../modules/08_dataloader/08_dataloader.ipynb 4
|
||||
class Dataset(ABC):
|
||||
"""
|
||||
Abstract base class for all datasets.
|
||||
@@ -84,7 +86,7 @@ class Dataset(ABC):
|
||||
pass
|
||||
### END SOLUTION
|
||||
|
||||
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 7
|
||||
# %% ../../modules/08_dataloader/08_dataloader.ipynb 7
|
||||
class TensorDataset(Dataset):
|
||||
"""
|
||||
Dataset wrapping tensors for supervised learning.
|
||||
@@ -161,7 +163,7 @@ class TensorDataset(Dataset):
|
||||
return tuple(Tensor(tensor.data[idx]) for tensor in self.tensors)
|
||||
### END SOLUTION
|
||||
|
||||
# %% ../../modules/source/08_dataloader/dataloader_dev.ipynb 10
|
||||
# %% ../../modules/08_dataloader/08_dataloader.ipynb 10
|
||||
class DataLoader:
|
||||
"""
|
||||
Data loader with batching and shuffling support.
|
||||
|
||||
Reference in New Issue
Block a user