# Progress & Data Management
Track Your Journey
Understanding progress tracking, data management, and reset commands
**Purpose**: Learn how TinyTorch tracks your progress, where your data lives, and how to manage it effectively.
## Your Learning Journey: Two Tracking Systems
TinyTorch uses a clean, simple approach to track your ML systems engineering journey:
```{mermaid}
:align: center
:caption: "**Progress Tracking Flow.** Build modules, export to package, unlock historical milestones, and track achievements through two parallel systems."
graph LR
A[Build Modules] --> B[Complete 01-20]
B --> C[Export to Package]
C --> D[Unlock Milestones]
D --> E[Achieve 1958-2018]
E --> F[Track Progress]
style A fill:#e3f2fd
style B fill:#fffbeb
style C fill:#f0fdf4
style D fill:#fef3c7
style E fill:#f3e5f5
style F fill:#e8eaf6
```
### The Two Systems
Module Progress
What you BUILD (01-20)
- Tensor, Autograd, Optimizers
- Layers, Training, DataLoader
- Convolutions, Transformers
- Your complete ML framework
Milestone Achievements
What you ACHIEVE (01-06)
- Perceptron (1958)
- XOR Crisis (1969)
- MLP Revival (1986)
- CNN Revolution (1998)
- Transformer Era (2017)
- MLPerf (2018)
**Simple relationship**:
- Complete modules → Unlock milestones → Achieve historical ML recreations
- Build capabilities → Validate with history → Track achievements
## Where Your Data Lives
All your progress is stored in the `.tito/` folder:
```
TinyTorch/
├── .tito/ ← Your progress data
│ ├── config.json ← User preferences
│ ├── progress.json ← Module completion (01-20)
│ ├── milestones.json ← Milestone achievements (01-06)
│ └── backups/ ← Automatic safety backups
│ ├── 01_tensor_YYYYMMDD_HHMMSS.py
│ ├── 02_activations_YYYYMMDD_HHMMSS.py
│ └── ...
├── modules/ ← Where you edit
├── tinytorch/ ← Where code exports
└── ...
```
### Understanding Each File
**`config.json`** - User Preferences
```json
{
"logo_theme": "standard"
}
```
- UI preferences
- Display settings
- Personal configuration
**`progress.json`** - Module Completion
```text
{
"version": "1.0",
"completed_modules": [1, 2, 3, 4, 5, 6, 7],
"completion_dates": {
"1": "2025-11-16T10:00:00",
"2": "2025-11-16T11:00:00",
...
}
}
```
- Tracks which modules (01-20) you've completed
- Records when you completed each
- Updated by `tito module complete XX`
**`milestones.json`** - Milestone Achievements
```json
{
"version": "1.0",
"completed_milestones": ["03"],
"completion_dates": {
"03": "2025-11-16T15:00:00"
}
}
```
- Tracks which milestones (01-06) you've achieved
- Records when you achieved each
- Updated by `tito milestone run XX`
**`backups/`** - Module Backups
- Automatic backups before operations
- Timestamped copies of your implementations
- Safety net for module development
- Format: `XX_name_YYYYMMDD_HHMMSS.py`
## Progress Views
### Module Progress: `tito module status`
```bash
tito module status
```
**Shows your module completion progress**:
```
╭─────────────── TinyTorch Progress ────────────────╮
│ │
│ Modules Completed: 7/20 (35%) │
│ Milestones Achieved: 1/6 (17%) │
│ Last Activity: Module 07 (2 hours ago) │
│ │
│ Next Steps: │
│ • Complete modules 08-09 to unlock Milestone 04 │
│ │
╰──────────────────────────────────────────────────────╯
Module Progress:
01 Tensor
02 Activations
03 Layers
04 Losses
05 DataLoader
06 Autograd
07 Optimizers
08 Training
09 Convolutions
10 Tokenization
...
Milestone Achievements:
03 - MLP Revival (1986)
04 - CNN Revolution (1998) [Ready after modules 08-09]
05 - Transformer Era (2017)
06 - MLPerf (2018)
```
**Use this to**:
- Check module completion
- See next recommended steps
- Track your learning journey
### Milestone Achievements: `tito milestone status`
```bash
tito milestone status
```
**Shows your milestone achievements**:
- Which historical recreations you've completed
- Which milestones are unlocked
- What prerequisites remain
## Data Management Commands
### Reset a Module
**Need to start a module over?** The reset command lets you reset a specific module cleanly.
#### Reset a Specific Module
```bash
tito module reset XX
```
**What this does**:
- Resets the specified module to its clean state
- Creates a backup of your current implementation
- Keeps other modules untouched
- Asks for confirmation before proceeding
**Example**:
```bash
tito module reset 03
```
**Example output**:
```
️ Warning: This will reset Module 03 (Layers)
This will:
• Backup current implementation
• Reset module to clean state
• Clear module completion status
Your code will be backed up to .tito/backups/
Continue? [y/N]: y
Creating backup at .tito/backups/03_layers_20251116_143000.py
Resetting module to clean state
Reset Complete!
You're ready to start fresh on Module 03.
Run: tito module start 03
```
#### Automatic Backups
Before any reset, TinyTorch automatically:
- Creates timestamped backup of your implementation
- Stores it in `.tito/backups/`
- Format: `XX_name_YYYYMMDD_HHMMSS.py`
- Allows manual restore if needed
## Data Safety & Recovery
### Automatic Backups
TinyTorch automatically backs up your work:
**When backups happen**:
1. **Before module start**: Backs up existing work
2. **Before reset**: Creates full `.tito/` backup
3. **Before module reset**: Saves current implementation
**Where backups go**:
```
.tito/backups/
├── 01_tensor_20251116_100000.py
├── 01_tensor_20251116_143000.py
├── 03_layers_20251115_180000.py
└── ...
```
**How to use backups**:
```bash
# Backups are timestamped - find the one you need
ls -la .tito/backups/
# Manually restore if needed
cp .tito/backups/03_layers_20251115_180000.py modules/03_layers/layers_dev.py
```
### What If .tito/ Is Deleted?
**No problem!** TinyTorch recovers gracefully:
```bash
# If .tito/ is deleted, next command recreates it
tito system health
```
**What happens**:
1. TinyTorch detects missing `.tito/` folder
2. Creates fresh folder structure
3. Initializes empty progress tracking
4. Your code in `modules/` and `tinytorch/` is safe
5. You can continue from where you left off
**Important**: Your actual code (source in `src/`, notebooks in `modules/`, package in `tinytorch/`) is separate from progress tracking (in `.tito/`). Deleting `.tito/` only resets progress tracking, not your implementations.
## Data Health Checks
### Verify Data Integrity
```bash
tito system health
```
**Now includes data health checks**:
```
╭────────── TinyTorch System Check ──────────╮
│ │
│ Environment setup │
│ Dependencies installed │
│ TinyTorch in development mode │
│ Data files intact │
│ .tito/progress.json valid │
│ .tito/milestones.json valid │
│ .tito/config.json valid │
│ Backups directory exists │
│ │
╰───────────────────────────────────────────────╯
All systems ready!
```
**If data is corrupted**:
```
Data files corrupted
.tito/progress.json is malformed
Fix by removing and recreating:
rm .tito/progress.json
tito system health # Recreates the file
Or restore from backup:
cp .tito_backup_YYYYMMDD/.tito/progress.json .tito/
```
## Best Practices
### Regular Progress Checks
**Good habits**:
1. **Check status regularly**:
```bash
tito module status
tito milestone status
```
See where you are, what's next
2. **Verify environment before work**:
```bash
tito system health
```
Catch issues early
3. **Let automatic backups work**:
- Don't disable them
- They're your safety net
- Cleanup happens automatically
4. **Reset modules when needed**:
```bash
tito module reset XX # Reset a specific module
```
5. **Version control for code**:
```bash
git commit -m "Completed Module 05: DataLoader"
```
`.tito/` is gitignored - use git for code versions
## Understanding What Gets Tracked
### Modules (Build Progress)
**Tracked when**: You run `tito module complete XX`
**What's recorded**:
- Module number (1-20)
- Completion timestamp
- Test results (passed/failed)
**Visible in**:
- `tito module status`
- `.tito/progress.json`
### Milestones (Achievement Progress)
**Tracked when**: You run `tito milestone run XX`
**What's recorded**:
- Milestone ID (01-06)
- Achievement timestamp
- Number of attempts (if multiple runs)
**Visible in**:
- `tito milestone status`
- `.tito/milestones.json`
### What's NOT Tracked
**TinyTorch does NOT track**:
- Your actual code implementations (source in `src/`, notebooks in `modules/`, package in `tinytorch/`)
- How long you spent on each module
- How many times you edited files
- Your test scores or grades
- Personal information
- Usage analytics
**Why**: TinyTorch is a local, offline learning tool. Your privacy is protected. All data stays on your machine.
## Common Data Scenarios
### Scenario 1: "I want to reset a specific module"
```bash
# Reset module 03 to start fresh
tito module reset 03
# Start working on it again
tito module start 03
```
**Result**: Module 03 reset to clean state, backup created, other modules untouched
### Scenario 2: "I want to re-run a milestone"
```bash
# Just run the milestone again
tito milestone run 03
```
**Result**: Milestone re-runs using your current implementations
### Scenario 3: "I accidentally deleted .tito/"
```bash
# Just run any tito command
tito system health
# OR
# If you have a backup
cp -r .tito_backup_YYYYMMDD/ .tito/
```
**Result**: `.tito/` folder recreated, either fresh or from backup
### Scenario 4: "I want to share my progress with a friend"
```bash
# Copy your progress folder
cp -r .tito/ ~/Desktop/my-tinytorch-progress/
```
**Result**: Friend can see your progress by copying to their `.tito/` folder
## FAQ
### Q: Will resetting delete my code?
**A**: No! Reset commands only affect progress tracking in `.tito/`. Your source code in `src/`, notebooks in `modules/`, and exported code in `tinytorch/` are never touched.
### Q: Can I manually edit progress.json?
**A**: Yes, but not recommended. Use `tito` commands instead. Manual edits might break validation.
### Q: What if I want to re-export a module?
**A**: Just run `tito module complete XX` again. It will re-run tests and re-export. Progress tracking remains unchanged.
### Q: How do I see my completion dates?
**A**: Run `tito module status` for a formatted view, or check `.tito/progress.json` and `.tito/milestones.json` directly.
### Q: Can I delete backups?
**A**: Yes, backups in `.tito/backups/` can be deleted manually. They're safety nets, not requirements.
### Q: Is my data shared anywhere?
**A**: No. TinyTorch is completely local. No data leaves your machine. No tracking, no analytics, no cloud sync.
## Next Steps
*Your progress is tracked, your data is safe, and your journey is yours. TinyTorch keeps track of what you've built and achieved - you focus on learning ML systems engineering.*