Progress & Data Management

Progress & Data Management#

Track Your Journey

Understanding progress tracking, data management, and reset commands

Purpose: Learn how TinyTorch tracks your progress, where your data lives, and how to manage it effectively.

Your Learning Journey: Two Tracking Systems#

TinyTorch uses a clean, simple approach to track your ML systems engineering journey:

        graph LR
    A[Build Modules] --> B[Complete 01-20]
    B --> C[Export to Package]
    C --> D[Unlock Milestones]
    D --> E[Achieve 1957-2018]
    E --> F[Track Progress]

    style A fill:#e3f2fd
    style B fill:#fffbeb
    style C fill:#f0fdf4
    style D fill:#fef3c7
    style E fill:#f3e5f5
    style F fill:#e8eaf6

The Two Systems#

📦 Module Progress

What you BUILD (01-20)

  • Tensor, Autograd, Optimizers
  • Layers, Training, DataLoader
  • Convolutions, Transformers
  • Your complete ML framework

🏆 Milestone Achievements

What you ACHIEVE (01-06)

  • Perceptron (1957)
  • MLP Revival (1986)
  • CNN Revolution (1998)
  • AlexNet Era (2012)
  • Transformer Era (2017)
  • MLPerf (2018)

Simple relationship:

  • Complete modules → Unlock milestones → Achieve historical ML recreations

  • Build capabilities → Validate with history → Track achievements

Where Your Data Lives#

All your progress is stored in the .tito/ folder:

TinyTorch/
├── .tito/                    ← Your progress data
│   ├── config.json           ← User preferences
│   ├── progress.json         ← Module completion (01-20)
│   ├── milestones.json       ← Milestone achievements (01-06)
│   └── backups/              ← Automatic safety backups
│       ├── 01_tensor_YYYYMMDD_HHMMSS.py
│       ├── 02_activations_YYYYMMDD_HHMMSS.py
│       └── ...
├── modules/                  ← Where you edit
├── tinytorch/                ← Where code exports
└── ...

Understanding Each File#

config.json - User Preferences

{
  "logo_theme": "standard"
}

  • UI preferences

  • Display settings

  • Personal configuration

progress.json - Module Completion

{
  "version": "1.0",
  "completed_modules": [1, 2, 3, 4, 5, 6, 7],
  "completion_dates": {
    "1": "2025-11-16T10:00:00",
    "2": "2025-11-16T11:00:00",
    ...
  }
}

  • Tracks which modules (01-20) you’ve completed

  • Records when you completed each

  • Updated by tito module complete XX

milestones.json - Milestone Achievements

{
  "version": "1.0",
  "completed_milestones": ["03"],
  "completion_dates": {
    "03": "2025-11-16T15:00:00"
  }
}

  • Tracks which milestones (01-06) you’ve achieved

  • Records when you achieved each

  • Updated by tito milestone run XX

backups/ - Module Backups

  • Automatic backups before operations

  • Timestamped copies of your implementations

  • Safety net for module development

  • Format: XX_name_YYYYMMDD_HHMMSS.py

Unified Progress View#

See Everything: tito status#

tito status

Shows your complete learning journey in one view:

╭─────────────── 📊 TinyTorch Progress ────────────────╮
│                                                      │
│  📦 Modules Completed: 7/20 (35%)                    │
│  🏆 Milestones Achieved: 1/6 (17%)                   │
│  📍 Last Activity: Module 07 (2 hours ago)           │
│                                                      │
│  Next Steps:                                         │
│    • Complete modules 08-09 to unlock Milestone 04   │
│                                                      │
╰──────────────────────────────────────────────────────╯

Module Progress:
  ✅ 01 Tensor
  ✅ 02 Activations
  ✅ 03 Layers
  ✅ 04 Losses
  ✅ 05 Autograd
  ✅ 06 Optimizers
  ✅ 07 Training
  🔒 08 DataLoader
  🔒 09 Convolutions
  🔒 10 Normalization
  ...

Milestone Achievements:
  ✅ 03 - MLP Revival (1986)
  🎯 04 - CNN Revolution (1998) [Ready after modules 08-09]
  🔒 05 - Transformer Era (2017)
  🔒 06 - MLPerf (2018)

Use this to:

  • Check overall progress

  • See next recommended steps

  • Understand milestone prerequisites

  • Track your learning journey

Data Management Commands#

Reset Your Progress#

Starting fresh? Reset commands let you start over cleanly.

Reset Everything#

tito reset all

What this does:

  • Clears all module completion

  • Clears all milestone achievements

  • Resets configuration to defaults

  • Keeps your code in modules/ safe

  • Asks for confirmation before proceeding

Example output:

⚠️  Warning: This will reset ALL progress

This will clear:
  • Module completion (7 modules)
  • Milestone achievements (1 milestone)
  • Configuration settings

Your code in modules/ will NOT be deleted.

Continue? [y/N]: y

✅ Creating backup at .tito_backup_20251116_143000/
✅ Clearing module progress
✅ Clearing milestone achievements
✅ Resetting configuration

🔄 Reset Complete!

You're ready to start fresh.
Run: tito module start 01

Reset Module Progress Only#

tito reset progress

What this does:

  • Clears module completion tracking only

  • Keeps milestone achievements

  • Keeps configuration

  • Useful for re-doing module workflow

Reset Milestone Achievements Only#

tito reset milestones

What this does:

  • Clears milestone achievements only

  • Keeps module completion

  • Keeps configuration

  • Useful for re-running historical recreations

Safety: Automatic Backups#

# Create backup before reset
tito reset all --backup

What this does:

  • Creates timestamped backup: .tito_backup_YYYYMMDD_HHMMSS/

  • Contains complete copy of .tito/ folder

  • Allows manual restore if needed

  • Automatic before any destructive operation

Data Safety & Recovery#

Automatic Backups#

TinyTorch automatically backs up your work:

When backups happen:

  1. Before module start: Backs up existing work

  2. Before reset: Creates full .tito/ backup

  3. Before module reset: Saves current implementation

Where backups go:

.tito/backups/
├── 01_tensor_20251116_100000.py
├── 01_tensor_20251116_143000.py
├── 03_layers_20251115_180000.py
└── ...

How to use backups:

# Backups are timestamped - find the one you need
ls -la .tito/backups/

# Manually restore if needed
cp .tito/backups/03_layers_20251115_180000.py modules/03_layers/layers_dev.py

What If .tito/ Is Deleted?#

No problem! TinyTorch recovers gracefully:

# If .tito/ is deleted, next command recreates it
tito system health

What happens:

  1. TinyTorch detects missing .tito/ folder

  2. Creates fresh folder structure

  3. Initializes empty progress tracking

  4. Your code in modules/ and tinytorch/ is safe

  5. You can continue from where you left off

Important: Your actual code (source in src/, notebooks in modules/, package in tinytorch/) is separate from progress tracking (in .tito/). Deleting .tito/ only resets progress tracking, not your implementations.

Data Health Checks#

Verify Data Integrity#

tito system health

Now includes data health checks:

╭────────── 🔍 TinyTorch System Check ──────────╮
│                                               │
│  ✅ Environment setup                         │
│  ✅ Dependencies installed                    │
│  ✅ TinyTorch in development mode             │
│  ✅ Data files intact                         │
│    ✓ .tito/progress.json valid               │
│    ✓ .tito/milestones.json valid             │
│    ✓ .tito/config.json valid                 │
│  ✅ Backups directory exists                  │
│                                               │
╰───────────────────────────────────────────────╯

All systems ready! 🚀

If data is corrupted:

❌ Data files corrupted
  ✗ .tito/progress.json is malformed

Fix:
  tito reset progress

Or restore from backup:
  cp .tito_backup_YYYYMMDD/.tito/progress.json .tito/

Best Practices#

Regular Progress Checks#

Good habits:

  1. Check status regularly:

    tito status

    See where you are, what’s next

  2. Verify environment before work:

    tito system health

    Catch issues early

  3. Let automatic backups work:

    • Don’t disable them

    • They’re your safety net

    • Cleanup happens automatically

  4. Backup before experiments:

    tito reset all --backup  # If trying something risky

  5. Version control for code:

    git commit -m "Completed Module 05: Autograd"

    .tito/ is gitignored - use git for code versions

Understanding What Gets Tracked#

Modules (Build Progress)#

Tracked when: You run tito module complete XX

What’s recorded:

  • Module number (1-20)

  • Completion timestamp

  • Test results (passed/failed)

Visible in:

  • tito module status

  • tito status

  • .tito/progress.json

Milestones (Achievement Progress)#

Tracked when: You run tito milestone run XX

What’s recorded:

  • Milestone ID (01-06)

  • Achievement timestamp

  • Number of attempts (if multiple runs)

Visible in:

  • tito milestone status

  • tito status

  • .tito/milestones.json

What’s NOT Tracked#

TinyTorch does NOT track:

  • Your actual code implementations (source in src/, notebooks in modules/, package in tinytorch/)

  • How long you spent on each module

  • How many times you edited files

  • Your test scores or grades

  • Personal information

  • Usage analytics

Why: TinyTorch is a local, offline learning tool. Your privacy is protected. All data stays on your machine.

Common Data Scenarios#

Scenario 1: “I want to start completely fresh”#

# Create backup first (recommended)
tito reset all --backup

# Or just reset
tito reset all

# Start from Module 01
tito module start 01

Result: Clean slate, progress tracking reset, your code untouched

Scenario 2: “I want to re-run milestones but keep module progress”#

# Reset only milestone achievements
tito reset milestones

# Re-run historical recreations
tito milestone run 03
tito milestone run 04

Result: Module completion preserved, milestone achievements reset

Scenario 3: “I accidentally deleted .tito/”#

# Just run any tito command
tito system health

# OR

# If you have a backup
cp -r .tito_backup_YYYYMMDD/ .tito/

Result: .tito/ folder recreated, either fresh or from backup

Scenario 4: “I want to share my progress with a friend”#

# Create backup with timestamp
tito reset all --backup  # (then cancel when prompted)

# Share the backup folder
cp -r .tito_backup_YYYYMMDD/ ~/Desktop/my-tinytorch-progress/

Result: Friend can see your progress by copying to their .tito/ folder

FAQ#

Q: Will resetting delete my code?#

A: No! Reset commands only affect progress tracking in .tito/. Your source code in src/, notebooks in modules/, and exported code in tinytorch/ are never touched.

Q: Can I manually edit progress.json?#

A: Yes, but not recommended. Use tito commands instead. Manual edits might break validation.

Q: What if I want to re-export a module?#

A: Just run tito module complete XX again. It will re-run tests and re-export. Progress tracking remains unchanged.

Q: How do I see my completion dates?#

A: Run tito status for a formatted view, or check .tito/progress.json and .tito/milestones.json directly.

Q: Can I delete backups?#

A: Yes, backups in .tito/backups/ can be deleted manually. They’re safety nets, not requirements.

Q: Is my data shared anywhere?#

A: No. TinyTorch is completely local. No data leaves your machine. No tracking, no analytics, no cloud sync.

Next Steps#

Keep Building!

Now that you understand data management, focus on what matters: building ML systems

Module Workflow → Milestone System →

Your progress is tracked, your data is safe, and your journey is yours. TinyTorch keeps track of what you’ve built and achieved - you focus on learning ML systems engineering.

Contents