github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-06-03 17:30:53 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

fix(milestones): Improve tinystories_gpt.py training output frequency

Browse Source 
- Changed logging from every 20 batches to every 10 batches
- Show first batch immediately for instant feedback
- Display both current loss and running average
- Format: 'Batch X/500 | Loss: X.XXXX | Avg: X.XXXX'

This provides continuous visual feedback during training so users can
see the model learning in real-time.
This commit is contained in:
Vijay Janapa Reddi
2025-10-28 12:21:30 -04:00
parent c88da0b031
commit 8338733af2
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
milestones/05_2017_transformer/tinystories_gpt.py
View File

@@ -354,9 +354,10 @@ def train_tinystories_gpt(model, train_loader, dataset, epochs=5, learning_rate=
epoch_loss += loss_value
batch_count += 1
# Progress
if (batch_idx + 1) % 20 == 0:
console.print(f" Batch {batch_idx+1}: Loss = [cyan]{loss_value:.4f}[/cyan]")
# Progress - show output frequently so user sees continuous training
if batch_idx == 0 or (batch_idx + 1) % 10 == 0 or (batch_idx + 1) % 50 == 0:
avg_loss = epoch_loss / batch_count
console.print(f" Batch {batch_idx+1}/500 | Loss: {loss_value:.4f} | Avg: {avg_loss:.4f}")
# Epoch summary
avg_loss = epoch_loss / max(1, batch_count)