mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-06-03 03:26:06 -05:00
Merge transformer-training into dev
Complete Milestone 05 - 2017 Transformer implementation Major Features: - TinyTalks interactive dashboard with rich CLI - Complete gradient flow fixes (13 tests passing) - Multiple training examples (5-min, 10-min, levels 1-2) - Milestone celebration card (perceptron style) - Comprehensive documentation Gradient Flow Fixes: - Fixed reshape, matmul (3D), embedding, sqrt, mean, sub, div, GELU - All transformer components now fully differentiable - Hybrid attention approach for educational clarity + gradients Training Results: - 10-min training: 96.6% loss improvement, 62.5% accuracy - 5-min training: 97.8% loss improvement, 66.7% accuracy - Working chatbot with coherent responses Files Added: - tinytalks_dashboard.py (main demo) - tinytalks_chatbot.py, tinytalks_dataset.py - level1_memorization.py, level2_patterns.py - Comprehensive docs and test suites Ready for student use 2>&1
This commit is contained in:
@@ -15,6 +15,12 @@
|
||||
#| default_exp text.tokenization
|
||||
#| export
|
||||
|
||||
import numpy as np
|
||||
from typing import List, Dict, Tuple, Optional, Set
|
||||
import json
|
||||
import re
|
||||
from collections import defaultdict, Counter
|
||||
|
||||
# %% [markdown]
|
||||
"""
|
||||
# Module 10: Tokenization - Converting Text to Numbers
|
||||
|
||||
Reference in New Issue
Block a user