# Machine Learning Systems
*Principles and Practices of Engineering Artificially Intelligent Systems*
English •
中文 •
日本語 •
한국어
[](https://github.com/harvard-edge/cs249r_book/actions/workflows/book-validate-dev.yml)
[](https://github.com/harvard-edge/cs249r_book/actions/workflows/tinytorch-validate-dev.yml)
[](https://github.com/harvard-edge/cs249r_book/blob/dev/LICENSE.md)
[](#-citation--license)
[](https://opencollective.com/mlsysbook)
**[📖 Read Online](https://mlsysbook.ai/book/)** •
**[📘 Volume I](https://mlsysbook.ai/vol1/)** •
**[📙 Volume II](https://mlsysbook.ai/vol2/)** •
**[Tiny🔥Torch](https://mlsysbook.ai/tinytorch/)** •
**[📄 PDF](https://mlsysbook.ai/book/assets/downloads/Machine-Learning-Systems.pdf)** •
**[🌐 Ecosystem](https://mlsysbook.org)**
📚 **Hardcopy edition coming 2026 with MIT Press.**
---
## Mission
**The world is rushing to build AI systems. It is not engineering them.**
That gap is what we mean by AI engineering.
**AI engineering is the discipline of building efficient, reliable, safe, and robust intelligent systems that operate in the real world, not just models in isolation.**
**Our mission:** Establish AI engineering as a foundational discipline, alongside software engineering and computer engineering, by teaching how to design, build, and evaluate end to end intelligent systems. The long term impact of AI will be shaped by engineers who can turn ideas into working, dependable systems.
---
## What’s in this repo
This repository is the open learning stack for AI systems engineering.
It includes the textbook source, TinyTorch, hardware kits, and upcoming co-labs that connect principles to runnable code and real devices.
---
## Start Here
Choose a path based on your goal.
**READ** Start with the [textbook](https://mlsysbook.ai/book/):
- [📘 Volume I: Introduction to Machine Learning Systems](https://mlsysbook.ai/vol1/) covers ML basics, development, optimization, and operations
- [📙 Volume II: Machine Learning Systems at Scale](https://mlsysbook.ai/vol2/) covers distributed systems, production deployment, and responsible AI
**BUILD** Start TinyTorch with the [getting started guide](https://mlsysbook.ai/tinytorch/getting-started.html). Begin with Module 01 and work up from CNNs to transformers and the MLPerf benchmarks.
**DEPLOY** Pick a [hardware kit](https://mlsysbook.ai/kits/) and run the labs on Arduino, Raspberry Pi, and other edge devices.
**CONNECT** Say hello in [Discussions](https://github.com/harvard-edge/cs249r_book/discussions). We will do our best to reply.
---
## The Learning Stack
The learning stack below shows how the textbook connects to hands on work and deployment. Read the textbook, then pick your path:
```
┌───────────────────────────────────────────────────────────────────────────────┐
│ │
│ MACHINE LEARNING SYSTEMS │
│ Read the Textbook │
│ │
│ Theory • Concepts • Best Practices │
│ │
└───────────────────────────────────────┬───────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
▼ ▼ ▼
┌───────────────────────────────────────────────────────────────────────────────┐
│ HANDS-ON ACTIVITIES │
│ (pick one or all) │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ │ │ │ │ │ │
│ │ SOFTWARE │ │ TINYTORCH │ │ HARDWARE │ │
│ │ CO-LABS │ │ FRAMEWORK │ │ LABS │ │
│ │ │ │ │ │ │ │
│ │ EXPLORE │ │ BUILD │ │ DEPLOY │ │
│ │ │ │ │ │ │ │
│ │ Run controlled │ │ Understand │ │ Engineer under │ │
│ │ experiments on │ │ frameworks by │ │ real constraints│ │
│ │ latency, memory,│ │ implementing │ │ memory, power, │ │
│ │ energy, cost │ │ them │ │ timing, safety │ │
│ │ │ │ │ │ │ │
│ │ (coming 2026) │ │ │ │ Arduino, Pi │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ EXPLORE BUILD DEPLOY │
│ │
└───────────────────────────────────────┬───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────────────────────────┐
│ │
│ AI OLYMPICS │
│ Prove Mastery │
│ │
│ Compete across all tracks • University teams • Public leaderboards │
│ │
│ (coming 2026) │
│ │
└───────────────────────────────────────────────────────────────────────────────┘
```
| | Component | What You Do | Link |
|--|-----------|-------------|------|
| **READ** | [📖 Textbook](https://mlsysbook.ai/book/) | Understand ML systems concepts | [book/](book/README.md) |
| | [📘 Volume I](https://mlsysbook.ai/vol1/) | Build, Optimize, Deploy | |
| | [📙 Volume II](https://mlsysbook.ai/vol2/) | Scale, Distribute, Govern | |
| **EXPLORE** | 🔮 Software Co-Labs | Run controlled experiments on latency, memory, energy, cost | *Coming 2026* |
| **BUILD** | [🔥 TinyTorch](https://mlsysbook.ai/tinytorch/) | Understand frameworks by implementing them | [tinytorch/](tinytorch/README.md) |
| **DEPLOY** | [🔧 Hardware Kits](https://mlsysbook.ai/kits/) | Engineer under real constraints: memory, power, timing, safety | [kits/](kits/README.md) |
| **PROVE** | 🏆 AI Olympics | Compete and benchmark across all tracks | *Coming 2026* |
**What each path teaches:**
- **EXPLORE** teaches *why* — Understand tradeoffs. Change batch sizes, precision, model architectures and see how latency, memory, and accuracy shift.
- **BUILD** teaches *how* — Understand internals. Implement autograd, optimizers, and attention from scratch to see how TensorFlow and PyTorch actually work.
- **DEPLOY** teaches *where* — Understand constraints. Face real memory limits, power budgets, and latency requirements on actual hardware.
---
## What You Will Learn
This textbook teaches you to think at the intersection of machine learning and systems engineering. Each chapter bridges algorithmic concepts with the infrastructure that makes them work in practice.
### The ML ↔ Systems Bridge
| ML Concept | Systems Concept | What You Learn |
|------------|-----------------|----------------|
| Model parameters | Memory constraints | How to fit large models on resource-limited devices |
| Inference latency | Hardware acceleration | How GPUs, TPUs, and accelerators execute neural networks |
| Training convergence | Compute efficiency | How mixed-precision and optimization techniques reduce cost |
| Model accuracy | Quantization and pruning | How to compress models while preserving performance |
| Data requirements | Pipeline infrastructure | How to build efficient data loading and preprocessing |
| Model deployment | MLOps practices | How to monitor, version, and update models in production |
| Privacy constraints | On-device learning | How to train and adapt models without sending data to the cloud |
### Book Structure
This textbook is organized into **two volumes** following the Hennessy & Patterson pedagogical model:
| Volume | Title | Focus | Link |
|--------|-------|-------|------|
| **Volume I** | Introduction to Machine Learning Systems | Single-machine ML systems, foundational principles | [mlsysbook.ai/vol1](https://mlsysbook.ai/vol1/) |
| **Volume II** | Machine Learning Systems at Scale | Distributed systems at production scale | [mlsysbook.ai/vol2](https://mlsysbook.ai/vol2/) |
**Volume I** teaches you to *understand* ML systems (like "Computer Organization and Design").
**Volume II** teaches you to *build* ML systems at scale (like "Computer Architecture").
The [full textbook](https://mlsysbook.ai/book/) combines both volumes for comprehensive coverage.
#### Volume I: Introduction to Machine Learning Systems
| Part | Focus | Chapters |
|------|-------|----------|
| **I. Foundations** | Core concepts | Introduction, ML Systems, ML Workflow, Data Engineering |
| **II. Development** | Building blocks | Neural Computation, Architectures, Frameworks, Training |
| **III. Optimization** | Making it fast | Data Selection, Model Compression, HW Acceleration, Benchmarking |
| **IV. Deployment** | Making it work | Model Serving, MLOps, Responsible Engineering |
### Volume II: Machine Learning Systems at Scale
| Part | Focus | Chapters |
|------|-------|----------|
| **V. The Fleet** | Physical infrastructure | Introduction, Compute Infrastructure, Network Fabrics, Data Storage |
| **VI. Distributed ML** | Algorithms of scale | Distributed Training, Communication, Fault Tolerance, Orchestration |
| **VII. Deployment at Scale** | Global serving | Performance Engineering, Inference, Edge Intelligence, Ops at Scale |
| **VIII. The Responsible Fleet** | Safety and governance | Security & Privacy, Robust AI, Sustainable AI, Responsible AI |
---
## What Makes This Different
This is a living textbook. We keep it updated as the field grows, with community input along the way.
AI may feel like it is moving at lightning speed, but the engineering building blocks that make it work do not change as quickly as the headlines. This project is built around those stable foundations.
Think of it like LEGO. New sets arrive all the time, but the bricks themselves stay the same. Once you learn how the bricks fit together, you can build anything. Here, those "AI bricks" are the solid systems principles that make AI work.
Whether you are reading a chapter, running a lab, or sharing feedback, you are helping make these ideas more accessible to the next learner.
### Research to Teaching Loop
We use the same loop for research and teaching: define the system problem, build a reference implementation, benchmark it, then turn it into curriculum and tooling so others can reproduce and extend it.
| Loop Step | Research Artifacts | Teaching Artifacts |
|-----------|-------------------|-------------------|
| **Measure** | Benchmarks, suites, metrics | Benchmarking chapter, assignments |
| **Build** | Reference systems, compilers, runtimes | TinyTorch modules, co-labs |
| **Deploy** | Hardware targets, constraints, reliability | Hardware labs, kits |
---
## Support This Work
We are working toward **1 million learners by 2030** so that AI engineering becomes a shared, teachable discipline, not a collection of isolated practices. Every star, share, and contribution helps move this effort forward.
### Why GitHub Stars Matter
*What gets measured gets improved.*
Each star is a learner, educator, or supporter who believes AI systems should be engineered with rigor and real world constraints in mind.
[](https://github.com/harvard-edge/cs249r_book/stargazers)
[](https://star-history.com/#harvard-edge/cs249r_book&Date)
1 learner → 10 learners → 100 learners → 1,000 learners → **10,000 learners** → 100,000 learners → **1M learners**
Stars are not the goal. They are a signal.
A visible, growing community makes it easier for universities, foundations, and industry partners to adopt this material, donate hardware, and fund workshops. That momentum lowers the barrier for the next institution, the next classroom, and the next cohort of learners.
Support raised through this signal flows into [Open Collective](https://opencollective.com/mlsysbook) and funds concrete outcomes such as TinyML4D workshops, hardware kits for underserved classrooms, and the infrastructure required to keep this resource free and open.
One click can unlock the next classroom, the next contributor, and the next generation of AI engineers.
### Fund the Mission
All contributions go to [Open Collective](https://opencollective.com/mlsysbook), a transparent fund that supports educational outreach.
[](https://opencollective.com/mlsysbook)
---
## Community and Resources
| Resource | Description |
|---|---|
| [📖 **Full Textbook**](https://mlsysbook.ai/book/) | Complete interactive online textbook (both volumes) |
| [📘 **Volume I**](https://mlsysbook.ai/vol1/) | Build, Optimize, Deploy |
| [📙 **Volume II**](https://mlsysbook.ai/vol2/) | Scale, Distribute, Govern |
| [🔥 **TinyTorch**](https://mlsysbook.ai/tinytorch/) | Build ML frameworks from scratch |
| [🔧 **Hardware Kits**](https://mlsysbook.ai/kits/) | Deploy to Arduino, Raspberry Pi, edge devices |
| [🌐 **Ecosystem**](https://mlsysbook.org) | Resources, workshops, and community |
| [💬 **Discussions**](https://github.com/harvard-edge/cs249r_book/discussions) | Questions and ideas |
---
## Contributing
We welcome contributions to the book, TinyTorch, and hardware kits!
| I want to... | Go here |
|--------------|---------|
| Fix a typo or improve a chapter | [book/docs/CONTRIBUTING.md](book/docs/CONTRIBUTING.md) |
| Add a TinyTorch module or fix a bug | [tinytorch/CONTRIBUTING.md](tinytorch/CONTRIBUTING.md) |
| Improve hardware labs | [kits/README.md](kits/README.md) |
| Report an issue | [GitHub Issues](https://github.com/harvard-edge/cs249r_book/issues) |
| Ask a question | [GitHub Discussions](https://github.com/harvard-edge/cs249r_book/discussions) |
---
## Citation & License
### Citation
```bibtex
@inproceedings{reddi2024mlsysbook,
title = {MLSysBook.AI: Principles and Practices of Machine Learning Systems Engineering},
author = {Reddi, Vijay Janapa},
booktitle = {2024 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS)},
pages = {41--42},
year = {2024},
organization = {IEEE},
url = {https://mlsysbook.org}
}
```
### License
This project uses a dual-license structure:
| Component | License | What It Means |
|-----------|---------|---------------|
| **Book content** | [CC BY-NC-ND 4.0](LICENSE.md) | Share freely with attribution; no commercial use; no derivatives |
| **TinyTorch code** | [Apache 2.0](tinytorch/LICENSE) | Use, modify, and distribute freely; includes patent protection |
The textbook content (chapters, figures, explanations) is educational material that should circulate with attribution and without commercial exploitation. The software framework is a tool designed to be easy for anyone to use, modify, or integrate into their own projects.
---
## Contributors
Thanks goes to these wonderful people who have contributed to making this resource better for everyone!
**Legend:** 🪲 Bug Hunter · 🧑💻 Code Contributor · ✍️ Doc Wizard · 🎨 Design Artist · 🧠 Idea Spark · 🔎 Code Reviewer · 🧪 Test Tinkerer · 🛠️ Tool Builder
### 📖 Textbook Contributors
---
### 🔥 TinyTorch Contributors
---
### 🛠️ Hardware Kits Contributors
---
### 🧪 Labs Contributors
---
**[⭐ Star us on GitHub](https://github.com/harvard-edge/cs249r_book#support-this-work) • [✉️ Subscribe](https://buttondown.email/mlsysbook) • [💬 Join discussions](https://github.com/harvard-edge/cs249r_book/discussions) • [🌐 Visit mlsysbook.ai](https://mlsysbook.ai/)**
**Made with ❤️ for AI engineers**
*in the making, around the world* 🌎
