Note

📌 Early release (2026)

MLSys·im shipped with the 2026 MLSysBook refresh. The analytical modeling framework, APIs, and lab integrations are actively iterated as we harden the package and teaching workflows.

Feedback — GitHub issues or pull requests.

MLSys·im: The Modeling Platform

A first-principles analytical modeling framework for ML systems.
Designed for education and early design-space reasoning before empirical benchmarking.

pip install mlsysim · Documentation · Getting Started · Issues

Start with a Python question

Use Engine.solve() for a generic single-device Roofline profile. Use from mlsysim.solvers import ServingModel for KV-aware language-model serving, or another specialized solver for training memory, distributed performance, energy, or cost. A Scenario binds workload and system choices to a reusable scorecard; it does not automatically select the most specialized solver.

The Python API guide (source) provides runnable examples, result meanings, and a map from public imports to implementation. Specifications, equations, solvers, and presentation remain separate; all textbook volumes use this same package.

🏗 The 5-Layer Analytical Stack

mlsysim implements a "Progressive Lowering" architecture, separating high-level workloads from the physical infrastructure that executes them.

Layer	Domain	Key Components
Layer A	Workload Representation `mlsysim.models`	FLOPs, parameters, and intensity. e.g., `Models.Language.Llama3_70B`, `Models.Vision.ResNet50`
Layer B	Hardware Registry `mlsysim.hardware`	Concrete specs for real-world silicon. e.g., `Hardware.Cloud.H100`, Jetson, ESP32
Layer C	Infrastructure `mlsysim.infrastructure`	Grid profiles and datacenter sustainability. e.g., PUE, Carbon Intensity, WUE
Layer D	Systems & Topology `mlsysim.systems`	Nodes, racks, fleet configurations, and network fabrics. e.g., `Systems.Racks.DGX_H100_4Node`, `Systems.Clusters.Frontier_8K`
Layer E	Execution & Resolvers `mlsysim.solvers`	The 3-tier math engine: Models, Solvers, and Optimizers (Design space search).
Layer F	Embodied & Physical AI `mlsysim.embodied`, `mlsysim.physics.robotics`	Cyber-physical platforms (AMRs, Autonomous Vehicles, Manipulators) and closed-form physics (stopping envelopes, impact forces, stiction, bus utilization, safety leases).

Scenarios.* sits above the stack as the runnable composition layer: it pairs a Models.* workload with a Hardware.*, Systems.*, or Embodied.* target and local constraints such as latency, power, or stopping envelopes. Non-executable real-world anchors live in ReferenceStats.*. There are no compatibility aliases between these namespaces: new and existing code should use the canonical path directly.

Quick Usage: Automation-Friendly CLI

mlsysim is a first-principles analytical modeling framework for ML systems. It provides a terminal UI for humans and strict JSON output for scripts, CI/CD pipelines, and validation tooling.

Accuracy note: Results depend on each model's assumptions and supplied efficiencies. A physical lower bound, a calibrated estimate, and a measured runtime are different kinds of evidence. Check sensitivity and feasibility before ranking designs, and use matched measurements for deployment decisions. See the calibration report for the scope of existing comparisons.

1. Explore the Registry (The Zoo)

Discover built-in hardware, models, and infrastructure without reading source code: mlsysim zoo hardware
mlsysim zoo models

2. Quick Evaluation (CLI Flags)

Evaluate the physics of a workload on a specific hardware node instantly: mlsysim eval Llama3_8B H100 --batch-size 32

3. Full-Stack Analytical Run (Infrastructure as Code)

Define your entire cluster and SLA constraints in a declarative mlsys.yaml file:

# example_cluster.yaml
version: "1.0"
name: "Llama-3 70B training audit"
workload:
  name: "Llama3_70B"
  batch_size: 4096
hardware:
  name: "H100"
  accelerators: 64
ops:
  region: "Quebec"
  duration_days: 14.0
constraints:
  assert:
    - metric: "performance.latency"
      max: 50.0

Then compile and evaluate the 3-lens scorecard (Feasibility, Performance, Macro): mlsysim eval example_cluster.yaml

4. CI/CD & Automation

Every command supports strict, schema-validated JSON output. If an assert constraint is violated, the CLI returns a semantic Exit Code 3.

# Export the JSON Schema for your IDE or validation tooling
mlsysim schema > schema.json

# Run an evaluation in a CI pipeline
tco=$(mlsysim --output json eval example_cluster.yaml | jq .m_tco_usd)

5. Design Space Search (Optimizers)

Use the Tier 3 Engineering Engine to automatically find the optimal configuration: mlsysim optimize parallelism example_cluster.yaml
mlsysim optimize placement example_cluster.yaml --carbon-tax 150

Stability & Integrity

Because this core powers a printed textbook, we enforce strict Invariant Verification. Registry constants are traceable to primary sources where available, and dimensional integrity is enforced via pint.

Release-Facing Modeling Workflows

TrainingMemoryModel: weights, gradients, optimizer state, activations, and communication buffers per accelerator.
ServingCapacityModel: first-pass replica sizing from QPS, target P99 latency, generated length, batching capacity, and queueing.
MoERoutingModel: MoE active-parameter and expert-parallel traffic sensitivity under hot-expert imbalance.

What This Tool Does Not Model

MLSys·im is an analytical modeling framework for first-pass reasoning, not a production serving or orchestration system. The 22 walls model physical and economic constraints that bound ML system performance. Several critical production concerns are deliberately out of scope:

Concern	Why it matters	Where to learn more
Data drift / distribution shift	The #1 cause of production ML failures — model accuracy degrades silently as input distributions change	Sculley et al. (2015), "Hidden Technical Debt in ML Systems"
Model versioning & rollback	Production requires running multiple versions, A/B testing, and safe rollback	Huyen (2022), Designing Machine Learning Systems
Monitoring & observability	You cannot manage what you cannot measure — prediction distributions, latency percentiles, error rates	Google SRE Book (2016); Huyen (2022)
Feature store freshness	Stale features silently degrade real-time models (recommendations, fraud detection)	Uber Michelangelo (2017)
Software bugs & misconfigurations	Most outages are caused by software, not hardware	Barroso et al. (2018)
Human factors	Team velocity, on-call burden, and organizational alignment often dominate outcomes	Brooks (1975), The Mythical Man-Month

Passing all 22 walls is necessary but not sufficient for a successful production deployment.

Students using this tool should understand that infrastructure physics (what mlsysim models) is one dimension of a multi-dimensional engineering challenge.

How to Cite

If you use mlsysim in your research or teaching, please cite:

@software{mlsysim2026,
  author       = {Janapa Reddi, Vijay},
  title        = {{MLSys$\cdot$im}: First-Principles Infrastructure Modeling for Machine Learning Systems},
  year         = {2026},
  url          = {https://mlsysbook.ai/mlsysim},
  version      = {0.3.1},
  institution  = {Harvard University}
}

Installation

MLSys·im is designed to be highly modular. Install only what you need:

# Core physics engine only (fastest, smallest footprint)
pip install mlsysim

# The CLI and YAML support are included in the base package.
# The [cli] extra is retained as a backward-compatible no-op.
pip install "mlsysim[cli]"

# Install plotting dependencies
pip install "mlsysim[viz]"

Python API Usage

The framework is just as useful inside a Python script or Jupyter Notebook. The SystemEvaluator provides a clean, unified entry point for full-stack analysis:

import mlsysim
from mlsysim.engine.evaluation import SystemEvaluator

# 1. Define the scenario
model = mlsysim.Models.Language.Llama3_8B
hardware = mlsysim.Hardware.Cloud.H100

# 2. Run the evaluation
evaluation = SystemEvaluator.evaluate(
    scenario_name="Llama-3 8B on H100",
    model_obj=model,
    hardware_obj=hardware,
    batch_size=32,
    precision="fp16",
    efficiency=0.45
)

# 3. View the formatted scorecard
print(evaluation.scorecard())

Efficiency Parameter Guide

The efficiency parameter (0.0–1.0) captures the gap between peak hardware performance and what your software stack actually achieves. Use these guidelines:

Scenario	Efficiency	Rationale
Training (Megatron-LM, large Transformer)	0.40–0.55	Well-optimized GEMM + FlashAttention
Training (PyTorch eager, small model)	0.08–0.15	Kernel launch overhead dominates
Inference decode, batch=1	0.01–0.05	Memory-bound; compute nearly idle
Inference decode, batch=32+	0.15–0.35	Batch amortizes weight loading
Inference prefill, long context	0.30–0.50	Compute-bound GEMM + attention
TinyML (TFLite Micro on ESP32)	0.05–0.15	Interpreter overhead, no tensor cores

Contributors

Thanks to these wonderful people for helping improve MLSys·im!

Legend: Bug Hunter · Code Contributor · Documentation Contributor · Design Contributor · Idea Contributor · Code Reviewer · Test Engineer · Tool Builder

_{Vijay Janapa Reddi} 🧑‍💻 🎨 ✍️ 🧠 maintenance	_Rocky 🪲 🧑‍💻 🎨 ✍️	_{Farhan Asghar} 🪲 🧑‍💻 🎨 ✍️	_{Sankur Kundu} 🧑‍💻 ✍️ 🧠 🧪	_Aadityansha 🪲 🧑‍💻 🧪	_{Shivtej Gaikwad} 🪲 🧑‍💻 🧪	_{Tong Lin} 🧑‍💻 ✍️ 🧪
_{lynnyulinlin-debug} 🪲 🧑‍💻 🧠	_{Nishitha-Rajesh} 🪲 🧑‍💻 ✍️	_{Peter Koellner} 🪲 ✍️	_{Zeljko Hrcek} 🧑‍💻	_nyxst4ck ✍️

Recognize a contributor: Comment on any issue or PR:

@all-contributors please add @username for code, doc, ideas, or bug

License

Code: Apache License 2.0 — free for commercial and non-commercial use, with patent grant and attribution requirement.

Documentation and textbook prose: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC-BY-NC-SA-4.0) — the tutorials and prose on mlsysbook.ai/mlsysim are part of the Machine Learning Systems textbook and carry its license.

The two licenses are intentionally separate: the Python package is permissively licensed so engineers and researchers can use it anywhere (including commercially), while the textbook prose retains its non-commercial protection to prevent republication as a derivative textbook.

Development and model contracts

See CONTRIBUTING.md for an isolated installation, the package layout, validation commands, and numerical-change review. The executable calibration report separates calibrated regressions from external validation. RELEASE.md describes the release gates and packaging checks.

README.md Unescape Escape