mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-21 00:23:30 -05:00
Vijay Janapa Reddi
cb0ae4082f
Run quartodoc build against the refactored package (engine/ extracted from core/, infra->infrastructure): regenerate the API reference so every stub points at the new paths. Adds engine.*/infrastructure.* stubs, removes the 41 stale core.solver.* / core.engine.* / core.scenarios.* / core.config.* / core.evaluation.* / infra*.qmd orphans, and refreshes the package pages (core now primitives-only) + index. Verified: all 47 documented symbols resolve in the new package; every index link resolves; zero stale mlsysim.core.<engine-mod> / mlsysim.infra references anywhere in docs/. (objects.json inventory is gitignored -- regenerated at build.)
44 lines
1.8 KiB
Plaintext
44 lines
1.8 KiB
Plaintext
# engine.solver.CheckpointModel { #mlsysim.engine.solver.CheckpointModel }
|
|
|
|
```python
|
|
engine.solver.CheckpointModel()
|
|
```
|
|
|
|
Analyzes the storage constraints and I/O burst penalties of saving model states.
|
|
|
|
Training massive models requires saving hundreds of gigabytes (Weights +
|
|
Optimizer States) to persistent storage. This model calculates the time
|
|
spent blocked on I/O, subtracting from the cluster's Model FLOPs Utilization.
|
|
|
|
Literature Source:
|
|
1. Eisenman et al. (2022), "Check-N-Run: A Checkpointing System for
|
|
Training Large Language Models."
|
|
|
|
## Methods
|
|
|
|
| Name | Description |
|
|
| --- | --- |
|
|
| [solve](#mlsysim.engine.solver.CheckpointModel.solve) | Solves for checkpoint size, write time, and resulting MFU penalty. |
|
|
|
|
### solve { #mlsysim.engine.solver.CheckpointModel.solve }
|
|
|
|
```python
|
|
engine.solver.CheckpointModel.solve(
|
|
model,
|
|
hardware,
|
|
optimizer='adam',
|
|
checkpoint_interval_hours=4.0,
|
|
n_writers=1,
|
|
filesystem_limit_gbs=500.0,
|
|
)
|
|
```
|
|
|
|
Solves for checkpoint size, write time, and resulting MFU penalty.
|
|
|
|
#### Parameters {.doc-section .doc-section-parameters}
|
|
|
|
| Name | Type | Description | Default |
|
|
|----------------------|--------|-------------------------------------------------------------------------------------------------------------------------------|-----------|
|
|
| n_writers | int | Number of parallel checkpoint writers (default 1). Distributed checkpointing (e.g., FSDP) shards the write across workers. | `1` |
|
|
| filesystem_limit_gbs | float | Maximum aggregate filesystem write bandwidth in GB/s (default 500). Prevents over-optimistic scaling when n_writers is large. | `500.0` |
|