mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-21 00:23:30 -05:00
Vijay Janapa Reddi
cb0ae4082f
Run quartodoc build against the refactored package (engine/ extracted from core/, infra->infrastructure): regenerate the API reference so every stub points at the new paths. Adds engine.*/infrastructure.* stubs, removes the 41 stale core.solver.* / core.engine.* / core.scenarios.* / core.config.* / core.evaluation.* / infra*.qmd orphans, and refreshes the package pages (core now primitives-only) + index. Verified: all 47 documented symbols resolve in the new package; every index link resolves; zero stale mlsysim.core.<engine-mod> / mlsysim.infra references anywhere in docs/. (objects.json inventory is gitignored -- regenerated at build.)
39 lines
994 B
Plaintext
39 lines
994 B
Plaintext
# engine.solver.ParallelismOptimizer { #mlsysim.engine.solver.ParallelismOptimizer }
|
|
|
|
```python
|
|
engine.solver.ParallelismOptimizer()
|
|
```
|
|
|
|
Searches for the optimal 3D/4D parallelism split (DP, TP, PP, EP).
|
|
|
|
Given a model architecture and a cluster size, this optimizer sweeps
|
|
the integer design space of parallelism degrees to find the
|
|
configuration that maximizes Model FLOPs Utilization (MFU).
|
|
|
|
Literature Source:
|
|
1. Narayanan et al. (2021), "Efficient Large-Scale Language Model
|
|
Training on GPU Clusters Using Megatron-LM."
|
|
|
|
## Methods
|
|
|
|
| Name | Description |
|
|
| --- | --- |
|
|
| [solve](#mlsysim.engine.solver.ParallelismOptimizer.solve) | Searches for the optimal parallelism split. |
|
|
|
|
### solve { #mlsysim.engine.solver.ParallelismOptimizer.solve }
|
|
|
|
```python
|
|
engine.solver.ParallelismOptimizer.solve(
|
|
model,
|
|
fleet,
|
|
batch_size,
|
|
precision='fp16',
|
|
efficiency=0.5,
|
|
max_tp=None,
|
|
max_pp=None,
|
|
overlap_comm=True,
|
|
)
|
|
```
|
|
|
|
Searches for the optimal parallelism split.
|