mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-21 00:23:30 -05:00
Vijay Janapa Reddi
cb0ae4082f
Run quartodoc build against the refactored package (engine/ extracted from core/, infra->infrastructure): regenerate the API reference so every stub points at the new paths. Adds engine.*/infrastructure.* stubs, removes the 41 stale core.solver.* / core.engine.* / core.scenarios.* / core.config.* / core.evaluation.* / infra*.qmd orphans, and refreshes the package pages (core now primitives-only) + index. Verified: all 47 documented symbols resolve in the new package; every index link resolves; zero stale mlsysim.core.<engine-mod> / mlsysim.infra references anywhere in docs/. (objects.json inventory is gitignored -- regenerated at build.)
59 lines
2.6 KiB
Plaintext
59 lines
2.6 KiB
Plaintext
# engine.solver.EfficiencyModel { #mlsysim.engine.solver.EfficiencyModel }
|
|
|
|
```python
|
|
engine.solver.EfficiencyModel()
|
|
```
|
|
|
|
Models the gap between peak and achieved FLOPS (Wall 3: Software Efficiency).
|
|
|
|
This model quantifies the software efficiency of a workload — the fraction
|
|
of peak hardware FLOPS that the software stack actually converts into useful
|
|
computation. It decomposes Model FLOPs Utilization (MFU) by workload type,
|
|
accounting for kernel fusion efficiency, SM occupancy, and memory access
|
|
patterns.
|
|
|
|
Literature Source:
|
|
1. Chowdhery et al. (2022), "PaLM: Scaling Language Modeling with Pathways."
|
|
(First systematic MFU reporting for large Transformers.)
|
|
2. Dao et al. (2022), "FlashAttention: Fast and Memory-Efficient Exact
|
|
Attention with IO-Awareness." (FlashAttention MFU improvement.)
|
|
3. NVIDIA (2023), "Hopper Architecture Tuning Guide." (SM Occupancy model.)
|
|
|
|
## Methods
|
|
|
|
| Name | Description |
|
|
| --- | --- |
|
|
| [solve](#mlsysim.engine.solver.EfficiencyModel.solve) | Estimates achievable MFU and FLOPS for a given workload type. |
|
|
|
|
### solve { #mlsysim.engine.solver.EfficiencyModel.solve }
|
|
|
|
```python
|
|
engine.solver.EfficiencyModel.solve(
|
|
model,
|
|
hardware,
|
|
workload_type='ffn',
|
|
use_flash_attention=False,
|
|
precision='fp16',
|
|
efficiency=0.5,
|
|
)
|
|
```
|
|
|
|
Estimates achievable MFU and FLOPS for a given workload type.
|
|
|
|
#### Parameters {.doc-section .doc-section-parameters}
|
|
|
|
| Name | Type | Description | Default |
|
|
|---------------------|--------------|------------------------------------------------------------------|------------|
|
|
| model | Workload | The model architecture to simulate. | _required_ |
|
|
| hardware | HardwareNode | The target hardware node. | _required_ |
|
|
| workload_type | str | The dominant kernel type ('attention', 'ffn', 'conv'). | `'ffn'` |
|
|
| use_flash_attention | bool | Whether FlashAttention is enabled (only applies to 'attention'). | `False` |
|
|
| precision | str | Numerical precision ('fp16', 'fp32', 'int8', 'int4'). | `'fp16'` |
|
|
| efficiency | float | Base compute efficiency factor (0.0 to 1.0). | `0.5` |
|
|
|
|
#### Returns {.doc-section .doc-section-returns}
|
|
|
|
| Name | Type | Description |
|
|
|--------|------------------|---------------------------------------------------------|
|
|
| | Dict\[str, Any\] | MFU estimate, achievable FLOPS, and overhead breakdown. |
|