mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-21 00:23:30 -05:00
Vijay Janapa Reddi
cb0ae4082f
Run quartodoc build against the refactored package (engine/ extracted from core/, infra->infrastructure): regenerate the API reference so every stub points at the new paths. Adds engine.*/infrastructure.* stubs, removes the 41 stale core.solver.* / core.engine.* / core.scenarios.* / core.config.* / core.evaluation.* / infra*.qmd orphans, and refreshes the package pages (core now primitives-only) + index. Verified: all 47 documented symbols resolve in the new package; every index link resolves; zero stale mlsysim.core.<engine-mod> / mlsysim.infra references anywhere in docs/. (objects.json inventory is gitignored -- regenerated at build.)
46 lines
1.9 KiB
Plaintext
46 lines
1.9 KiB
Plaintext
# engine.solver.WeightStreamingModel { #mlsysim.engine.solver.WeightStreamingModel }
|
|
|
|
```python
|
|
engine.solver.WeightStreamingModel()
|
|
```
|
|
|
|
Analyzes Wafer-Scale inference (e.g., Cerebras CS-3) using Weight Streaming.
|
|
|
|
Instead of holding weights in HBM and streaming activations (the GPU Memory Wall),
|
|
this architecture holds massive activation batches on-wafer (SRAM) and streams
|
|
the model weights from external MemoryX nodes.
|
|
|
|
The bottleneck shifts from Memory Bandwidth to Injection Interconnect Bandwidth.
|
|
|
|
Literature Source:
|
|
1. Lie et al. (2022), "Cerebras Architecture Deep Dive: First Look Inside
|
|
the Hardware/Software Co-Design for Deep Learning."
|
|
|
|
## Methods
|
|
|
|
| Name | Description |
|
|
| --- | --- |
|
|
| [solve](#mlsysim.engine.solver.WeightStreamingModel.solve) | Simulates Weight Streaming throughput and SRAM feasibility. |
|
|
|
|
### solve { #mlsysim.engine.solver.WeightStreamingModel.solve }
|
|
|
|
```python
|
|
engine.solver.WeightStreamingModel.solve(
|
|
model,
|
|
hardware,
|
|
seq_len,
|
|
batch_size=1,
|
|
precision='fp16',
|
|
efficiency=0.5,
|
|
phase='decode',
|
|
)
|
|
```
|
|
|
|
Simulates Weight Streaming throughput and SRAM feasibility.
|
|
|
|
#### Parameters {.doc-section .doc-section-parameters}
|
|
|
|
| Name | Type | Description | Default |
|
|
|--------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|
|
|
| phase | str | Inference phase: 'prefill' or 'decode' (default 'decode'). - prefill: processes all S tokens in parallel (compute-heavy, O(S^2) attention) - decode: processes one token at a time per request (memory-bound) | `'decode'` |
|