mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-21 00:23:30 -05:00
Vijay Janapa Reddi
cb0ae4082f
Run quartodoc build against the refactored package (engine/ extracted from core/, infra->infrastructure): regenerate the API reference so every stub points at the new paths. Adds engine.*/infrastructure.* stubs, removes the 41 stale core.solver.* / core.engine.* / core.scenarios.* / core.config.* / core.evaluation.* / infra*.qmd orphans, and refreshes the package pages (core now primitives-only) + index. Verified: all 47 documented symbols resolve in the new package; every index link resolves; zero stale mlsysim.core.<engine-mod> / mlsysim.infra references anywhere in docs/. (objects.json inventory is gitignored -- regenerated at build.)
44 lines
1.8 KiB
Plaintext
44 lines
1.8 KiB
Plaintext
# engine.solver.DataModel { #mlsysim.engine.solver.DataModel }
|
|
|
|
```python
|
|
engine.solver.DataModel()
|
|
```
|
|
|
|
Analyzes the 'Data Wall' — the throughput bottleneck between storage and compute.
|
|
|
|
This model simulates the data pipeline constraints, comparing the data demand
|
|
of a workload (e.g., training tokens or high-resolution video frames)
|
|
against the physical bandwidth of the storage hierarchy and IO interconnects.
|
|
|
|
Literature Source:
|
|
1. Janapa Reddi et al. (2025), "Machine Learning Systems," Chapter 4 (Data Engineering).
|
|
2. Beitzel et al. (2024), "The Data Wall: Scaling Laws for Data Ingestion in AI."
|
|
3. Mohan et al. (2022), "Analyzing and Mitigating Data Bottlenecks in Deep Learning Training."
|
|
|
|
## Methods
|
|
|
|
| Name | Description |
|
|
| --- | --- |
|
|
| [solve](#mlsysim.engine.solver.DataModel.solve) | Solves for data pipeline feasibility. |
|
|
|
|
### solve { #mlsysim.engine.solver.DataModel.solve }
|
|
|
|
```python
|
|
engine.solver.DataModel.solve(workload_data_rate, hardware)
|
|
```
|
|
|
|
Solves for data pipeline feasibility.
|
|
|
|
#### Parameters {.doc-section .doc-section-parameters}
|
|
|
|
| Name | Type | Description | Default |
|
|
|--------------------|--------------|-----------------------------------------------------------|------------|
|
|
| workload_data_rate | Quantity | The required data ingestion rate (e.g., TB/hour or GB/s). | _required_ |
|
|
| hardware | HardwareNode | The hardware node with storage and interconnect specs. | _required_ |
|
|
|
|
#### Returns {.doc-section .doc-section-returns}
|
|
|
|
| Name | Type | Description |
|
|
|--------|------------------|---------------------------------------------------------------|
|
|
| | Dict\[str, Any\] | Pipeline metrics including utilization and stall probability. |
|