diff --git a/mlsysim/docs/api-stability.md b/mlsysim/docs/api-stability.md index 7dfcd3fd4b..7ee5b29217 100644 --- a/mlsysim/docs/api-stability.md +++ b/mlsysim/docs/api-stability.md @@ -88,7 +88,7 @@ Flat aliases at the registry root (for example bare `H100` or `ResNet50` leaf na The CLI still resolves short names (`mlsysim eval Llama3_8B H100`) for convenience. Solvers not listed in `mlsysim.__init__` (for example `CompressionModel`, `MoERoutingModel`) -import from `mlsysim.engine.solver`. Workload types import from `mlsysim.models.types`. +import from `mlsysim.solvers`. Workload types import from `mlsysim.models.types`. ### Scenario Registry @@ -99,8 +99,8 @@ from mlsysim import ReferenceStats, Scenarios `Scenarios.*` is the executable scenario registry: each entry composes an existing `Models.*` workload, a `Hardware.*` or `Systems.*` target, and scenario-local constraints such as latency or power budget. `ReferenceStats.*` -holds non-executable sourced anchors used by the book, such as mobile power -envelopes, Waymo data-rate ranges, and TinyML case-study measurements. +holds non-executable sourced anchors, such as mobile power envelopes, Waymo +data-rate ranges, and TinyML case-study measurements. There are no compatibility aliases between these namespaces. Use `Scenarios.SmartDoorbell` for an executable case and diff --git a/mlsysim/docs/api/engine.calibration.qmd b/mlsysim/docs/api/engine.calibration.qmd index a3685eae64..ea5066433d 100644 --- a/mlsysim/docs/api/engine.calibration.qmd +++ b/mlsysim/docs/api/engine.calibration.qmd @@ -4,6 +4,6 @@ Parameters for analytical solvers and the roofline engine. -These values tune ``mlsysim.engine.solver`` models and ``mlsysim.engine.engine.Engine`` -when callers omit explicit arguments. Not appendix-facing — use ``Literature.*``, -``Systems.*``, and ``Infrastructure.*`` for book-cited numbers. +These values tune ``mlsysim.solvers`` models and ``mlsysim.engine.engine.Engine`` +when callers omit explicit arguments. Use ``Literature.*``, ``Systems.*``, and +``Infrastructure.*`` for sourced domain reference values. diff --git a/mlsysim/docs/api/engine.solver.ForwardModel.qmd b/mlsysim/docs/api/engine.solver.ForwardModel.qmd deleted file mode 100644 index 3d9a24a4ae..0000000000 --- a/mlsysim/docs/api/engine.solver.ForwardModel.qmd +++ /dev/null @@ -1,7 +0,0 @@ -# engine.solver.ForwardModel { #mlsysim.engine.solver.ForwardModel } - -```python -engine.solver.ForwardModel() -``` - -Forward-evaluating mechanistic engine (Y = f(X)). diff --git a/mlsysim/docs/api/engine.solver.PlacementOptimizer.qmd b/mlsysim/docs/api/engine.solver.PlacementOptimizer.qmd deleted file mode 100644 index c7fca86b6e..0000000000 --- a/mlsysim/docs/api/engine.solver.PlacementOptimizer.qmd +++ /dev/null @@ -1,27 +0,0 @@ -# engine.solver.PlacementOptimizer { #mlsysim.engine.solver.PlacementOptimizer } - -```python -engine.solver.PlacementOptimizer() -``` - -Finds the optimal datacenter location to minimize TCO and Carbon. - -## Methods - -| Name | Description | -| --- | --- | -| [solve](#mlsysim.engine.solver.PlacementOptimizer.solve) | Determines the optimal data center location to minimize TCO and carbon taxes. | - -### solve { #mlsysim.engine.solver.PlacementOptimizer.solve } - -```python -engine.solver.PlacementOptimizer.solve( - fleet, - duration_days, - regions=['US_Avg', 'Quebec', 'Iowa'], - carbon_tax_per_ton=100.0, - mfu=1.0, -) -``` - -Determines the optimal data center location to minimize TCO and carbon taxes. diff --git a/mlsysim/docs/api/hardware.registry.qmd b/mlsysim/docs/api/hardware.registry.qmd index 723aa720a2..737442bd34 100644 --- a/mlsysim/docs/api/hardware.registry.qmd +++ b/mlsysim/docs/api/hardware.registry.qmd @@ -8,9 +8,9 @@ | Name | Description | | --- | --- | -| [CloudHardware](#mlsysim.hardware.registry.CloudHardware) | Datacenter-scale accelerators (Volume II). | -| [EdgeHardware](#mlsysim.hardware.registry.EdgeHardware) | Robotics and Industrial Edge (Volume I). | -| [MobileHardware](#mlsysim.hardware.registry.MobileHardware) | Smartphone and handheld devices (Volume I). | +| [CloudHardware](#mlsysim.hardware.registry.CloudHardware) | Datacenter-scale accelerators. | +| [EdgeHardware](#mlsysim.hardware.registry.EdgeHardware) | Robotics and industrial edge systems. | +| [MobileHardware](#mlsysim.hardware.registry.MobileHardware) | Smartphone and handheld devices. | | [TinyHardware](#mlsysim.hardware.registry.TinyHardware) | Microcontrollers and sub-watt devices. | | [WorkstationHardware](#mlsysim.hardware.registry.WorkstationHardware) | Personal computing systems used for local development. | @@ -20,7 +20,7 @@ hardware.registry.CloudHardware() ``` -Datacenter-scale accelerators (Volume II). +Datacenter-scale accelerators. ### EdgeHardware { #mlsysim.hardware.registry.EdgeHardware } @@ -28,7 +28,7 @@ Datacenter-scale accelerators (Volume II). hardware.registry.EdgeHardware() ``` -Robotics and Industrial Edge (Volume I). +Robotics and industrial edge systems. ### MobileHardware { #mlsysim.hardware.registry.MobileHardware } @@ -36,7 +36,7 @@ Robotics and Industrial Edge (Volume I). hardware.registry.MobileHardware() ``` -Smartphone and handheld devices (Volume I). +Smartphone and handheld devices. ### TinyHardware { #mlsysim.hardware.registry.TinyHardware } diff --git a/mlsysim/docs/api/index.qmd b/mlsysim/docs/api/index.qmd index 4d1001b3de..5a53cd0b81 100644 --- a/mlsysim/docs/api/index.qmd +++ b/mlsysim/docs/api/index.qmd @@ -16,6 +16,7 @@ Primary objects and resolvers. | [ops](ops.qmd#mlsysim.ops) | | | [core](core.qmd#mlsysim.core) | | | [engine](engine.qmd#mlsysim.engine) | | +| [solvers](solvers.qmd#mlsysim.solvers) | Canonical public solver import surface. | | [core.provenance.Provenance](core.provenance.Provenance.qmd#mlsysim.core.provenance.Provenance) | How we know a numeric value (package audit trail; not BibTeX). | | [core.provenance.ProvenanceKind](core.provenance.ProvenanceKind.qmd#mlsysim.core.provenance.ProvenanceKind) | | | [core.provenance.Sourced](core.provenance.Sourced.qmd#mlsysim.core.provenance.Sourced) | Scalar with mandatory ``Provenance``. Subclasses ``float`` so appendix | @@ -23,33 +24,33 @@ Primary objects and resolvers. | [fmt.fmt](fmt.fmt.qmd#mlsysim.fmt.fmt) | Format a Pint Quantity (or plain number) for narrative text. | | [fmt.fmt_int](fmt.fmt_int.qmd#mlsysim.fmt.fmt_int) | Format a value as an integer for narrative text. | | [physics](physics.qmd#mlsysim.physics) | Canonical physics and accounting formulas for ML systems. | -| [engine.solver.SingleNodeModel](engine.solver.SingleNodeModel.qmd#mlsysim.engine.solver.SingleNodeModel) | Resolves single-node hardware Roofline bounds and feasibility. | -| [engine.solver.NetworkRooflineModel](engine.solver.NetworkRooflineModel.qmd#mlsysim.engine.solver.NetworkRooflineModel) | Analyzes the Distributed Performance Bounds (The Network Wall). | -| [engine.solver.EfficiencyModel](engine.solver.EfficiencyModel.qmd#mlsysim.engine.solver.EfficiencyModel) | Models the gap between peak and achieved FLOPS (Wall 3: Software Efficiency). | -| [engine.solver.ForwardModel](engine.solver.ForwardModel.qmd#mlsysim.engine.solver.ForwardModel) | Forward-evaluating mechanistic engine (Y = f(X)). | -| [engine.solver.ServingModel](engine.solver.ServingModel.qmd#mlsysim.engine.solver.ServingModel) | Analyzes the two-phase LLM serving lifecycle: Pre-fill vs. Decoding. | -| [engine.solver.TrainingMemoryModel](engine.solver.TrainingMemoryModel.qmd#mlsysim.engine.solver.TrainingMemoryModel) | Decomposes per-accelerator training memory into teachable components. | -| [engine.solver.ServingCapacityModel](engine.solver.ServingCapacityModel.qmd#mlsysim.engine.solver.ServingCapacityModel) | Sizes an LLM serving deployment from a QPS and tail-latency target. | -| [engine.solver.ContinuousBatchingModel](engine.solver.ContinuousBatchingModel.qmd#mlsysim.engine.solver.ContinuousBatchingModel) | Analyzes production LLM serving with Continuous Batching and PagedAttention. | -| [engine.solver.WeightStreamingModel](engine.solver.WeightStreamingModel.qmd#mlsysim.engine.solver.WeightStreamingModel) | Analyzes Wafer-Scale inference (e.g., Cerebras CS-3) using Weight Streaming. | -| [engine.solver.TailLatencyModel](engine.solver.TailLatencyModel.qmd#mlsysim.engine.solver.TailLatencyModel) | Analyzes queueing delays and P99 tail latency for deployed inference models. | -| [engine.solver.DataModel](engine.solver.DataModel.qmd#mlsysim.engine.solver.DataModel) | Analyzes the 'Data Wall' — the throughput bottleneck between storage and compute. | -| [engine.solver.TransformationModel](engine.solver.TransformationModel.qmd#mlsysim.engine.solver.TransformationModel) | Quantifies the CPU preprocessing bottleneck (Wall 9: Transformation). | -| [engine.solver.TopologyModel](engine.solver.TopologyModel.qmd#mlsysim.engine.solver.TopologyModel) | Models bisection bandwidth for different network topologies (Wall 10). | -| [engine.solver.ScalingModel](engine.solver.ScalingModel.qmd#mlsysim.engine.solver.ScalingModel) | Analyzes the 'Scaling Physics' of model training (Chinchilla Laws). | -| [engine.solver.InferenceScalingModel](engine.solver.InferenceScalingModel.qmd#mlsysim.engine.solver.InferenceScalingModel) | Models inference-time compute scaling (Wall 12: Reasoning/CoT Cost). | -| [engine.solver.CompressionModel](engine.solver.CompressionModel.qmd#mlsysim.engine.solver.CompressionModel) | Analyzes model compression trade-offs (Accuracy vs. Efficiency). | -| [engine.solver.DistributedModel](engine.solver.DistributedModel.qmd#mlsysim.engine.solver.DistributedModel) | Resolves fleet-wide communication, synchronization, and pipelining constraints. | -| [engine.solver.MoERoutingModel](engine.solver.MoERoutingModel.qmd#mlsysim.engine.solver.MoERoutingModel) | Models first-order MoE routing imbalance and expert-parallel all-to-all cost. | -| [engine.solver.ReliabilityModel](engine.solver.ReliabilityModel.qmd#mlsysim.engine.solver.ReliabilityModel) | Calculates Mean Time Between Failures (MTBF) and optimal checkpointing intervals. | -| [engine.solver.OrchestrationModel](engine.solver.OrchestrationModel.qmd#mlsysim.engine.solver.OrchestrationModel) | Analyzes Cluster Orchestration and Queueing (Little's Law). | -| [engine.solver.EconomicsModel](engine.solver.EconomicsModel.qmd#mlsysim.engine.solver.EconomicsModel) | Calculates Total Cost of Ownership (TCO) including Capex and Opex. | -| [engine.solver.SustainabilityModel](engine.solver.SustainabilityModel.qmd#mlsysim.engine.solver.SustainabilityModel) | Calculates Datacenter-scale Sustainability metrics. | -| [engine.solver.CheckpointModel](engine.solver.CheckpointModel.qmd#mlsysim.engine.solver.CheckpointModel) | Analyzes the storage constraints and I/O burst penalties of saving model states. | -| [engine.solver.ResponsibleEngineeringModel](engine.solver.ResponsibleEngineeringModel.qmd#mlsysim.engine.solver.ResponsibleEngineeringModel) | Models the computational cost of responsible AI practices (Wall 20: Safety). | -| [engine.solver.SensitivitySolver](engine.solver.SensitivitySolver.qmd#mlsysim.engine.solver.SensitivitySolver) | Identifies the binding constraint via numerical sensitivity analysis (Wall 21). | -| [engine.solver.SynthesisSolver](engine.solver.SynthesisSolver.qmd#mlsysim.engine.solver.SynthesisSolver) | Given an SLA, synthesizes the required hardware specs (Wall 22: Inverse Solve). | -| [engine.solver.ParallelismOptimizer](engine.solver.ParallelismOptimizer.qmd#mlsysim.engine.solver.ParallelismOptimizer) | Searches for the optimal 3D/4D parallelism split (DP, TP, PP, EP). | -| [engine.solver.BatchingOptimizer](engine.solver.BatchingOptimizer.qmd#mlsysim.engine.solver.BatchingOptimizer) | Finds the maximum batch size that satisfies a P99 latency SLA. | -| [engine.solver.PlacementOptimizer](engine.solver.PlacementOptimizer.qmd#mlsysim.engine.solver.PlacementOptimizer) | Finds the optimal datacenter location to minimize TCO and Carbon. | +| [solvers.SingleNodeModel](solvers.SingleNodeModel.qmd#mlsysim.solvers.SingleNodeModel) | Resolves single-node hardware Roofline bounds and feasibility. | +| [solvers.NetworkRooflineModel](solvers.NetworkRooflineModel.qmd#mlsysim.solvers.NetworkRooflineModel) | Analyzes the Distributed Performance Bounds (The Network Wall). | +| [solvers.EfficiencyModel](solvers.EfficiencyModel.qmd#mlsysim.solvers.EfficiencyModel) | Models the gap between peak and achieved FLOPS (Wall 3: Software Efficiency). | +| [solvers.ForwardModel](solvers.ForwardModel.qmd#mlsysim.solvers.ForwardModel) | Forward-evaluating mechanistic engine (Y = f(X)). | +| [solvers.ServingModel](solvers.ServingModel.qmd#mlsysim.solvers.ServingModel) | Analyzes the two-phase LLM serving lifecycle: Pre-fill vs. Decoding. | +| [solvers.TrainingMemoryModel](solvers.TrainingMemoryModel.qmd#mlsysim.solvers.TrainingMemoryModel) | Decomposes per-accelerator training memory into teachable components. | +| [solvers.ServingCapacityModel](solvers.ServingCapacityModel.qmd#mlsysim.solvers.ServingCapacityModel) | Sizes an LLM serving deployment from a QPS and tail-latency target. | +| [solvers.ContinuousBatchingModel](solvers.ContinuousBatchingModel.qmd#mlsysim.solvers.ContinuousBatchingModel) | Analyzes production LLM serving with Continuous Batching and PagedAttention. | +| [solvers.WeightStreamingModel](solvers.WeightStreamingModel.qmd#mlsysim.solvers.WeightStreamingModel) | Analyzes Wafer-Scale inference (e.g., Cerebras CS-3) using Weight Streaming. | +| [solvers.TailLatencyModel](solvers.TailLatencyModel.qmd#mlsysim.solvers.TailLatencyModel) | Analyzes queueing delays and P99 tail latency for deployed inference models. | +| [solvers.DataModel](solvers.DataModel.qmd#mlsysim.solvers.DataModel) | Analyzes the 'Data Wall' — the throughput bottleneck between storage and compute. | +| [solvers.TransformationModel](solvers.TransformationModel.qmd#mlsysim.solvers.TransformationModel) | Quantifies the CPU preprocessing bottleneck (Wall 9: Transformation). | +| [solvers.TopologyModel](solvers.TopologyModel.qmd#mlsysim.solvers.TopologyModel) | Models bisection bandwidth for different network topologies (Wall 10). | +| [solvers.ScalingModel](solvers.ScalingModel.qmd#mlsysim.solvers.ScalingModel) | Analyzes the 'Scaling Physics' of model training (Chinchilla Laws). | +| [solvers.InferenceScalingModel](solvers.InferenceScalingModel.qmd#mlsysim.solvers.InferenceScalingModel) | Models inference-time compute scaling (Wall 12: Reasoning/CoT Cost). | +| [solvers.CompressionModel](solvers.CompressionModel.qmd#mlsysim.solvers.CompressionModel) | Analyzes model compression trade-offs (Accuracy vs. Efficiency). | +| [solvers.DistributedModel](solvers.DistributedModel.qmd#mlsysim.solvers.DistributedModel) | Resolves fleet-wide communication, synchronization, and pipelining constraints. | +| [solvers.MoERoutingModel](solvers.MoERoutingModel.qmd#mlsysim.solvers.MoERoutingModel) | Models first-order MoE routing imbalance and expert-parallel all-to-all cost. | +| [solvers.ReliabilityModel](solvers.ReliabilityModel.qmd#mlsysim.solvers.ReliabilityModel) | Calculates Mean Time Between Failures (MTBF) and optimal checkpointing intervals. | +| [solvers.OrchestrationModel](solvers.OrchestrationModel.qmd#mlsysim.solvers.OrchestrationModel) | Analyzes Cluster Orchestration and Queueing (Little's Law). | +| [solvers.EconomicsModel](solvers.EconomicsModel.qmd#mlsysim.solvers.EconomicsModel) | Calculates Total Cost of Ownership (TCO) including Capex and Opex. | +| [solvers.SustainabilityModel](solvers.SustainabilityModel.qmd#mlsysim.solvers.SustainabilityModel) | Calculates Datacenter-scale Sustainability metrics. | +| [solvers.CheckpointModel](solvers.CheckpointModel.qmd#mlsysim.solvers.CheckpointModel) | Analyzes the storage constraints and I/O burst penalties of saving model states. | +| [solvers.ResponsibleEngineeringModel](solvers.ResponsibleEngineeringModel.qmd#mlsysim.solvers.ResponsibleEngineeringModel) | Models the computational cost of responsible AI practices (Wall 20: Safety). | +| [solvers.SensitivitySolver](solvers.SensitivitySolver.qmd#mlsysim.solvers.SensitivitySolver) | Identifies the binding constraint via numerical sensitivity analysis (Wall 21). | +| [solvers.SynthesisSolver](solvers.SynthesisSolver.qmd#mlsysim.solvers.SynthesisSolver) | Given an SLA, synthesizes the required hardware specs (Wall 22: Inverse Solve). | +| [solvers.ParallelismOptimizer](solvers.ParallelismOptimizer.qmd#mlsysim.solvers.ParallelismOptimizer) | Searches for the optimal 3D/4D parallelism split (DP, TP, PP, EP). | +| [solvers.BatchingOptimizer](solvers.BatchingOptimizer.qmd#mlsysim.solvers.BatchingOptimizer) | Finds the maximum batch size that satisfies a P99 latency SLA. | +| [solvers.PlacementOptimizer](solvers.PlacementOptimizer.qmd#mlsysim.solvers.PlacementOptimizer) | Finds the optimal datacenter location to minimize TCO and Carbon. | | [engine.dse.DSE](engine.dse.DSE.qmd#mlsysim.engine.dse.DSE) | Declarative Design Space Exploration (DSE) Engine. | diff --git a/mlsysim/docs/api/engine.solver.BatchingOptimizer.qmd b/mlsysim/docs/api/solvers.BatchingOptimizer.qmd similarity index 60% rename from mlsysim/docs/api/engine.solver.BatchingOptimizer.qmd rename to mlsysim/docs/api/solvers.BatchingOptimizer.qmd index 28920e0a03..71ddc12654 100644 --- a/mlsysim/docs/api/engine.solver.BatchingOptimizer.qmd +++ b/mlsysim/docs/api/solvers.BatchingOptimizer.qmd @@ -1,7 +1,7 @@ -# engine.solver.BatchingOptimizer { #mlsysim.engine.solver.BatchingOptimizer } +# solvers.BatchingOptimizer { #mlsysim.solvers.BatchingOptimizer } ```python -engine.solver.BatchingOptimizer() +solvers.BatchingOptimizer() ``` Finds the maximum batch size that satisfies a P99 latency SLA. @@ -13,12 +13,12 @@ model to find the optimal balance between throughput and tail latency. | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.BatchingOptimizer.solve) | Determines the maximum batch size that satisfies a P99 tail latency SLA. | +| [solve](#mlsysim.solvers.BatchingOptimizer.solve) | Determines the maximum batch size that satisfies a P99 tail latency SLA. | -### solve { #mlsysim.engine.solver.BatchingOptimizer.solve } +### solve { #mlsysim.solvers.BatchingOptimizer.solve } ```python -engine.solver.BatchingOptimizer.solve( +solvers.BatchingOptimizer.solve( model, hardware, seq_len, diff --git a/mlsysim/docs/api/engine.solver.CheckpointModel.qmd b/mlsysim/docs/api/solvers.CheckpointModel.qmd similarity index 81% rename from mlsysim/docs/api/engine.solver.CheckpointModel.qmd rename to mlsysim/docs/api/solvers.CheckpointModel.qmd index 59454ed2af..e6ddc00b86 100644 --- a/mlsysim/docs/api/engine.solver.CheckpointModel.qmd +++ b/mlsysim/docs/api/solvers.CheckpointModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.CheckpointModel { #mlsysim.engine.solver.CheckpointModel } +# solvers.CheckpointModel { #mlsysim.solvers.CheckpointModel } ```python -engine.solver.CheckpointModel() +solvers.CheckpointModel() ``` Analyzes the storage constraints and I/O burst penalties of saving model states. @@ -18,12 +18,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.CheckpointModel.solve) | Solves for checkpoint size, write time, and resulting MFU penalty. | +| [solve](#mlsysim.solvers.CheckpointModel.solve) | Solves for checkpoint size, write time, and resulting MFU penalty. | -### solve { #mlsysim.engine.solver.CheckpointModel.solve } +### solve { #mlsysim.solvers.CheckpointModel.solve } ```python -engine.solver.CheckpointModel.solve( +solvers.CheckpointModel.solve( model, hardware, optimizer='adam', diff --git a/mlsysim/docs/api/engine.solver.CompressionModel.qmd b/mlsysim/docs/api/solvers.CompressionModel.qmd similarity index 92% rename from mlsysim/docs/api/engine.solver.CompressionModel.qmd rename to mlsysim/docs/api/solvers.CompressionModel.qmd index 2909e10a38..86625277cb 100644 --- a/mlsysim/docs/api/engine.solver.CompressionModel.qmd +++ b/mlsysim/docs/api/solvers.CompressionModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.CompressionModel { #mlsysim.engine.solver.CompressionModel } +# solvers.CompressionModel { #mlsysim.solvers.CompressionModel } ```python -engine.solver.CompressionModel() +solvers.CompressionModel() ``` Analyzes model compression trade-offs (Accuracy vs. Efficiency). @@ -21,12 +21,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.CompressionModel.solve) | Solves for compression gains and estimated accuracy impact. | +| [solve](#mlsysim.solvers.CompressionModel.solve) | Solves for compression gains and estimated accuracy impact. | -### solve { #mlsysim.engine.solver.CompressionModel.solve } +### solve { #mlsysim.solvers.CompressionModel.solve } ```python -engine.solver.CompressionModel.solve( +solvers.CompressionModel.solve( model, hardware, method='quantization', diff --git a/mlsysim/docs/api/engine.solver.ContinuousBatchingModel.qmd b/mlsysim/docs/api/solvers.ContinuousBatchingModel.qmd similarity index 67% rename from mlsysim/docs/api/engine.solver.ContinuousBatchingModel.qmd rename to mlsysim/docs/api/solvers.ContinuousBatchingModel.qmd index dc0ec65990..eb6c3f3aa5 100644 --- a/mlsysim/docs/api/engine.solver.ContinuousBatchingModel.qmd +++ b/mlsysim/docs/api/solvers.ContinuousBatchingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ContinuousBatchingModel { #mlsysim.engine.solver.ContinuousBatchingModel } +# solvers.ContinuousBatchingModel { #mlsysim.solvers.ContinuousBatchingModel } ```python -engine.solver.ContinuousBatchingModel() +solvers.ContinuousBatchingModel() ``` Analyzes production LLM serving with Continuous Batching and PagedAttention. @@ -20,12 +20,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ContinuousBatchingModel.solve) | Calculates continuous batching throughput and PagedAttention memory. | +| [solve](#mlsysim.solvers.ContinuousBatchingModel.solve) | Calculates continuous batching throughput and PagedAttention memory. | -### solve { #mlsysim.engine.solver.ContinuousBatchingModel.solve } +### solve { #mlsysim.solvers.ContinuousBatchingModel.solve } ```python -engine.solver.ContinuousBatchingModel.solve( +solvers.ContinuousBatchingModel.solve( model, hardware, seq_len, diff --git a/mlsysim/docs/api/engine.solver.DataModel.qmd b/mlsysim/docs/api/solvers.DataModel.qmd similarity index 84% rename from mlsysim/docs/api/engine.solver.DataModel.qmd rename to mlsysim/docs/api/solvers.DataModel.qmd index a84d826a48..ce150307a6 100644 --- a/mlsysim/docs/api/engine.solver.DataModel.qmd +++ b/mlsysim/docs/api/solvers.DataModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.DataModel { #mlsysim.engine.solver.DataModel } +# solvers.DataModel { #mlsysim.solvers.DataModel } ```python -engine.solver.DataModel() +solvers.DataModel() ``` Analyzes the 'Data Wall' — the throughput bottleneck between storage and compute. @@ -19,12 +19,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.DataModel.solve) | Solves for data pipeline feasibility. | +| [solve](#mlsysim.solvers.DataModel.solve) | Solves for data pipeline feasibility. | -### solve { #mlsysim.engine.solver.DataModel.solve } +### solve { #mlsysim.solvers.DataModel.solve } ```python -engine.solver.DataModel.solve(workload_data_rate, hardware) +solvers.DataModel.solve(workload_data_rate, hardware) ``` Solves for data pipeline feasibility. diff --git a/mlsysim/docs/api/engine.solver.DistributedModel.qmd b/mlsysim/docs/api/solvers.DistributedModel.qmd similarity index 94% rename from mlsysim/docs/api/engine.solver.DistributedModel.qmd rename to mlsysim/docs/api/solvers.DistributedModel.qmd index 81190dac07..990e3ae3b6 100644 --- a/mlsysim/docs/api/engine.solver.DistributedModel.qmd +++ b/mlsysim/docs/api/solvers.DistributedModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.DistributedModel { #mlsysim.engine.solver.DistributedModel } +# solvers.DistributedModel { #mlsysim.solvers.DistributedModel } ```python -engine.solver.DistributedModel() +solvers.DistributedModel() ``` Resolves fleet-wide communication, synchronization, and pipelining constraints. @@ -23,12 +23,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.DistributedModel.solve) | Calculates distributed training performance using the 3D/4D Parallelism model. | +| [solve](#mlsysim.solvers.DistributedModel.solve) | Calculates distributed training performance using the 3D/4D Parallelism model. | -### solve { #mlsysim.engine.solver.DistributedModel.solve } +### solve { #mlsysim.solvers.DistributedModel.solve } ```python -engine.solver.DistributedModel.solve( +solvers.DistributedModel.solve( model, fleet, batch_size=1, diff --git a/mlsysim/docs/api/engine.solver.EconomicsModel.qmd b/mlsysim/docs/api/solvers.EconomicsModel.qmd similarity index 86% rename from mlsysim/docs/api/engine.solver.EconomicsModel.qmd rename to mlsysim/docs/api/solvers.EconomicsModel.qmd index 40bf110e30..86ea2b1033 100644 --- a/mlsysim/docs/api/engine.solver.EconomicsModel.qmd +++ b/mlsysim/docs/api/solvers.EconomicsModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.EconomicsModel { #mlsysim.engine.solver.EconomicsModel } +# solvers.EconomicsModel { #mlsysim.solvers.EconomicsModel } ```python -engine.solver.EconomicsModel() +solvers.EconomicsModel() ``` Calculates Total Cost of Ownership (TCO) including Capex and Opex. @@ -19,12 +19,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.EconomicsModel.solve) | Calculates the TCO for a fleet over a specified duration. | +| [solve](#mlsysim.solvers.EconomicsModel.solve) | Calculates the TCO for a fleet over a specified duration. | -### solve { #mlsysim.engine.solver.EconomicsModel.solve } +### solve { #mlsysim.solvers.EconomicsModel.solve } ```python -engine.solver.EconomicsModel.solve( +solvers.EconomicsModel.solve( fleet, duration_days, kwh_price=None, diff --git a/mlsysim/docs/api/engine.solver.EfficiencyModel.qmd b/mlsysim/docs/api/solvers.EfficiencyModel.qmd similarity index 87% rename from mlsysim/docs/api/engine.solver.EfficiencyModel.qmd rename to mlsysim/docs/api/solvers.EfficiencyModel.qmd index d2e88bff14..e47f2b5e9f 100644 --- a/mlsysim/docs/api/engine.solver.EfficiencyModel.qmd +++ b/mlsysim/docs/api/solvers.EfficiencyModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.EfficiencyModel { #mlsysim.engine.solver.EfficiencyModel } +# solvers.EfficiencyModel { #mlsysim.solvers.EfficiencyModel } ```python -engine.solver.EfficiencyModel() +solvers.EfficiencyModel() ``` Models the gap between peak and achieved FLOPS (Wall 3: Software Efficiency). @@ -23,12 +23,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.EfficiencyModel.solve) | Estimates achievable MFU and FLOPS for a given workload type. | +| [solve](#mlsysim.solvers.EfficiencyModel.solve) | Estimates achievable MFU and FLOPS for a given workload type. | -### solve { #mlsysim.engine.solver.EfficiencyModel.solve } +### solve { #mlsysim.solvers.EfficiencyModel.solve } ```python -engine.solver.EfficiencyModel.solve( +solvers.EfficiencyModel.solve( model, hardware, workload_type='ffn', diff --git a/mlsysim/docs/api/solvers.ForwardModel.qmd b/mlsysim/docs/api/solvers.ForwardModel.qmd new file mode 100644 index 0000000000..730c9231c1 --- /dev/null +++ b/mlsysim/docs/api/solvers.ForwardModel.qmd @@ -0,0 +1,7 @@ +# solvers.ForwardModel { #mlsysim.solvers.ForwardModel } + +```python +solvers.ForwardModel() +``` + +Forward-evaluating mechanistic engine (Y = f(X)). diff --git a/mlsysim/docs/api/engine.solver.InferenceScalingModel.qmd b/mlsysim/docs/api/solvers.InferenceScalingModel.qmd similarity index 86% rename from mlsysim/docs/api/engine.solver.InferenceScalingModel.qmd rename to mlsysim/docs/api/solvers.InferenceScalingModel.qmd index 93713c6247..794dbe4441 100644 --- a/mlsysim/docs/api/engine.solver.InferenceScalingModel.qmd +++ b/mlsysim/docs/api/solvers.InferenceScalingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.InferenceScalingModel { #mlsysim.engine.solver.InferenceScalingModel } +# solvers.InferenceScalingModel { #mlsysim.solvers.InferenceScalingModel } ```python -engine.solver.InferenceScalingModel() +solvers.InferenceScalingModel() ``` Models inference-time compute scaling (Wall 12: Reasoning/CoT Cost). @@ -22,12 +22,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.InferenceScalingModel.solve) | Solves for inference-time reasoning cost. | +| [solve](#mlsysim.solvers.InferenceScalingModel.solve) | Solves for inference-time reasoning cost. | -### solve { #mlsysim.engine.solver.InferenceScalingModel.solve } +### solve { #mlsysim.solvers.InferenceScalingModel.solve } ```python -engine.solver.InferenceScalingModel.solve( +solvers.InferenceScalingModel.solve( model, hardware, reasoning_steps=8, diff --git a/mlsysim/docs/api/engine.solver.MoERoutingModel.qmd b/mlsysim/docs/api/solvers.MoERoutingModel.qmd similarity index 65% rename from mlsysim/docs/api/engine.solver.MoERoutingModel.qmd rename to mlsysim/docs/api/solvers.MoERoutingModel.qmd index 1956440273..a49239a3f9 100644 --- a/mlsysim/docs/api/engine.solver.MoERoutingModel.qmd +++ b/mlsysim/docs/api/solvers.MoERoutingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.MoERoutingModel { #mlsysim.engine.solver.MoERoutingModel } +# solvers.MoERoutingModel { #mlsysim.solvers.MoERoutingModel } ```python -engine.solver.MoERoutingModel() +solvers.MoERoutingModel() ``` Models first-order MoE routing imbalance and expert-parallel all-to-all cost. @@ -15,12 +15,12 @@ volume. It does not simulate a router or token dispatcher. | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.MoERoutingModel.solve) | Estimate effective active parameters and optional EP all-to-all latency. | +| [solve](#mlsysim.solvers.MoERoutingModel.solve) | Estimate effective active parameters and optional EP all-to-all latency. | -### solve { #mlsysim.engine.solver.MoERoutingModel.solve } +### solve { #mlsysim.solvers.MoERoutingModel.solve } ```python -engine.solver.MoERoutingModel.solve( +solvers.MoERoutingModel.solve( model, batch_size, seq_len, diff --git a/mlsysim/docs/api/engine.solver.NetworkRooflineModel.qmd b/mlsysim/docs/api/solvers.NetworkRooflineModel.qmd similarity index 68% rename from mlsysim/docs/api/engine.solver.NetworkRooflineModel.qmd rename to mlsysim/docs/api/solvers.NetworkRooflineModel.qmd index 6c241df948..13dd0dc253 100644 --- a/mlsysim/docs/api/engine.solver.NetworkRooflineModel.qmd +++ b/mlsysim/docs/api/solvers.NetworkRooflineModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.NetworkRooflineModel { #mlsysim.engine.solver.NetworkRooflineModel } +# solvers.NetworkRooflineModel { #mlsysim.solvers.NetworkRooflineModel } ```python -engine.solver.NetworkRooflineModel() +solvers.NetworkRooflineModel() ``` Analyzes the Distributed Performance Bounds (The Network Wall). @@ -20,12 +20,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.NetworkRooflineModel.solve) | Solves for the distributed performance bound. | +| [solve](#mlsysim.solvers.NetworkRooflineModel.solve) | Solves for the distributed performance bound. | -### solve { #mlsysim.engine.solver.NetworkRooflineModel.solve } +### solve { #mlsysim.solvers.NetworkRooflineModel.solve } ```python -engine.solver.NetworkRooflineModel.solve( +solvers.NetworkRooflineModel.solve( model, fleet, precision='fp16', diff --git a/mlsysim/docs/api/engine.solver.OrchestrationModel.qmd b/mlsysim/docs/api/solvers.OrchestrationModel.qmd similarity index 85% rename from mlsysim/docs/api/engine.solver.OrchestrationModel.qmd rename to mlsysim/docs/api/solvers.OrchestrationModel.qmd index 768bb427ff..03e33b4a3d 100644 --- a/mlsysim/docs/api/engine.solver.OrchestrationModel.qmd +++ b/mlsysim/docs/api/solvers.OrchestrationModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.OrchestrationModel { #mlsysim.engine.solver.OrchestrationModel } +# solvers.OrchestrationModel { #mlsysim.solvers.OrchestrationModel } ```python -engine.solver.OrchestrationModel() +solvers.OrchestrationModel() ``` Analyzes Cluster Orchestration and Queueing (Little's Law). @@ -24,12 +24,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.OrchestrationModel.solve) | Solves for cluster wait times and utilization. | +| [solve](#mlsysim.solvers.OrchestrationModel.solve) | Solves for cluster wait times and utilization. | -### solve { #mlsysim.engine.solver.OrchestrationModel.solve } +### solve { #mlsysim.solvers.OrchestrationModel.solve } ```python -engine.solver.OrchestrationModel.solve( +solvers.OrchestrationModel.solve( fleet, arrival_rate_jobs_per_day, avg_job_duration_days, diff --git a/mlsysim/docs/api/engine.solver.ParallelismOptimizer.qmd b/mlsysim/docs/api/solvers.ParallelismOptimizer.qmd similarity index 66% rename from mlsysim/docs/api/engine.solver.ParallelismOptimizer.qmd rename to mlsysim/docs/api/solvers.ParallelismOptimizer.qmd index 4a27f82af9..036e25dc78 100644 --- a/mlsysim/docs/api/engine.solver.ParallelismOptimizer.qmd +++ b/mlsysim/docs/api/solvers.ParallelismOptimizer.qmd @@ -1,7 +1,7 @@ -# engine.solver.ParallelismOptimizer { #mlsysim.engine.solver.ParallelismOptimizer } +# solvers.ParallelismOptimizer { #mlsysim.solvers.ParallelismOptimizer } ```python -engine.solver.ParallelismOptimizer() +solvers.ParallelismOptimizer() ``` Searches for the optimal 3D/4D parallelism split (DP, TP, PP, EP). @@ -18,12 +18,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ParallelismOptimizer.solve) | Searches for the optimal parallelism split. | +| [solve](#mlsysim.solvers.ParallelismOptimizer.solve) | Searches for the optimal parallelism split. | -### solve { #mlsysim.engine.solver.ParallelismOptimizer.solve } +### solve { #mlsysim.solvers.ParallelismOptimizer.solve } ```python -engine.solver.ParallelismOptimizer.solve( +solvers.ParallelismOptimizer.solve( model, fleet, batch_size, diff --git a/mlsysim/docs/api/solvers.PlacementOptimizer.qmd b/mlsysim/docs/api/solvers.PlacementOptimizer.qmd new file mode 100644 index 0000000000..8d41d765f0 --- /dev/null +++ b/mlsysim/docs/api/solvers.PlacementOptimizer.qmd @@ -0,0 +1,27 @@ +# solvers.PlacementOptimizer { #mlsysim.solvers.PlacementOptimizer } + +```python +solvers.PlacementOptimizer() +``` + +Finds the optimal datacenter location to minimize TCO and Carbon. + +## Methods + +| Name | Description | +| --- | --- | +| [solve](#mlsysim.solvers.PlacementOptimizer.solve) | Determines the optimal data center location to minimize TCO and carbon taxes. | + +### solve { #mlsysim.solvers.PlacementOptimizer.solve } + +```python +solvers.PlacementOptimizer.solve( + fleet, + duration_days, + regions=['US_Avg', 'Quebec', 'Iowa'], + carbon_tax_per_ton=100.0, + mfu=1.0, +) +``` + +Determines the optimal data center location to minimize TCO and carbon taxes. diff --git a/mlsysim/docs/api/engine.solver.ReliabilityModel.qmd b/mlsysim/docs/api/solvers.ReliabilityModel.qmd similarity index 85% rename from mlsysim/docs/api/engine.solver.ReliabilityModel.qmd rename to mlsysim/docs/api/solvers.ReliabilityModel.qmd index f536ba974b..0633419abe 100644 --- a/mlsysim/docs/api/engine.solver.ReliabilityModel.qmd +++ b/mlsysim/docs/api/solvers.ReliabilityModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ReliabilityModel { #mlsysim.engine.solver.ReliabilityModel } +# solvers.ReliabilityModel { #mlsysim.solvers.ReliabilityModel } ```python -engine.solver.ReliabilityModel() +solvers.ReliabilityModel() ``` Calculates Mean Time Between Failures (MTBF) and optimal checkpointing intervals. @@ -21,12 +21,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ReliabilityModel.solve) | Calculates reliability and checkpointing metrics for a fleet. | +| [solve](#mlsysim.solvers.ReliabilityModel.solve) | Calculates reliability and checkpointing metrics for a fleet. | -### solve { #mlsysim.engine.solver.ReliabilityModel.solve } +### solve { #mlsysim.solvers.ReliabilityModel.solve } ```python -engine.solver.ReliabilityModel.solve( +solvers.ReliabilityModel.solve( fleet, job_duration_hours, checkpoint_time_s=60.0, diff --git a/mlsysim/docs/api/engine.solver.ResponsibleEngineeringModel.qmd b/mlsysim/docs/api/solvers.ResponsibleEngineeringModel.qmd similarity index 60% rename from mlsysim/docs/api/engine.solver.ResponsibleEngineeringModel.qmd rename to mlsysim/docs/api/solvers.ResponsibleEngineeringModel.qmd index ee9cb1fe4b..17d7a90574 100644 --- a/mlsysim/docs/api/engine.solver.ResponsibleEngineeringModel.qmd +++ b/mlsysim/docs/api/solvers.ResponsibleEngineeringModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ResponsibleEngineeringModel { #mlsysim.engine.solver.ResponsibleEngineeringModel } +# solvers.ResponsibleEngineeringModel { #mlsysim.solvers.ResponsibleEngineeringModel } ```python -engine.solver.ResponsibleEngineeringModel() +solvers.ResponsibleEngineeringModel() ``` Models the computational cost of responsible AI practices (Wall 20: Safety). @@ -17,12 +17,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ResponsibleEngineeringModel.solve) | Calculates the overhead of responsible engineering practices. | +| [solve](#mlsysim.solvers.ResponsibleEngineeringModel.solve) | Calculates the overhead of responsible engineering practices. | -### solve { #mlsysim.engine.solver.ResponsibleEngineeringModel.solve } +### solve { #mlsysim.solvers.ResponsibleEngineeringModel.solve } ```python -engine.solver.ResponsibleEngineeringModel.solve( +solvers.ResponsibleEngineeringModel.solve( base_training_time, epsilon=1.0, delta=1e-05, diff --git a/mlsysim/docs/api/engine.solver.ScalingModel.qmd b/mlsysim/docs/api/solvers.ScalingModel.qmd similarity index 81% rename from mlsysim/docs/api/engine.solver.ScalingModel.qmd rename to mlsysim/docs/api/solvers.ScalingModel.qmd index 2ca1467455..cda14a287c 100644 --- a/mlsysim/docs/api/engine.solver.ScalingModel.qmd +++ b/mlsysim/docs/api/solvers.ScalingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ScalingModel { #mlsysim.engine.solver.ScalingModel } +# solvers.ScalingModel { #mlsysim.solvers.ScalingModel } ```python -engine.solver.ScalingModel() +solvers.ScalingModel() ``` Analyzes the 'Scaling Physics' of model training (Chinchilla Laws). @@ -19,12 +19,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ScalingModel.solve) | Solves for compute-optimal model and dataset parameters. | +| [solve](#mlsysim.solvers.ScalingModel.solve) | Solves for compute-optimal model and dataset parameters. | -### solve { #mlsysim.engine.solver.ScalingModel.solve } +### solve { #mlsysim.solvers.ScalingModel.solve } ```python -engine.solver.ScalingModel.solve(compute_budget, target_model_size=None) +solvers.ScalingModel.solve(compute_budget, target_model_size=None) ``` Solves for compute-optimal model and dataset parameters. diff --git a/mlsysim/docs/api/engine.solver.SensitivitySolver.qmd b/mlsysim/docs/api/solvers.SensitivitySolver.qmd similarity index 63% rename from mlsysim/docs/api/engine.solver.SensitivitySolver.qmd rename to mlsysim/docs/api/solvers.SensitivitySolver.qmd index 6283cc20f9..6ad7e62337 100644 --- a/mlsysim/docs/api/engine.solver.SensitivitySolver.qmd +++ b/mlsysim/docs/api/solvers.SensitivitySolver.qmd @@ -1,7 +1,7 @@ -# engine.solver.SensitivitySolver { #mlsysim.engine.solver.SensitivitySolver } +# solvers.SensitivitySolver { #mlsysim.solvers.SensitivitySolver } ```python -engine.solver.SensitivitySolver() +solvers.SensitivitySolver() ``` Identifies the binding constraint via numerical sensitivity analysis (Wall 21). @@ -17,12 +17,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.SensitivitySolver.solve) | Solves for sensitivities and identifies the binding constraint. | +| [solve](#mlsysim.solvers.SensitivitySolver.solve) | Solves for sensitivities and identifies the binding constraint. | -### solve { #mlsysim.engine.solver.SensitivitySolver.solve } +### solve { #mlsysim.solvers.SensitivitySolver.solve } ```python -engine.solver.SensitivitySolver.solve( +solvers.SensitivitySolver.solve( model, hardware, precision='fp16', diff --git a/mlsysim/docs/api/engine.solver.ServingCapacityModel.qmd b/mlsysim/docs/api/solvers.ServingCapacityModel.qmd similarity index 65% rename from mlsysim/docs/api/engine.solver.ServingCapacityModel.qmd rename to mlsysim/docs/api/solvers.ServingCapacityModel.qmd index 550c7d8695..d2464808a3 100644 --- a/mlsysim/docs/api/engine.solver.ServingCapacityModel.qmd +++ b/mlsysim/docs/api/solvers.ServingCapacityModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ServingCapacityModel { #mlsysim.engine.solver.ServingCapacityModel } +# solvers.ServingCapacityModel { #mlsysim.solvers.ServingCapacityModel } ```python -engine.solver.ServingCapacityModel() +solvers.ServingCapacityModel() ``` Sizes an LLM serving deployment from a QPS and tail-latency target. @@ -15,12 +15,12 @@ not a request-level scheduler. | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ServingCapacityModel.solve) | Return the minimum replica count that satisfies the target P99. | +| [solve](#mlsysim.solvers.ServingCapacityModel.solve) | Return the minimum replica count that satisfies the target P99. | -### solve { #mlsysim.engine.solver.ServingCapacityModel.solve } +### solve { #mlsysim.solvers.ServingCapacityModel.solve } ```python -engine.solver.ServingCapacityModel.solve( +solvers.ServingCapacityModel.solve( model, hardware, qps, diff --git a/mlsysim/docs/api/engine.solver.ServingModel.qmd b/mlsysim/docs/api/solvers.ServingModel.qmd similarity index 96% rename from mlsysim/docs/api/engine.solver.ServingModel.qmd rename to mlsysim/docs/api/solvers.ServingModel.qmd index aad5781744..791ac3bafc 100644 --- a/mlsysim/docs/api/engine.solver.ServingModel.qmd +++ b/mlsysim/docs/api/solvers.ServingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.ServingModel { #mlsysim.engine.solver.ServingModel } +# solvers.ServingModel { #mlsysim.solvers.ServingModel } ```python -engine.solver.ServingModel() +solvers.ServingModel() ``` Analyzes the two-phase LLM serving lifecycle: Pre-fill vs. Decoding. @@ -20,12 +20,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.ServingModel.solve) | Solves for LLM serving performance. | +| [solve](#mlsysim.solvers.ServingModel.solve) | Solves for LLM serving performance. | -### solve { #mlsysim.engine.solver.ServingModel.solve } +### solve { #mlsysim.solvers.ServingModel.solve } ```python -engine.solver.ServingModel.solve( +solvers.ServingModel.solve( model, hardware, seq_len, diff --git a/mlsysim/docs/api/engine.solver.SingleNodeModel.qmd b/mlsysim/docs/api/solvers.SingleNodeModel.qmd similarity index 66% rename from mlsysim/docs/api/engine.solver.SingleNodeModel.qmd rename to mlsysim/docs/api/solvers.SingleNodeModel.qmd index dc0c518bf2..74689dc3cb 100644 --- a/mlsysim/docs/api/engine.solver.SingleNodeModel.qmd +++ b/mlsysim/docs/api/solvers.SingleNodeModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.SingleNodeModel { #mlsysim.engine.solver.SingleNodeModel } +# solvers.SingleNodeModel { #mlsysim.solvers.SingleNodeModel } ```python -engine.solver.SingleNodeModel() +solvers.SingleNodeModel() ``` Resolves single-node hardware Roofline bounds and feasibility. @@ -17,12 +17,12 @@ Performance Model for Floating-Point Programs and Multicore Architectures." | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.SingleNodeModel.solve) | Calculates the performance profile for a single hardware node. | +| [solve](#mlsysim.solvers.SingleNodeModel.solve) | Calculates the performance profile for a single hardware node. | -### solve { #mlsysim.engine.solver.SingleNodeModel.solve } +### solve { #mlsysim.solvers.SingleNodeModel.solve } ```python -engine.solver.SingleNodeModel.solve( +solvers.SingleNodeModel.solve( model, hardware, batch_size=1, diff --git a/mlsysim/docs/api/engine.solver.SustainabilityModel.qmd b/mlsysim/docs/api/solvers.SustainabilityModel.qmd similarity index 70% rename from mlsysim/docs/api/engine.solver.SustainabilityModel.qmd rename to mlsysim/docs/api/solvers.SustainabilityModel.qmd index 4f59751286..caecdddf53 100644 --- a/mlsysim/docs/api/engine.solver.SustainabilityModel.qmd +++ b/mlsysim/docs/api/solvers.SustainabilityModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.SustainabilityModel { #mlsysim.engine.solver.SustainabilityModel } +# solvers.SustainabilityModel { #mlsysim.solvers.SustainabilityModel } ```python -engine.solver.SustainabilityModel() +solvers.SustainabilityModel() ``` Calculates Datacenter-scale Sustainability metrics. @@ -22,12 +22,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.SustainabilityModel.solve) | Calculates energy, carbon, and water footprint for a fleet operation. | +| [solve](#mlsysim.solvers.SustainabilityModel.solve) | Calculates energy, carbon, and water footprint for a fleet operation. | -### solve { #mlsysim.engine.solver.SustainabilityModel.solve } +### solve { #mlsysim.solvers.SustainabilityModel.solve } ```python -engine.solver.SustainabilityModel.solve( +solvers.SustainabilityModel.solve( fleet, duration_days, datacenter=None, diff --git a/mlsysim/docs/api/engine.solver.SynthesisSolver.qmd b/mlsysim/docs/api/solvers.SynthesisSolver.qmd similarity index 63% rename from mlsysim/docs/api/engine.solver.SynthesisSolver.qmd rename to mlsysim/docs/api/solvers.SynthesisSolver.qmd index efe8ae4cdd..0122c7339f 100644 --- a/mlsysim/docs/api/engine.solver.SynthesisSolver.qmd +++ b/mlsysim/docs/api/solvers.SynthesisSolver.qmd @@ -1,7 +1,7 @@ -# engine.solver.SynthesisSolver { #mlsysim.engine.solver.SynthesisSolver } +# solvers.SynthesisSolver { #mlsysim.solvers.SynthesisSolver } ```python -engine.solver.SynthesisSolver() +solvers.SynthesisSolver() ``` Given an SLA, synthesizes the required hardware specs (Wall 22: Inverse Solve). @@ -17,12 +17,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.SynthesisSolver.solve) | Synthesizes hardware requirements from an SLA target. | +| [solve](#mlsysim.solvers.SynthesisSolver.solve) | Synthesizes hardware requirements from an SLA target. | -### solve { #mlsysim.engine.solver.SynthesisSolver.solve } +### solve { #mlsysim.solvers.SynthesisSolver.solve } ```python -engine.solver.SynthesisSolver.solve( +solvers.SynthesisSolver.solve( model, target_latency, precision='fp16', diff --git a/mlsysim/docs/api/engine.solver.TailLatencyModel.qmd b/mlsysim/docs/api/solvers.TailLatencyModel.qmd similarity index 86% rename from mlsysim/docs/api/engine.solver.TailLatencyModel.qmd rename to mlsysim/docs/api/solvers.TailLatencyModel.qmd index 8ef9963c9f..6d31d225ab 100644 --- a/mlsysim/docs/api/engine.solver.TailLatencyModel.qmd +++ b/mlsysim/docs/api/solvers.TailLatencyModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.TailLatencyModel { #mlsysim.engine.solver.TailLatencyModel } +# solvers.TailLatencyModel { #mlsysim.solvers.TailLatencyModel } ```python -engine.solver.TailLatencyModel() +solvers.TailLatencyModel() ``` Analyzes queueing delays and P99 tail latency for deployed inference models. @@ -16,12 +16,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.TailLatencyModel.solve) | Solves for P50 and P99 tail latencies under variable load. | +| [solve](#mlsysim.solvers.TailLatencyModel.solve) | Solves for P50 and P99 tail latencies under variable load. | -### solve { #mlsysim.engine.solver.TailLatencyModel.solve } +### solve { #mlsysim.solvers.TailLatencyModel.solve } ```python -engine.solver.TailLatencyModel.solve( +solvers.TailLatencyModel.solve( arrival_rate_qps, service_latency_ms, num_replicas=1, diff --git a/mlsysim/docs/api/engine.solver.TopologyModel.qmd b/mlsysim/docs/api/solvers.TopologyModel.qmd similarity index 83% rename from mlsysim/docs/api/engine.solver.TopologyModel.qmd rename to mlsysim/docs/api/solvers.TopologyModel.qmd index cf309a3bbe..0a136712c9 100644 --- a/mlsysim/docs/api/engine.solver.TopologyModel.qmd +++ b/mlsysim/docs/api/solvers.TopologyModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.TopologyModel { #mlsysim.engine.solver.TopologyModel } +# solvers.TopologyModel { #mlsysim.solvers.TopologyModel } ```python -engine.solver.TopologyModel() +solvers.TopologyModel() ``` Models bisection bandwidth for different network topologies (Wall 10). @@ -23,12 +23,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.TopologyModel.solve) | Solves for effective network bandwidth under a given topology. | +| [solve](#mlsysim.solvers.TopologyModel.solve) | Solves for effective network bandwidth under a given topology. | -### solve { #mlsysim.engine.solver.TopologyModel.solve } +### solve { #mlsysim.solvers.TopologyModel.solve } ```python -engine.solver.TopologyModel.solve(fabric, topology='fat_tree', num_nodes=64) +solvers.TopologyModel.solve(fabric, topology='fat_tree', num_nodes=64) ``` Solves for effective network bandwidth under a given topology. diff --git a/mlsysim/docs/api/engine.solver.TrainingMemoryModel.qmd b/mlsysim/docs/api/solvers.TrainingMemoryModel.qmd similarity index 81% rename from mlsysim/docs/api/engine.solver.TrainingMemoryModel.qmd rename to mlsysim/docs/api/solvers.TrainingMemoryModel.qmd index f580793744..7c1b287c97 100644 --- a/mlsysim/docs/api/engine.solver.TrainingMemoryModel.qmd +++ b/mlsysim/docs/api/solvers.TrainingMemoryModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.TrainingMemoryModel { #mlsysim.engine.solver.TrainingMemoryModel } +# solvers.TrainingMemoryModel { #mlsysim.solvers.TrainingMemoryModel } ```python -engine.solver.TrainingMemoryModel() +solvers.TrainingMemoryModel() ``` Decomposes per-accelerator training memory into teachable components. @@ -21,12 +21,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.TrainingMemoryModel.solve) | Estimate per-accelerator training memory. | +| [solve](#mlsysim.solvers.TrainingMemoryModel.solve) | Estimate per-accelerator training memory. | -### solve { #mlsysim.engine.solver.TrainingMemoryModel.solve } +### solve { #mlsysim.solvers.TrainingMemoryModel.solve } ```python -engine.solver.TrainingMemoryModel.solve( +solvers.TrainingMemoryModel.solve( model, hardware, batch_size, diff --git a/mlsysim/docs/api/engine.solver.TransformationModel.qmd b/mlsysim/docs/api/solvers.TransformationModel.qmd similarity index 85% rename from mlsysim/docs/api/engine.solver.TransformationModel.qmd rename to mlsysim/docs/api/solvers.TransformationModel.qmd index 49128ab495..0fcdec9a0c 100644 --- a/mlsysim/docs/api/engine.solver.TransformationModel.qmd +++ b/mlsysim/docs/api/solvers.TransformationModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.TransformationModel { #mlsysim.engine.solver.TransformationModel } +# solvers.TransformationModel { #mlsysim.solvers.TransformationModel } ```python -engine.solver.TransformationModel() +solvers.TransformationModel() ``` Quantifies the CPU preprocessing bottleneck (Wall 9: Transformation). @@ -22,12 +22,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.TransformationModel.solve) | Solves for CPU preprocessing bottleneck. | +| [solve](#mlsysim.solvers.TransformationModel.solve) | Solves for CPU preprocessing bottleneck. | -### solve { #mlsysim.engine.solver.TransformationModel.solve } +### solve { #mlsysim.solvers.TransformationModel.solve } ```python -engine.solver.TransformationModel.solve( +solvers.TransformationModel.solve( batch_size, sample_size_bytes, cpu_throughput, diff --git a/mlsysim/docs/api/engine.solver.WeightStreamingModel.qmd b/mlsysim/docs/api/solvers.WeightStreamingModel.qmd similarity index 81% rename from mlsysim/docs/api/engine.solver.WeightStreamingModel.qmd rename to mlsysim/docs/api/solvers.WeightStreamingModel.qmd index 0e638d50b5..e1049ea07c 100644 --- a/mlsysim/docs/api/engine.solver.WeightStreamingModel.qmd +++ b/mlsysim/docs/api/solvers.WeightStreamingModel.qmd @@ -1,7 +1,7 @@ -# engine.solver.WeightStreamingModel { #mlsysim.engine.solver.WeightStreamingModel } +# solvers.WeightStreamingModel { #mlsysim.solvers.WeightStreamingModel } ```python -engine.solver.WeightStreamingModel() +solvers.WeightStreamingModel() ``` Analyzes Wafer-Scale inference (e.g., Cerebras CS-3) using Weight Streaming. @@ -20,12 +20,12 @@ Literature Source: | Name | Description | | --- | --- | -| [solve](#mlsysim.engine.solver.WeightStreamingModel.solve) | Simulates Weight Streaming throughput and SRAM feasibility. | +| [solve](#mlsysim.solvers.WeightStreamingModel.solve) | Simulates Weight Streaming throughput and SRAM feasibility. | -### solve { #mlsysim.engine.solver.WeightStreamingModel.solve } +### solve { #mlsysim.solvers.WeightStreamingModel.solve } ```python -engine.solver.WeightStreamingModel.solve( +solvers.WeightStreamingModel.solve( model, hardware, seq_len, diff --git a/mlsysim/docs/api/solvers.qmd b/mlsysim/docs/api/solvers.qmd new file mode 100644 index 0000000000..957fcd14e5 --- /dev/null +++ b/mlsysim/docs/api/solvers.qmd @@ -0,0 +1,7 @@ +# solvers { #mlsysim.solvers } + +Canonical public import surface for MLSysIM solver protocols, analytical +models, solvers, and optimizers. + +Use `from mlsysim.solvers import ServingModel` for solver classes. The +implementation remains split by domain under `mlsysim.engine.solvers`. diff --git a/mlsysim/docs/blog/how-much-memory-llama3.md b/mlsysim/docs/blog/how-much-memory-llama3.md index b377ab59a8..2ca660a90a 100644 --- a/mlsysim/docs/blog/how-much-memory-llama3.md +++ b/mlsysim/docs/blog/how-much-memory-llama3.md @@ -61,7 +61,7 @@ But at 32K context? KV cache is 1.28 GB per request → only **35 requests**. ## The Full Picture ```python -from mlsysim.engine.solver import ServingModel +from mlsysim.solvers import ServingModel result = ServingModel().solve( mlsysim.Models.Language.Llama3_70B, diff --git a/mlsysim/docs/config/_quarto-html.yml b/mlsysim/docs/config/_quarto-html.yml index d4fc48e1c2..ee898c5835 100644 --- a/mlsysim/docs/config/_quarto-html.yml +++ b/mlsysim/docs/config/_quarto-html.yml @@ -211,7 +211,7 @@ website: - text: "Core" href: api/core.qmd - text: "Solvers" - href: api/engine.qmd + href: api/solvers.qmd # ── BLOG ─────────────────────────────────────────── - section: "Blog" @@ -312,33 +312,33 @@ quartodoc: - fmt.fmt - fmt.fmt_int - physics - - engine.solver.SingleNodeModel - - engine.solver.NetworkRooflineModel - - engine.solver.EfficiencyModel - - engine.solver.ForwardModel - - engine.solver.ServingModel - - engine.solver.TrainingMemoryModel - - engine.solver.ServingCapacityModel - - engine.solver.ContinuousBatchingModel - - engine.solver.WeightStreamingModel - - engine.solver.TailLatencyModel - - engine.solver.DataModel - - engine.solver.TransformationModel - - engine.solver.TopologyModel - - engine.solver.ScalingModel - - engine.solver.InferenceScalingModel - - engine.solver.CompressionModel - - engine.solver.DistributedModel - - engine.solver.MoERoutingModel - - engine.solver.ReliabilityModel - - engine.solver.OrchestrationModel - - engine.solver.EconomicsModel - - engine.solver.SustainabilityModel - - engine.solver.CheckpointModel - - engine.solver.ResponsibleEngineeringModel - - engine.solver.SensitivitySolver - - engine.solver.SynthesisSolver - - engine.solver.ParallelismOptimizer - - engine.solver.BatchingOptimizer - - engine.solver.PlacementOptimizer + - solvers.SingleNodeModel + - solvers.NetworkRooflineModel + - solvers.EfficiencyModel + - solvers.ForwardModel + - solvers.ServingModel + - solvers.TrainingMemoryModel + - solvers.ServingCapacityModel + - solvers.ContinuousBatchingModel + - solvers.WeightStreamingModel + - solvers.TailLatencyModel + - solvers.DataModel + - solvers.TransformationModel + - solvers.TopologyModel + - solvers.ScalingModel + - solvers.InferenceScalingModel + - solvers.CompressionModel + - solvers.DistributedModel + - solvers.MoERoutingModel + - solvers.ReliabilityModel + - solvers.OrchestrationModel + - solvers.EconomicsModel + - solvers.SustainabilityModel + - solvers.CheckpointModel + - solvers.ResponsibleEngineeringModel + - solvers.SensitivitySolver + - solvers.SynthesisSolver + - solvers.ParallelismOptimizer + - solvers.BatchingOptimizer + - solvers.PlacementOptimizer - engine.dse.DSE diff --git a/mlsysim/docs/empirical-calibration.md b/mlsysim/docs/empirical-calibration.md index 2682ca4411..ba1db2e48f 100644 --- a/mlsysim/docs/empirical-calibration.md +++ b/mlsysim/docs/empirical-calibration.md @@ -138,7 +138,7 @@ values minimize error against published benchmarks: ```python import mlsysim -from mlsysim.engine.solver import SingleNodeModel, ServingModel +from mlsysim.solvers import SingleNodeModel, ServingModel from mlsysim.physics import calc_transformer_training_flops # Config 1: ResNet-50 / A100 / training diff --git a/mlsysim/docs/extending-the-engine.qmd b/mlsysim/docs/extending-the-engine.qmd index 97b39dc26f..95617f30bf 100644 --- a/mlsysim/docs/extending-the-engine.qmd +++ b/mlsysim/docs/extending-the-engine.qmd @@ -37,7 +37,7 @@ Every resolver follows the same pattern: declare inputs (`requires`), declare ou Let's build a custom `PowerEfficiencyModel` that calculates TFLOPs per Watt. ```python -from mlsysim.engine.solver import ForwardModel +from mlsysim.solvers import ForwardModel from mlsysim.engine.results import SolverResult from mlsysim.core.constants import Q_ from mlsysim.hardware.types import HardwareNode @@ -76,7 +76,7 @@ class PowerEfficiencyModel(ForwardModel): A solver algebraically inverts an equation. For example, if $T = \frac{W}{BW}$, and we have a target $T$, we solve for $BW = \frac{W}{T}$. ```python -from mlsysim.engine.solver import BaseSolver +from mlsysim.solvers import BaseSolver from mlsysim.engine.results import SolverResult from mlsysim.models.types import Workload from mlsysim.core.types import Quantity @@ -108,7 +108,7 @@ An Optimizer explores a design space. It MUST inherit from `BaseOptimizer` and i Let's build a `CheapestHardwareOptimizer` that searches the `HardwareZoo` for the cheapest chip that satisfies a minimum TFLOP requirement. ```python -from mlsysim.engine.solver import BaseOptimizer +from mlsysim.solvers import BaseOptimizer from mlsysim.engine.results import OptimizerResult from mlsysim.hardware.registry import Hardware from typing import Dict, Any @@ -162,7 +162,7 @@ MLSys·im doesn't just evaluate models in a vacuum. It uses a **Composable Pipel ```python from mlsysim.engine.pipeline import Pipeline -from mlsysim.engine.solver import DistributedModel, EconomicsModel +from mlsysim.solvers import DistributedModel, EconomicsModel # Build a pipeline with your custom solver in the middle my_pipeline = Pipeline([ diff --git a/mlsysim/docs/for-engineers.qmd b/mlsysim/docs/for-engineers.qmd index 14f5b30dfd..41751e589d 100644 --- a/mlsysim/docs/for-engineers.qmd +++ b/mlsysim/docs/for-engineers.qmd @@ -58,7 +58,7 @@ The `ServingModel` models the [two-phase LLM inference lifecycle]({{< var slides ```python import mlsysim -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel serving = ServingModel() result = serving.solve( @@ -103,7 +103,7 @@ The `DistributedModel` models [3D parallelism]({{< var slides_latest >}}/vol2_05 ```python import mlsysim -from mlsysim import DistributedModel +from mlsysim.solvers import DistributedModel dist = DistributedModel() result = dist.solve( @@ -131,7 +131,7 @@ The core solvers are designed to chain. Here are three common engineering workfl ```python import mlsysim -from mlsysim import ServingModel, EconomicsModel +from mlsysim.solvers import ServingModel, EconomicsModel # Step 1: Does it fit and what's the latency? serving = ServingModel() @@ -157,7 +157,7 @@ print(f" OpEx: ${cost.total_opex_usd:,.0f}") ```python import mlsysim -from mlsysim import SustainabilityModel +from mlsysim.solvers import SustainabilityModel sustain = SustainabilityModel() for grid in [mlsysim.Infrastructure.Grids.Quebec, mlsysim.Infrastructure.Grids.US_Avg, @@ -178,7 +178,7 @@ For the theory behind PUE, carbon intensity, and the energy hierarchy, see the [ ```python import mlsysim -from mlsysim import ReliabilityModel +from mlsysim.solvers import ReliabilityModel rel = ReliabilityModel() result = rel.solve( diff --git a/mlsysim/docs/math.qmd b/mlsysim/docs/math.qmd index 3f06f97b72..003c81076e 100644 --- a/mlsysim/docs/math.qmd +++ b/mlsysim/docs/math.qmd @@ -16,7 +16,7 @@ or follow the **Slide Deck** links to the full lecture treatment with worked exa ## 1. The Roofline Model (Single-Node Performance) {#sec-roofline} -*Implemented in [`mlsysim.engine.solver.SingleNodeModel`](api/engine.solver.SingleNodeModel.qmd).* +*Implemented in [`mlsysim.solvers.SingleNodeModel`](api/solvers.SingleNodeModel.qmd).* **Slide Deck:** [Hardware Acceleration (Vol I, Ch 11)]({{< var slides_latest >}}/vol1_11_hw_acceleration.pdf){target="_blank"} ::: {.callout-note appearance="simple" icon=false} @@ -72,7 +72,7 @@ Source: Slide deck exercise, Vol I Ch 11. ## 2. Distributed Training (3D/4D Parallelism) {#sec-distributed} -*Implemented in [`mlsysim.engine.solver.DistributedModel`](api/engine.solver.DistributedModel.qmd).* +*Implemented in [`mlsysim.solvers.DistributedModel`](api/solvers.DistributedModel.qmd).* **Slide Decks:** [Distributed Training (Vol II, Ch 5)]({{< var slides_latest >}}/vol2_05_distributed_training.pdf){target="_blank"} | [Collective Communication (Vol II, Ch 6)]({{< var slides_latest >}}/vol2_06_collective_communication.pdf){target="_blank"} Real distributed training involves complex interactions between computation, communication, and scheduling. Empirical profiling requires access to expensive multi-GPU clusters and takes hours per configuration. MLSYSIM decomposes the problem into independent overheads — each governed by a closed-form equation — letting you evaluate thousands of parallelism configurations in seconds. @@ -232,7 +232,7 @@ The actual training took ~34 days, consistent with scaling efficiency losses at ### 3.1 Training Memory Accounting {#sec-training-memory} -*Implemented in [`mlsysim.engine.solver.TrainingMemoryModel`](api/engine.solver.TrainingMemoryModel.qmd).* +*Implemented in [`mlsysim.solvers.TrainingMemoryModel`](api/solvers.TrainingMemoryModel.qmd).* Inference memory is mostly weights plus KV cache. Training memory adds gradients, optimizer state, saved activations, and communication buffers: @@ -256,7 +256,7 @@ Tensor, pipeline, and expert parallelism first shard the model states across mod ## 4. LLM Serving Lifecycle {#sec-serving} -*Implemented in [`mlsysim.engine.solver.ServingModel`](api/engine.solver.ServingModel.qmd).* +*Implemented in [`mlsysim.solvers.ServingModel`](api/solvers.ServingModel.qmd).* **Slide Decks:** [Model Serving (Vol I, Ch 13)]({{< var slides_latest >}}/vol1_13_model_serving.pdf){target="_blank"} | [Inference at Scale (Vol II, Ch 10)]({{< var slides_latest >}}/vol2_10_inference.pdf){target="_blank"} LLM autoregressive inference has two physically distinct phases. Understanding which phase dominates is critical for capacity planning. @@ -363,7 +363,7 @@ This estimates a coarse scheduling proxy used by chunked-prefill systems: smalle ### 4.5 Serving Capacity Planning {#sec-serving-capacity} -*Implemented in [`mlsysim.engine.solver.ServingCapacityModel`](api/engine.solver.ServingCapacityModel.qmd).* +*Implemented in [`mlsysim.solvers.ServingCapacityModel`](api/solvers.ServingCapacityModel.qmd).* Capacity planning composes three first-order quantities: @@ -400,7 +400,7 @@ A 70B model costing \$2M to train, serving 1M daily users at 50 requests/day, co ## 5. Datacenter Sustainability {#sec-sustainability} -*Implemented in [`mlsysim.engine.solver.SustainabilityModel`](api/engine.solver.SustainabilityModel.qmd).* +*Implemented in [`mlsysim.solvers.SustainabilityModel`](api/solvers.SustainabilityModel.qmd).* **Slide Deck:** [Sustainable AI (Vol II, Ch 15)]({{< var slides_latest >}}/vol2_15_sustainable_ai.pdf){target="_blank"} ### 5.1 Total Energy @@ -439,7 +439,7 @@ Source: Sustainable AI slides (Vol II, Ch 15). ## 6. Total Cost of Ownership (TCO) {#sec-tco} -*Implemented in [`mlsysim.engine.solver.EconomicsModel`](api/engine.solver.EconomicsModel.qmd).* +*Implemented in [`mlsysim.solvers.EconomicsModel`](api/solvers.EconomicsModel.qmd).* **Slide Deck:** [Compute Infrastructure (Vol II, Ch 2)]({{< var slides_latest >}}/vol2_02_compute_infrastructure.pdf){target="_blank"} $$ @@ -466,7 +466,7 @@ Source: Compute Infrastructure slides (Vol II, Ch 2). ## 7. Cluster Reliability (The Young-Daly Model) {#sec-reliability} -*Implemented in [`mlsysim.engine.solver.ReliabilityModel`](api/engine.solver.ReliabilityModel.qmd).* +*Implemented in [`mlsysim.solvers.ReliabilityModel`](api/solvers.ReliabilityModel.qmd).* **Slide Deck:** [Fault Tolerance (Vol II, Ch 7)]({{< var slides_latest >}}/vol2_07_fault_tolerance.pdf){target="_blank"} ::: {.callout-note appearance="simple" icon=false} diff --git a/mlsysim/docs/models-and-solvers.qmd b/mlsysim/docs/models-and-solvers.qmd index 04ace14ace..2b005929d8 100644 --- a/mlsysim/docs/models-and-solvers.qmd +++ b/mlsysim/docs/models-and-solvers.qmd @@ -40,44 +40,44 @@ Use these when you want to ask: *"What happens if I run this exact setup?"* ### Domain 1 — Node (Single-Accelerator Resources) | Model | Key Inputs | Key Outputs | Best For | |:------|:-----------|:------------|:---------| -| [**`SingleNodeModel`**](api/engine.solver.SingleNodeModel.qmd) | model, hardware, batch_size | latency, throughput, bottleneck | "Is my model memory-bound?" | -| [**`TrainingMemoryModel`**](api/engine.solver.TrainingMemoryModel.qmd) | model, hardware, batch_size, seq_len | weights, gradients, optimizer, activations | "Why does training need so much more memory?" | -| [**`EfficiencyModel`**](api/engine.solver.EfficiencyModel.qmd) | model, hardware, workload_type | MFU, achievable FLOPS | "What MFU will my workload achieve?" | -| [**`ServingModel`**](api/engine.solver.ServingModel.qmd) | model, hardware, seq_len | TTFT, ITL, KV-cache footprint, decode stall proxy | "Can I serve this LLM on this GPU?" | -| [**`ServingCapacityModel`**](api/engine.solver.ServingCapacityModel.qmd) | model, hardware, QPS, target P99 | replicas, capacity, queue wait | "How many replicas do I need for this SLA?" | -| [**`ContinuousBatchingModel`**](api/engine.solver.ContinuousBatchingModel.qmd) | model, hardware, seq_len, max_batch | throughput, fragmentation | "What throughput with PagedAttention?" | -| [**`WeightStreamingModel`**](api/engine.solver.WeightStreamingModel.qmd) | model, hardware, seq_len, batch_size | throughput, optimal_batch | "Cerebras wafer-scale inference?" | -| [**`TailLatencyModel`**](api/engine.solver.TailLatencyModel.qmd) | arrival_rate, service_latency, replicas | P50, P99 wait times | "Will I meet P99 latency SLAs?" | +| [**`SingleNodeModel`**](api/solvers.SingleNodeModel.qmd) | model, hardware, batch_size | latency, throughput, bottleneck | "Is my model memory-bound?" | +| [**`TrainingMemoryModel`**](api/solvers.TrainingMemoryModel.qmd) | model, hardware, batch_size, seq_len | weights, gradients, optimizer, activations | "Why does training need so much more memory?" | +| [**`EfficiencyModel`**](api/solvers.EfficiencyModel.qmd) | model, hardware, workload_type | MFU, achievable FLOPS | "What MFU will my workload achieve?" | +| [**`ServingModel`**](api/solvers.ServingModel.qmd) | model, hardware, seq_len | TTFT, ITL, KV-cache footprint, decode stall proxy | "Can I serve this LLM on this GPU?" | +| [**`ServingCapacityModel`**](api/solvers.ServingCapacityModel.qmd) | model, hardware, QPS, target P99 | replicas, capacity, queue wait | "How many replicas do I need for this SLA?" | +| [**`ContinuousBatchingModel`**](api/solvers.ContinuousBatchingModel.qmd) | model, hardware, seq_len, max_batch | throughput, fragmentation | "What throughput with PagedAttention?" | +| [**`WeightStreamingModel`**](api/solvers.WeightStreamingModel.qmd) | model, hardware, seq_len, batch_size | throughput, optimal_batch | "Cerebras wafer-scale inference?" | +| [**`TailLatencyModel`**](api/solvers.TailLatencyModel.qmd) | arrival_rate, service_latency, replicas | P50, P99 wait times | "Will I meet P99 latency SLAs?" | ### Domain 2 — Data (Movement & Pipelines) | Model | Key Inputs | Key Outputs | Best For | |:------|:-----------|:------------|:---------| -| [**`DataModel`**](api/engine.solver.DataModel.qmd) | workload_data_rate, hardware | utilization, is_stalled | "Is my storage/IO the bottleneck?" | -| [**`TransformationModel`**](api/engine.solver.TransformationModel.qmd) | batch_size, cpu_throughput | transform_time, is_bottleneck | "Is CPU preprocessing starving my GPU?" | -| [**`TopologyModel`**](api/engine.solver.TopologyModel.qmd) | fabric, topology, num_nodes | effective_bw, bisection_bw | "What topology should I use?" | +| [**`DataModel`**](api/solvers.DataModel.qmd) | workload_data_rate, hardware | utilization, is_stalled | "Is my storage/IO the bottleneck?" | +| [**`TransformationModel`**](api/solvers.TransformationModel.qmd) | batch_size, cpu_throughput | transform_time, is_bottleneck | "Is CPU preprocessing starving my GPU?" | +| [**`TopologyModel`**](api/solvers.TopologyModel.qmd) | fabric, topology, num_nodes | effective_bw, bisection_bw | "What topology should I use?" | ### Domain 3 — Algorithm (Scaling & Compression) | Model | Key Inputs | Key Outputs | Best For | |:------|:-----------|:------------|:---------| -| [**`ScalingModel`**](api/engine.solver.ScalingModel.qmd) | compute_budget | optimal_params, optimal_tokens | "What is my optimal model size?" | -| [**`InferenceScalingModel`**](api/engine.solver.InferenceScalingModel.qmd) | model, hardware, reasoning_steps | total_reasoning_time | "How much does CoT reasoning cost?" | -| [**`CompressionModel`**](api/engine.solver.CompressionModel.qmd) | model, hardware, method | accuracy_delta, compression_ratio | "Is quantization/pruning worth it?" | +| [**`ScalingModel`**](api/solvers.ScalingModel.qmd) | compute_budget | optimal_params, optimal_tokens | "What is my optimal model size?" | +| [**`InferenceScalingModel`**](api/solvers.InferenceScalingModel.qmd) | model, hardware, reasoning_steps | total_reasoning_time | "How much does CoT reasoning cost?" | +| [**`CompressionModel`**](api/solvers.CompressionModel.qmd) | model, hardware, method | accuracy_delta, compression_ratio | "Is quantization/pruning worth it?" | ### Domain 4 — Fleet (Multi-Node Coordination) | Model | Key Inputs | Key Outputs | Best For | |:------|:-----------|:------------|:---------| -| [**`DistributedModel`**](api/engine.solver.DistributedModel.qmd) | model, fleet, tp/pp/dp sizes | scaling efficiency, comm overhead | "How many GPUs do I actually need?" | -| [**`MoERoutingModel`**](api/engine.solver.MoERoutingModel.qmd) | sparse model, batch, seq_len, EP | active experts, routed bytes, all-to-all | "How much does hot-expert imbalance cost?" | -| [**`ReliabilityModel`**](api/engine.solver.ReliabilityModel.qmd) | fleet, job_duration | MTBF, failure probability | "Will my training job complete?" | -| [**`OrchestrationModel`**](api/engine.solver.OrchestrationModel.qmd) | fleet, arrival_rate, avg_duration | avg_wait_time, utilization | "How busy is my cluster?" | +| [**`DistributedModel`**](api/solvers.DistributedModel.qmd) | model, fleet, tp/pp/dp sizes | scaling efficiency, comm overhead | "How many GPUs do I actually need?" | +| [**`MoERoutingModel`**](api/solvers.MoERoutingModel.qmd) | sparse model, batch, seq_len, EP | active experts, routed bytes, all-to-all | "How much does hot-expert imbalance cost?" | +| [**`ReliabilityModel`**](api/solvers.ReliabilityModel.qmd) | fleet, job_duration | MTBF, failure probability | "Will my training job complete?" | +| [**`OrchestrationModel`**](api/solvers.OrchestrationModel.qmd) | fleet, arrival_rate, avg_duration | avg_wait_time, utilization | "How busy is my cluster?" | ### Domain 5 — Ops (Economics, Sustainability & Safety) | Model | Key Inputs | Key Outputs | Best For | |:------|:-----------|:------------|:---------| -| [**`EconomicsModel`**](api/engine.solver.EconomicsModel.qmd) | fleet, duration_days, kwh_price | CapEx, OpEx, total TCO | "What will this cost over 3 years?" | -| [**`SustainabilityModel`**](api/engine.solver.SustainabilityModel.qmd) | fleet, duration_days, datacenter | energy, carbon (kg CO₂e), water | "Where should I train to minimize carbon?" | -| [**`CheckpointModel`**](api/engine.solver.CheckpointModel.qmd) | model, hardware, optimizer | checkpoint_size, MFU penalty | "How much MFU do I lose to checkpoints?" | -| [**`ResponsibleEngineeringModel`**](api/engine.solver.ResponsibleEngineeringModel.qmd) | base_training_time, epsilon | dp_slowdown | "What is the cost of differential privacy?" | +| [**`EconomicsModel`**](api/solvers.EconomicsModel.qmd) | fleet, duration_days, kwh_price | CapEx, OpEx, total TCO | "What will this cost over 3 years?" | +| [**`SustainabilityModel`**](api/solvers.SustainabilityModel.qmd) | fleet, duration_days, datacenter | energy, carbon (kg CO₂e), water | "Where should I train to minimize carbon?" | +| [**`CheckpointModel`**](api/solvers.CheckpointModel.qmd) | model, hardware, optimizer | checkpoint_size, MFU penalty | "How much MFU do I lose to checkpoints?" | +| [**`ResponsibleEngineeringModel`**](api/solvers.ResponsibleEngineeringModel.qmd) | base_training_time, epsilon | dp_slowdown | "What is the cost of differential privacy?" | --- @@ -87,8 +87,8 @@ Use these when you want to ask: *"What exact number do I need to hit my target?" | Solver | Key Inputs | Key Outputs | Best For | |:-------|:-----------|:------------|:---------| -| [**`SensitivitySolver`**](api/engine.solver.SensitivitySolver.qmd) | model, hardware, perturbation_pct | sensitivities, binding_constraint | "Which parameter should I invest in?" | -| [**`SynthesisSolver`**](api/engine.solver.SynthesisSolver.qmd) | model, target_latency | required_bw, required_flops | "What hardware do I need for this SLA?" | +| [**`SensitivitySolver`**](api/solvers.SensitivitySolver.qmd) | model, hardware, perturbation_pct | sensitivities, binding_constraint | "Which parameter should I invest in?" | +| [**`SynthesisSolver`**](api/solvers.SynthesisSolver.qmd) | model, target_latency | required_bw, required_flops | "What hardware do I need for this SLA?" | --- @@ -109,7 +109,8 @@ Real-world questions require **chaining** multiple tiers. Here are three common ### "Can I serve Llama-70B on 4 H100s within budget?" ```python -from mlsysim import ServingModel, EconomicsModel, Hardware, Models, Systems +from mlsysim import Hardware, Models, Systems +from mlsysim.solvers import ServingModel, EconomicsModel serving = ServingModel().solve( model=Models.Language.Llama3_70B, @@ -124,7 +125,7 @@ print(f"KV-Cache: {serving.kv_cache_size}") ```python from mlsysim import Hardware, Models -from mlsysim.engine.solver import SensitivitySolver +from mlsysim.solvers import SensitivitySolver sensitivity = SensitivitySolver().solve( model=Models.Language.Llama3_8B, @@ -141,7 +142,7 @@ print(f"FLOPS sensitivity: {sensitivity.flops_sensitivity:.3f}") ```python from mlsysim import Models -from mlsysim.engine.solver import SynthesisSolver +from mlsysim.solvers import SynthesisSolver from mlsysim.core.constants import Q_ synthesis = SynthesisSolver().solve( diff --git a/mlsysim/docs/solver-guide.qmd b/mlsysim/docs/solver-guide.qmd index f25ecc06ca..90c2e23f82 100644 --- a/mlsysim/docs/solver-guide.qmd +++ b/mlsysim/docs/solver-guide.qmd @@ -10,39 +10,39 @@ MLSys·im provides specialized analytical resolvers for different classes of ML ## Start With Your Question **"How fast will my model run on this GPU?"** -: Use the [**SingleNodeModel**](api/engine.solver.SingleNodeModel.qmd). It applies the roofline model to determine whether your workload is compute-bound or memory-bound and returns latency, throughput, and bottleneck classification. +: Use the [**SingleNodeModel**](api/solvers.SingleNodeModel.qmd). It applies the roofline model to determine whether your workload is compute-bound or memory-bound and returns latency, throughput, and bottleneck classification. : *Lecture slides:* [Hardware Acceleration](https://mlsysbook.ai/slides/vol1.html) (Vol I, Ch 11) · [Benchmarking](https://mlsysbook.ai/slides/vol1.html) (Vol I, Ch 12) **"How fast will my LLM generate tokens?"** -: Use the [**ServingModel**](api/engine.solver.ServingModel.qmd). It models the two distinct phases of autoregressive inference: the compute-bound prefill (TTFT) and the memory-bound decode (ITL), plus KV-cache memory pressure, phase splitting, prompt caching, speculative decode, and an optional chunked-prefill stall proxy. +: Use the [**ServingModel**](api/solvers.ServingModel.qmd). It models the two distinct phases of autoregressive inference: the compute-bound prefill (TTFT) and the memory-bound decode (ITL), plus KV-cache memory pressure, phase splitting, prompt caching, speculative decode, and an optional chunked-prefill stall proxy. : *Lecture slides:* [Model Serving](https://mlsysbook.ai/slides/vol1.html) (Vol I, Ch 13) · [Inference at Scale](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 10) **"How much memory do I need for training?"** -: Use the [**TrainingMemoryModel**](api/engine.solver.TrainingMemoryModel.qmd). It separates weights, gradients, optimizer state, activations, and communication buffers so training memory is not confused with inference memory. +: Use the [**TrainingMemoryModel**](api/solvers.TrainingMemoryModel.qmd). It separates weights, gradients, optimizer state, activations, and communication buffers so training memory is not confused with inference memory. : *Lecture slides:* [Training](https://mlsysbook.ai/slides/vol1.html) (Vol I, Ch 8) · [Distributed Training](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 5) **"How many serving replicas do I need for this SLA?"** -: Use the [**ServingCapacityModel**](api/engine.solver.ServingCapacityModel.qmd). It composes serving latency, continuous-batching capacity, and queueing pressure into a replica-count estimate. +: Use the [**ServingCapacityModel**](api/solvers.ServingCapacityModel.qmd). It composes serving latency, continuous-batching capacity, and queueing pressure into a replica-count estimate. : *Lecture slides:* [Model Serving](https://mlsysbook.ai/slides/vol1.html) (Vol I, Ch 13) · [Inference at Scale](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 10) **"How does performance scale across multiple GPUs?"** -: Use the [**DistributedModel**](api/engine.solver.DistributedModel.qmd). It decomposes workloads using 3D/4D parallelism (DP, TP, PP, EP) and calculates communication overhead, pipeline bubbles, and scaling efficiency. +: Use the [**DistributedModel**](api/solvers.DistributedModel.qmd). It decomposes workloads using 3D/4D parallelism (DP, TP, PP, EP) and calculates communication overhead, pipeline bubbles, and scaling efficiency. : *Lecture slides:* [Distributed Training](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 5) · [Collective Communication](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 6) · [Network Fabrics](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 3) **"How much does MoE routing imbalance hurt?"** -: Use the [**MoERoutingModel**](api/engine.solver.MoERoutingModel.qmd). It keeps MoE modeling first-order: total parameters set memory, active parameters set compute, and a routing-imbalance factor inflates expert-parallel all-to-all traffic. +: Use the [**MoERoutingModel**](api/solvers.MoERoutingModel.qmd). It keeps MoE modeling first-order: total parameters set memory, active parameters set compute, and a routing-imbalance factor inflates expert-parallel all-to-all traffic. : *Lecture slides:* [Distributed Training](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 5) · [Inference at Scale](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 10) **"How much will this cost to run?"** -: Use the [**EconomicsModel**](api/engine.solver.EconomicsModel.qmd). It calculates Total Cost of Ownership: CapEx (hardware purchase), OpEx (energy + maintenance), and total TCO over a specified duration. +: Use the [**EconomicsModel**](api/solvers.EconomicsModel.qmd). It calculates Total Cost of Ownership: CapEx (hardware purchase), OpEx (energy + maintenance), and total TCO over a specified duration. : *Lecture slides:* [Compute Infrastructure](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 2) **"What is the carbon footprint?"** -: Use the [**SustainabilityModel**](api/engine.solver.SustainabilityModel.qmd). It computes energy consumption (factoring in PUE), carbon emissions (using regional grid intensity), and water usage across datacenter locations. +: Use the [**SustainabilityModel**](api/solvers.SustainabilityModel.qmd). It computes energy consumption (factoring in PUE), carbon emissions (using regional grid intensity), and water usage across datacenter locations. : *Lecture slides:* [Sustainable AI](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 15) **"How often will my cluster fail during training?"** -: Use the [**ReliabilityModel**](api/engine.solver.ReliabilityModel.qmd). It estimates fleet-wide MTBF, failure probability for a given job duration, and the Young-Daly optimal checkpoint interval. +: Use the [**ReliabilityModel**](api/solvers.ReliabilityModel.qmd). It estimates fleet-wide MTBF, failure probability for a given job duration, and the Young-Daly optimal checkpoint interval. : *Lecture slides:* [Fault Tolerance](https://mlsysbook.ai/slides/vol2.html) (Vol II, Ch 7) --- @@ -51,15 +51,15 @@ MLSys·im provides specialized analytical resolvers for different classes of ML | Solver | Key Inputs | Key Outputs | Best For | |:-------|:-----------|:------------|:---------| -| [**SingleNodeModel**](api/engine.solver.SingleNodeModel.qmd) | `model`, `hardware`, `batch_size`, `precision` | latency, throughput, bottleneck, MFU | "Is my model memory-bound?" | -| [**ServingModel**](api/engine.solver.ServingModel.qmd) | `model`, `hardware`, `seq_len`, `batch_size` | TTFT, ITL, KV-cache size, decode stall proxy, feasibility | "Can I serve this LLM on this GPU?" | -| [**TrainingMemoryModel**](api/engine.solver.TrainingMemoryModel.qmd) | `model`, `hardware`, `batch_size`, `seq_len` | memory breakdown, feasibility | "Why does training not fit?" | -| [**ServingCapacityModel**](api/engine.solver.ServingCapacityModel.qmd) | `model`, `hardware`, `qps`, `target_p99_latency_ms` | replicas, QPS capacity, queue wait | "How many replicas do I need?" | -| [**DistributedModel**](api/engine.solver.DistributedModel.qmd) | `model`, `fleet`, `tp_size`, `pp_size`, `ep_size` | scaling efficiency, communication overhead | "How many GPUs do I actually need?" | -| [**MoERoutingModel**](api/engine.solver.MoERoutingModel.qmd) | sparse `model`, `batch_size`, `seq_len`, `ep_size` | active experts, routed bytes, all-to-all latency | "What is the MoE routing tax?" | -| [**EconomicsModel**](api/engine.solver.EconomicsModel.qmd) | `fleet`, `duration_days`, `kwh_price` | CapEx, OpEx, total TCO | "What will this cost over 3 years?" | -| [**SustainabilityModel**](api/engine.solver.SustainabilityModel.qmd) | `fleet`, `duration_days`, `datacenter` | energy (kWh), carbon (kg CO₂e), water (L) | "Where should I train to minimize carbon?" | -| [**ReliabilityModel**](api/engine.solver.ReliabilityModel.qmd) | `fleet`, `job_duration_hours`, `checkpoint_time_s` | MTBF, failure probability, checkpoint interval | "Will my training job complete?" | +| [**SingleNodeModel**](api/solvers.SingleNodeModel.qmd) | `model`, `hardware`, `batch_size`, `precision` | latency, throughput, bottleneck, MFU | "Is my model memory-bound?" | +| [**ServingModel**](api/solvers.ServingModel.qmd) | `model`, `hardware`, `seq_len`, `batch_size` | TTFT, ITL, KV-cache size, decode stall proxy, feasibility | "Can I serve this LLM on this GPU?" | +| [**TrainingMemoryModel**](api/solvers.TrainingMemoryModel.qmd) | `model`, `hardware`, `batch_size`, `seq_len` | memory breakdown, feasibility | "Why does training not fit?" | +| [**ServingCapacityModel**](api/solvers.ServingCapacityModel.qmd) | `model`, `hardware`, `qps`, `target_p99_latency_ms` | replicas, QPS capacity, queue wait | "How many replicas do I need?" | +| [**DistributedModel**](api/solvers.DistributedModel.qmd) | `model`, `fleet`, `tp_size`, `pp_size`, `ep_size` | scaling efficiency, communication overhead | "How many GPUs do I actually need?" | +| [**MoERoutingModel**](api/solvers.MoERoutingModel.qmd) | sparse `model`, `batch_size`, `seq_len`, `ep_size` | active experts, routed bytes, all-to-all latency | "What is the MoE routing tax?" | +| [**EconomicsModel**](api/solvers.EconomicsModel.qmd) | `fleet`, `duration_days`, `kwh_price` | CapEx, OpEx, total TCO | "What will this cost over 3 years?" | +| [**SustainabilityModel**](api/solvers.SustainabilityModel.qmd) | `fleet`, `duration_days`, `datacenter` | energy (kWh), carbon (kg CO₂e), water (L) | "Where should I train to minimize carbon?" | +| [**ReliabilityModel**](api/solvers.ReliabilityModel.qmd) | `fleet`, `job_duration_hours`, `checkpoint_time_s` | MTBF, failure probability, checkpoint interval | "Will my training job complete?" | --- @@ -74,7 +74,7 @@ explicit. ```python import mlsysim -from mlsysim import SingleNodeModel +from mlsysim.solvers import SingleNodeModel solver = SingleNodeModel() profile = solver.solve( @@ -91,7 +91,7 @@ print(f"MFU: {profile.mfu:.1%}") ```python import mlsysim -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel serving = ServingModel() result = serving.solve( @@ -110,7 +110,7 @@ print(f"Fits: {result.feasible}") ```python import mlsysim -from mlsysim import TrainingMemoryModel +from mlsysim.solvers import TrainingMemoryModel memory = TrainingMemoryModel().solve( model=mlsysim.Models.Language.Llama3_8B, @@ -131,7 +131,7 @@ print(f"Fits: {memory.feasible}") ```python import mlsysim -from mlsysim import ServingCapacityModel +from mlsysim.solvers import ServingCapacityModel capacity = ServingCapacityModel().solve( model=mlsysim.Models.Language.Llama3_8B, @@ -150,7 +150,7 @@ print(f"Util: {capacity.utilization:.1%}") ```python from mlsysim import Systems, ureg -from mlsysim.engine.solver import MoERoutingModel +from mlsysim.solvers import MoERoutingModel from mlsysim.models.types import SparseTransformerWorkload moe = SparseTransformerWorkload( @@ -182,7 +182,8 @@ print(f"All-to-all: {routing.all_to_all_latency:~.2f}") ```python import mlsysim -from mlsysim import DistributedModel, Systems +from mlsysim import Systems +from mlsysim.solvers import DistributedModel dist = DistributedModel() result = dist.solve( @@ -204,7 +205,7 @@ MLSYSIM does not provide a built-in sweep function. Instead, use a simple Python ```python import mlsysim -from mlsysim import SingleNodeModel +from mlsysim.solvers import SingleNodeModel solver = SingleNodeModel() targets = [ @@ -336,4 +337,4 @@ Use `pint.Quantity` for all physical calculations so that unit errors are imposs --- -*For the equations behind each solver, see [Math Foundations](math.qmd). For full API details, see the [Solver API Reference](api/engine.qmd).* +*For the equations behind each solver, see [Math Foundations](math.qmd). For full API details, see the [Solver API Reference](api/solvers.qmd).* diff --git a/mlsysim/docs/tutorials/01_pipeline_callbacks.qmd b/mlsysim/docs/tutorials/01_pipeline_callbacks.qmd index 20ad65ccaa..e08a2d7728 100644 --- a/mlsysim/docs/tutorials/01_pipeline_callbacks.qmd +++ b/mlsysim/docs/tutorials/01_pipeline_callbacks.qmd @@ -26,7 +26,7 @@ First, import the necessary modules. We will use the `Pipeline` orchestrator dir ```python import mlsysim from mlsysim.engine.pipeline import Pipeline -from mlsysim.engine.solver import DistributedModel, EconomicsModel +from mlsysim.solvers import DistributedModel, EconomicsModel ``` ## 2. The Composable Pipeline diff --git a/mlsysim/docs/tutorials/02_two_phases.qmd b/mlsysim/docs/tutorials/02_two_phases.qmd index 0fe0d931fd..823d76cb39 100644 --- a/mlsysim/docs/tutorials/02_two_phases.qmd +++ b/mlsysim/docs/tutorials/02_two_phases.qmd @@ -61,7 +61,7 @@ import mlsysim # installed via `pip install mlsysim` (see workflow) ```python import mlsysim -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel ``` In the previous tutorials, you used `Engine.solve`, which models inference as a single @@ -74,7 +74,7 @@ different bottlenecks. The `ServingModel` models each phase separately, giving y ## 2. First Serving Prediction ```{python} -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel # Llama-3 8B: 8B parameters, 32 layers, 4096 hidden_dim model = mlsysim.Models.Language.Llama3_8B diff --git a/mlsysim/docs/tutorials/03_kv_cache.qmd b/mlsysim/docs/tutorials/03_kv_cache.qmd index a27d0c6bc3..8d25c956a2 100644 --- a/mlsysim/docs/tutorials/03_kv_cache.qmd +++ b/mlsysim/docs/tutorials/03_kv_cache.qmd @@ -71,8 +71,8 @@ Engine = mlsysim.Engine ```python import mlsysim -from mlsysim import ServingModel -from mlsysim.engine.solver import ContinuousBatchingModel +from mlsysim.solvers import ServingModel +from mlsysim.solvers import ContinuousBatchingModel ``` --- @@ -82,7 +82,7 @@ from mlsysim.engine.solver import ContinuousBatchingModel Let's start with a single user at a modest 2K context and see how memory breaks down: ```{python} -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel model = mlsysim.Models.Language.Llama3_8B hardware = mlsysim.Hardware.Cloud.H100 @@ -195,7 +195,7 @@ on demand, PagedAttention maps KV-cache blocks to GPU memory on demand, eliminat fragmentation and fitting more concurrent requests: ```{python} -from mlsysim.engine.solver import ContinuousBatchingModel +from mlsysim.solvers import ContinuousBatchingModel cb_solver = ContinuousBatchingModel() diff --git a/mlsysim/docs/tutorials/04_starving_the_gpu.qmd b/mlsysim/docs/tutorials/04_starving_the_gpu.qmd index d7b757da09..c706ec6fa7 100644 --- a/mlsysim/docs/tutorials/04_starving_the_gpu.qmd +++ b/mlsysim/docs/tutorials/04_starving_the_gpu.qmd @@ -58,8 +58,8 @@ Engine = mlsysim.Engine ```python import mlsysim -from mlsysim import SingleNodeModel, DataModel -from mlsysim.engine.solver import TransformationModel +from mlsysim.solvers import SingleNodeModel, DataModel +from mlsysim.solvers import TransformationModel ``` --- @@ -76,7 +76,7 @@ First, establish how fast the A100 processes a ResNet-50 training step in isolat data loading, no preprocessing, just pure compute: ```{python} -from mlsysim import SingleNodeModel +from mlsysim.solvers import SingleNodeModel from mlsysim.core.constants import Q_ from mlsysim.show import table, info @@ -107,7 +107,7 @@ ImageNet images average ~500 KB each (JPEG compressed). At batch 256, the GPU de burst of data every step. Can the storage subsystem supply it? ```{python} -from mlsysim import DataModel +from mlsysim.solvers import DataModel sample_size = Q_("500 KB") # Average ImageNet JPEG batch_size = 256 @@ -139,7 +139,7 @@ normalization. A typical CPU worker processes ImageNet images at ~250 MB/s. With total CPU throughput is ~2 GB/s: ```{python} -from mlsysim.engine.solver import TransformationModel +from mlsysim.solvers import TransformationModel transform_solver = TransformationModel() cpu_throughput = Q_("2 GB/s") # 8 workers x 250 MB/s each diff --git a/mlsysim/docs/tutorials/05_quantization.qmd b/mlsysim/docs/tutorials/05_quantization.qmd index 5e1f93db5b..4d6c6c80c8 100644 --- a/mlsysim/docs/tutorials/05_quantization.qmd +++ b/mlsysim/docs/tutorials/05_quantization.qmd @@ -64,8 +64,8 @@ Engine = mlsysim.Engine ```python import mlsysim -from mlsysim import ServingModel, SingleNodeModel -from mlsysim.engine.solver import CompressionModel +from mlsysim.solvers import ServingModel, SingleNodeModel +from mlsysim.solvers import CompressionModel ``` --- @@ -77,7 +77,7 @@ must reload the entire model from HBM. Fewer bytes per parameter means fewer byt load means lower inter-token latency: ```{python} -from mlsysim import ServingModel +from mlsysim.solvers import ServingModel from mlsysim.show import table, info model = mlsysim.Models.Language.Llama3_8B @@ -112,7 +112,7 @@ Now let's try the same optimization on a compute-bound workload — ResNet-50 tr at batch 256 on the A100: ```{python} -from mlsysim import SingleNodeModel +from mlsysim.solvers import SingleNodeModel train_model = mlsysim.Models.Vision.ResNet50 train_hw = mlsysim.Hardware.Cloud.A100 @@ -198,7 +198,7 @@ Quantization is not free — it trades accuracy for speed. The `CompressionModel this trade-off: ```{python} -from mlsysim.engine.solver import CompressionModel +from mlsysim.solvers import CompressionModel comp_solver = CompressionModel() diff --git a/mlsysim/docs/tutorials/06_scaling_1000_gpus.qmd b/mlsysim/docs/tutorials/06_scaling_1000_gpus.qmd index 938840e8a3..2fd680f2ae 100644 --- a/mlsysim/docs/tutorials/06_scaling_1000_gpus.qmd +++ b/mlsysim/docs/tutorials/06_scaling_1000_gpus.qmd @@ -72,7 +72,8 @@ Let's start with the simplest case: 8 GPUs in a single DGX node. No network fabric, no cross-node communication. This is our ceiling for scaling efficiency. ```{python} -from mlsysim import DistributedModel, Models, Systems +from mlsysim import Models, Systems +from mlsysim.solvers import DistributedModel from mlsysim.systems.types import Fleet, Node, NetworkFabric from mlsysim.core.constants import Q_ from mlsysim.show import table, info @@ -171,7 +172,7 @@ Let's now ask a different question: how often does a 512-GPU or 1024-GPU cluster experience a hardware failure? The `ReliabilityModel` models cluster-level MTBF. ```{python} -from mlsysim import ReliabilityModel +from mlsysim.solvers import ReliabilityModel rel_solver = ReliabilityModel() @@ -211,7 +212,7 @@ Now the key comparison. Over a 30-day training run, how much total time is lost to communication overhead vs. checkpoint overhead? ```{python} -from mlsysim.engine.solver import CheckpointModel +from mlsysim.solvers import CheckpointModel ckpt_solver = CheckpointModel() job_hours = 30 * 24 # 30 days diff --git a/mlsysim/docs/tutorials/07_geography.qmd b/mlsysim/docs/tutorials/07_geography.qmd index f93c1c0318..03dfb62d3a 100644 --- a/mlsysim/docs/tutorials/07_geography.qmd +++ b/mlsysim/docs/tutorials/07_geography.qmd @@ -69,7 +69,8 @@ Let's run the same training job in two locations: Quebec (hydroelectric) and Pol is where the electricity comes from. ```{python} -from mlsysim import SustainabilityModel, Systems +from mlsysim import Systems +from mlsysim.solvers import SustainabilityModel from mlsysim.systems.types import Fleet, Node, NetworkFabric from mlsysim.core.constants import Q_ from mlsysim.show import table, info @@ -200,7 +201,7 @@ taxes or cap-and-trade) changes the economics of datacenter location. Let's comp TCO with a carbon price of $50/tonne. ```{python} -from mlsysim import EconomicsModel +from mlsysim.solvers import EconomicsModel econ = EconomicsModel() carbon_price = 50 # USD per tonne CO2 diff --git a/mlsysim/docs/tutorials/08_nine_million_dollar.qmd b/mlsysim/docs/tutorials/08_nine_million_dollar.qmd index 6cd993aac7..03c1aa34c8 100644 --- a/mlsysim/docs/tutorials/08_nine_million_dollar.qmd +++ b/mlsysim/docs/tutorials/08_nine_million_dollar.qmd @@ -69,7 +69,8 @@ First, establish the baseline: a GPT-4 scale model served on an H100, no reasoni This gives us the per-query TTFT and ITL that everything else builds on. ```{python} -from mlsysim import Models, Hardware, ServingModel +from mlsysim import Models, Hardware +from mlsysim.solvers import ServingModel from mlsysim.show import table, info model = Models.Language.GPT4 @@ -101,7 +102,7 @@ Now sweep reasoning depth using the `InferenceScalingModel`. Each step generates tokens of intermediate reasoning. Watch how the cost multiplier grows. ```{python} -from mlsysim.engine.solver import InferenceScalingModel +from mlsysim.solvers import InferenceScalingModel cot_solver = InferenceScalingModel() K_values = [1, 4, 8, 16] @@ -142,7 +143,7 @@ Per-query cost is interesting. Fleet-level cost is what matters. Let's compute the annual infrastructure cost of serving 100 queries per second at K=1 vs. K=8. ```{python} -from mlsysim import EconomicsModel +from mlsysim.solvers import EconomicsModel from mlsysim.systems.types import Fleet, Node, NetworkFabric from mlsysim.core.constants import Q_ diff --git a/mlsysim/docs/tutorials/09_sensitivity.qmd b/mlsysim/docs/tutorials/09_sensitivity.qmd index ee53229f96..6ad9cb2424 100644 --- a/mlsysim/docs/tutorials/09_sensitivity.qmd +++ b/mlsysim/docs/tutorials/09_sensitivity.qmd @@ -53,8 +53,8 @@ import mlsysim ```python import mlsysim -from mlsysim import ServingModel -from mlsysim.engine.solver import SensitivitySolver, SynthesisSolver +from mlsysim.solvers import ServingModel +from mlsysim.solvers import SensitivitySolver, SynthesisSolver from mlsysim.core.constants import Q_ ``` @@ -66,8 +66,8 @@ We analyze **Llama-3.1-70B** inference on an **NVIDIA A100** --- a common deploy scenario where procurement decisions have real budget implications. ```{python} -from mlsysim import ServingModel -from mlsysim.engine.solver import SensitivitySolver, SynthesisSolver +from mlsysim.solvers import ServingModel +from mlsysim.solvers import SensitivitySolver, SynthesisSolver from mlsysim.core.constants import Q_ from mlsysim.show import table, info diff --git a/mlsysim/docs/tutorials/10_gpu_vs_wafer.qmd b/mlsysim/docs/tutorials/10_gpu_vs_wafer.qmd index 71778e599d..d6cdb69706 100644 --- a/mlsysim/docs/tutorials/10_gpu_vs_wafer.qmd +++ b/mlsysim/docs/tutorials/10_gpu_vs_wafer.qmd @@ -65,8 +65,8 @@ import mlsysim ```python import mlsysim -from mlsysim import SingleNodeModel -from mlsysim.engine.solver import WeightStreamingModel, SensitivitySolver +from mlsysim.solvers import SingleNodeModel +from mlsysim.solvers import WeightStreamingModel, SensitivitySolver ``` --- @@ -78,8 +78,8 @@ weights reach compute become the dominant factor. At batch size 1, each decode s reload the entire model from HBM. ```{python} -from mlsysim import SingleNodeModel -from mlsysim.engine.solver import WeightStreamingModel, SensitivitySolver +from mlsysim.solvers import SingleNodeModel +from mlsysim.solvers import WeightStreamingModel, SensitivitySolver from mlsysim.show import table, info, banner model = mlsysim.Models.Language.GPT3 diff --git a/mlsysim/docs/tutorials/11_training_memory_capacity_moe.qmd b/mlsysim/docs/tutorials/11_training_memory_capacity_moe.qmd index 145e1f79a2..379fc197f7 100644 --- a/mlsysim/docs/tutorials/11_training_memory_capacity_moe.qmd +++ b/mlsysim/docs/tutorials/11_training_memory_capacity_moe.qmd @@ -37,7 +37,8 @@ gradients, optimizer state, activations, and communication buffers. That is why model that fits for inference can fail during training. ```{python} -from mlsysim import Hardware, Models, TrainingMemoryModel +from mlsysim import Hardware, Models +from mlsysim.solvers import TrainingMemoryModel from mlsysim.show import table memory = TrainingMemoryModel().solve( @@ -79,7 +80,7 @@ A serving deployment is not sized by TTFT alone. You need base request latency, per-replica token capacity, and queueing pressure under load. ```{python} -from mlsysim import ServingCapacityModel +from mlsysim.solvers import ServingCapacityModel capacity = ServingCapacityModel().solve( model=Models.Language.Llama3_8B, @@ -119,7 +120,7 @@ the routed payload. ```{python} from mlsysim import Systems, ureg -from mlsysim.engine.solver import MoERoutingModel +from mlsysim.solvers import MoERoutingModel from mlsysim.models.types import SparseTransformerWorkload moe = SparseTransformerWorkload( diff --git a/mlsysim/docs/tutorials/12_design_space_exploration.qmd b/mlsysim/docs/tutorials/12_design_space_exploration.qmd index 3b4a3397f0..8495e0b682 100644 --- a/mlsysim/docs/tutorials/12_design_space_exploration.qmd +++ b/mlsysim/docs/tutorials/12_design_space_exploration.qmd @@ -27,7 +27,7 @@ Import the necessary classes. The `DSE` (Design Space Explorer) is our Tier 3 en ```python import mlsysim from mlsysim.engine.dse import DSE -from mlsysim.engine.solver import DistributedModel, EconomicsModel +from mlsysim.solvers import DistributedModel, EconomicsModel from mlsysim.engine.pipeline import Pipeline ``` diff --git a/mlsysim/docs/tutorials/12_full_stack_audit.qmd b/mlsysim/docs/tutorials/12_full_stack_audit.qmd index 6e81d34923..96c0e748c1 100644 --- a/mlsysim/docs/tutorials/12_full_stack_audit.qmd +++ b/mlsysim/docs/tutorials/12_full_stack_audit.qmd @@ -138,7 +138,7 @@ First, classify the per-GPU forward-backward pass. Is each GPU compute-bound or memory-bound during training? ```{python} -from mlsysim import SingleNodeModel +from mlsysim.solvers import SingleNodeModel node_solver = SingleNodeModel() node_result = node_solver.solve( @@ -168,7 +168,7 @@ preprocessing pipeline actually deliver data that fast? If not, the GPUs stall - "compute-bound" becomes a meaningless label. ```{python} -from mlsysim import DataModel +from mlsysim.solvers import DataModel # Estimate data demand per step: 4 samples/GPU * 512 GPUs * 2048 tokens * 2 bytes ≈ 8 MB/step # At ~1 step/sec, this is ~8 MB/s — tokenized text is compact @@ -203,7 +203,7 @@ Is our training budget compute-optimal? The Chinchilla scaling law says D = 20P (tokens = 20x parameters) for optimal allocation. ```{python} -from mlsysim.engine.solver import ScalingModel +from mlsysim.solvers import ScalingModel # MFU (Model FLOP Utilization): the fraction of peak hardware FLOP/s that goes # to useful model computation (excluding communication, idle time, overhead). @@ -243,7 +243,7 @@ The distributed solver models AllReduce overhead and pipeline bubbles. The reliability solver computes cluster MTBF and optimal checkpoint intervals. ```{python} -from mlsysim import DistributedModel, ReliabilityModel +from mlsysim.solvers import DistributedModel, ReliabilityModel # 3D parallelism: TP=8 (within node), PP=1, DP=64 dist_solver = DistributedModel() @@ -292,7 +292,7 @@ quantifies these costs. The economics solver combines CapEx, OpEx, and sustainability into a single financial model. ```{python} -from mlsysim import EconomicsModel, SustainabilityModel +from mlsysim.solvers import EconomicsModel, SustainabilityModel # 30-day training run econ_solver = EconomicsModel() @@ -338,7 +338,7 @@ infrastructure geography is a first-class engineering variable. Finally, confirm the binding constraint and derive minimum hardware for a 14-day completion target. ```{python} -from mlsysim.engine.solver import SensitivitySolver, SynthesisSolver +from mlsysim.solvers import SensitivitySolver, SynthesisSolver # Sensitivity: confirm compute is the binding constraint for training sens_solver = SensitivitySolver() diff --git a/mlsysim/docs/tutorials/distributed.qmd b/mlsysim/docs/tutorials/distributed.qmd index 02e7ce4817..7b8094e99f 100644 --- a/mlsysim/docs/tutorials/distributed.qmd +++ b/mlsysim/docs/tutorials/distributed.qmd @@ -48,11 +48,11 @@ import mlsysim ```python import mlsysim -from mlsysim import DistributedModel +from mlsysim.solvers import DistributedModel ``` ```{python} -from mlsysim import DistributedModel +from mlsysim.solvers import DistributedModel # Llama-3.1-70B: the model requires distributed training — too large for a single GPU model = mlsysim.Models.Language.Llama3_70B diff --git a/mlsysim/docs/zoo/datasets.qmd b/mlsysim/docs/zoo/datasets.qmd index 4d6b67ac60..b48e8193d1 100644 --- a/mlsysim/docs/zoo/datasets.qmd +++ b/mlsysim/docs/zoo/datasets.qmd @@ -31,5 +31,5 @@ cifar = mlsysim.Datasets.CIFAR10 mnist = mlsysim.Datasets.MNIST ``` -Pair dataset profiles with [`DataModel`](../api/engine.solver.DataModel.qmd) and the +Pair dataset profiles with [`DataModel`](../api/solvers.DataModel.qmd) and the *Data Engineering* textbook chapters when reasoning about ingestion bandwidth and epoch time. diff --git a/mlsysim/docs/zoo/infra.qmd b/mlsysim/docs/zoo/infra.qmd index 6e00aa3009..a131fc926b 100644 --- a/mlsysim/docs/zoo/infra.qmd +++ b/mlsysim/docs/zoo/infra.qmd @@ -88,4 +88,4 @@ The *Sustainable AI* chapter uses these grid profiles to quantify the carbon foo --- -*Note: For carbon and water usage formulas, see the [SustainabilityModel API Reference](../api/engine.solver.SustainabilityModel.qmd).* +*Note: For carbon and water usage formulas, see the [SustainabilityModel API Reference](../api/solvers.SustainabilityModel.qmd).* diff --git a/mlsysim/examples/03_heterogeneous_cluster.py b/mlsysim/examples/03_heterogeneous_cluster.py index 4eb9d34a6d..391f76ea38 100644 --- a/mlsysim/examples/03_heterogeneous_cluster.py +++ b/mlsysim/examples/03_heterogeneous_cluster.py @@ -6,7 +6,7 @@ distributed training performance + economics. from mlsysim.systems.types import Fleet, Node, NetworkFabric from mlsysim.hardware.registry import Hardware from mlsysim.models.registry import Models -from mlsysim.engine.solver import DistributedModel, EconomicsModel +from mlsysim.solvers import DistributedModel, EconomicsModel from mlsysim.core.constants import Q_ # 1. Define the Workload diff --git a/mlsysim/examples/06_multi_objective_pareto.py b/mlsysim/examples/06_multi_objective_pareto.py index 1d1c670aee..770200062f 100644 --- a/mlsysim/examples/06_multi_objective_pareto.py +++ b/mlsysim/examples/06_multi_objective_pareto.py @@ -13,7 +13,7 @@ single "best" design — you have a frontier of feasible operating points, and the SLA chooses one of them. """ import mlsysim -from mlsysim.engine.solver import ServingModel, TailLatencyModel +from mlsysim.solvers import ServingModel, TailLatencyModel def main(): diff --git a/mlsysim/mlsysim/cli/commands/optimize.py b/mlsysim/mlsysim/cli/commands/optimize.py index 5edf35bf51..e2d11ddd72 100644 --- a/mlsysim/mlsysim/cli/commands/optimize.py +++ b/mlsysim/mlsysim/cli/commands/optimize.py @@ -6,7 +6,7 @@ from mlsysim.cli.context import OUTPUT_FORMAT_HELP, resolve_output_format from mlsysim.cli.schemas import MlsysPlanSchema from mlsysim.cli.exceptions import ExitCode, exit_with_code, error_shield from mlsysim.cli.renderers import render_optimization -from mlsysim.engine.solver import ParallelismOptimizer, BatchingOptimizer, PlacementOptimizer +from mlsysim.solvers import ParallelismOptimizer, BatchingOptimizer, PlacementOptimizer optimize_app = typer.Typer( help="[Tier 3] Search the design space for optimal configurations.", diff --git a/mlsysim/mlsysim/cli/commands/serve.py b/mlsysim/mlsysim/cli/commands/serve.py index e15bbaf921..77145d1908 100644 --- a/mlsysim/mlsysim/cli/commands/serve.py +++ b/mlsysim/mlsysim/cli/commands/serve.py @@ -48,7 +48,7 @@ def serve_main( "Use 'mlsysim eval' for non-Transformer workloads." ) - from mlsysim.engine.solver import ServingModel + from mlsysim.solvers import ServingModel solver = ServingModel() result = solver.solve( model=model_obj, diff --git a/mlsysim/mlsysim/engine/pipeline.py b/mlsysim/mlsysim/engine/pipeline.py index ed5809441f..22c77bc53a 100644 --- a/mlsysim/mlsysim/engine/pipeline.py +++ b/mlsysim/mlsysim/engine/pipeline.py @@ -11,7 +11,7 @@ between each stage and `run()` to execute the chain. Example ------- >>> from mlsysim.engine.pipeline import Pipeline ->>> from mlsysim.engine.solver import ScalingModel, DistributedModel, EconomicsModel +>>> from mlsysim.solvers import ScalingModel, DistributedModel, EconomicsModel >>> pipe = Pipeline([ScalingModel(), DistributedModel(), EconomicsModel()]) >>> pipe.explain() # Shows the DAG and identifies gaps >>> result = pipe.run(compute_budget=Q_("1e21 FLOP"), fleet=cluster) diff --git a/mlsysim/mlsysim/solvers.py b/mlsysim/mlsysim/solvers.py index 284f1a096f..eb1b420603 100644 --- a/mlsysim/mlsysim/solvers.py +++ b/mlsysim/mlsysim/solvers.py @@ -6,8 +6,12 @@ Usage: """ from .engine.solver import ( + BaseOptimizer, + BaseResolver, + BaseSolver, ForwardModel, SingleNodeModel, + NetworkRooflineModel, DistributedModel, ReliabilityModel, SustainabilityModel, @@ -37,8 +41,12 @@ from .engine.solver import ( ) __all__ = [ + "BaseOptimizer", + "BaseResolver", + "BaseSolver", "ForwardModel", "SingleNodeModel", + "NetworkRooflineModel", "DistributedModel", "ReliabilityModel", "SustainabilityModel", diff --git a/mlsysim/paper/scripts/validate_anchors.py b/mlsysim/paper/scripts/validate_anchors.py index 52be5ac4d0..984f08a67e 100644 --- a/mlsysim/paper/scripts/validate_anchors.py +++ b/mlsysim/paper/scripts/validate_anchors.py @@ -18,7 +18,7 @@ sys.path.insert(0, str(project_root)) import mlsysim # noqa: E402 from mlsysim.core.constants import Q_ # noqa: E402 -from mlsysim.engine.solver import ( # noqa: E402 +from mlsysim.solvers import ( # noqa: E402 DistributedModel, ScalingModel, ParallelismOptimizer, diff --git a/mlsysim/tests/test_solver_module_exports.py b/mlsysim/tests/test_solver_module_exports.py index 443bfebcde..4db791bfb8 100644 --- a/mlsysim/tests/test_solver_module_exports.py +++ b/mlsysim/tests/test_solver_module_exports.py @@ -1,5 +1,6 @@ import mlsysim import mlsysim.ops as ops +import mlsysim.solvers as public_solvers from mlsysim.engine import solver from mlsysim.engine.solvers import ( BatchingOptimizer, @@ -74,6 +75,11 @@ def test_solver_implementations_live_in_domain_modules(): assert solver.CompressionModel.__module__.endswith(".solvers.compression") +def test_public_solver_module_exports_protocol_and_solver_classes(): + for name in solver.__all__: + assert getattr(public_solvers, name) is getattr(solver, name) + + def test_package_root_does_not_reexport_solver_aliases(): root_only = ( "SingleNodeModel", diff --git a/mlsysim/tutorial/cheatsheet.md b/mlsysim/tutorial/cheatsheet.md index c5730ea5a9..d25646d4dc 100644 --- a/mlsysim/tutorial/cheatsheet.md +++ b/mlsysim/tutorial/cheatsheet.md @@ -146,7 +146,7 @@ result = DistributedModel().solve( ```python from mlsysim import Hardware, Models -from mlsysim.engine.solver import CompressionModel +from mlsysim.solvers import CompressionModel result = CompressionModel().solve( model=Models.Language.Llama3_8B, diff --git a/mlsysim/tutorial/exercises.md b/mlsysim/tutorial/exercises.md index 464fd7d128..a07c2ce218 100644 --- a/mlsysim/tutorial/exercises.md +++ b/mlsysim/tutorial/exercises.md @@ -144,7 +144,7 @@ from INT4 quantization, and understand the accuracy trade-off. ```python from mlsysim import Engine, Hardware, Models -from mlsysim.engine.solver import CompressionModel +from mlsysim.solvers import CompressionModel compress = CompressionModel() model = Models.Language.Llama3_8B @@ -219,7 +219,7 @@ What is the optimal compression point? ```python from mlsysim import Models, Systems -from mlsysim.engine.solver import ParallelismOptimizer +from mlsysim.solvers import ParallelismOptimizer optimizer = ParallelismOptimizer() model = Models.Language.Llama3_70B @@ -584,7 +584,7 @@ from mlsysim import ( ServingModel, EconomicsModel, SustainabilityModel, Hardware, Models, Infrastructure ) -from mlsysim.engine.solver import CompressionModel +from mlsysim.solvers import CompressionModel from mlsysim.systems.types import Fleet from mlsysim.systems.registry import Nodes, Fabrics diff --git a/mlsysim/tutorial/slides/tutorial_module2.tex b/mlsysim/tutorial/slides/tutorial_module2.tex index 358c3dcee5..1df8cb99eb 100644 --- a/mlsysim/tutorial/slides/tutorial_module2.tex +++ b/mlsysim/tutorial/slides/tutorial_module2.tex @@ -163,7 +163,7 @@ print(f"Speedup: {base_result.itl / spec_result.itl:.2f}x") \begin{frame}[fragile]{Wall 12: Inference-Time Compute (The Reasoning Wall)} With models like OpenAI o1, compute scaling shifts from training to inference. The model generates $K$ hidden reasoning steps before answering. \begin{lstlisting}[language=Python] -from mlsysim.engine.solver import InferenceScalingModel +from mlsysim.solvers import InferenceScalingModel reasoning_solver = InferenceScalingModel() result = reasoning_solver.solve( diff --git a/mlsysim/tutorial/slides/tutorial_module4.tex b/mlsysim/tutorial/slides/tutorial_module4.tex index 7392ef8dbf..c17f85c97a 100644 --- a/mlsysim/tutorial/slides/tutorial_module4.tex +++ b/mlsysim/tutorial/slides/tutorial_module4.tex @@ -145,7 +145,7 @@ H100 SXM & 989\,T & 700\,W \\ \begin{frame}[fragile]{Wall 21: Sensitivity Analysis} \note{[3 min] ``Which knob should I turn next?'' The parameter with the largest partial derivative.} \begin{lstlisting} -from mlsysim.engine.solver import SensitivitySolver +from mlsysim.solvers import SensitivitySolver solver = SensitivitySolver() result = solver.solve(