cs249r_book/mlsysim/docs/api/solvers.ServingCapacityModel.qmd

# solvers.ServingCapacityModel { #mlsysim.solvers.ServingCapacityModel }

```python
solvers.ServingCapacityModel()
```

Sizes an LLM serving deployment from a QPS and tail-latency target.

The model deliberately composes existing first-order pieces: ``ServingModel``
for TTFT/ITL, ``ContinuousBatchingModel`` for per-replica token capacity,
and ``TailLatencyModel`` for queueing pressure. It is a capacity planner,
not a request-level scheduler.

## Methods

| Name | Description |
| --- | --- |
| [solve](#mlsysim.solvers.ServingCapacityModel.solve) | Return the minimum replica count that satisfies the target P99. |

### solve { #mlsysim.solvers.ServingCapacityModel.solve }

```python
solvers.ServingCapacityModel.solve(
    model,
    hardware,
    qps,
    target_p99_latency_ms,
    seq_len=2048,
    output_tokens=128,
    max_batch_size=32,
    precision='fp16',
    efficiency=0.5,
    max_replicas=1024,
    service_time_cv=1.0,
)
```

Return the minimum replica count that satisfies the target P99.