# solvers.BatchingOptimizer { #mlsysim.solvers.BatchingOptimizer }

```python
solvers.BatchingOptimizer()
```

Finds the maximum batch size that satisfies a P99 latency SLA.

Searches the continuous batching design space using an M/M/c queueing
model to find the optimal balance between throughput and tail latency.

## Methods

| Name | Description |
| --- | --- |
| [solve](#mlsysim.solvers.BatchingOptimizer.solve) | Determines the maximum batch size that satisfies a P99 tail latency SLA. |

### solve { #mlsysim.solvers.BatchingOptimizer.solve }

```python
solvers.BatchingOptimizer.solve(
    model,
    hardware,
    seq_len,
    sla_latency_ms,
    arrival_rate_qps,
    num_replicas=1,
    precision='fp16',
    efficiency=0.5,
    max_search_batch=256,
)
```

Determines the maximum batch size that satisfies a P99 tail latency SLA.