Jesse Gross 0f23b7bff5 mlxrunner: Cancel in-flight requests when the client disconnects ...

Currently, a canceled request can result in computation continuing
in the background to completion. It can also trigger a deadlock
when there is nobody to read the output tokens and the pipeline
cannot continue to the next request.

2026-02-25 14:00:42 -08:00

4.5 KiB

Raw Blame History

View Raw