Daniel Hiltgen and GitHub
db5266a71d
mlxrunner: batch speculative verification ( #19663 )
...
* mlxrunner: batch speculative verification
Keep drafting active for concurrent generation by sharing target verification while preserving per-request sampling and cache rollback. Learn draft depth for each concurrent workload separately from isolated requests, and preserve probes that short requests never execute.
Materialize owned target features before a reader can pause, and cover acceptance, cancellation, handoffs, mixed decoding and retained-memory boundaries.
* address comments
2026-11-16 08:43:54 -08:00
..
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-07 16:36:53 -08:00
2026-10-04 07:45:15 -07:00
2026-11-16 07:11:09 -08:00
2026-10-03 13:17:21 -07:00
2026-10-03 13:17:21 -07:00
2026-11-10 13:24:12 -08:00
2026-10-22 11:36:42 -07:00
2026-09-22 11:24:11 -07:00
2026-10-22 11:36:42 -07:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-10-03 13:17:21 -07:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-10-12 09:28:22 -07:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-06 13:39:21 -08:00
2026-11-10 13:24:12 -08:00
2026-11-16 08:43:54 -08:00
2026-11-16 08:43:54 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-16 08:43:54 -08:00
2026-11-16 08:43:54 -08:00
2026-11-16 08:43:54 -08:00
2026-11-16 08:43:54 -08:00
2026-10-26 16:04:43 -07:00
2026-11-16 08:43:54 -08:00
2026-11-10 13:24:12 -08:00
2026-11-10 13:24:12 -08:00
2026-11-07 16:36:53 -08:00