Daniel HiltgenandGitHub a5b659e61b mlx: compact restored recurrent state after speculative rollback (#18955) ...

* mlx: compact restored recurrent state after speculative rollback

Per-token recurrent captures can leave a restored state pointing into a larger buffer. If a request ends before another model step, the live cache retains that buffer even though it accounts for only the selected state.

Track when a restored state may share that backing buffer. At request close, compact marked states in one batch before retaining them in the prefix cache. Clear the marker when a normal model step replaces the state, and cover restoration and buffer detachment in tests.

* address comments

2026-10-12 09:28:22 -07:00

..

cache_test.go

…

cache.go

mlx: System one support (#18701)

2026-10-03 13:17:21 -07:00

hidden_test.go

mlx: System one support (#18701)

2026-10-03 13:17:21 -07:00

hidden.go

mlx: System one support (#18701)

2026-10-03 13:17:21 -07:00

kvcache.go

…

lazy_test.go

…

recurrent_test.go

mlx: compact restored recurrent state after speculative rollback (#18955)

2026-10-12 09:28:22 -07:00

recurrent.go

mlx: compact restored recurrent state after speculative rollback (#18955)

2026-10-12 09:28:22 -07:00

rotating_attention_test.go

…

rotating_multiturn_test.go

…

rotating.go

…

snapshot_capture_test.go

…