Patrick Devine 15e6076d79 mlx: Gemma4 MTP speculative decoding (#15980) ...

This change adds support for MTP (multi-token prediction) speculative decoding for the
gemma4 model family.

It includes:
  * support for importing safetensors based gemma4 draft models with `ollama create`
  * a new DRAFT command in the Modelfile for specifying draft models
  * a --quantize-draft flag for the ollama create command to quantize the draft model
  * cache support for speculation
  * changes to the rotating cache to be able to handle MTP correctly
  * sampling support for draft model token prediction

---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>

2026-05-05 08:55:04 -07:00

..

agent

…

cmd

…

create

…

imagegen

…

internal/mlxthread

…

mlxrunner

…

models

…

safetensors

…

server

…

tokenizer

…

tools

…