Replace the prerelease checkpoint path with Gradium's GA format, including default-voice import and speech conditioning. Set the 0.40.9 minimum, update the speech docs, and test import, inference setup, and rejection of the old checkpoint.
* mlxrunner: add Phonon text-to-speech support
Add Phonon text-to-speech through the existing MLX runner, with an
OpenAI-compatible speech endpoint and CLI playback.
Include model loading, text preprocessing, streaming audio, and
regression tests.
* docs: add speech models
* fix min version
---------
Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
Preserve source tool JSON alongside decoded API values so Flint can retain
numeric types and schema order in prompts. Existing API marshaling is unchanged.
Assemble ModelOpt expert banks during safetensors import, and reject direct
Flint NVFP4 creation before conversion. Other models retain their existing
import and quantization paths.
Add optional learned sinks and device causal masks to shared MLX attention.
Existing callers retain the no-sink and host-mask paths; Flint opts into the
new behavior.
* ci: shorten Windows Vulkan build path
Keep the nested shader-generator compiler probes below Windows path limits in checkouts with longer repository names.
* mlx: fix Nemotron bugs and support global scales
Accept published Nemotron layer names while retaining legacy aliases, and
fix layer-count inference and detached MTP quantization. Stabilize Mamba
softplus, preserve FP32 vision normalization, and compile Mamba gate and
norm scaling.
Support per-expert F32 global scales and preserve activation-scale
companions in ModelOpt and compressed-tensors NVFP4 imports. Normalize
reciprocal globals and reject invalid scales. Keep activation scales
separate from weight-only inference.
* mlx: add Nemotron 3.5 Super
* minor adjustments
* Add Kev decision scoring to MLX
Load Kev models with their FP32 decision head and release metadata.
Support System One scoring with shared-state caching and Kev-specific limits.
* address comments
Implements the EmbeddingGemma2Model architecture on the MLX runner: 24-layer
bidirectional text encoder with PLE, shared gemma4 vision/audio towers,
mean-pool + L2 output. /api/embed accepts per-item media via input dicts.
* mlxrunner: restore System One scoring
Carry the MLX implementation and its tests from the original System One
commit. The API/compiler and llama-server implementation already exist on
the release branch, so retain those versions. Adapt the optional model test
to the renamed decision package and its explicit rendering/context setup.
(cherry picked from commit 26e546ddb3)
* Enable MLX System One and add regression coverage
* mlxrunner: add Laya support for System One
Import Laya checkpoints and implement bidirectional decision scoring.
* mlxrunner: release cached buffers after System One scoring
* decision: match Tev1 and Nimble publisher prompt formats
Select Tev1 prompts using renderer metadata and fix JSON formatting, with regression coverage for GGUF and MLX.
* mlx: support Clef and Clef Flash decision models
Add model import, quantization safeguards, and text/image scoring with the Clef joint decision head.
* support vision capabilities for decision models
* Make Clef scoring cancellable between prefill chunks
Preserve joint-head inputs and test cancellation, cleanup, and recovery.
* mlx: separate decision preparation and readout from forward
Let serial and batched scoring share model-owned preparation and readout
without duplicating architecture-specific logic in the runner.
* mlx: add Strands Decider support
* implement caching
---------
Co-authored-by: ParthSareen <parth.sareen@ollama.com>
The MLX runner is the only Go inference runner left and is no longer
experimental, so its packages leave x/. The bindings become a top-level
mlx package beside the carried patches in mlx/compat, mirroring how
llama/ holds the llama.cpp integration, and the runner becomes mlxrunner
with the architectures nested under the package they implement.
Subpackages move with their parent unless listed.
x/mlxrunner/mlx mlx
x/internal/mlxthread mlx/mlxthread
x/internal/mlxthreadtest mlx/mlxthread/mlxthreadtest
x/internal/mlxtest mlx/mlxtest
x/quant mlx/quant
mlx/compat/*.patch mlx/compat/mlx-c (MLX patches go in mlx/compat/mlx)
x/mlxrunner mlxrunner
x/models/nn mlxrunner/nn
x/models/<arch> mlxrunner/model/<arch>
x/mlxrunner/imports.go mlxrunner/model/architectures (new package)
x/create create
x/safetensors fs/safetensors
x/tokenizer mlxrunner/tokenizer
Every package keeps its name, so the Go changes are the import path
rewrites the moves force, and the CMake, Dockerfile, CI cache keys, drift
check and Darwin payload script follow the new paths. Four edits are not
paths: the runner's blank architecture imports become the package
mlxrunner/model/architectures, so the list to extend for a new model sits
beside the architecture directories; a depguard rule keeps the two test
harnesses out of non-test code, as the x/internal placement used to; the
CI change filter's two entries for the long-deleted x/imagegen/mlx now
name the bindings' CMake project and the carried patches, so a change to
either builds the payload; and the tokenizer parity test reads its
fixtures from its own testdata instead of walking out of x/.
x/server and x/imagegen/manifest stay for the next two commits.
Daniel HiltgenandGitHub
Patrick DevineandGitHub
Jesse Gross