Fixes for running LFM3 GGUFs on the llama-server runner, plus GGUF
text-to-speech that turns on once llama.cpp is bumped past b11901.
- BOS: honor an explicit tokenizer.ggml.add_bos_token. LFM3 GGUFs (and
lfm2:24b) set it to false and expect the template's BOS, which the
lfm2 default was stripping. The lfm2 default still applies when the
key is absent. On lfm2:24b, WikiText PPL goes from 81.3 to 72.0.
- repeat_last_n -1 now maps to the context size; llama-server rejects
negative values.
- Audio-only projectors (LFM3-Audio) no longer advertise vision, so
clients don't send images the model can't take.
- /v1/audio/speech on GGUF models calls llama-server /tts chunk by
chunk. Renderers opt in through a new SpeechPrompter (voices,
voice parsing, chunking), so other speech models can add their own.
The LFM3 voice and chunking helpers move from the MLX runner into
model/renderers and are shared by both runners.
The MLX runner is the only Go inference runner left and is no longer
experimental, so its packages leave x/. The bindings become a top-level
mlx package beside the carried patches in mlx/compat, mirroring how
llama/ holds the llama.cpp integration, and the runner becomes mlxrunner
with the architectures nested under the package they implement.
Subpackages move with their parent unless listed.
x/mlxrunner/mlx mlx
x/internal/mlxthread mlx/mlxthread
x/internal/mlxthreadtest mlx/mlxthread/mlxthreadtest
x/internal/mlxtest mlx/mlxtest
x/quant mlx/quant
mlx/compat/*.patch mlx/compat/mlx-c (MLX patches go in mlx/compat/mlx)
x/mlxrunner mlxrunner
x/models/nn mlxrunner/nn
x/models/<arch> mlxrunner/model/<arch>
x/mlxrunner/imports.go mlxrunner/model/architectures (new package)
x/create create
x/safetensors fs/safetensors
x/tokenizer mlxrunner/tokenizer
Every package keeps its name, so the Go changes are the import path
rewrites the moves force, and the CMake, Dockerfile, CI cache keys, drift
check and Darwin payload script follow the new paths. Four edits are not
paths: the runner's blank architecture imports become the package
mlxrunner/model/architectures, so the list to extend for a new model sits
beside the architecture directories; a depguard rule keeps the two test
harnesses out of non-test code, as the x/internal placement used to; the
CI change filter's two entries for the long-deleted x/imagegen/mlx now
name the bindings' CMake project and the carried patches, so a change to
either builds the payload; and the tokenizer parity test reads its
fixtures from its own testdata instead of walking out of x/.
x/server and x/imagegen/manifest stay for the next two commits.
Daniel HiltgenandGitHub
Jesse Gross