* MLX, MLX-C: version bump
The MLX-C pin now provides its own bindings for gather_qmm global_scale
and fast::gated_delta_update, so those two carried patches drop out.
Carry 0001-mlx-c-sdpa-window-size instead: sliding-window attention
added a window_size argument to fast::scaled_dot_product_attention
(ml-explore/mlx#4734) that MLX-C has not bound yet, to be regenerated
from MLX-C once the upstream binding PR merges. The SDPA wrappers gain
a window size argument; all call sites default to the full context.
* mlxrunner: dispatch sliding-window prefills to the native SDPA window
Sliding layers no longer materialize a causal∩sliding mask for prefill
chunks past the window: resolve() dispatches {causal, window} natively
when the rotating cache's L>1 concat has linearized the ring and the
mask is otherwise pure causal, via a new MaskApplier.NativeWindow hint.
Vision relaxations, padding, model arrays, and decode keep the
materialized path. No model changes; sinks compose natively.
Preserve source tool JSON alongside decoded API values so Flint can retain
numeric types and schema order in prompts. Existing API marshaling is unchanged.
Assemble ModelOpt expert banks during safetensors import, and reject direct
Flint NVFP4 creation before conversion. Other models retain their existing
import and quantization paths.
Add optional learned sinks and device causal masks to shared MLX attention.
Existing callers retain the no-sink and host-mask paths; Flint opts into the
new behavior.
* Add Kev decision scoring to MLX
Load Kev models with their FP32 decision head and release metadata.
Support System One scoring with shared-state caching and Kev-specific limits.
* address comments
* mlx: speed up Qwen 3.8 prompt processing
Use MLX's gated-delta kernel for long scans and fold dense MLP global scales into SwiGLU.
* address comments
nn.go held every layer type in the package apart from attention,
recurrence and rope: the Linear and Embedding interfaces with their dense
and quantized types, Conv1d, RMSNorm, LayerNorm and MultiLinear, with one
test file to match. Finding a layer meant scanning the file named after
the package.
Each layer kind gets its own file: linear.go and embedding.go hold the
interface and the dense and quantized types, conv.go, norm.go and
multilinear.go take the rest, and nn_test.go splits the same way. The
Layer and MultiLinearLayer interfaces go; nothing implemented or accepted
them. No code changes otherwise.
The MLX runner is the only Go inference runner left and is no longer
experimental, so its packages leave x/. The bindings become a top-level
mlx package beside the carried patches in mlx/compat, mirroring how
llama/ holds the llama.cpp integration, and the runner becomes mlxrunner
with the architectures nested under the package they implement.
Subpackages move with their parent unless listed.
x/mlxrunner/mlx mlx
x/internal/mlxthread mlx/mlxthread
x/internal/mlxthreadtest mlx/mlxthread/mlxthreadtest
x/internal/mlxtest mlx/mlxtest
x/quant mlx/quant
mlx/compat/*.patch mlx/compat/mlx-c (MLX patches go in mlx/compat/mlx)
x/mlxrunner mlxrunner
x/models/nn mlxrunner/nn
x/models/<arch> mlxrunner/model/<arch>
x/mlxrunner/imports.go mlxrunner/model/architectures (new package)
x/create create
x/safetensors fs/safetensors
x/tokenizer mlxrunner/tokenizer
Every package keeps its name, so the Go changes are the import path
rewrites the moves force, and the CMake, Dockerfile, CI cache keys, drift
check and Darwin payload script follow the new paths. Four edits are not
paths: the runner's blank architecture imports become the package
mlxrunner/model/architectures, so the list to extend for a new model sits
beside the architecture directories; a depguard rule keeps the two test
harnesses out of non-test code, as the x/internal placement used to; the
CI change filter's two entries for the long-deleted x/imagegen/mlx now
name the bindings' CMake project and the carried patches, so a change to
either builds the payload; and the tokenizer parity test reads its
fixtures from its own testdata instead of walking out of x/.
x/server and x/imagegen/manifest stay for the next two commits.
Patrick DevineandGitHub
Daniel HiltgenandGitHub
Jesse Gross