* MLX: reduce Qwen3.8 Flash Next memory use
For NVFP4 creation, retain source precision for sensitive QSA projections
and non-expert MTP weights. Keep routed experts and PLE at NVFP4, and
promote other eligible weights to MXFP8.
Keep NVFP4 PLE tables host-backed, transferring only selected packed rows
and their scales to MLX. Honor scalar and per-expert NVFP4 global scales
in MoE, and per-shard global scales in host-backed PLE.
Use compressed QSA block selections in fused indexed Metal attention for
long prefill batches, retaining the graph fallback. Co-locate component
loading and forward methods, with explicit quantization and cache ownership.
* mlx: skip materializing host-backed tensors
Let models exclude tensors already owned by host storage from MLX loading.
Avoid allocating and discarding the full Qwen PLE tables at startup while
preserving the resident fallback.
* address comments
DFlash2 adds block-local dynamic convolutions and a candidate selector
to DFlash. The selector proposes each token conditioned on its
predecessor and returns each position's candidate distribution, which
verification uses to keep the output exact. Like the reference, the
draft samples with the request's temperature only.
A DFlash2 session drafts seven tokens every round on the existing
snapshot verify path instead of following the depth controller, whose
shallower picks lose more accepted tokens than they save in verify time.
qwen3.5-family targets now expose the layer taps, token embeddings and
raw logits a DFlash draft needs.
model is one package with three jobs: the contract between the runner
and the architectures, the opened checkpoint, and building nn layers
from checkpoint tensors. Its files did not say which was which. base.go
carried the folded package's name over the interfaces and the registry,
root.go held the safetensors header scan next to Root, and quant.go
mixed the nvfp4 global-scale helpers with quant parameter resolution.
base.go becomes model.go, named for what it holds. root.go keeps Root
and Open; TensorQuantInfo and the header scan join quant.go, so
everything the checkpoint says about quantization is read and resolved
in one file. The global-scale helpers move to globalscale.go with their
tests. Root.Close, a no-op with one caller, goes. No code changes
otherwise.
Daniel HiltgenandGitHub
Jesse Gross