mirror of
https://github.com/ollama/ollama.git
synced 2026-11-12 03:46:14 -06:00
Daniel HiltgenandGitHub
3fbc609be4
Fixes for running LFM3 GGUFs on the llama-server runner, plus GGUF text-to-speech that turns on once llama.cpp is bumped past b11901. - BOS: honor an explicit tokenizer.ggml.add_bos_token. LFM3 GGUFs (and lfm2:24b) set it to false and expect the template's BOS, which the lfm2 default was stripping. The lfm2 default still applies when the key is absent. On lfm2:24b, WikiText PPL goes from 81.3 to 72.0. - repeat_last_n -1 now maps to the context size; llama-server rejects negative values. - Audio-only projectors (LFM3-Audio) no longer advertise vision, so clients don't send images the model can't take. - /v1/audio/speech on GGUF models calls llama-server /tts chunk by chunk. Renderers opt in through a new SpeechPrompter (voices, voice parsing, chunking), so other speech models can add their own. The LFM3 voice and chunking helpers move from the MLX runner into model/renderers and are shared by both runners.
26 lines
748 B
Go
26 lines
748 B
Go
package llm
|
|
|
|
import (
|
|
"context"
|
|
|
|
"github.com/ollama/ollama/model/renderers"
|
|
)
|
|
|
|
// SpeechRequest asks a speech model to read Input aloud.
|
|
type SpeechRequest struct {
|
|
Input string `json:"input"`
|
|
Voice string `json:"voice,omitempty"`
|
|
// Prompter maps the request onto llama-server's /tts; runners with their
|
|
// own speech models ignore it.
|
|
Prompter renderers.SpeechPrompter `json:"-"`
|
|
}
|
|
|
|
// SpeechSampleRate is the rate of the PCM a Speaker streams.
|
|
const SpeechSampleRate = 24000
|
|
|
|
// Speaker is implemented by runners that serve speech models. Speech streams
|
|
// SpeechSampleRate mono 16-bit little-endian PCM to fn as it is generated.
|
|
type Speaker interface {
|
|
Speech(ctx context.Context, req SpeechRequest, fn func(pcm []byte) error) error
|
|
}
|