github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15952] Anthropic compat endpoint: no way to invoke 'max' thinking mode (e.g. on deepseek-v4-pro) #87846

New Issue
Open
opened 2026-05-10 06:26:51 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-10 06:26:51 -05:00
Owner
Copy Link

Originally created by @ankitvgupta on GitHub (May 4, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15952

Summary

The Anthropic-compatible endpoint at /v1/messages only exposes binary thinking control. There's no way to request the higher-effort modes (high, max) that the native /api/chat endpoint supports for reasoning models like deepseek-v4-pro:cloud.

Current Behavior

In anthropic/anthropic.go:375-378:

var think *api.ThinkValue
if r.Thinking != nil && r.Thinking.Type == "enabled" {
    think = &api.ThinkValue{Value: true}
}

ThinkValue.Value is hardcoded to true, so the request is always equivalent to standard "Thinking" mode regardless of what the client sends. There is no path to "max".

Empirical Evidence

Same prompt (deepseek-v4-pro:cloud, math problem requiring reasoning), comparing endpoints:

Native /api/chat — think parameter is honored:

think value thinking chars produced
false 0
true 1338
"low" 1245
"high" 1122
"max" 2199

Anthropic-compat /v1/messages — n=6 distributions (mean):

Setting Mean thinking chars
baseline 1301
options: {think: "max"} 1502
options: {think: "low"} 1379
options: {think: false} 1659

The /v1/messages distributions are statistically indistinguishable — options.think: false should produce 0 chars if respected, but produces 1659 (higher than baseline). The compat layer drops the field entirely; Go's silent unknown-field handling means there's no error either.

Why This Matters

Tools like Claude Code (and our app's Claude Agent SDK integration) communicate with Ollama Cloud exclusively through the Anthropic-compat endpoint via ANTHROPIC_BASE_URL. They have no way to construct native /api/chat requests. As a result, those tools cannot access max-thinking on capable models even though the underlying inference engine supports it. This caps the quality available to Anthropic-protocol clients well below what direct CLI users can achieve.

This is distinct from #15287 (which is about correctly handling "thinking.type": "disabled" — the absence/disabled case). This issue is about needing more granular enabled levels.

It's also distinct from #15831 (which proposes "max" for the top-level think on the native API) — that's a different endpoint.

Proposed Fix

Add a non-standard reasoning_effort extension at the top level of the Anthropic compat request, mirroring what already works on the OpenAI-compat endpoint (openai/openai.go accepts reasoning_effort/reasoning.effort with values "low" | "medium" | "high" | "max" | "none").

There's precedent for adding Ollama-specific extensions to /v1/messages — see #14891, which proposes num_ctx for the same endpoint.

Sketched change in FromMessagesRequest:

type MessagesRequest struct {
    // ... existing fields ...
    ReasoningEffort *string `json:"reasoning_effort,omitempty"`
}

// In FromMessagesRequest:
var think *api.ThinkValue
if r.Thinking != nil && r.Thinking.Type == "enabled" {
    think = &api.ThinkValue{Value: true}
    // explicit override beats the Anthropic-spec on/off
    if r.ReasoningEffort != nil {
        effort := *r.ReasoningEffort
        if !slices.Contains([]string{"high", "medium", "low", "max", "none"}, effort) {
            return nil, fmt.Errorf("invalid reasoning_effort: %q", effort)
        }
        if effort == "none" {
            think = &api.ThinkValue{Value: false}
        } else {
            think = &api.ThinkValue{Value: effort}
        }
    }
}

This preserves the Anthropic spec's budget_tokens semantics untouched (it's an upper-bound budget, not a level signal — using it for level mapping would conflict with its actual meaning when other clients eventually pass it through).

Reproduction

# Native /api/chat — works
curl https://ollama.com/api/chat -H "Authorization: Bearer $KEY" \
  -d '{"model":"deepseek-v4-pro:cloud","messages":[{"role":"user","content":"hard problem"}],"stream":false,"think":"max"}'
# response.message.thinking is ~2200 chars

# Anthropic compat /v1/messages — no way to invoke "max"
curl https://ollama.com/v1/messages -H "Authorization: Bearer $KEY" -H "anthropic-version: 2023-06-01" \
  -d '{"model":"deepseek-v4-pro:cloud","max_tokens":8192,"messages":[{"role":"user","content":"hard problem"}],"reasoning_effort":"max"}'
# Currently: reasoning_effort dropped silently, thinking is at default depth

Happy to put up a PR if there's interest in the proposed shape.

Originally created by @ankitvgupta on GitHub (May 4, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15952 ## Summary The Anthropic-compatible endpoint at `/v1/messages` only exposes binary thinking control. There's no way to request the higher-effort modes (`high`, `max`) that the native `/api/chat` endpoint supports for reasoning models like `deepseek-v4-pro:cloud`. ## Current Behavior In [`anthropic/anthropic.go:375-378`](https://github.com/ollama/ollama/blob/main/anthropic/anthropic.go#L375-L378): ```go var think *api.ThinkValue if r.Thinking != nil && r.Thinking.Type == "enabled" { think = &api.ThinkValue{Value: true} } ``` `ThinkValue.Value` is hardcoded to `true`, so the request is always equivalent to standard "Thinking" mode regardless of what the client sends. There is no path to `"max"`. ## Empirical Evidence Same prompt (`deepseek-v4-pro:cloud`, math problem requiring reasoning), comparing endpoints: **Native `/api/chat`** — `think` parameter is honored: | `think` value | thinking chars produced | |---|---| | `false` | 0 | | `true` | 1338 | | `"low"` | 1245 | | `"high"` | 1122 | | **`"max"`** | **2199** | **Anthropic-compat `/v1/messages`** — n=6 distributions (mean): | Setting | Mean thinking chars | |---|---| | baseline | 1301 | | `options: {think: "max"}` | 1502 | | `options: {think: "low"}` | 1379 | | `options: {think: false}` | 1659 | The `/v1/messages` distributions are statistically indistinguishable — `options.think: false` should produce 0 chars if respected, but produces 1659 (higher than baseline). The compat layer drops the field entirely; Go's silent unknown-field handling means there's no error either. ## Why This Matters Tools like Claude Code (and our app's Claude Agent SDK integration) communicate with Ollama Cloud exclusively through the Anthropic-compat endpoint via `ANTHROPIC_BASE_URL`. They have no way to construct native `/api/chat` requests. As a result, those tools cannot access max-thinking on capable models even though the underlying inference engine supports it. This caps the quality available to Anthropic-protocol clients well below what direct CLI users can achieve. This is distinct from #15287 (which is about correctly handling `"thinking.type": "disabled"` — the absence/disabled case). This issue is about needing more granular **enabled** levels. It's also distinct from #15831 (which proposes `"max"` for the top-level `think` on the native API) — that's a different endpoint. ## Proposed Fix Add a non-standard `reasoning_effort` extension at the top level of the Anthropic compat request, mirroring what already works on the OpenAI-compat endpoint (`openai/openai.go` accepts `reasoning_effort`/`reasoning.effort` with values `"low" | "medium" | "high" | "max" | "none"`). There's precedent for adding Ollama-specific extensions to `/v1/messages` — see [#14891](https://github.com/ollama/ollama/pull/14891), which proposes `num_ctx` for the same endpoint. Sketched change in `FromMessagesRequest`: ```go type MessagesRequest struct { // ... existing fields ... ReasoningEffort *string `json:"reasoning_effort,omitempty"` } // In FromMessagesRequest: var think *api.ThinkValue if r.Thinking != nil && r.Thinking.Type == "enabled" { think = &api.ThinkValue{Value: true} // explicit override beats the Anthropic-spec on/off if r.ReasoningEffort != nil { effort := *r.ReasoningEffort if !slices.Contains([]string{"high", "medium", "low", "max", "none"}, effort) { return nil, fmt.Errorf("invalid reasoning_effort: %q", effort) } if effort == "none" { think = &api.ThinkValue{Value: false} } else { think = &api.ThinkValue{Value: effort} } } } ``` This preserves the Anthropic spec's `budget_tokens` semantics untouched (it's an upper-bound budget, not a level signal — using it for level mapping would conflict with its actual meaning when other clients eventually pass it through). ## Reproduction ```bash # Native /api/chat — works curl https://ollama.com/api/chat -H "Authorization: Bearer $KEY" \ -d '{"model":"deepseek-v4-pro:cloud","messages":[{"role":"user","content":"hard problem"}],"stream":false,"think":"max"}' # response.message.thinking is ~2200 chars # Anthropic compat /v1/messages — no way to invoke "max" curl https://ollama.com/v1/messages -H "Authorization: Bearer $KEY" -H "anthropic-version: 2023-06-01" \ -d '{"model":"deepseek-v4-pro:cloud","max_tokens":8192,"messages":[{"role":"user","content":"hard problem"}],"reasoning_effort":"max"}' # Currently: reasoning_effort dropped silently, thinking is at default depth ``` Happy to put up a PR if there's interest in the proposed shape.
GiteaMirror added the feature request label 2026-05-10 06:26:51 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87846
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?