github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15645] v1/messages :cloud — Kimi K2.5 native tokens leak, nested objects stringified on Qwen/GLM #87669

New Issue
Open
opened 2026-05-10 06:13:46 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-10 06:13:46 -05:00
Owner
Copy Link

Originally created by @john-savepoint on GitHub (Apr 17, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15645

Ollama :cloud Anthropic-compat: Kimi K2.5 native tokens leak, nested objects stringified on Qwen/GLM

Summary

Using Ollama 0.20.7 as an Anthropic Messages API shim (/v1/messages endpoint) with :cloud models, two related bugs break multi-agent tool-calling workflows (Claude Code agent teams):

  1. Kimi K2.5 native tool-call tokens pass through as plain text. The model emits <|tool_call_begin|>functions.X:0<|tool_call_argument_begin|>{…}<|tool_call_end|><|tool_calls_section_end|> in what becomes an assistant text block instead of a properly-formed tool_use block. Subsequent requests replay the transcript and the Anthropic validator rejects with "tool_use block missing required 'name' field".

  2. Nested JSON objects in tool-call parameters get serialized as strings on Qwen3.x, GLM-5/5.1 (and others). The model emits {"message": {"type": "shutdown_response", …}} but the response arrives with "message": "{\"type\":\"shutdown_response\",…}" (a JSON string). Anthropic's validator then requires summary because typeof message === 'string', and any downstream dispatch keyed on message.type (structured protocol messages) never fires.

Both failure modes break Claude Code's SendMessage tool and the shutdown_request / shutdown_response lifecycle for in-process teammates.

Environment

  • Ollama: 0.20.7
  • Backend: Ollama Cloud (:cloud model variants — remote_host: https://ollama.com:443)
  • Client: Claude Code 2.1.110 with ANTHROPIC_BASE_URL=http://localhost:11434 and ANTHROPIC_AUTH_TOKEN=ollama
  • Platform: WSL2 (Linux 6.6.87.2-microsoft-standard-WSL2)

Affected models (confirmed via direct testing)

Model Plain-text SendMessage Structured shutdown_response Failure mode
kimi-k2.5:cloud partial ❌ Native <|tool_call_begin|>…<|tool_call_end|> tokens leak as raw text
qwen3-coder-next:cloud ✅ ❌ Nested object → JSON string
qwen3.5:cloud, qwen3.5:397b-cloud ✅ ❌ Nested object → JSON string
glm-5.1:cloud ✅ ❌ Nested object → JSON string
minimax-m2.7:cloud ✅ ✅ Works correctly (control)
deepseek-v3.1:671b-cloud ✅ ✅ (inferred, has Ollama parser) Works correctly

Reproduction

Case 1: Kimi K2.5 native tokens

curl -sS -X POST http://127.0.0.1:11434/v1/messages \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  -d '{
    "model": "kimi-k2.5:cloud",
    "max_tokens": 200,
    "tools": [{"name":"get_weather","description":"Get weather","input_schema":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}],
    "messages": [{"role":"user","content":"what is the weather in tokyo"}]
  }'

Observed (when Kimi decides to tool-call): an assistant content block of type text containing:

<|im_assistant|><think>…</think>

<tool_call_begin>functions.get_weather:0<|tool_call_argument_begin>{"city":"tokyo"}<|tool_call_end|><|tool_calls_section_end|>

Expected: a proper Anthropic tool_use block:

{
  "type": "tool_use",
  "id": "toolu_…",
  "name": "get_weather",
  "input": {"city": "tokyo"}
}

When the client replays this transcript in the next request, Ollama's own Anthropic-compat validator at middleware/anthropic.go:447 rejects with:

anthropic: tool_use block missing name … 
error: "tool_use block missing required 'name' field"

Case 2: Qwen3-coder-next nested-object stringification

curl -sS -X POST http://127.0.0.1:11434/v1/messages \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  -d '{
    "model": "qwen3-coder-next:cloud",
    "max_tokens": 500,
    "tools": [{"name":"send_structured","description":"Send a structured message","input_schema":{"type":"object","properties":{"to":{"type":"string"},"message":{"oneOf":[{"type":"string"},{"type":"object"}]}},"required":["to","message"]}}],
    "messages": [{"role":"user","content":"call send_structured with to=bob and message={\"type\":\"ping\",\"id\":1}"}]
  }'

Observed: the returned tool_use block has:

{"input": {"to": "bob", "message": "{\"type\":\"ping\",\"id\":1}"}}

(message is a string, not an object.)

Expected:

{"input": {"to": "bob", "message": {"type": "ping", "id": 1}}}

Source-code analysis

Gap 1: No Kimi K2 parser

model/parsers/ in Ollama 0.20.7 contains parsers for: cogito, deepseek3, functiongemma, gemma4, glm46, glm47, glmocr, lfm2, ministral, nemotron3nano, olmo3, olmo3_think, qwen3, qwen35, qwen3coder.

No kimi.go exists. parsers.go has no case "kimi" branch. Grep for "kimi" / "K2" across src/ only returns unrelated launch-helper tests (cline, opencode, openclaw).

Kimi K2's native tool-call tokens (<|im_assistant|>, <|tool_call_begin|>, <|tool_call_argument_begin|>, <|tool_call_end|>, <|tool_calls_section_end|>, functions.NAME:N invocation syntax) are not recognized by any existing parser.

Gap 2: :cloud passthrough bypasses local parsers

server/cloud_proxy.go:73 (cloudPassthroughMiddleware) detects modelRef.Source == modelSourceCloud, normalizes the model name, and calls proxyCloudRequest(...) which forwards the request raw to ollama.com. The response is returned as-is without any local parser running. Even if a Kimi parser existed locally in model/parsers/, it would not fire for :cloud models.

This means the fix must land on the cloud backend (ollama.com), not just the local binary.

Gap 3: Nested-object handling in Anthropic-compat shim

The Anthropic-compat layer at anthropic/anthropic.go + middleware/anthropic.go appears to lose type fidelity when translating tool-call arguments from the internal api.ToolCall shape to Anthropic tool_use blocks for models that emit JSON-string arguments. The input field should be an object, but when the model's native tool-call format represents arguments as a string (as Qwen/GLM do via some routers), that string is embedded directly instead of being JSON-parsed first.

middleware/anthropic.go:440-448 does enforce that outgoing tool_use blocks have a name field, which catches some malformations, but doesn't re-parse stringified JSON objects in input values.

Impact on Claude Code agent teams

Claude Code's SendMessage tool (src/tools/SendMessageTool/SendMessageTool.ts) has a Zod discriminated union on message:

  • String → plain-text coordination, requires summary
  • Object with type: "shutdown_request" | "shutdown_response" | "plan_approval_response" → structured protocol messages, no summary

handleShutdownApproval at line 305 fires task.abortController.abort() only when message.type === "shutdown_response" and message is an object. If the router stringifies the object, validateInput at line 667-674 demands summary, and even if the client supplies one, message.type is undefined (because message is a string), so the abort never fires. The in-process teammate becomes a zombie.

Real observed sessions: agent-a3300a9cf343c54a5.jsonl (kimi-k2.5), bc0d05bd-095f-4037-ad9b-4a0b9e663264 (qwen3-coder-next), and several others in the same pattern. Teammates accumulate 40+ summary is required when message is a string errors in infinite retry loops because their shutdown path is broken at the router layer.

Proposed fixes

  1. Kimi K2/K2.5 parser (model/parsers/kimi.go) that recognizes Kimi's native tool-call token sequence (<|tool_call_begin|>functions.NAME:N<|tool_call_argument_begin|>JSON<|tool_call_end|><|tool_calls_section_end|>) and emits api.ToolCall objects. This must be deployed both locally and on the cloud backend since :cloud bypasses local parsers.

  2. Nested-object preservation in Anthropic-compat translation. When a model's native tool-call argument representation is a JSON string, the Anthropic-compat shim should json.Unmarshal it and embed the resulting object as input rather than embedding the string literally. This affects Qwen3.x, GLM-5.1, and likely others.

  3. Validation / logging on the cloud passthrough for unrecognized token sequences in text blocks — early rejection with a clear error beats a validation failure two turns later with a confusing "missing name" message.

Why this is worth fixing

Ollama Cloud is positioned as a drop-in Anthropic-compat endpoint (the /v1/messages path exists precisely for this). Third-party multi-agent frameworks like Claude Code's agent teams (CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1) rely on structured tool-call JSON to implement peer-to-peer coordination, plan approvals, and graceful shutdown. Without the fix, any workflow that relies on nested objects in tool inputs breaks on Kimi/Qwen/GLM — which are some of the most interesting models in the Ollama Cloud catalog.

Workarounds (client-side)

Until fixed, Claude Code users can:

  • Switch teammate models to minimax-m2.7:cloud or deepseek-v3.1:671b-cloud (confirmed working).
  • Use TaskStop with the internal agentId instead of SendMessage shutdown_request for teardown — bypasses the nested-object path entirely.
  • Install a PostToolUseFailure hook that detects stringified JSON in failed SendMessage calls and coaches the model to switch approaches.

None of these are substitutes for fixing the shim.

Attachments / references

  • Internal agent ID format regex (for those investigating task_id / agentId routing): src/types/ids.ts:35 — /^a(?:.+-)?[0-9a-f]{16}$/
  • SendMessage validation logic: src/tools/SendMessageTool/SendMessageTool.ts:667-674
  • Deferred-tool gate: src/tools/ToolSearchTool/prompt.ts:62
  • Claude Code agent-teams feature flag: CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1

Happy to provide raw .jsonl transcripts from the failing sessions if helpful.

Originally created by @john-savepoint on GitHub (Apr 17, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15645 # Ollama `:cloud` Anthropic-compat: Kimi K2.5 native tokens leak, nested objects stringified on Qwen/GLM ## Summary Using Ollama 0.20.7 as an Anthropic Messages API shim (`/v1/messages` endpoint) with `:cloud` models, two related bugs break multi-agent tool-calling workflows (Claude Code agent teams): 1. **Kimi K2.5 native tool-call tokens pass through as plain text.** The model emits `<|tool_call_begin|>functions.X:0<|tool_call_argument_begin|>{…}<|tool_call_end|><|tool_calls_section_end|>` in what becomes an assistant `text` block instead of a properly-formed `tool_use` block. Subsequent requests replay the transcript and the Anthropic validator rejects with `"tool_use block missing required 'name' field"`. 2. **Nested JSON objects in tool-call parameters get serialized as strings on Qwen3.x, GLM-5/5.1 (and others).** The model emits `{"message": {"type": "shutdown_response", …}}` but the response arrives with `"message": "{\"type\":\"shutdown_response\",…}"` (a JSON string). Anthropic's validator then requires `summary` because `typeof message === 'string'`, and any downstream dispatch keyed on `message.type` (structured protocol messages) never fires. Both failure modes break Claude Code's `SendMessage` tool and the `shutdown_request` / `shutdown_response` lifecycle for in-process teammates. ## Environment - Ollama: `0.20.7` - Backend: Ollama Cloud (`:cloud` model variants — `remote_host: https://ollama.com:443`) - Client: Claude Code `2.1.110` with `ANTHROPIC_BASE_URL=http://localhost:11434` and `ANTHROPIC_AUTH_TOKEN=ollama` - Platform: WSL2 (Linux 6.6.87.2-microsoft-standard-WSL2) ## Affected models (confirmed via direct testing) | Model | Plain-text `SendMessage` | Structured shutdown_response | Failure mode | |---|---|---|---| | `kimi-k2.5:cloud` | partial | ❌ | Native `<\|tool_call_begin\|>…<\|tool_call_end\|>` tokens leak as raw text | | `qwen3-coder-next:cloud` | ✅ | ❌ | Nested object → JSON string | | `qwen3.5:cloud`, `qwen3.5:397b-cloud` | ✅ | ❌ | Nested object → JSON string | | `glm-5.1:cloud` | ✅ | ❌ | Nested object → JSON string | | `minimax-m2.7:cloud` | ✅ | ✅ | Works correctly (control) | | `deepseek-v3.1:671b-cloud` | ✅ | ✅ (inferred, has Ollama parser) | Works correctly | ## Reproduction ### Case 1: Kimi K2.5 native tokens ```bash curl -sS -X POST http://127.0.0.1:11434/v1/messages \ -H 'Content-Type: application/json' \ -H 'anthropic-version: 2023-06-01' \ -d '{ "model": "kimi-k2.5:cloud", "max_tokens": 200, "tools": [{"name":"get_weather","description":"Get weather","input_schema":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}], "messages": [{"role":"user","content":"what is the weather in tokyo"}] }' ``` **Observed** (when Kimi decides to tool-call): an assistant `content` block of type `text` containing: ``` <|im_assistant|><think>…</think> <tool_call_begin>functions.get_weather:0<|tool_call_argument_begin>{"city":"tokyo"}<|tool_call_end|><|tool_calls_section_end|> ``` **Expected:** a proper Anthropic `tool_use` block: ```json { "type": "tool_use", "id": "toolu_…", "name": "get_weather", "input": {"city": "tokyo"} } ``` When the client replays this transcript in the next request, Ollama's own Anthropic-compat validator at `middleware/anthropic.go:447` rejects with: ``` anthropic: tool_use block missing name … error: "tool_use block missing required 'name' field" ``` ### Case 2: Qwen3-coder-next nested-object stringification ```bash curl -sS -X POST http://127.0.0.1:11434/v1/messages \ -H 'Content-Type: application/json' \ -H 'anthropic-version: 2023-06-01' \ -d '{ "model": "qwen3-coder-next:cloud", "max_tokens": 500, "tools": [{"name":"send_structured","description":"Send a structured message","input_schema":{"type":"object","properties":{"to":{"type":"string"},"message":{"oneOf":[{"type":"string"},{"type":"object"}]}},"required":["to","message"]}}], "messages": [{"role":"user","content":"call send_structured with to=bob and message={\"type\":\"ping\",\"id\":1}"}] }' ``` **Observed**: the returned `tool_use` block has: ```json {"input": {"to": "bob", "message": "{\"type\":\"ping\",\"id\":1}"}} ``` (`message` is a string, not an object.) **Expected**: ```json {"input": {"to": "bob", "message": {"type": "ping", "id": 1}}} ``` ## Source-code analysis ### Gap 1: No Kimi K2 parser `model/parsers/` in Ollama 0.20.7 contains parsers for: cogito, deepseek3, functiongemma, gemma4, glm46, glm47, glmocr, lfm2, ministral, nemotron3nano, olmo3, olmo3_think, qwen3, qwen35, qwen3coder. **No `kimi.go` exists.** `parsers.go` has no `case "kimi"` branch. Grep for "kimi" / "K2" across `src/` only returns unrelated launch-helper tests (cline, opencode, openclaw). Kimi K2's native tool-call tokens (`<|im_assistant|>`, `<|tool_call_begin|>`, `<|tool_call_argument_begin|>`, `<|tool_call_end|>`, `<|tool_calls_section_end|>`, `functions.NAME:N` invocation syntax) are not recognized by any existing parser. ### Gap 2: `:cloud` passthrough bypasses local parsers `server/cloud_proxy.go:73` (`cloudPassthroughMiddleware`) detects `modelRef.Source == modelSourceCloud`, normalizes the model name, and calls `proxyCloudRequest(...)` which forwards the request raw to `ollama.com`. The response is returned as-is without any local parser running. Even if a Kimi parser existed locally in `model/parsers/`, it would not fire for `:cloud` models. This means the fix must land on the cloud backend (`ollama.com`), not just the local binary. ### Gap 3: Nested-object handling in Anthropic-compat shim The Anthropic-compat layer at `anthropic/anthropic.go` + `middleware/anthropic.go` appears to lose type fidelity when translating tool-call arguments from the internal `api.ToolCall` shape to Anthropic `tool_use` blocks for models that emit JSON-string arguments. The `input` field should be an object, but when the model's native tool-call format represents arguments as a string (as Qwen/GLM do via some routers), that string is embedded directly instead of being JSON-parsed first. `middleware/anthropic.go:440-448` does enforce that outgoing `tool_use` blocks have a `name` field, which catches some malformations, but doesn't re-parse stringified JSON objects in `input` values. ## Impact on Claude Code agent teams Claude Code's `SendMessage` tool (`src/tools/SendMessageTool/SendMessageTool.ts`) has a Zod discriminated union on `message`: - String → plain-text coordination, requires `summary` - Object with `type: "shutdown_request" | "shutdown_response" | "plan_approval_response"` → structured protocol messages, no `summary` `handleShutdownApproval` at line 305 fires `task.abortController.abort()` **only** when `message.type === "shutdown_response"` and `message` is an object. If the router stringifies the object, `validateInput` at line 667-674 demands `summary`, and even if the client supplies one, `message.type` is `undefined` (because `message` is a string), so the abort never fires. The in-process teammate becomes a zombie. Real observed sessions: `agent-a3300a9cf343c54a5.jsonl` (kimi-k2.5), `bc0d05bd-095f-4037-ad9b-4a0b9e663264` (qwen3-coder-next), and several others in the same pattern. Teammates accumulate 40+ `summary is required when message is a string` errors in infinite retry loops because their shutdown path is broken at the router layer. ## Proposed fixes 1. **Kimi K2/K2.5 parser** (`model/parsers/kimi.go`) that recognizes Kimi's native tool-call token sequence (`<|tool_call_begin|>functions.NAME:N<|tool_call_argument_begin|>JSON<|tool_call_end|><|tool_calls_section_end|>`) and emits `api.ToolCall` objects. This must be deployed both locally and on the cloud backend since `:cloud` bypasses local parsers. 2. **Nested-object preservation in Anthropic-compat translation.** When a model's native tool-call argument representation is a JSON string, the Anthropic-compat shim should `json.Unmarshal` it and embed the resulting object as `input` rather than embedding the string literally. This affects Qwen3.x, GLM-5.1, and likely others. 3. **Validation / logging** on the cloud passthrough for unrecognized token sequences in text blocks — early rejection with a clear error beats a validation failure two turns later with a confusing "missing name" message. ## Why this is worth fixing Ollama Cloud is positioned as a drop-in Anthropic-compat endpoint (the `/v1/messages` path exists precisely for this). Third-party multi-agent frameworks like Claude Code's agent teams (`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`) rely on structured tool-call JSON to implement peer-to-peer coordination, plan approvals, and graceful shutdown. Without the fix, any workflow that relies on nested objects in tool inputs breaks on Kimi/Qwen/GLM — which are some of the most interesting models in the Ollama Cloud catalog. ## Workarounds (client-side) Until fixed, Claude Code users can: - Switch teammate models to `minimax-m2.7:cloud` or `deepseek-v3.1:671b-cloud` (confirmed working). - Use `TaskStop` with the internal `agentId` instead of `SendMessage` `shutdown_request` for teardown — bypasses the nested-object path entirely. - Install a `PostToolUseFailure` hook that detects stringified JSON in failed `SendMessage` calls and coaches the model to switch approaches. None of these are substitutes for fixing the shim. ## Attachments / references - Internal agent ID format regex (for those investigating `task_id` / `agentId` routing): `src/types/ids.ts:35` — `/^a(?:.+-)?[0-9a-f]{16}$/` - SendMessage validation logic: `src/tools/SendMessageTool/SendMessageTool.ts:667-674` - Deferred-tool gate: `src/tools/ToolSearchTool/prompt.ts:62` - Claude Code agent-teams feature flag: `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1` Happy to provide raw `.jsonl` transcripts from the failing sessions if helpful.
GiteaMirror added the cloud label 2026-05-10 06:13:46 -05:00
GiteaMirror commented 2026-05-10 06:13:48 -05:00
Author
Owner
Copy Link

@PureBlissAK commented on GitHub (Apr 18, 2026):

🤖 Automated Triage & Analysis Report

Issue: #15645
Analyzed: 2026-04-18T18:13:41.717714

Analysis

  • Type: unknown
  • Severity: medium
  • Components: unknown

Implementation Plan

  • Effort: medium
  • Steps:

This issue has been triaged and marked for implementation.

<!-- gh-comment-id:4274294767 --> @PureBlissAK commented on GitHub (Apr 18, 2026): <!-- ollama-issue-orchestrator:v1 issue:15645 --> ## 🤖 Automated Triage & Analysis Report **Issue**: #15645 **Analyzed**: 2026-04-18T18:13:41.717714 ### Analysis - **Type**: unknown - **Severity**: medium - **Components**: unknown ### Implementation Plan - **Effort**: medium - **Steps**: *This issue has been triaged and marked for implementation.*
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label cloud
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87669
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?