github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #15241] gemma4:31b - Tool Call Fails #71809

New Issue
Closed
opened 2026-05-05 02:36:18 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-05-05 02:36:18 -05:00
Owner
Copy Link

Originally created by @Glitch3dPenguin on GitHub (Apr 2, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15241

What is the issue?

gemma4 seems to fail a tool call.

Version running is: ollama:0.20.0-rc1

Relevant log output

source=gemma4.go:287 msg="gemma4 tool call parsing failed" error="invalid character 'p' after object key:value pair" content="call:subagent{*****************}"

OS

Docker

GPU

Nvidia

CPU

AMD

Ollama version

0.20.0-rc1

Originally created by @Glitch3dPenguin on GitHub (Apr 2, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15241 ### What is the issue? gemma4 seems to fail a tool call. Version running is: ollama:0.20.0-rc1 ### Relevant log output ```shell source=gemma4.go:287 msg="gemma4 tool call parsing failed" error="invalid character 'p' after object key:value pair" content="call:subagent{*****************}" ``` ### OS Docker ### GPU Nvidia ### CPU AMD ### Ollama version 0.20.0-rc1
GiteaMirror added the bug label 2026-05-05 02:36:18 -05:00
GiteaMirror commented 2026-05-05 02:36:22 -05:00
Author
Owner
Copy Link

@chathaway-codes commented on GitHub (Apr 3, 2026):

Same issue; I believe the error message can answer this:

time=2026-04-03T02:05:16.223Z level=WARN source=gemma4.go:287 msg="gemma4 tool call parsing failed" error="invalid character 'U' after object key:value pair" content="call:terminal{cd:<|\"|>dir<|\"|>,command:<|\"|>git add task1.yaml && git commit -m \"Update task1.yaml to use gemma4:31b\"<|\"|>}"

The issue is this line; if the command contains a quote, it will fail because it replaces <"> with ", and doesn't escape " when converting to the string that is passed to the JSON parser

<!-- gh-comment-id:4181555415 --> @chathaway-codes commented on GitHub (Apr 3, 2026): Same issue; I believe the error message can answer this: ``` time=2026-04-03T02:05:16.223Z level=WARN source=gemma4.go:287 msg="gemma4 tool call parsing failed" error="invalid character 'U' after object key:value pair" content="call:terminal{cd:<|\"|>dir<|\"|>,command:<|\"|>git add task1.yaml && git commit -m \"Update task1.yaml to use gemma4:31b\"<|\"|>}" ``` The issue is [this line](https://github.com/ollama/ollama/blob/3536ef58f613b9f448a040b09417aa057e52bc2d/model/parsers/gemma4.go#L358); if the command contains a quote, it will fail because it replaces `<">` with `"`, and doesn't escape `"` when converting to the string that is passed to the JSON parser
GiteaMirror commented 2026-05-05 02:36:22 -05:00
Author
Owner
Copy Link

@drifkin commented on GitHub (Apr 3, 2026):

Thanks for reporting! I've got what I think is an initial fix up at #15254, but still testing it

<!-- gh-comment-id:4181661932 --> @drifkin commented on GitHub (Apr 3, 2026): Thanks for reporting! I've got what I think is an initial fix up at #15254, but still testing it
GiteaMirror commented 2026-05-05 02:36:23 -05:00
Author
Owner
Copy Link

@chathaway-codes commented on GitHub (Apr 3, 2026):

No pressure to take this, but I already had the code mostly finished before I had to step away :). Your testing is much more thorough than mine, although I'm not 100% about some of the details of the test cases.

In particular, this one:

<|tool_call>call:search{query:<|"|>say \"hello\"<|"|>}<tool_call|>

It's not clear to me what the correct behavior is; PR 15255 uses strconv.Quote to do the quoting.

https://github.com/ollama/ollama/pull/15255

<!-- gh-comment-id:4181854966 --> @chathaway-codes commented on GitHub (Apr 3, 2026): No pressure to take this, but I already had the code mostly finished before I had to step away :). Your testing is much more thorough than mine, although I'm not 100% about some of the details of the test cases. In particular, this one: > `<|tool_call>call:search{query:<|"|>say \"hello\"<|"|>}<tool_call|>` It's not clear to me what the correct behavior is; PR 15255 uses strconv.Quote to do the quoting. https://github.com/ollama/ollama/pull/15255
GiteaMirror commented 2026-05-05 02:36:23 -05:00
Author
Owner
Copy Link

@drifkin commented on GitHub (Apr 3, 2026):

@chathaway-codes thanks for the PR, appreciate it! I'm a little uncertain of the exact correct behavior for that test case, I'm trying to experimentally figure it out right now, thanks for highlighting that. Models aren't always consistent about this sort of thing (which makes it impossible to solve it 100% of the time), but from my empirical testing, I think you're right that I was handling that case differently from what the model prefers to do. Checking some of the other escapes right now to see if it's consistent.

<!-- gh-comment-id:4181977936 --> @drifkin commented on GitHub (Apr 3, 2026): @chathaway-codes thanks for the PR, appreciate it! I'm a little uncertain of the exact correct behavior for that test case, I'm trying to experimentally figure it out right now, thanks for highlighting that. Models aren't always consistent about this sort of thing (which makes it impossible to solve it 100% of the time), but from my empirical testing, I think you're right that I was handling that case differently from what the model prefers to do. Checking some of the other escapes right now to see if it's consistent.
GiteaMirror commented 2026-05-05 02:36:24 -05:00
Author
Owner
Copy Link

@drifkin commented on GitHub (Apr 3, 2026):

@chathaway-codes I pushed up that fix to my PR, will go with mine for now since I tested it more, but I've still got some other edge cases to look at tomorrow. Thanks again for pointing out that test case, was super helpful. Added you as a co-author on that commit since it wouldn't have happened tonight without you!

<!-- gh-comment-id:4182025774 --> @drifkin commented on GitHub (Apr 3, 2026): @chathaway-codes I pushed up that fix to my PR, will go with mine for now since I tested it more, but I've still got some other edge cases to look at tomorrow. Thanks again for pointing out that test case, was super helpful. Added you as a co-author on that commit since it wouldn't have happened tonight without you!
GiteaMirror commented 2026-05-05 02:36:25 -05:00
Author
Owner
Copy Link

@daniel-farina commented on GitHub (Apr 3, 2026):

I got things working by merging some PRs, I'll keep on posting my findings there

https://gist.github.com/daniel-farina/87dc1c394b94e45bb700d27e9ea03193

<!-- gh-comment-id:4183480666 --> @daniel-farina commented on GitHub (Apr 3, 2026): I got things working by merging some PRs, I'll keep on posting my findings there https://gist.github.com/daniel-farina/87dc1c394b94e45bb700d27e9ea03193
GiteaMirror commented 2026-05-05 02:36:27 -05:00
Author
Owner
Copy Link

@tryblackjack commented on GitHub (Apr 5, 2026):

Workaround for Gemma-4 Tools API Bug in Ollama (#15241)

Hey everyone.

A lot of us are hitting the failed to load crash when trying to use the native Tools API (tools payload) with the new gemma4:31b models in Ollama.

Waiting for the core team to rewrite the Go parser and push an official patch might take a while. So, my AI assistant (JARVIS) and I put together a simple, elegant Python shim. It completely bypasses the bug and lets you use agentic workflows with Gemma 4 right now.

P.S. Co-authored by my AI assistant, Gem bot Gemini. If we fuck this up, we fuck up together =)

The Fix (TL;DR)

If Ollama crashes when it sees the tools array, we simply don't send it.

Instead, we use a fallback method: we tell Gemma in the system prompt what tools are available and force it to output tool calls as raw JSON text. Then, we catch that JSON with a regex parser before it hits your main agent loop.

Here is how to implement it in 3 steps:

Step 1: Update the System Prompt

Remove tools from your API payload if the model is Gemma 4. Instead, inject this into your System Prompt:

<tool_usage_protocol>
The native Tools API is currently unavailable. To use a tool, you MUST output a JSON block strictly in this format:
<tool_call>
{"tool_name": "name_of_tool", "parameters": {"key": "value"}}
</tool_call>
</tool_usage_protocol>

AVAILABLE TOOLS:

  • "file_manager": {"action": "read|write", "path": "string"}
  • "bash_executor": {"command": "string"}

Step 2: The Python Parser (gemma_tools_shim.py)

Drop this lightweight script into your project. It extracts the JSON from Gemma's raw text response.

import json
import re

def extract_tool_call(text: str):
"""
Looks for a tool call block inside Gemma's raw text output.
"""
match = re.search(r"{.*?}", text, flags=re.S)
if not match:
return None

try:
    data = json.loads(match.group())
    # Verify it actually looks like a tool call
    if isinstance(data, dict) and any(k in data for k in ("tool", "tool_name", "name")):
        return data
except Exception:
    return None
    
return None

Step 3: Wrap Your Router

Now, just wrap your Ollama API call with a simple condition:

from gemma_tools_shim import extract_tool_call

def chat_with_model(model_name, messages, tools_list):
is_gemma4 = "gemma4" in model_name.lower()

# Build base payload
payload = {
    "model": model_name,
    "messages": messages,
    "stream": False
}

# Only use the native Tools API if it's NOT Gemma 4
if not is_gemma4:
    payload["tools"] = tools_list
    
# Make the request...
# response = requests.post(...)

reply_text = response.json().get("message", {}).get("content", "")

# If it's Gemma, run the text through our shim
if is_gemma4:
    tool_call = extract_tool_call(reply_text)
    if tool_call:
        return execute_tool(tool_call) # Your tool execution logic goes here
        
return reply_text

Conclusion

This is a classic Graceful Degradation pattern. You can keep writing your agentic code today. Once the Ollama devs close Issue #15241, just delete the is_gemma4 check, and your code will automatically switch back to the native API without any architectural rewrites.

Hope this saves you some debugging hours!

<!-- gh-comment-id:4189214964 --> @tryblackjack commented on GitHub (Apr 5, 2026): Workaround for Gemma-4 Tools API Bug in Ollama (#15241) Hey everyone. A lot of us are hitting the failed to load crash when trying to use the native Tools API (tools payload) with the new gemma4:31b models in Ollama. Waiting for the core team to rewrite the Go parser and push an official patch might take a while. So, my AI assistant (JARVIS) and I put together a simple, elegant Python shim. It completely bypasses the bug and lets you use agentic workflows with Gemma 4 right now. P.S. Co-authored by my AI assistant, Gem bot Gemini. If we fuck this up, we fuck up together =) The Fix (TL;DR) If Ollama crashes when it sees the tools array, we simply don't send it. Instead, we use a fallback method: we tell Gemma in the system prompt what tools are available and force it to output tool calls as raw JSON text. Then, we catch that JSON with a regex parser before it hits your main agent loop. Here is how to implement it in 3 steps: Step 1: Update the System Prompt Remove tools from your API payload if the model is Gemma 4. Instead, inject this into your System Prompt: <tool_usage_protocol> The native Tools API is currently unavailable. To use a tool, you MUST output a JSON block strictly in this format: <tool_call> {"tool_name": "name_of_tool", "parameters": {"key": "value"}} </tool_call> </tool_usage_protocol> AVAILABLE TOOLS: - "file_manager": {"action": "read|write", "path": "string"} - "bash_executor": {"command": "string"} Step 2: The Python Parser (gemma_tools_shim.py) Drop this lightweight script into your project. It extracts the JSON from Gemma's raw text response. import json import re def extract_tool_call(text: str): """ Looks for a tool call block inside Gemma's raw text output. """ match = re.search(r"\{.*?\}", text, flags=re.S) if not match: return None try: data = json.loads(match.group()) # Verify it actually looks like a tool call if isinstance(data, dict) and any(k in data for k in ("tool", "tool_name", "name")): return data except Exception: return None return None Step 3: Wrap Your Router Now, just wrap your Ollama API call with a simple condition: from gemma_tools_shim import extract_tool_call def chat_with_model(model_name, messages, tools_list): is_gemma4 = "gemma4" in model_name.lower() # Build base payload payload = { "model": model_name, "messages": messages, "stream": False } # Only use the native Tools API if it's NOT Gemma 4 if not is_gemma4: payload["tools"] = tools_list # Make the request... # response = requests.post(...) reply_text = response.json().get("message", {}).get("content", "") # If it's Gemma, run the text through our shim if is_gemma4: tool_call = extract_tool_call(reply_text) if tool_call: return execute_tool(tool_call) # Your tool execution logic goes here return reply_text Conclusion This is a classic Graceful Degradation pattern. You can keep writing your agentic code today. Once the Ollama devs close Issue #15241, just delete the is_gemma4 check, and your code will automatically switch back to the native API without any architectural rewrites. Hope this saves you some debugging hours!
GiteaMirror commented 2026-05-05 02:36:28 -05:00
Author
Owner
Copy Link

@drifkin commented on GitHub (Apr 6, 2026):

@tryblackjack: this issue is already fixed, no need for a workaround

<!-- gh-comment-id:4189967556 --> @drifkin commented on GitHub (Apr 6, 2026): @tryblackjack: this issue is already fixed, no need for a workaround
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#71809
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?