[GH-ISSUE #15315] gemma4:e4b with ollama 0.20.1 still has tool parsing errors #35555

New Issue

GiteaMirror · 2026-04-22T20:07:36-05:00

GiteaMirror commented

2026-04-22 20:07:36 -05:00

Originally created by @KeuntekLee on GitHub (Apr 4, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15315

Originally assigned to: @drifkin on GitHub.

What is the issue?

After updating with ollama version 0.20.1 for fixing gemma4 tool parsing error(#15254), ollama still show errors on tool calling.
Using gemma4:e4b with opencode, oh-my-opencode, ollama 0.20.1

Relevant log output

time=2026-04-04T13:58:18.757+09:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '`' looking for beginning of value" content="call:write{content:\n\n# Project Style Guide for Autonomous Agents' Code Generation (AGENTS.md)\n\nThis document captures the *de facto* coding standards observed across the `src/` and `components/` source code, designed to ensure consistency for all generated code and modules consumed by the agent system.

time=2026-04-04T13:57:33.487+09:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '\\'' looking for beginning of value" content="call:grep{include:<|\"|>*.py<|\"|>,output_mode:<|\"|>content<|\"|>,path:<|\"|>/data/robotics/experiment1<|\"|>,pattern:':\\s*\\w+'<|\"|>}"

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.20.1

Originally created by @KeuntekLee on GitHub (Apr 4, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15315 Originally assigned to: @drifkin on GitHub. ### What is the issue? After updating with ollama version 0.20.1 for fixing gemma4 tool parsing error(#15254), ollama still show errors on tool calling. Using gemma4:e4b with opencode, oh-my-opencode, ollama 0.20.1 ### Relevant log output ```shell time=2026-04-04T13:58:18.757+09:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '`' looking for beginning of value" content="call:write{content:\n\n# Project Style Guide for Autonomous Agents' Code Generation (AGENTS.md)\n\nThis document captures the *de facto* coding standards observed across the `src/` and `components/` source code, designed to ensure consistency for all generated code and modules consumed by the agent system. time=2026-04-04T13:57:33.487+09:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '\\'' looking for beginning of value" content="call:grep{include:<|\"|>*.py<|\"|>,output_mode:<|\"|>content<|\"|>,path:<|\"|>/data/robotics/experiment1<|\"|>,pattern:':\\s*\\w+'<|\"|>}" ``` ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.20.1

GiteaMirror added the bug label 2026-04-22 20:07:36 -05:00

GiteaMirror commented

2026-04-22 20:07:37 -05:00

@Habitante commented on GitHub (Apr 4, 2026):

Root cause analysis

The underlying issue is that Gemma 4's tool call format is not JSON — it's a custom serialization:

<|tool_call>call:func_name{key:<|"|>value<|"|>,num:42}<tool_call|>

Rules:

String values are delimited by <|"|> (special token 52), not regular " quotes
Keys are unquoted (bare identifiers)
Non-string values (numbers, booleans) are bare
Content between <|"|> delimiters can contain any characters: double quotes, single quotes, backticks, backslashes, newlines, etc.

The current parser appears to do a naive <|"|> → " replacement and then call json.Unmarshal. This breaks whenever the string content contains characters that aren't valid unescaped JSON — which is exactly what both error messages in this issue show:

invalid character ''` — backtick inside a markdown code block argument
invalid character '\'' — single quote inside a regex pattern argument like ':\s*\w+'

PR #15255 fixed one case (internal double quotes), but the problem is more fundamental: you can't convert Gemma 4's format to JSON with string replacement alone, because the content between <|"|> delimiters is arbitrary text that needs proper JSON string escaping (backslashes, control chars, etc.) before it can be placed between " quotes.

What a correct parser looks like

vLLM's gemma4_tool_parser.py has a working reference implementation. Their approach:

Extract function name and raw argument string from call:name{...}
Parse the key:value pairs with a custom tokenizer that understands <|"|> as string delimiters
For string values (between <|"|> delimiters): capture the raw content, then properly escape it for JSON
For bare values: parse as number/boolean/null directly
Build a native dict/map, then serialize to JSON via json.dumps() / json.Marshal

The critical step is #3 — the raw content between <|"|> delimiters must be JSON-escaped (\n → \\n, " → \", \ → \\, etc.) before being placed into a JSON string value.

Reproduction (minimal, no app needed)

This reliably triggers the bug because print("hello world") contains double quotes that generate <|"|> escaping:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:26b",
  "messages": [{"role": "user", "content": "use run_script to print hello world"}],
  "stream": false,
  "tools": [{
    "type": "function",
    "function": {
      "name": "run_script",
      "description": "Execute Python code",
      "parameters": {
        "type": "object",
        "properties": {
          "code": {"type": "string", "description": "Python code"}
        },
        "required": ["code"]
      }
    }
  }]
}'

Expected: tool_calls with print("hello world")
Actual: Empty content, done_reason: "stop", no tool_calls field. The WARN log shows parsing failed.

Contrast with a quote-free argument that works fine:

# Same setup, but "compute 2+2" generates print(2 + 2) — no quotes, no <|"|> escaping
curl ... "content": "use run_script to compute 2+2" ...

This returns a valid tool_calls response because print(2 + 2) has no internal quotes.

Additional failure modes from systematic testing

User prompt	Generated code	Quotes in args?	Result
"compute 2+2"	`print(2 + 2)`	No	✅ Works
"fibonacci up to 20"	`def fibonacci(n):...`	Yes (in strings)	✅ Works sometimes
"print hello world"	`print("hello world")`	Yes	❌ Fails silently
"build a calculator app"	Full tkinter program	Many	❌ Fails (1618 eval tokens, all thinking)
"generate 10 random passwords"	Full script	Yes	❌ Falls back to markdown text

The pattern: any tool argument containing characters that need JSON escaping will fail. Simple numeric expressions work; anything with string literals, regex patterns, shell commands, or multi-line code fails.

References

vLLM's working parser: vllm/tool_parsers/gemma4_tool_parser.py and gemma4_utils.py
llama.cpp has the same issue: ggml-org/llama.cpp#21316
mlx-lm has no parser at all yet: ml-explore/mlx-lm#1096

@Habitante commented on GitHub (Apr 4, 2026): ## Root cause analysis The underlying issue is that Gemma 4's tool call format is **not JSON** — it's a custom serialization: ``` <|tool_call>call:func_name{key:<|"|>value<|"|>,num:42}<tool_call|> ``` Rules: - String values are delimited by `<|"|>` (special token 52), **not** regular `"` quotes - Keys are **unquoted** (bare identifiers) - Non-string values (numbers, booleans) are bare - Content between `<|"|>` delimiters can contain **any characters**: double quotes, single quotes, backticks, backslashes, newlines, etc. The current parser appears to do a naive `<|"|>` → `"` replacement and then call `json.Unmarshal`. This breaks whenever the string content contains characters that aren't valid unescaped JSON — which is exactly what both error messages in this issue show: - `invalid character '`'` — backtick inside a markdown code block argument - `invalid character '\''` — single quote inside a regex pattern argument like `':\s*\w+'` PR #15255 fixed one case (internal double quotes), but the problem is more fundamental: **you can't convert Gemma 4's format to JSON with string replacement alone**, because the content between `<|"|>` delimiters is arbitrary text that needs proper JSON string escaping (backslashes, control chars, etc.) before it can be placed between `"` quotes. ## What a correct parser looks like vLLM's `gemma4_tool_parser.py` has a working reference implementation. Their approach: 1. Extract function name and raw argument string from `call:name{...}` 2. Parse the `key:value` pairs with a **custom tokenizer** that understands `<|"|>` as string delimiters 3. For string values (between `<|"|>` delimiters): capture the raw content, then properly escape it for JSON 4. For bare values: parse as number/boolean/null directly 5. Build a native dict/map, then serialize to JSON via `json.dumps()` / `json.Marshal` The critical step is #3 — the raw content between `<|"|>` delimiters must be JSON-escaped (`\n` → `\\n`, `"` → `\"`, `\` → `\\`, etc.) before being placed into a JSON string value. ## Reproduction (minimal, no app needed) This reliably triggers the bug because `print("hello world")` contains double quotes that generate `<|"|>` escaping: ```bash curl http://localhost:11434/api/chat -d '{ "model": "gemma4:26b", "messages": [{"role": "user", "content": "use run_script to print hello world"}], "stream": false, "tools": [{ "type": "function", "function": { "name": "run_script", "description": "Execute Python code", "parameters": { "type": "object", "properties": { "code": {"type": "string", "description": "Python code"} }, "required": ["code"] } } }] }' ``` **Expected:** `tool_calls` with `print("hello world")` **Actual:** Empty content, `done_reason: "stop"`, no `tool_calls` field. The WARN log shows parsing failed. Contrast with a quote-free argument that works fine: ```bash # Same setup, but "compute 2+2" generates print(2 + 2) — no quotes, no <|"|> escaping curl ... "content": "use run_script to compute 2+2" ... ``` This returns a valid `tool_calls` response because `print(2 + 2)` has no internal quotes. ## Additional failure modes from systematic testing | User prompt | Generated code | Quotes in args? | Result | |---|---|---|---| | "compute 2+2" | `print(2 + 2)` | No | ✅ Works | | "fibonacci up to 20" | `def fibonacci(n):...` | Yes (in strings) | ✅ Works sometimes | | "print hello world" | `print("hello world")` | Yes | ❌ Fails silently | | "build a calculator app" | Full tkinter program | Many | ❌ Fails (1618 eval tokens, all thinking) | | "generate 10 random passwords" | Full script | Yes | ❌ Falls back to markdown text | The pattern: **any tool argument containing characters that need JSON escaping will fail**. Simple numeric expressions work; anything with string literals, regex patterns, shell commands, or multi-line code fails. ## References - vLLM's working parser: [`vllm/tool_parsers/gemma4_tool_parser.py`](https://github.com/vllm-project/vllm/blob/main/vllm/tool_parsers/gemma4_tool_parser.py) and `gemma4_utils.py` - llama.cpp has the same issue: [ggml-org/llama.cpp#21316](https://github.com/ggml-org/llama.cpp/issues/21316) - mlx-lm has no parser at all yet: [ml-explore/mlx-lm#1096](https://github.com/ml-explore/mlx-lm/issues/1096)

GiteaMirror commented

2026-04-22 20:07:38 -05:00

@drifkin commented on GitHub (Apr 6, 2026):

Thanks for reporting. These look like the model emitted invalid tool calls:

The first log shows is missing the gemma4-style <|"|> quotes around the value
The second log is missing the opening gemma4-style quote, but does have the closing one

Sometimes very small models will make mistakes in tool calling, but perhaps there are some other issues at play, we'll investigate. Depending on what we find, we may make the tool parser try to repair these bad calls. Thanks again for reporting, the logs were extremely helpful.

@drifkin commented on GitHub (Apr 6, 2026): Thanks for reporting. These look like the model emitted invalid tool calls: - The first log shows is missing the gemma4-style `<|"|>` quotes around the value - The second log is missing the opening gemma4-style quote, but _does_ have the closing one Sometimes very small models will make mistakes in tool calling, but perhaps there are some other issues at play, we'll investigate. Depending on what we find, we may make the tool parser try to repair these bad calls. Thanks again for reporting, the logs were extremely helpful.

GiteaMirror commented

2026-04-22 20:07:39 -05:00

@AbeEstrada commented on GitHub (Apr 6, 2026):

What is the issue?

Error while calling a tool, I'm using gemma4:26b-a4b-it-q4_K_M

I hope this is relevant and can help fix this issue

Relevant log output

time=2026-04-05T22:48:05.951-06:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '<' looking for beginning of value" content="call:bash{command:<|\"|>find . -maxdepth 3 -not -path '*/.*'</div>"

OS

macOS 15.7.4

GPU

M2 Max

CPU

M2 Max

Ollama version

0.20.2

@AbeEstrada commented on GitHub (Apr 6, 2026): ### What is the issue? Error while calling a tool, I'm using `gemma4:26b-a4b-it-q4_K_M` I hope this is relevant and can help fix this issue ### Relevant log output ```shell time=2026-04-05T22:48:05.951-06:00 level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '<' looking for beginning of value" content="call:bash{command:<|\"|>find . -maxdepth 3 -not -path '*/.*'</div>" ``` ### OS macOS 15.7.4 ### GPU M2 Max ### CPU M2 Max ### Ollama version 0.20.2

GiteaMirror commented

2026-04-22 20:07:39 -05:00

@AdriRayns commented on GitHub (Apr 6, 2026):

I've also been dealing with that error all weekend while running the 31b model. I ran a lot of tests to see if I could adjust the behavior using the modelfile, stops, templates, temperature, and a prompt to “force” it to make the tool calls correctly... but nothing worked. I saw that there were open issues over the weekend, and I figured it was something “common” to other users.

@AdriRayns commented on GitHub (Apr 6, 2026): I've also been dealing with that error all weekend while running the 31b model. I ran a lot of tests to see if I could adjust the behavior using the modelfile, stops, templates, temperature, and a prompt to “force” it to make the tool calls correctly... but nothing worked. I saw that there were open issues over the weekend, and I figured it was something “common” to other users.

GiteaMirror commented

2026-04-22 20:07:40 -05:00

@BaneusCatrix commented on GitHub (Apr 6, 2026):

Same:

time=2026-04-06T19:16:04.667Z level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '<' looking for beginning of value" content="call:bash{command:<|\"|>ls}"

ollama: 0.20.2
model: gemma4:26b
os: cachyos
gpu: Nvidia RTX 4080

@BaneusCatrix commented on GitHub (Apr 6, 2026): Same: `time=2026-04-06T19:16:04.667Z level=WARN source=gemma4.go:293 msg="gemma4 tool call parsing failed" error="invalid character '<' looking for beginning of value" content="call:bash{command:<|\"|>ls}"` ollama: 0.20.2 model: gemma4:26b os: cachyos gpu: Nvidia RTX 4080

GiteaMirror commented

2026-04-22 20:07:41 -05:00

@aalzehla commented on GitHub (Apr 6, 2026):

same issue here
MacBook Pro M1
LM Studio 0.4.9+1
Model: gemma4:26b

@aalzehla commented on GitHub (Apr 6, 2026): same issue here MacBook Pro M1 LM Studio 0.4.9+1 Model: gemma4:26b

GiteaMirror commented

2026-04-22 20:07:42 -05:00

@AdriRayns commented on GitHub (Apr 7, 2026):

I can confirm that the new version 0.20.3 resolves this issue (which was discussed at https://github.com/ollama/ollama/issues/15241)

I think this issue should be closed.

@AdriRayns commented on GitHub (Apr 7, 2026): I can confirm that the new version 0.20.3 resolves this issue (which was discussed at https://github.com/ollama/ollama/issues/15241) I think this issue should be closed.

GiteaMirror commented

2026-04-22 20:07:42 -05:00

@kevincox commented on GitHub (Apr 7, 2026):

Forgive me if I am missing something, but isn't this being handled at the wrong later. IIUC Gemma 4 has a specific token for tool call arguments. But this token is currently being mapped to the text <|"|> then this text is being parsed for cases of that replacement string.

But this means that the argument parsing will also interpret other tokens that "evaluate" to <|"|> as tool call argument delimiters. I think this will cause issues if the delimiter appears inside of the tool call.

It seems that Google did something really clever here to make the delimiters clear and unambiguous but the way Ollama is handling this defeats that by doing this logic after converting from tokens to strings rather than before.

So the new patch probably "mostly works" but isn't fully correct.

@kevincox commented on GitHub (Apr 7, 2026): Forgive me if I am missing something, but isn't this being handled at the wrong later. IIUC Gemma 4 has a specific token for tool call arguments. But this token is currently being mapped to the text `<|"|>` then this text is being parsed for cases of that replacement string. But this means that the argument parsing will also interpret other tokens that "evaluate" to `<|"|>` as tool call argument delimiters. I think this will cause issues if the delimiter appears inside of the tool call. It seems that Google did something really clever here to make the delimiters clear and unambiguous but the way Ollama is handling this defeats that by doing this logic after converting from tokens to strings rather than before. So the new patch probably "mostly works" but isn't fully correct.

GiteaMirror commented

2026-04-22 20:07:43 -05:00

@kevincox commented on GitHub (Apr 7, 2026):

Also I'm still getting issues with tool calls on 0.20.3 It may be a touch better, but still fails very often on gemma4:e4b.

Apr 07 13:09:59 kevinryzen ollama[402661]: time=2026-04-07T13:09:59.783-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="expected '{' in tool call" content="call:ls -F src/<|\"|>,description:<|\"|>List files and directories inside the src directory<|\"|>}"
Apr 07 13:10:00 kevinryzen ollama[402661]: time=2026-04-07T13:10:00.818-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="expected '{' in tool call" content="call:ls -F web/<|\"|>,description:<|\"|>List files and directories inside the web directory<|\"|>}"

@kevincox commented on GitHub (Apr 7, 2026): Also I'm still getting issues with tool calls on 0.20.3 It may be a touch better, but still fails very often on `gemma4:e4b`. ``` Apr 07 13:09:59 kevinryzen ollama[402661]: time=2026-04-07T13:09:59.783-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="expected '{' in tool call" content="call:ls -F src/<|\"|>,description:<|\"|>List files and directories inside the src directory<|\"|>}" Apr 07 13:10:00 kevinryzen ollama[402661]: time=2026-04-07T13:10:00.818-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="expected '{' in tool call" content="call:ls -F web/<|\"|>,description:<|\"|>List files and directories inside the web directory<|\"|>}" ```

GiteaMirror commented

2026-04-22 20:07:44 -05:00

@drifkin commented on GitHub (Apr 7, 2026):

Hi @kevincox you're right that operating at the token-level rather than string-level is more correct, and I'm similarly very happy about this new special token approach, escaping is one of the trickiest issues for models and I think this will ultimately help that issue a lot. Switching everything to token-based will be fairly invasive, but it is on my list of things to investigate. However, I don't think that this is the root cause of any of the issues in this thread, or any failed tool calls I've looked at so far. They've all been from actually malformed tool calls. (Though I'm sure I could easily repro the token v. string problem by trying to use gemma4 to work on this gemma4 parser, for example!)

Thanks for the additional two logs, I'll work on a repair for those as well. Notice that they're not ambiguous, but rather malformed:

call:ls -F web/<|\"|>,

is missing the opening <|"|>, but has a closing one.

@drifkin commented on GitHub (Apr 7, 2026): Hi @kevincox you're right that operating at the token-level rather than string-level is more correct, and I'm similarly very happy about this new special token approach, escaping is one of the trickiest issues for models and I think this will ultimately help that issue a lot. Switching everything to token-based will be fairly invasive, but it is on my list of things to investigate. However, I don't think that this is the root cause of any of the issues in this thread, or any failed tool calls I've looked at so far. They've all been from actually malformed tool calls. (Though I'm sure I could easily repro the token v. string problem by trying to use gemma4 to work on this gemma4 parser, for example!) Thanks for the additional two logs, I'll work on a repair for those as well. Notice that they're not ambiguous, but rather malformed: ``` call:ls -F web/<|\"|>, ``` is missing the opening `<|"|>`, but has a closing one.

GiteaMirror commented

2026-04-22 20:07:45 -05:00

@drifkin commented on GitHub (Apr 7, 2026):

@kevincox: I took a closer look and those examples are more malformed than I thought. They're actually missing the tool name itself (e.g., if the intended tool name wasbash, then call:bash{command:<|"|>ls -F web/<|"|>,description:<|"|>...<|"|>})

I wonder if you can provide more example logs to see if there are any other repairable cases or if they're all in this very malformed state? Or if you have a straightforward repro, that would be helpful too.

@drifkin commented on GitHub (Apr 7, 2026): @kevincox: I took a closer look and those examples are more malformed than I thought. They're actually missing the tool name itself (e.g., if the intended tool name was`bash`, then `call:bash{command:<|"|>ls -F web/<|"|>,description:<|"|>...<|"|>}`) I wonder if you can provide more example logs to see if there are any other repairable cases or if they're all in this very malformed state? Or if you have a straightforward repro, that would be helpful too.

GiteaMirror commented

2026-04-22 20:07:45 -05:00

@kevincox commented on GitHub (Apr 7, 2026):

I can't seem to find any super simple cases. I'm mostly using opencode, I'm trying to figure out what logging is available there. I found a different case here:

Apr 07 19:33:02 kevinryzen ollama[402661]: time=2026-04-07T19:33:02.146-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'r' looking for beginning of value\nrepair failed to produce valid JSON arguments" content="call:edit{filePath:<|\"|>src/robot_test.rs<|\"|>,newString:r#\"#[cfg(test)]\nmod tests {\n    use super::Robot;\n\n    #[test]\n    fn test_from_str_all_colors() {\n        // Test known string representations\n        assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n\n        assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n\n        assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n\n        assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n\n        assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n    }\n\n    #[test]\n    fn test_from_str_unknown() {\n        // Test unknown or malformed strings\n        assert_eq!(Robot::from_str(\"5\").unwrap_err(), \"Unknown robot\");\n        assert_eq!(Robot::from_str(\"unknown_color\").unwrap_err(), \"Unknown robot\");\n    }\n\n     #[test]\n    fn test_from_u8_conversion() {\n        assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n    }\n}\n\"#,oldString:r#\"#[cfg(test)]\nmod robot_test {\n    use super::Robot;\n\n    // Helper function to test FromStr implementation coverage\n    #[test]\n    fn test_from_str_black() {\n        assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n    }\n\n    #[test]\n    fn test_from_str_red() {\n        assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n    }\n\n    #[test]\n    fn test_from_str_green() {\n        assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n    }\n\n    #[test]\n    fn test_from_str_yellow() {\n        assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n    }\n\n    #[test]\n    fn test_from_str_blue() {\n        assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n    }\n\n    #[test]\n    fn test_from_str_unknown() {\n        // Test unknown or malformed strings\n        assert!(Robot::from_str(\"5\").is_err());\n        assert!(Robot::from_str(\"unknown_color\").is_err());\n    }\n\n    // Test the u8 conversion\n    #[test]\n    fn test_from_u8() {\n        assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n        // Test an invalid u8 value which should fail (assuming the TryFrom implementation correctly handles this, \n        // current implementation relies on ROBOTS array size, testing failure mode directly is hard without access to internal logic)\n    }\n}\n\"#,oldString:r#\"#[cfg(test)]\nmod robot_test {\n    use super::Robot;\n\n    // Helper function to test FromStr implementation coverage\n    #[test]\n    fn test_from_str_black() {\n        assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n    }\n\n    #[test]\n    fn test_from_str_red() {\n        assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n        assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n    }\n\n    #[test]\n    fn test_from_str_green() {\n        assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n        assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n    }\n\n    #[test]\n    fn test_from_str_yellow() {\n        assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n        assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n    }\n\n    #[test]\n    fn test_from_str_blue() {\n        assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n        assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n    }\n\n    #[test]\n    fn test_from_str_unknown() {\n        // Test unknown or malformed strings\n        assert!(Robot::from_str(\"5\").is_err());\n        assert!(Robot::from_str(\"unknown_color\").is_err());\n    }\n\n    // Test the u8 conversion\n    #[test]\n    fn test_from_u8() {\n        assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n        assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n        // Test an invalid u8 value which should fail (assuming the TryFrom implementation correctly handles this, \n        // current implementation relies on ROBOTS array size, testing failure mode directly is hard without access to internal logic)\n    }\n}\n\"#,replaceAll:true}"

It does seem unexpected that tool calling is so inconsistent. I know it is a fairly small model but it seems that this behaviour should be well baked in. I'll try to gather more evidence and post soon.

@kevincox commented on GitHub (Apr 7, 2026): I can't seem to find any super simple cases. I'm mostly using opencode, I'm trying to figure out what logging is available there. I found a different case here: ``` Apr 07 19:33:02 kevinryzen ollama[402661]: time=2026-04-07T19:33:02.146-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'r' looking for beginning of value\nrepair failed to produce valid JSON arguments" content="call:edit{filePath:<|\"|>src/robot_test.rs<|\"|>,newString:r#\"#[cfg(test)]\nmod tests {\n use super::Robot;\n\n #[test]\n fn test_from_str_all_colors() {\n // Test known string representations\n assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n\n assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n\n assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n\n assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n\n assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n }\n\n #[test]\n fn test_from_str_unknown() {\n // Test unknown or malformed strings\n assert_eq!(Robot::from_str(\"5\").unwrap_err(), \"Unknown robot\");\n assert_eq!(Robot::from_str(\"unknown_color\").unwrap_err(), \"Unknown robot\");\n }\n\n #[test]\n fn test_from_u8_conversion() {\n assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n }\n}\n\"#,oldString:r#\"#[cfg(test)]\nmod robot_test {\n use super::Robot;\n\n // Helper function to test FromStr implementation coverage\n #[test]\n fn test_from_str_black() {\n assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n }\n\n #[test]\n fn test_from_str_red() {\n assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n }\n\n #[test]\n fn test_from_str_green() {\n assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n }\n\n #[test]\n fn test_from_str_yellow() {\n assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n }\n\n #[test]\n fn test_from_str_blue() {\n assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n }\n\n #[test]\n fn test_from_str_unknown() {\n // Test unknown or malformed strings\n assert!(Robot::from_str(\"5\").is_err());\n assert!(Robot::from_str(\"unknown_color\").is_err());\n }\n\n // Test the u8 conversion\n #[test]\n fn test_from_u8() {\n assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n // Test an invalid u8 value which should fail (assuming the TryFrom implementation correctly handles this, \n // current implementation relies on ROBOTS array size, testing failure mode directly is hard without access to internal logic)\n }\n}\n\"#,oldString:r#\"#[cfg(test)]\nmod robot_test {\n use super::Robot;\n\n // Helper function to test FromStr implementation coverage\n #[test]\n fn test_from_str_black() {\n assert_eq!(Robot::from_str(\"0\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"k\").unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_str(\"black\").unwrap(), Robot::BLACK);\n }\n\n #[test]\n fn test_from_str_red() {\n assert_eq!(Robot::from_str(\"1\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"r\").unwrap(), Robot::RED);\n assert_eq!(Robot::from_str(\"red\").unwrap(), Robot::RED);\n }\n\n #[test]\n fn test_from_str_green() {\n assert_eq!(Robot::from_str(\"2\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"g\").unwrap(), Robot::GREEN);\n assert_eq!(Robot::from_str(\"green\").unwrap(), Robot::GREEN);\n }\n\n #[test]\n fn test_from_str_yellow() {\n assert_eq!(Robot::from_str(\"3\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"y\").unwrap(), Robot::YELLOW);\n assert_eq!(Robot::from_str(\"yellow\").unwrap(), Robot::YELLOW);\n }\n\n #[test]\n fn test_from_str_blue() {\n assert_eq!(Robot::from_str(\"4\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"u\").unwrap(), Robot::BLUE);\n assert_eq!(Robot::from_str(\"blue\").unwrap(), Robot::BLUE);\n }\n\n #[test]\n fn test_from_str_unknown() {\n // Test unknown or malformed strings\n assert!(Robot::from_str(\"5\").is_err());\n assert!(Robot::from_str(\"unknown_color\").is_err());\n }\n\n // Test the u8 conversion\n #[test]\n fn test_from_u8() {\n assert_eq!(Robot::from_u8(0).unwrap(), Robot::BLACK);\n assert_eq!(Robot::from_u8(4).unwrap(), Robot::BLUE);\n // Test an invalid u8 value which should fail (assuming the TryFrom implementation correctly handles this, \n // current implementation relies on ROBOTS array size, testing failure mode directly is hard without access to internal logic)\n }\n}\n\"#,replaceAll:true}" ``` It does seem unexpected that tool calling is so inconsistent. I know it is a fairly small model but it seems that this behaviour should be well baked in. I'll try to gather more evidence and post soon.

GiteaMirror commented

2026-04-22 20:07:47 -05:00

@drifkin commented on GitHub (Apr 7, 2026):

thanks so much! super helpful. I'll see if I can repro in opencode as well.

@drifkin commented on GitHub (Apr 7, 2026): thanks so much! super helpful. I'll see if I can repro in opencode as well.

GiteaMirror commented

2026-04-22 20:07:48 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

Opencode's instructions to enable debug logging don't seem to work, so I don't now how to get a reproducable example. But I'm trying to get a working example with just the Ollama API. For now I'll log failed tool calls in this message.

time=2026-04-07T20:05:17.165-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx,vue}\", path: \".\"}"

@kevincox commented on GitHub (Apr 8, 2026): Opencode's instructions to enable debug logging don't seem to work, so I don't now how to get a reproducable example. But I'm trying to get a working example with just the Ollama API. For now I'll log failed tool calls in this message. ``` time=2026-04-07T20:05:17.165-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx,vue}\", path: \".\"}" ```

GiteaMirror commented

2026-04-22 20:07:49 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

Ok, found a relatively small example:

curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{
	"model": "gemma4:e4b",
	"messages": [
		{"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
		{"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun."},
			{"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."}
	],
	"stream": false,
	"tools": [{
		"type": "function",
		"function": {
			"name": "write_file",
			"description": "Write the data to a file.",
			"parameters": {
				"type": "object",
				"required": ["file", "contents"],
				"properties": {
					"file": {"type": "string", "description": "The path of the file"},
					"contents": {"type": "string", "description": "The new contents of the file"}
				}
			}
		}
	}, {
		"name": "glob",
		"description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
		"parameters": {
			"type": "object",
			"properties": {
				"path": {
					"description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.",
					"type": "string"
				},
				"pattern": {
					"description": "The glob pattern to match files against",
					"type": "string",
					"required": "true"
				}
			},
			"required": [
				"pattern"
			]
		}
	}],
	"options": {
		"seed": 0
	}
}'

@kevincox commented on GitHub (Apr 8, 2026): Ok, found a relatively small example: ``` curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun."}, {"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."} ], "stream": false, "tools": [{ "type": "function", "function": { "name": "write_file", "description": "Write the data to a file.", "parameters": { "type": "object", "required": ["file", "contents"], "properties": { "file": {"type": "string", "description": "The path of the file"}, "contents": {"type": "string", "description": "The new contents of the file"} } } } }, { "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "properties": { "path": { "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.", "type": "string" }, "pattern": { "description": "The glob pattern to match files against", "type": "string", "required": "true" } }, "required": [ "pattern" ] } }], "options": { "seed": 0 } }' ```

GiteaMirror commented

2026-04-22 20:07:50 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

This has a different failure mode:

curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{
        "model": "gemma4:e4b",
        "messages": [
                {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
                {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun."}
        ],
        "stream": false,
        "tools": [{
                "name": "glob",
                "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
                "parameters": {
                        "type": "object",
                        "properties": {
                                "path": {
                                        "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.",
                                        "type": "string"
                                },
                                "pattern": {
                                        "description": "The glob pattern to match files against",
                                        "type": "string",
                                        "required": "true"
                                }
                        },
                        "required": [
                                "pattern"
                        ]
                }
        }],
        "options": {
                "seed": 4
        }
}'

time=2026-04-07T20:35:49.967-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '<' after object key:value pair\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*web-ui*js\"<|\"|>}"
time=2026-04-07T20:35:50.449-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '<' after object key:value pair\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*solver*js\"<|\"|>}"

@kevincox commented on GitHub (Apr 8, 2026): This has a different failure mode: ``` curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun."} ], "stream": false, "tools": [{ "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "properties": { "path": { "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.", "type": "string" }, "pattern": { "description": "The glob pattern to match files against", "type": "string", "required": "true" } }, "required": [ "pattern" ] } }], "options": { "seed": 4 } }' ``` ``` time=2026-04-07T20:35:49.967-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '<' after object key:value pair\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*web-ui*js\"<|\"|>}" time=2026-04-07T20:35:50.449-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '<' after object key:value pair\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*solver*js\"<|\"|>}" ```

GiteaMirror commented

2026-04-22 20:07:50 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{
        "model": "gemma4:e4b",
        "messages": [
                {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
                {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."}
        ],
        "stream": false,
        "options": {
                "seed": 15
        }
}'

time=2026-04-07T20:41:49.652-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="unexpected end of JSON input\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx}''}"

@kevincox commented on GitHub (Apr 8, 2026): ``` curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."} ], "stream": false, "options": { "seed": 15 } }' ``` ``` time=2026-04-07T20:41:49.652-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="unexpected end of JSON input\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx}''}" ```

GiteaMirror commented

2026-04-22 20:07:51 -05:00

@drifkin commented on GitHub (Apr 8, 2026):

@kevincox thanks so much! I was able to repro (though for me I had to use a different seed)

these look much more repairable, so I'll take care of these at least

@drifkin commented on GitHub (Apr 8, 2026): @kevincox thanks so much! I was able to repro (though for me I had to use a different seed) these look much more repairable, so I'll take care of these at least

GiteaMirror commented

2026-04-22 20:07:52 -05:00

@drifkin commented on GitHub (Apr 8, 2026):

One thing that's interesting about all these examples is that "tools" is either not provided, or it contains invalid entries (e.g., the first one has a valid tool definition for write_file, but then glob is missing the { "type": "function", "function}: { ... } } "wrapper".

Beyond us needing better warnings instead of silently failing to parse unexpected tool shapes, I wonder if this could be the main source of these remaining malformed tools: if the model is told to make a tool call, but doesn't "see" the definition of such a tool, it starts hallucinating and is worse at following its trained format. Your final example is particularly helpful since it only defines the tool in the system message, rather than in the format it expects.

If this is the case, and you're still seeing issues with opencode, I wonder if there's perhaps a bug somewhere that's dropping some of their tool definitions (or they're incorrect?). Or it could just be that this is an easier form to try to get a repro for

@drifkin commented on GitHub (Apr 8, 2026): One thing that's interesting about all these examples is that `"tools"` is either not provided, or it contains invalid entries (e.g., the first one has a valid tool definition for `write_file`, but then `glob` is missing the `{ "type": "function", "function}: { ... } }` "wrapper". Beyond us needing better warnings instead of silently failing to parse unexpected tool shapes, I wonder if this could be the main source of these remaining malformed tools: if the model is told to make a tool call, but doesn't "see" the definition of such a tool, it starts hallucinating and is worse at following its trained format. Your final example is particularly helpful since it only defines the tool in the system message, rather than in the format it expects. If this is the case, and you're still seeing issues with opencode, I wonder if there's perhaps a bug somewhere that's dropping some of their tool definitions (or they're incorrect?). Or it could just be that this is an easier form to try to get a repro for

GiteaMirror commented

2026-04-22 20:07:52 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

Those are surely not the case with opencode. That was just me trying to make an API call too quickly. It seems that it doesn't really matter if the definition is there or not. I noticed the error when I switched to trying llama.cpp (which doesn't seem to have tool call issues at all, but I need more testing to say for sure). So I would assume it is a red herring.

@kevincox commented on GitHub (Apr 8, 2026): Those are surely not the case with opencode. That was just me trying to make an API call too quickly. It seems that it doesn't really matter if the definition is there or not. I noticed the error when I switched to trying llama.cpp (which doesn't seem to have tool call issues at all, but I need more testing to say for sure). So I would assume it is a red herring.

GiteaMirror commented

2026-04-22 20:07:53 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

Yeah, found a case with a valid tool call definition (I think)

curl -if "http://localhost:11434/api/chat?seed=$seed" -H "Content-Type: application/json" -d '{
  "model": "gemma4:e4b",
  "messages": [
        {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
        {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."},
        {"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."}
        ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "write_file",
        "description": "Write the data to a file.",
        "parameters": {
          "type": "object",
          "required": ["file", "contents"],
          "properties": {
            "file": {"type": "string", "description": "The path of the file"},
            "contents": {"type": "string", "description": "The new contents of the file"}
          }
        }
      }
    },
{
      "type": "function",
      "function": {
  "name": "glob",
  "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
  "parameters": {
    "type": "object",
    "parameters": {
      "path": {
        "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.",
        "type": "string"
      },
      "pattern": {
        "description": "The glob pattern to match files against",
        "type": "string",
        "required": true
      }
    },
    "required": [
      "pattern"
    ]
  }}
}
  ],
  "options": {
    "seed": 19
  }
}'

time=2026-04-07T21:35:01.840-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*/*.js\", path: \"src\"}"

Might have been coincidence but it did take a lot longer to find a seed. So maybe this makes the model less likely to fail the call.

@kevincox commented on GitHub (Apr 8, 2026): Yeah, found a case with a valid tool call definition (I think) ``` curl -if "http://localhost:11434/api/chat?seed=$seed" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."}, {"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."} ], "stream": false, "tools": [ { "type": "function", "function": { "name": "write_file", "description": "Write the data to a file.", "parameters": { "type": "object", "required": ["file", "contents"], "properties": { "file": {"type": "string", "description": "The path of the file"}, "contents": {"type": "string", "description": "The new contents of the file"} } } } }, { "type": "function", "function": { "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "parameters": { "path": { "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.", "type": "string" }, "pattern": { "description": "The glob pattern to match files against", "type": "string", "required": true } }, "required": [ "pattern" ] }} } ], "options": { "seed": 19 } }' ``` ``` time=2026-04-07T21:35:01.840-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*/*.js\", path: \"src\"}" ``` Might have been coincidence but it did take a lot longer to find a seed. So maybe this makes the model less likely to fail the call.

GiteaMirror commented

2026-04-22 20:07:54 -05:00

@drifkin commented on GitHub (Apr 8, 2026):

The arguments belong under function.parameters.properties instead of function.parameters.parameters, so this example still only has the tool partially defined. Here's a corrected one:

curl -if "http://localhost:11434/api/chat" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e4b",
    "messages": [
      {
        "role": "system",
        "content": "You have a `glob` tool available."
      },
      {
        "role": "user",
        "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."
      },
      {
        "role": "assistant",
        "content": "The files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that are not directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add flashy animations to the solver part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that is where core application logic often resides, or check for a `components/` directory if one exists. Since there is a `src/` directory, I will explore that for components related to the game or solving."
      }
    ],
    "stream": false,
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "write_file",
          "description": "Write the data to a file.",
          "parameters": {
            "type": "object",
            "properties": {
              "file": {
                "type": "string",
                "description": "The path of the file"
              },
              "contents": {
                "type": "string",
                "description": "The new contents of the file"
              }
            },
            "required": ["file", "contents"]
          }
        }
      },
      {
        "type": "function",
        "function": {
          "name": "glob",
          "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
          "parameters": {
            "type": "object",
            "properties": {
              "path": {
                "type": "string",
                "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided."
              },
              "pattern": {
                "type": "string",
                "description": "The glob pattern to match files against"
              }
            },
            "required": ["pattern"]
          }
        }
      }
    ],
    "options": {
      "seed": 19
    }
  }'

@drifkin commented on GitHub (Apr 8, 2026): The arguments belong under `function.parameters.properties` instead of `function.parameters.parameters`, so this example still only has the tool partially defined. Here's a corrected one: <details> ``` curl -if "http://localhost:11434/api/chat" \ -H "Content-Type: application/json" \ -d '{ "model": "gemma4:e4b", "messages": [ { "role": "system", "content": "You have a `glob` tool available." }, { "role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project." }, { "role": "assistant", "content": "The files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that are not directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add flashy animations to the solver part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that is where core application logic often resides, or check for a `components/` directory if one exists. Since there is a `src/` directory, I will explore that for components related to the game or solving." } ], "stream": false, "tools": [ { "type": "function", "function": { "name": "write_file", "description": "Write the data to a file.", "parameters": { "type": "object", "properties": { "file": { "type": "string", "description": "The path of the file" }, "contents": { "type": "string", "description": "The new contents of the file" } }, "required": ["file", "contents"] } } }, { "type": "function", "function": { "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "properties": { "path": { "type": "string", "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided." }, "pattern": { "type": "string", "description": "The glob pattern to match files against" } }, "required": ["pattern"] } } } ], "options": { "seed": 19 } }' ``` </details>

GiteaMirror commented

2026-04-22 20:07:54 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

Thanks for the fixing. Some better validation here would definitely be a UX improvement :)

But after a bit of seed-searching I quickly found an example:

curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{
  "model": "gemma4:e4b",
  "messages": [
        {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
        {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."},
        {"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."}
        ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "write_file",
        "description": "Write the data to a file.",
        "parameters": {
          "type": "object",
          "required": ["file", "contents"],
          "properties": {
            "file": {"type": "string", "description": "The path of the file"},
            "contents": {"type": "string", "description": "The new contents of the file"}
          }
        }
      }
    },
{
      "type": "function",
      "function": {
  "name": "glob",
  "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
  "parameters": {
    "type": "object",
    "parameters": {
      "path": {
        "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.",
        "type": "string"
      },
      "pattern": {
        "description": "The glob pattern to match files against",
        "type": "string",
        "required": true
      }
    },
    "required": [
      "pattern"
    ]
  }}
}
  ],
  "options": {
    "seed": 25
  }
}'

time=2026-04-08T08:05:01.998-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '\\'' after top-level value\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.js\"}'}"

@kevincox commented on GitHub (Apr 8, 2026): Thanks for the fixing. Some better validation here would definitely be a UX improvement :) But after a bit of seed-searching I quickly found an example: ``` curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."}, {"role": "assistant", "content": "he files suggest `web/index.html` is the entry point, and `web/service-worker.js` handles caching. The actual UI and game logic components are likely managed by JavaScript/JSX/TSX files that aren''t directly in the root `web/` folder, but perhaps loaded dynamically or in a different directory relative to service-worker.js.\n\nTo add *flashy animations* to the *solver* part of the web UI, I need to find the main component that renders the game board/solver interface.\n\nI will check the `src/` directory now, as that''s where core application logic often resides, or check for a `components/` directory if one exists. Since there''s a `src/` directory, I''ll explore that for components related to the game or solving."} ], "stream": false, "tools": [ { "type": "function", "function": { "name": "write_file", "description": "Write the data to a file.", "parameters": { "type": "object", "required": ["file", "contents"], "properties": { "file": {"type": "string", "description": "The path of the file"}, "contents": {"type": "string", "description": "The new contents of the file"} } } } }, { "type": "function", "function": { "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "parameters": { "path": { "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.", "type": "string" }, "pattern": { "description": "The glob pattern to match files against", "type": "string", "required": true } }, "required": [ "pattern" ] }} } ], "options": { "seed": 25 } }' ``` ``` time=2026-04-08T08:05:01.998-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '\\'' after top-level value\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.js\"}'}" ```

GiteaMirror commented

2026-04-22 20:07:56 -05:00

@kevincox commented on GitHub (Apr 8, 2026):

And a slightly cut-down case:

curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{
  "model": "gemma4:e4b",
  "messages": [
    {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"},
    {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."}
  ],
  "stream": false,
  "tools": [{
    "type": "function",
    "function": {
      "name": "glob",
      "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead",
      "parameters": {
        "type": "object",
        "parameters": {
          "path": {
            "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.",
            "type": "string"
          },
          "pattern": {
            "description": "The glob pattern to match files against",
            "type": "string",
            "required": true
          }
        },
        "required": [
          "pattern"
        ]
      }
    }
  }],
  "options": {
    "seed": 68
  }
}'

time=2026-04-08T08:21:34.880-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx}\", path: \".\"}"

But I really don't think the specific examples are too important. It just shows that cases like this are easy to hit. I've also seen examples with other tool calls.

@kevincox commented on GitHub (Apr 8, 2026): And a slightly cut-down case: ``` curl -if "http://localhost:11434/api/chat" -H "Content-Type: application/json" -d '{ "model": "gemma4:e4b", "messages": [ {"role": "system", "content": "You have a `glob` tool available: { \"name\": \"glob\", \"description\": \"Fast file pattern matching tool that works with any codebase size\\n- Supports glob patterns like \\\"**/*.js\\\" or \\\"src/**/*.ts\\\"\\n- Returns matching file paths sorted by modification time\\n- Use this tool when you need to find files by name patterns\\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead\", \"parameters\": { \"type\": \"object\", \"properties\": { \"path\": { \"description\": \"The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \\\"undefined\\\" or \\\"null\\\" - simply omit it for the default behavior. Must be a valid directory path if provided.\", \"type\": \"string\" }, \"pattern\": { \"description\": \"The glob pattern to match files against\", \"type\": \"string\", \"required\": \"true\" } }, \"required\": [ \"pattern\" ] } }"}, {"role": "user", "content": "Look at the web-ui and add a few flashy animations to make using the solver to play games more fun. Start by using your `glob` tool to scope out the files in the project."} ], "stream": false, "tools": [{ "type": "function", "function": { "name": "glob", "description": "Fast file pattern matching tool that works with any codebase size\n- Supports glob patterns like \"**/*.js\" or \"src/**/*.ts\"\n- Returns matching file paths sorted by modification time\n- Use this tool when you need to find files by name patterns\n- When you are doing an open-ended search that may require multiple rounds of globbing and grepping, use the Task tool instead", "parameters": { "type": "object", "parameters": { "path": { "description": "The directory to search in. If not specified, the current working directory will be used. IMPORTANT: Omit this field to use the default directory. DO NOT enter \"undefined\" or \"null\" - simply omit it for the default behavior. Must be a valid directory path if provided.", "type": "string" }, "pattern": { "description": "The glob pattern to match files against", "type": "string", "required": true } }, "required": [ "pattern" ] } } }], "options": { "seed": 68 } }' ``` ``` time=2026-04-08T08:21:34.880-04:00 level=WARN source=gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character 'p' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:glob{pattern: \"**/*.{js,jsx,ts,tsx}\", path: \".\"}" ``` But I really don't think the specific examples are too important. It just shows that cases like this are easy to hit. I've also seen examples with other tool calls.

GiteaMirror commented

2026-04-22 20:07:57 -05:00

@warabe1122 commented on GitHub (Apr 9, 2026):

Environment

macOS Tahoe 26.3.1 (a)
Mac Studio M2 Max 64GB
Ollama 0.20.4
Model: gemma4:26b (Q4_K_M)
Client: OpenClaw (openclaw 2026.4.1) using /api/chat

Reproduction

Using OpenClaw's writer agent with gemma4:26b to edit a ~15K-character Markdown file. The model generates correct edit content but the tool call JSON parsing fails every time.

Log output (Ollama 0.20.4)

Attempt 1

gemma4.go:299 msg="gemma4 tool call parsing failed"
error="invalid character '<' after object key:value pair"

Attempt 2

gemma4.go:299 msg="gemma4 tool call parsing failed"
error="invalid character ']' after object key:value pair\nrepair failed to produce valid JSON arguments"

In both cases the raw content field shows Gemma4 using its native <|"|> string delimiters inside call:edit{edits:[{newText:<|"|>...

Key observation

The model IS generating semantically correct edits. The content inside newText / oldText is valid and relevant. The failure is in Ollama's gemma4.go parser not handling the <|"|> string delimiters that Gemma4 uses in its native tool call format.

This is blocking all file-editing tool calls with gemma4:26b, making it unusable as an agentic writer despite the model's actual capability being fine.

Updated from 0.20.0 to 0.20.4, same issue persists.

@warabe1122 commented on GitHub (Apr 9, 2026): ## Environment - macOS Tahoe 26.3.1 (a) - Mac Studio M2 Max 64GB - Ollama 0.20.4 - Model: gemma4:26b (Q4_K_M) - Client: OpenClaw (openclaw 2026.4.1) using `/api/chat` ## Reproduction Using OpenClaw's writer agent with gemma4:26b to edit a ~15K-character Markdown file. The model generates correct edit content but the tool call JSON parsing fails every time. ## Log output (Ollama 0.20.4) ### Attempt 1 ``` gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character '<' after object key:value pair" ``` ### Attempt 2 ``` gemma4.go:299 msg="gemma4 tool call parsing failed" error="invalid character ']' after object key:value pair\nrepair failed to produce valid JSON arguments" ``` In both cases the raw `content` field shows Gemma4 using its native `<|"|>` string delimiters inside `call:edit{edits:[{newText:<|"|>...` ## Key observation The model IS generating semantically correct edits. The content inside `newText` / `oldText` is valid and relevant. The failure is in Ollama's `gemma4.go` parser not handling the `<|"|>` string delimiters that Gemma4 uses in its native tool call format. This is blocking all file-editing tool calls with gemma4:26b, making it unusable as an agentic writer despite the model's actual capability being fine. Updated from 0.20.0 to 0.20.4, same issue persists.

GiteaMirror commented

2026-04-22 20:07:58 -05:00

@drifkin commented on GitHub (Apr 9, 2026):

Key observation

The model IS generating semantically correct edits. The content inside newText / oldText is valid and relevant. The failure is in Ollama's gemma4.go parser not handling the <|"|> string delimiters that Gemma4 uses in its native tool call format.

This is blocking all file-editing tool calls with gemma4:26b, making it unusable as an agentic writer despite the model's actual capability being fine.

Updated from 0.20.0 to 0.20.4, same issue persists.

Do you have an example raw call so I can check whether it is indeed well-formed? And if it is, then I can fix whatever gap the parser has (this would be the first well-formed example provided in this thread that misparses).

@drifkin commented on GitHub (Apr 9, 2026): > ## Key observation > The model IS generating semantically correct edits. The content inside `newText` / `oldText` is valid and relevant. The failure is in Ollama's `gemma4.go` parser not handling the `<|"|>` string delimiters that Gemma4 uses in its native tool call format. > > This is blocking all file-editing tool calls with gemma4:26b, making it unusable as an agentic writer despite the model's actual capability being fine. > > Updated from 0.20.0 to 0.20.4, same issue persists. Do you have an example raw call so I can check whether it is indeed well-formed? And if it is, then I can fix whatever gap the parser has (this would be the first well-formed example provided in this thread that misparses).

GiteaMirror commented

2026-04-22 20:07:59 -05:00

@warabe1122 commented on GitHub (Apr 10, 2026):

Thanks for jumping on this! Here are captured raw parser failures from my logs. Environment:

Ollama 0.20.4 (same issue on 0.20.0)
gemma4:26b Q4_K_M
macOS / M2 Max
Agent harness: openclaude / OpenClaw writer agent editing markdown files

Note: I've since uninstalled gemma4:26b, so I can't capture fresh traffic right now, but the content= field in the warnings below is the exact text the parser was given. Happy to reinstall and capture a full /api/chat request/response pair on whichever pattern is most useful.

Pattern A — simple exec, embedded quotes

[gemma4.go:287] invalid character 'v' after object key:value pair
content: call:exec{command:<|"|>ls -F "vault/"<|"|>}

[gemma4.go:287] invalid character '"' after object key:value pair
content: call:exec{command:<|"|>summarize "https://www.youtube.com/watch?v=Rl9CF8mslaQ" --youtube auto --extract<|"|>}

Semantically both are correct — ls -F "vault/" and the summarize invocation are exactly what I asked for. The gap looks specific to <|"|>-delimited string values that themselves contain ".

Pattern C — file edit, `edits:[{newText, oldText}, ...]`

This is the one that made gemma4:26b unusable as a writer agent. Note it's hitting gemma4.go:299 and the error mentions the repair path, so it's going through repair and still failing:

[gemma4.go:299] invalid character ']' after object key:value pair
                repair failed to produce valid JSON arguments
timestamp: 2026-04-09T16:49:47+09:00
content: call:edit{edits:[
  {newText:<|"|># Gemma4 + openclaude speed optimization guide...<|"|>,
   oldText:<|"|># Gemma4 + openclaude speed optimization guide...<|"|>},
  {newText:<|"|>## 14. Methods tried but not adopted\n\n### 1. Model quantization...<|"|>,
   oldText:<|"|>## 14. Methods tried but not adopted\n<|"|>}
]}

Semantically this is a perfectly valid {newText, oldText} edit pair list targeting a markdown file. The gap looks like (a) <|"|>-delimited strings containing newlines and ", combined with (b) the array-of-objects shape around edits:[...]. I have multiple captures of this exact pattern — happy to drop the raw log lines in a gist if that helps.

Let me know what's most useful — raw log gist, or a fresh /api/chat capture on Pattern A or C once I reinstall. Thanks for looking at this.

@warabe1122 commented on GitHub (Apr 10, 2026): Thanks for jumping on this! Here are captured raw parser failures from my logs. Environment: - Ollama 0.20.4 (same issue on 0.20.0) - `gemma4:26b` Q4_K_M - macOS / M2 Max - Agent harness: openclaude / OpenClaw writer agent editing markdown files Note: I've since uninstalled gemma4:26b, so I can't capture fresh traffic right now, but the `content=` field in the warnings below is the exact text the parser was given. Happy to reinstall and capture a full `/api/chat` request/response pair on whichever pattern is most useful. ## Pattern A — simple exec, embedded quotes ``` [gemma4.go:287] invalid character 'v' after object key:value pair content: call:exec{command:<|"|>ls -F "vault/"<|"|>} ``` ``` [gemma4.go:287] invalid character '"' after object key:value pair content: call:exec{command:<|"|>summarize "https://www.youtube.com/watch?v=Rl9CF8mslaQ" --youtube auto --extract<|"|>} ``` Semantically both are correct — `ls -F "vault/"` and the `summarize` invocation are exactly what I asked for. The gap looks specific to `<|"|>`-delimited string values that themselves contain `"`. ## Pattern C — file edit, `edits:[{newText, oldText}, ...]` This is the one that made gemma4:26b unusable as a writer agent. Note it's hitting `gemma4.go:299` and the error mentions the repair path, so it's going through repair and still failing: ``` [gemma4.go:299] invalid character ']' after object key:value pair repair failed to produce valid JSON arguments timestamp: 2026-04-09T16:49:47+09:00 content: call:edit{edits:[ {newText:<|"|># Gemma4 + openclaude speed optimization guide...<|"|>, oldText:<|"|># Gemma4 + openclaude speed optimization guide...<|"|>}, {newText:<|"|>## 14. Methods tried but not adopted\n\n### 1. Model quantization...<|"|>, oldText:<|"|>## 14. Methods tried but not adopted\n<|"|>} ]} ``` Semantically this is a perfectly valid `{newText, oldText}` edit pair list targeting a markdown file. The gap looks like (a) `<|"|>`-delimited strings containing newlines and `"`, combined with (b) the array-of-objects shape around `edits:[...]`. I have multiple captures of this exact pattern — happy to drop the raw log lines in a gist if that helps. --- Let me know what's most useful — raw log gist, or a fresh `/api/chat` capture on Pattern A or C once I reinstall. Thanks for looking at this.

GiteaMirror commented

2026-04-22 20:08:00 -05:00

@drifkin commented on GitHub (Apr 11, 2026):

thanks @warabe1122 for the logs, I'll take a look at them soon. Earlier today Google released a set of changes to their Gemma 4 format, and we have a new prerelease up that incorporates those same changes: https://github.com/ollama/ollama/releases/tag/v0.20.6-rc0

I'd be curious if using this pre-release fixes some of the tool calling issues you've been running into.

@drifkin commented on GitHub (Apr 11, 2026): thanks @warabe1122 for the logs, I'll take a look at them soon. Earlier today Google released a set of changes to their Gemma 4 format, and we have a new prerelease up that incorporates those same changes: https://github.com/ollama/ollama/releases/tag/v0.20.6-rc0 I'd be curious if using this pre-release fixes some of the tool calling issues you've been running into.

GiteaMirror commented

2026-04-22 20:08:02 -05:00

@drifkin commented on GitHub (Apr 11, 2026):

@warabe1122: looking more closely at your logs: Pattern A is almost certainly fixed by Ollama 0.20.4, so I wonder if maybe you're still running a stale (i.e., 0.20.0) server? Running ollama --version can help check this.

Pattern C is still a problem (assuming v0.20.6-rc0 and later will still generate such a call), appears to be from those newlines. I don't think the upstream parser would handle that either, but I'll either modify ours to accept it or make it part of the repair process, I don't see any reason not to. If you could post a full trace in a gist it would be useful to make sure that the newline is the only issue. Thank you!

@drifkin commented on GitHub (Apr 11, 2026): @warabe1122: looking more closely at your logs: Pattern A is almost certainly fixed by Ollama 0.20.4, so I wonder if maybe you're still running a stale (i.e., 0.20.0) server? Running `ollama --version` can help check this. Pattern C is still a problem (assuming v0.20.6-rc0 and later will still generate such a call), appears to be from those newlines. I don't think the upstream parser would handle that either, but I'll either modify ours to accept it or make it part of the repair process, I don't see any reason not to. If you could post a full trace in a gist it would be useful to make sure that the newline is the only issue. Thank you!

GiteaMirror commented

2026-04-22 20:08:03 -05:00

@warabe1122 commented on GitHub (Apr 11, 2026):

You were right — I double-checked my log headers:

server-4.log: Listening on 127.0.0.1:11434 (version 0.20.0) — this is where Pattern A came from (2026-04-03)
server-1.log: Listening on 127.0.0.1:11434 (version 0.20.4) — this is where Pattern C came from (2026-04-09)

So Pattern A was on stale 0.20.0 and is consistent with being fixed by 0.20.4. Sorry for the noise there.

Pattern C is on 0.20.4 — full trace in a gist here:

https://gist.github.com/warabe1122/d2835d3c2b2adabcdc72c816b933b26e

It includes the raw log line, a decoded/pretty-printed version of the content= field so the structure is easy to read, and notes on what I think is going on (newlines inside <|"|>, plus the edits:[ ... ] array-of-objects shape). The edited file was a Japanese markdown doc, so the strings contain CJK + **bold** + embedded ", in case any of those matter.

I'll reinstall gemma4:26b on v0.20.6-rc0 and try to reproduce Pattern C with a fresh /api/chat capture — will post back if I can get a clean trace on the prerelease. Thanks for looking into this!

@warabe1122 commented on GitHub (Apr 11, 2026): You were right — I double-checked my log headers: - `server-4.log`: `Listening on 127.0.0.1:11434 (version 0.20.0)` — this is where **Pattern A** came from (2026-04-03) - `server-1.log`: `Listening on 127.0.0.1:11434 (version 0.20.4)` — this is where **Pattern C** came from (2026-04-09) So Pattern A was on stale 0.20.0 and is consistent with being fixed by 0.20.4. Sorry for the noise there. Pattern C is on 0.20.4 — full trace in a gist here: https://gist.github.com/warabe1122/d2835d3c2b2adabcdc72c816b933b26e It includes the raw log line, a decoded/pretty-printed version of the `content=` field so the structure is easy to read, and notes on what I think is going on (newlines inside `<|"|>`, plus the `edits:[ ... ]` array-of-objects shape). The edited file was a Japanese markdown doc, so the strings contain CJK + `**bold**` + embedded `"`, in case any of those matter. I'll reinstall `gemma4:26b` on `v0.20.6-rc0` and try to reproduce Pattern C with a fresh `/api/chat` capture — will post back if I can get a clean trace on the prerelease. Thanks for looking into this!

GiteaMirror commented

2026-04-22 20:08:04 -05:00

@drifkin commented on GitHub (Apr 11, 2026):

Great, thanks so much! It wouldn't surprise me if it still repros, it might just be the model seeing a lot of newlines and then getting confused about the tool calling format itself not usually having newlines. I have a fix up at https://github.com/ollama/ollama/pull/15494 that relaxes the parser into allowing whitespace before tool call keys, which I expect to fix this issue. You could run that branch yourself via go run . serve if you're curious, but should have a second release candidate up soon-ish that has that patch in it, will post here when it's ready.

@drifkin commented on GitHub (Apr 11, 2026): Great, thanks so much! It wouldn't surprise me if it still repros, it might just be the model seeing a lot of newlines and then getting confused about the tool calling format itself not usually having newlines. I have a fix up at https://github.com/ollama/ollama/pull/15494 that relaxes the parser into allowing whitespace before tool call keys, which I expect to fix this issue. You could run that branch yourself via `go run . serve` if you're curious, but should have a second release candidate up soon-ish that has that patch in it, will post here when it's ready.

GiteaMirror commented

2026-04-22 20:08:04 -05:00

@warabe1122 commented on GitHub (Apr 11, 2026):

Thanks, that matches what I was seeing. I'll hold off on building from the branch and wait for the next RC — happy to re-test Pattern C against it with a fresh /api/chat capture once it drops. Appreciate the quick turnaround!

@warabe1122 commented on GitHub (Apr 11, 2026): Thanks, that matches what I was seeing. I'll hold off on building from the branch and wait for the next RC — happy to re-test Pattern C against it with a fresh `/api/chat` capture once it drops. Appreciate the quick turnaround!

GiteaMirror commented

2026-04-22 20:08:05 -05:00

@drifkin commented on GitHub (Apr 12, 2026):

@warabe1122: rc1 is up at https://github.com/ollama/ollama/releases/tag/v0.20.6-rc1

@drifkin commented on GitHub (Apr 12, 2026): @warabe1122: rc1 is up at https://github.com/ollama/ollama/releases/tag/v0.20.6-rc1

GiteaMirror commented

2026-04-22 20:08:06 -05:00

@emansom commented on GitHub (Apr 12, 2026):

@drifkin Consider looking at LiteRT-LM and how it parses and emits Gemma 4 stuff. It is the canonical reference implementation:

https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/conversation/model_data_processor/gemma4_data_processor.cc
https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/conversation/model_data_processor/gemma4_data_processor_test.cc

@emansom commented on GitHub (Apr 12, 2026): @drifkin Consider looking at LiteRT-LM and how it parses and emits Gemma 4 stuff. It is the canonical reference implementation: https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/conversation/model_data_processor/gemma4_data_processor.cc https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/conversation/model_data_processor/gemma4_data_processor_test.cc

GiteaMirror commented

2026-04-22 20:08:08 -05:00

@warabe1122 commented on GitHub (Apr 12, 2026):

Tested on v0.20.6-rc1 — Pattern C is fixed!

Environment

Ollama v0.20.6-rc1 (binary from release assets, running via ollama serve)
gemma4:26b (fresh pull, ID 5571076f3d70)
macOS / M2 Max 64GB

Test 1 — English, short multiline `edits:[...]`

Sent a /api/chat request with a tool schema matching the original failure: edit tool with edits array of {newText, oldText} pairs. Model returned a well-formed tool call with two edit pairs. Parser accepted it cleanly.

Test 2 — Japanese, long multiline `edits:[...]`

Same structure but with Japanese markdown content, **bold** markup, and multi-line newText/oldText values containing \n — this is the closest match to the original Pattern C failure. Model returned a well-formed tool call, parser accepted it.

Server log

Zero gemma4 tool call parsing failed warnings in the server log across both tests.

Looks like #15494 resolved it. Thanks for the quick turnaround @drifkin!

@warabe1122 commented on GitHub (Apr 12, 2026): Tested on `v0.20.6-rc1` — Pattern C is fixed! ## Environment - Ollama `v0.20.6-rc1` (binary from [release assets](https://github.com/ollama/ollama/releases/tag/v0.20.6-rc1), running via `ollama serve`) - `gemma4:26b` (fresh pull, ID `5571076f3d70`) - macOS / M2 Max 64GB ## Test 1 — English, short multiline `edits:[...]` Sent a `/api/chat` request with a tool schema matching the original failure: `edit` tool with `edits` array of `{newText, oldText}` pairs. Model returned a well-formed tool call with two edit pairs. Parser accepted it cleanly. ## Test 2 — Japanese, long multiline `edits:[...]` Same structure but with Japanese markdown content, `**bold**` markup, and multi-line `newText`/`oldText` values containing `\n` — this is the closest match to the original Pattern C failure. Model returned a well-formed tool call, parser accepted it. ## Server log Zero `gemma4 tool call parsing failed` warnings in the server log across both tests. --- Looks like #15494 resolved it. Thanks for the quick turnaround @drifkin!

GiteaMirror commented

2026-04-22 20:08:09 -05:00

@CamJN commented on GitHub (Apr 14, 2026):

Using ollama 0.20.7:

With the js library:

const response = await ollama.chat({
    model: "gemma4:e4b",
    stream: false,
    think: false,
    keep_alive: '24h',
    options: {
      temperature: 0.8,
    },
    format: JSON.stringify(z.toJSONSchema(InvoiceSchema)),
    messages: [
      { role: "system", content: t.my_message_string, },
      { role: "user", content: t.parse_template_string.replace('{context_str}', context), },
    ]
  });

The response.message.content is "```json\n{\n "json details": "elided"\n}\n```" which has erroneous ```json and ``` markers.

@CamJN commented on GitHub (Apr 14, 2026): Using ollama 0.20.7: With the js library: ```javascript const response = await ollama.chat({ model: "gemma4:e4b", stream: false, think: false, keep_alive: '24h', options: { temperature: 0.8, }, format: JSON.stringify(z.toJSONSchema(InvoiceSchema)), messages: [ { role: "system", content: t.my_message_string, }, { role: "user", content: t.parse_template_string.replace('{context_str}', context), }, ] }); ``` The `response.message.content` is <code>"\```json\n{\n "json details": "elided"\n}\n\```"</code> which has erroneous <code>\```json</code> and <code>\```</code> markers.

GiteaMirror commented

2026-04-22 20:08:09 -05:00

@CamJN commented on GitHub (Apr 14, 2026):

gemma4:31b has the same issue, and additionally does not follow the schema.

@CamJN commented on GitHub (Apr 14, 2026): `gemma4:31b` has the same issue, and additionally does not follow the schema.

GiteaMirror commented

2026-04-22 20:08:10 -05:00

@drifkin commented on GitHub (Apr 14, 2026):

can you open a different issue for that @CamJN, I don't think you're doing anything with tool calls, right?

@drifkin commented on GitHub (Apr 14, 2026): can you open a different issue for that @CamJN, I don't think you're doing anything with tool calls, right?

GiteaMirror commented

2026-04-22 20:08:11 -05:00

@emansom commented on GitHub (Apr 14, 2026):

@drifkin Could you explain to me why Ollama has specific Gemma 4 workarounds and testing, when that is already in llama.cpp project? It seems like a duplication of effort to me, which could result in false positives and even harder to debug issues later.

Or is Ollama using a different parser? Different from the default PEG parser in llama.cpp?
And why?

Fix in one place, different behavior/broken in other place?
Shouldn't llama.cpp be functional by default, and Ollama acting as a gateway? Not a duct-tape?

https://github.com/search?q=repo%3Aggml-org%2Fllama.cpp+gemma4&type=code
https://github.com/ggml-org/llama.cpp/pull/21760
https://github.com/ggml-org/llama.cpp/pull/21418
https://github.com/ggml-org/llama.cpp/pull/21326

@emansom commented on GitHub (Apr 14, 2026): @drifkin Could you explain to me why Ollama has specific Gemma 4 workarounds and testing, when that is already in llama.cpp project? It seems like a duplication of effort to me, which could result in false positives and even harder to debug issues later. Or is Ollama using a different parser? Different from the default PEG parser in llama.cpp? And why? Fix in one place, different behavior/broken in other place? Shouldn't llama.cpp be functional by default, and Ollama acting as a gateway? Not a duct-tape? https://github.com/search?q=repo%3Aggml-org%2Fllama.cpp+gemma4&type=code https://github.com/ggml-org/llama.cpp/pull/21760 https://github.com/ggml-org/llama.cpp/pull/21418 https://github.com/ggml-org/llama.cpp/pull/21326

GiteaMirror commented

2026-04-22 20:08:12 -05:00

@drifkin commented on GitHub (Apr 14, 2026):

yup, we use our own renderers and parsers, for quite a while now. This all predates the parsers you're mentioning. We have several underlying inference engines and having the parsers at a different layer is useful to us.

@drifkin commented on GitHub (Apr 14, 2026): yup, we use our own renderers and parsers, for quite a while now. This all predates the parsers you're mentioning. We have several underlying inference engines and having the parsers at a different layer is useful to us.

GiteaMirror commented

2026-04-22 20:08:12 -05:00

@emansom commented on GitHub (Apr 14, 2026):

yup, we use our own renderers and parsers, for quite a while now. This all predates the parsers you're mentioning. We have several underlying inference engines and having the parsers at a different layer is useful to us.

I see. Appreciate the insight/context!

Are you aware of the LLGuidance support in llama.cpp (and other inference engines)?

https://github.com/ggml-org/llama.cpp/blob/master/docs/llguidance.md

Ollama could utilize that to achieve better performance, and enforce better parsers; while reducing code complexity (Lark syntax vs custom parsers)?

I'm positive @mmoskal can concur.

@emansom commented on GitHub (Apr 14, 2026): > yup, we use our own renderers and parsers, for quite a while now. This all predates the parsers you're mentioning. We have several underlying inference engines and having the parsers at a different layer is useful to us. I see. Appreciate the insight/context! Are you aware of the LLGuidance support in llama.cpp (and [other inference engines](https://github.com/guidance-ai/llguidance?tab=readme-ov-file#integrations))? https://github.com/ggml-org/llama.cpp/blob/master/docs/llguidance.md Ollama could utilize that to achieve better performance, and enforce better parsers; while reducing code complexity ([Lark syntax](https://github.com/guidance-ai/llguidance/blob/main/docs/syntax.md) vs custom parsers)? I'm positive @mmoskal can concur.

GiteaMirror commented

2026-04-22 20:08:13 -05:00

@drifkin commented on GitHub (Apr 15, 2026):

thanks for the links! We do want to do more with structured outputs/constrained decoding in the future. And particularly for tool calls.

@drifkin commented on GitHub (Apr 15, 2026): thanks for the links! We do want to do more with structured outputs/constrained decoding in the future. And particularly for tool calls.

GiteaMirror commented

2026-04-22 20:08:14 -05:00

@emansom commented on GitHub (Apr 15, 2026):

thanks for the links! We do want to do more with structured outputs/constrained decoding in the future. And particularly for tool calls.

I see @ParthSareen also had some thoughts about it
https://parthsareen.com/writings/sampling/

Seems it would be very beneficial for guiding tool calls:
https://github.com/guidance-ai/llguidance/blob/main/docs/toolcalls.md

@emansom commented on GitHub (Apr 15, 2026): > thanks for the links! We do want to do more with structured outputs/constrained decoding in the future. And particularly for tool calls. I see @ParthSareen also had some thoughts about it https://parthsareen.com/writings/sampling/ Seems it would be very beneficial for guiding tool calls: https://github.com/guidance-ai/llguidance/blob/main/docs/toolcalls.md

GiteaMirror commented

2026-04-22 20:08:14 -05:00

@jk105jk105 commented on GitHub (Apr 17, 2026):

Hi, I'm still getting msg="gemma4 tool call parsing failed".

Full error

level=WARN source=gemma4.go:306 msg="gemma4 tool call parsing failed" error="invalid character '@' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:es_search{body:{aggs:{unique_source_ips:{terms:{field:<|\"|>client_ip.keyword<|\"|>,size:1000}}},query:{bool:{must:[{match:{message:<|\"|>openvpn<|\"|>}},{term:{\"sysloghost.keyword\":\"testhost\"}}],filter:[{range:{@timestamp:{gte:<|\"|>2026-03-01T00:00:00Z<|\"|>,lt:<|\"|>2026-04-01T00:00:00Z<|\"|>}}}]}},sort:[{@timestamp:{\"order\":\"desc\"}}],size:0},path:<|\"|>logstash-2026.03.05-000059/_search<|\"|>}"

Also the invalid character is inconsistent.

Context: I'm trying to have gemma4 generate a elasticsearch query so that I can use in a search API tool call.

System
ollama: 0.21.0
os: linux
model: gemma4:eb4

Let me know if you need more information. Thanks!

@jk105jk105 commented on GitHub (Apr 17, 2026): Hi, I'm still getting msg="gemma4 tool call parsing failed". Full error ``` level=WARN source=gemma4.go:306 msg="gemma4 tool call parsing failed" error="invalid character '@' looking for beginning of object key string\nrepair failed to produce valid JSON arguments" content="call:es_search{body:{aggs:{unique_source_ips:{terms:{field:<|\"|>client_ip.keyword<|\"|>,size:1000}}},query:{bool:{must:[{match:{message:<|\"|>openvpn<|\"|>}},{term:{\"sysloghost.keyword\":\"testhost\"}}],filter:[{range:{@timestamp:{gte:<|\"|>2026-03-01T00:00:00Z<|\"|>,lt:<|\"|>2026-04-01T00:00:00Z<|\"|>}}}]}},sort:[{@timestamp:{\"order\":\"desc\"}}],size:0},path:<|\"|>logstash-2026.03.05-000059/_search<|\"|>}" ``` Also the invalid character is inconsistent. Context: I'm trying to have gemma4 generate a elasticsearch query so that I can use in a search API tool call. System ollama: 0.21.0 os: linux model: gemma4:eb4 Let me know if you need more information. Thanks!

GiteaMirror commented

2026-04-22 20:08:15 -05:00

@PureBlissAK commented on GitHub (Apr 18, 2026):

🤖 Automated Triage & Analysis Report

Issue: #15315
Analyzed: 2026-04-18T18:22:38.782881

Analysis

Type: unknown
Severity: medium
Components: unknown

Implementation Plan

Effort: medium
Steps:

This issue has been triaged and marked for implementation.

@PureBlissAK commented on GitHub (Apr 18, 2026):  ## 🤖 Automated Triage & Analysis Report **Issue**: #15315 **Analyzed**: 2026-04-18T18:22:38.782881 ### Analysis - **Type**: unknown - **Severity**: medium - **Components**: unknown ### Implementation Plan - **Effort**: medium - **Steps**: *This issue has been triaged and marked for implementation.*

Sign in to join this conversation.

Branches Tags

main

parth-mlx-decode-checkpoints

dhiltgen/ci

hoyyeva/editor-config-repair

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

hoyyeva/launch-backup-ux

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

brucemacd/download-before-remove

parth/update-claude-docs

parth-anthropic-reference-images-path

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#35555

[GH-ISSUE #15315] gemma4:e4b with ollama 0.20.1 still has tool parsing errors #35555

What is the issue?

Relevant log output

OS

GPU

CPU

Ollama version

Root cause analysis

What a correct parser looks like

Reproduction (minimal, no app needed)

Additional failure modes from systematic testing

References

What is the issue?

Relevant log output

OS

GPU

CPU

Ollama version

Environment

Reproduction

Log output (Ollama 0.20.4)

Attempt 1

Attempt 2

Key observation

Key observation

Pattern A — simple exec, embedded quotes

Pattern C — file edit, edits:[{newText, oldText}, ...]

Environment

Test 1 — English, short multiline edits:[...]

Test 2 — Japanese, long multiline edits:[...]

Server log

🤖 Automated Triage & Analysis Report

Analysis

Implementation Plan

Pattern C — file edit, `edits:[{newText, oldText}, ...]`

Test 1 — English, short multiline `edits:[...]`

Test 2 — Japanese, long multiline `edits:[...]`