github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15719] Bug Report: gemma4:26b infinite tool call loop via LiteLLM proxy — Ollama 0.20.6/0.20.7 #72082

New Issue
Open
opened 2026-05-05 03:25:52 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-05 03:25:52 -05:00
Owner
Copy Link

Originally created by @roxlukas on GitHub (Apr 20, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15719

What is the issue?

Bug Report: gemma4:26b infinite tool call loop via LiteLLM proxy — Ollama 0.20.6/0.20.7

Date: 2026-04-17
Reporter: legalos.ai project
Severity: Critical — production regression, renders tool calling unusable

Summary

After upgrading Ollama from 0.20.5 → 0.20.6/0.20.7, gemma4:26b enters an
infinite loop when tool results are sent via a LiteLLM proxy 1.82.3 The model ignores
the tool result and re-issues the identical tool call on every iteration.

The bug does not reproduce when calling Ollama directly (bypassing LiteLLM).
Downgrading to Ollama 0.20.5 resolves the issue.

Environment

Component Version
Ollama 0.20.7 (also 0.20.6)
Ollama model gemma4:26b (pulled 2026-04-10)
LiteLLM 1.82.3
OS Linux

Steps to Reproduce

Minimal reproducer (Python, via LiteLLM proxy)

import json, urllib.request

LITELLM_URL = 'http://<litellm-host>/v1/chat/completions'
LITELLM_KEY = '<your-key>'

tools = [{
    'type': 'function',
    'function': {
        'name': 'get_weather',
        'description': 'Get current weather for a city',
        'parameters': {
            'type': 'object',
            'properties': {'city': {'type': 'string'}},
            'required': ['city']
        }
    }
}]

def call(messages, tools=None):
    payload = {'model': 'ollama/gemma4:26b', 'messages': messages, 'stream': False}
    if tools:
        payload['tools'] = tools
    data = json.dumps(payload).encode()
    req = urllib.request.Request(LITELLM_URL, data=data, headers={
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {LITELLM_KEY}'
    })
    with urllib.request.urlopen(req, timeout=120) as r:
        return json.loads(r.read())['choices'][0]['message']

# Step 1: model generates tool call
m1 = call([{'role': 'user', 'content': 'Weather in Warsaw?'}], tools)
print('Step 1 tool_calls:', m1.get('tool_calls'))  # → [get_weather(city=Warsaw)]

# Step 2: send tool result, expect text answer
msgs = [
    {'role': 'user', 'content': 'Weather in Warsaw?'},
    {'role': 'assistant', 'content': '', 'tool_calls': m1['tool_calls']},
    {'role': 'tool', 'tool_call_id': m1['tool_calls'][0]['id'],
     'content': '{"temperature": "15C", "conditions": "sunny"}'}
]
m2 = call(msgs, tools)
print('Step 2 content:', m2.get('content'))        # → '' (BUG: should be text answer)
print('Step 2 tool_calls:', m2.get('tool_calls'))  # → [get_weather(city=Warsaw)] again!

Direct Ollama call (works correctly)

import json, urllib.request

OLLAMA_URL = 'http://<ollama-host>:11434/api/chat'

def ollama_call(messages, tools):
    payload = {'model': 'gemma4:26b', 'stream': False, 'tools': tools, 'messages': messages}
    data = json.dumps(payload).encode()
    req = urllib.request.Request(OLLAMA_URL, data=data,
                                 headers={'Content-Type': 'application/json'})
    with urllib.request.urlopen(req, timeout=120) as r:
        return json.loads(r.read())['message']

# Same scenario — Ollama returns correct text answer on step 2
m1 = ollama_call([{'role': 'user', 'content': 'Weather in Warsaw?'}], tools)
msgs = [
    {'role': 'user', 'content': 'Weather in Warsaw?'},
    {'role': 'assistant', 'content': '', 'tool_calls': m1['tool_calls']},
    {'role': 'tool', 'content': '{"temperature": "15C", "conditions": "sunny"}'}
]
m2 = ollama_call(msgs, tools)
print('content:', m2['content'])  # → "The weather in Warsaw is 15°C and sunny." ✓

Observed vs Expected Behavior

Ollama 0.20.7 via LiteLLM — BUG

Step 1: content='', tool_calls=[get_weather(city=Warsaw)]   ← model calls tool
Step 2: content='', tool_calls=[get_weather(city=Warsaw)]   ← BUG: ignores result, loops
Step 3: content='', tool_calls=[get_weather(city=Warsaw)]   ← same
Step 4: content='', tool_calls=[get_weather(city=Warsaw)]   ← same
Step 5: content='', tool_calls=[get_weather(city=Warsaw)]   ← same

Ollama 0.20.5 via LiteLLM — OK

Step 1: content='', tool_calls=[get_weather(city=Warsaw)]   ← model calls tool
Step 2: content='The weather in Warsaw is 15°C and sunny.'  ← correct text answer ✓

Ollama 0.20.7 directly (no proxy) — OK

Step 1: content='', tool_calls=[get_weather(city=Warsaw)]   ← model calls tool
Step 2: content='The weather in Warsaw is 15°C and sunny.'  ← correct text answer ✓

Evidence from Production Logs

Real production data showing the loop pattern (LLM call log, n8n.llm_log):

logID  promptTokens  completionTokens  tool_called
2358   5232          18                calendar_read(overdue=true)
2359   5237          18                calendar_read(overdue=true)   ← +5 tokens only
2360   5239          18                calendar_read(overdue=true)   ← +2 tokens
2361   5241          18                calendar_read(overdue=true)   ← +2 tokens
2362   5243          18                calendar_read(overdue=true)   ← +2 tokens

Key observation: promptTokens grows by only 2–5 per iteration, but the tool result
JSON is ~80 tokens. This confirms the tool result content was not being injected into
the model's context — only the tool_call metadata was appended.

The same loop pattern was observed across multiple tools:

  • calendar_read — identical args, empty result {events:[], tasks:[]}
  • law_lookup — identical args, non-empty result (model ignored rich legal articles)
  • dokuwiki_search — identical args, empty result (no wiki pages found)
  • deep_research — identical args (multi-step internal tool)

Hypothesis

Ollama 0.20.6 changelog states:

"Gemma 4 tool calling ability is improved and updated to use Google's latest post-launch fixes"

This likely changed the chat template used to format tool result messages for the
model. LiteLLM 1.82.3 transforms OpenAI-format tool messages before forwarding to
Ollama, and this transformation is now incompatible with the updated template.

When calling Ollama directly, the native /api/chat format is used correctly.
When going through LiteLLM, the tool result message format sent to Ollama no longer
matches what gemma4:26b's updated template expects — causing the model to not "see"
the result and re-issue the same tool call.

Additional note: LiteLLM's model info for gemma4:26b reports
"supports_function_calling": false, which may cause LiteLLM to apply a fallback
prompt-injection path instead of passing tool messages natively.

Workaround

Downgrade Ollama to 0.20.5.

Related

Relevant log output

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.20.7

Originally created by @roxlukas on GitHub (Apr 20, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15719 ### What is the issue? # Bug Report: gemma4:26b infinite tool call loop via LiteLLM proxy — Ollama 0.20.6/0.20.7 **Date:** 2026-04-17 **Reporter:** legalos.ai project **Severity:** Critical — production regression, renders tool calling unusable ## Summary After upgrading Ollama from **0.20.5 → 0.20.6/0.20.7**, `gemma4:26b` enters an infinite loop when tool results are sent via a **LiteLLM proxy** 1.82.3 The model ignores the tool result and re-issues the identical tool call on every iteration. The bug does **not** reproduce when calling Ollama directly (bypassing LiteLLM). Downgrading to Ollama 0.20.5 resolves the issue. --- ## Environment | Component | Version | |--------------|-------------------------------| | Ollama | 0.20.7 (also 0.20.6) | | Ollama model | `gemma4:26b` (pulled 2026-04-10) | | LiteLLM | 1.82.3 | | OS | Linux | --- ## Steps to Reproduce ### Minimal reproducer (Python, via LiteLLM proxy) ```python import json, urllib.request LITELLM_URL = 'http://<litellm-host>/v1/chat/completions' LITELLM_KEY = '<your-key>' tools = [{ 'type': 'function', 'function': { 'name': 'get_weather', 'description': 'Get current weather for a city', 'parameters': { 'type': 'object', 'properties': {'city': {'type': 'string'}}, 'required': ['city'] } } }] def call(messages, tools=None): payload = {'model': 'ollama/gemma4:26b', 'messages': messages, 'stream': False} if tools: payload['tools'] = tools data = json.dumps(payload).encode() req = urllib.request.Request(LITELLM_URL, data=data, headers={ 'Content-Type': 'application/json', 'Authorization': f'Bearer {LITELLM_KEY}' }) with urllib.request.urlopen(req, timeout=120) as r: return json.loads(r.read())['choices'][0]['message'] # Step 1: model generates tool call m1 = call([{'role': 'user', 'content': 'Weather in Warsaw?'}], tools) print('Step 1 tool_calls:', m1.get('tool_calls')) # → [get_weather(city=Warsaw)] # Step 2: send tool result, expect text answer msgs = [ {'role': 'user', 'content': 'Weather in Warsaw?'}, {'role': 'assistant', 'content': '', 'tool_calls': m1['tool_calls']}, {'role': 'tool', 'tool_call_id': m1['tool_calls'][0]['id'], 'content': '{"temperature": "15C", "conditions": "sunny"}'} ] m2 = call(msgs, tools) print('Step 2 content:', m2.get('content')) # → '' (BUG: should be text answer) print('Step 2 tool_calls:', m2.get('tool_calls')) # → [get_weather(city=Warsaw)] again! ``` ### Direct Ollama call (works correctly) ```python import json, urllib.request OLLAMA_URL = 'http://<ollama-host>:11434/api/chat' def ollama_call(messages, tools): payload = {'model': 'gemma4:26b', 'stream': False, 'tools': tools, 'messages': messages} data = json.dumps(payload).encode() req = urllib.request.Request(OLLAMA_URL, data=data, headers={'Content-Type': 'application/json'}) with urllib.request.urlopen(req, timeout=120) as r: return json.loads(r.read())['message'] # Same scenario — Ollama returns correct text answer on step 2 m1 = ollama_call([{'role': 'user', 'content': 'Weather in Warsaw?'}], tools) msgs = [ {'role': 'user', 'content': 'Weather in Warsaw?'}, {'role': 'assistant', 'content': '', 'tool_calls': m1['tool_calls']}, {'role': 'tool', 'content': '{"temperature": "15C", "conditions": "sunny"}'} ] m2 = ollama_call(msgs, tools) print('content:', m2['content']) # → "The weather in Warsaw is 15°C and sunny." ✓ ``` --- ## Observed vs Expected Behavior ### Ollama 0.20.7 via LiteLLM — **BUG** ``` Step 1: content='', tool_calls=[get_weather(city=Warsaw)] ← model calls tool Step 2: content='', tool_calls=[get_weather(city=Warsaw)] ← BUG: ignores result, loops Step 3: content='', tool_calls=[get_weather(city=Warsaw)] ← same Step 4: content='', tool_calls=[get_weather(city=Warsaw)] ← same Step 5: content='', tool_calls=[get_weather(city=Warsaw)] ← same ``` ### Ollama 0.20.5 via LiteLLM — **OK** ``` Step 1: content='', tool_calls=[get_weather(city=Warsaw)] ← model calls tool Step 2: content='The weather in Warsaw is 15°C and sunny.' ← correct text answer ✓ ``` ### Ollama 0.20.7 directly (no proxy) — **OK** ``` Step 1: content='', tool_calls=[get_weather(city=Warsaw)] ← model calls tool Step 2: content='The weather in Warsaw is 15°C and sunny.' ← correct text answer ✓ ``` --- ## Evidence from Production Logs Real production data showing the loop pattern (LLM call log, `n8n.llm_log`): ``` logID promptTokens completionTokens tool_called 2358 5232 18 calendar_read(overdue=true) 2359 5237 18 calendar_read(overdue=true) ← +5 tokens only 2360 5239 18 calendar_read(overdue=true) ← +2 tokens 2361 5241 18 calendar_read(overdue=true) ← +2 tokens 2362 5243 18 calendar_read(overdue=true) ← +2 tokens ``` **Key observation:** `promptTokens` grows by only 2–5 per iteration, but the tool result JSON is ~80 tokens. This confirms the tool result content was not being injected into the model's context — only the tool_call metadata was appended. The same loop pattern was observed across multiple tools: - `calendar_read` — identical args, empty result `{events:[], tasks:[]}` - `law_lookup` — identical args, non-empty result (model ignored rich legal articles) - `dokuwiki_search` — identical args, empty result (no wiki pages found) - `deep_research` — identical args (multi-step internal tool) --- ## Hypothesis Ollama 0.20.6 changelog states: > *"Gemma 4 tool calling ability is improved and updated to use Google's latest post-launch fixes"* This likely changed the **chat template** used to format tool result messages for the model. LiteLLM 1.82.3 transforms OpenAI-format `tool` messages before forwarding to Ollama, and this transformation is now incompatible with the updated template. When calling Ollama directly, the native `/api/chat` format is used correctly. When going through LiteLLM, the tool result message format sent to Ollama no longer matches what `gemma4:26b`'s updated template expects — causing the model to not "see" the result and re-issue the same tool call. **Additional note:** LiteLLM's model info for `gemma4:26b` reports `"supports_function_calling": false`, which may cause LiteLLM to apply a fallback prompt-injection path instead of passing tool messages natively. --- ## Workaround Downgrade Ollama to **0.20.5**. --- ## Related - LiteLLM bug report: see `docs/bug-reports/litellm-gemma4-tool-messages.md` - Ollama changelog: https://github.com/ollama/ollama/releases/tag/v0.20.6 ### Relevant log output ```shell ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.20.7
GiteaMirror added the bug label 2026-05-05 03:25:52 -05:00
GiteaMirror commented 2026-05-05 03:25:56 -05:00
Author
Owner
Copy Link

@roxlukas commented on GitHub (Apr 20, 2026):

https://github.com/BerriAI/litellm/issues/26094

<!-- gh-comment-id:4279726797 --> @roxlukas commented on GitHub (Apr 20, 2026): https://github.com/BerriAI/litellm/issues/26094
GiteaMirror commented 2026-05-05 03:25:57 -05:00
Author
Owner
Copy Link

@roxlukas commented on GitHub (Apr 21, 2026):

A fix has been done on LiteLLM side. Will close this issue once confirmed that it's working end to end

<!-- gh-comment-id:4286134973 --> @roxlukas commented on GitHub (Apr 21, 2026): A fix has been done on LiteLLM side. Will close this issue once confirmed that it's working end to end
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#72082
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?