[GH-ISSUE #9773] API empty response when stream false and tools calls #102635

New Issue

Closed

opened 2026-05-17 23:57:11 -05:00 by GiteaMirror · 8 comments

GiteaMirror commented 2026-05-17 23:57:11 -05:00

Owner

Copy Link

Copy Source

Originally created by @Seniorsimo on GitHub (Feb 10, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9773

Bug Report

Installation Method

Docker

Environment

• Open WebUI Version: v0.5.10

• Ollama (if applicable): 0.5.7-0-ga420a45-dirty

• Operating System: Windows 11

Confirmation:

• I have read and followed all the instructions provided in the README.md.
• I am on the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.
• I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

When a model should generate a tool call, it should generate it

• if called with stream: true
• if called with stream: false

Actual Behavior:

When called with stream: false, no tool call and an empty response is generated as if the tool data isn't added.

Description

Tool call in API when stream is false seems not working and the response is empty

Reproduction Details

Stream true

Request

curl -XPOST -H 'Authorization: Bearer sk-YOUR-API-KEY' -H "Content-type: application/json" -d '{
        "messages": [
                {
                        "content": "You are a helpful assistant",
                        "role": "system"
                }, {
                        "content": "It feels so cold today in NY.",
                        "role": "user"
                }
        ],
        "model": "pc.qwen2.5:14b",
        "n": 1,
        "stream": true,
        "temperature": 0.7,
        "tools": [
                {
                        "type": "function",
                        "function": {
                                "name": "weather",
                                "description": "Retrieve the weather for a city.",
                                "parameters": {
                                        "properties": {
                                                "city": {
                                                        "description": "The city to inspect current weather",
                                                        "type": "string"
                                                }
                                        },
                                        "required": ["city"],
                                        "type": "object"
                                }
                        }
                }
        ]
}' 'http://localhost:8080/api/chat/completions'

Response

{
	"id": "qwen2.5:14b-92f77e7e-baa5-454a-aa84-4a8647ef3044", 
	"created": 1739210359, 
	"model": "qwen2.5:14b", 
	"choices": [
		{
			"index": 0, 
			"logprobs": null, 
			"finish_reason": null, 
			"delta": {
				"tool_calls": [
					{
						"index": 0, 
						"id": "call_df2ee15e-e8ee-4709-b179-e0f2df19fa91", 
						"type": "function", 
						"function": {
							"name": "weather", 
							"arguments": "{\"city\":\"New York\"}"
						}
					}
				]
			}
		}
	], 
	"object": "chat.completion.chunk"
}

Stream false

Request

curl -XPOST -H 'Authorization: Bearer sk-YOUR-API-KEY' -H "Content-type: application/json" -d '{
        "messages": [
                {
                        "content": "You are a helpful assistant",
                        "role": "system"
                }, {
                        "content": "It feels so cold today in NY.",
                        "role": "user"
                }
        ],
        "model": "pc.qwen2.5:14b",
        "n": 1,
        "stream": false,
        "temperature": 0.7,
        "tools": [
                {
                        "type": "function",
                        "function": {
                                "name": "weather",
                                "description": "Retrieve the weather for a city.",
                                "parameters": {
                                        "properties": {
                                                "city": {
                                                        "description": "The city to inspect current weather",
                                                        "type": "string"
                                                }
                                        },
                                        "required": ["city"],
                                        "type": "object"
                                }
                        }
                }
        ]
}' 'http://localhost:8080/api/chat/completions'

Response

{
	"id":"qwen2.5:14b-a7679dcc-3f1e-48cd-9e13-8d170943b44a",
	"created":1739219565,
	"model":"qwen2.5:14b",
	"choices":[
		{
			"index":0,
			"logprobs":null,
			"finish_reason":"stop",
			"message":{
				"content":"",
				"role":"assistant"
			}
		}
	],
	"object":"chat.completion",
	"usage":{
		"response_token/s":47.56,
		"prompt_token/s":19500.0,
		"total_duration":802440073,
		"load_duration":10723211,
		"prompt_eval_count":156,
		"prompt_eval_duration":8000000,
		"eval_count":37,
		"eval_duration":778000000,
		"approximate_total":"0h0m0s"
	}
}

Logs and Screenshots

Browser Console Logs:
not applicable

Docker Container Logs:
INFO: 10.88.5.1:0 - "POST /api/chat/completions HTTP/1.1" 200 OK

Screenshots/Screen Recordings (if applicable):
not applicable

Additional Information

Sorry to disturb again.
Seems like the tools field isn't populated.
This time I don't have a solution yet, but I will try to address.

I let you know ;-)

Originally created by @Seniorsimo on GitHub (Feb 10, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/9773 # Bug Report ## Installation Method Docker ## Environment - **Open WebUI Version:** v0.5.10 - **Ollama (if applicable):** 0.5.7-0-ga420a45-dirty - **Operating System:** Windows 11 **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: When a model should generate a tool call, it should generate it - if called with stream: true - if called with stream: false ## Actual Behavior: When called with stream: false, no tool call and an empty response is generated as if the tool data isn't added. ## Description Tool call in API when stream is false seems not working and the response is empty ## Reproduction Details *Stream true* Request ```bash curl -XPOST -H 'Authorization: Bearer sk-YOUR-API-KEY' -H "Content-type: application/json" -d '{ "messages": [ { "content": "You are a helpful assistant", "role": "system" }, { "content": "It feels so cold today in NY.", "role": "user" } ], "model": "pc.qwen2.5:14b", "n": 1, "stream": true, "temperature": 0.7, "tools": [ { "type": "function", "function": { "name": "weather", "description": "Retrieve the weather for a city.", "parameters": { "properties": { "city": { "description": "The city to inspect current weather", "type": "string" } }, "required": ["city"], "type": "object" } } } ] }' 'http://localhost:8080/api/chat/completions' ``` Response ```json { "id": "qwen2.5:14b-92f77e7e-baa5-454a-aa84-4a8647ef3044", "created": 1739210359, "model": "qwen2.5:14b", "choices": [ { "index": 0, "logprobs": null, "finish_reason": null, "delta": { "tool_calls": [ { "index": 0, "id": "call_df2ee15e-e8ee-4709-b179-e0f2df19fa91", "type": "function", "function": { "name": "weather", "arguments": "{\"city\":\"New York\"}" } } ] } } ], "object": "chat.completion.chunk" } ``` *Stream false* Request ```bash curl -XPOST -H 'Authorization: Bearer sk-YOUR-API-KEY' -H "Content-type: application/json" -d '{ "messages": [ { "content": "You are a helpful assistant", "role": "system" }, { "content": "It feels so cold today in NY.", "role": "user" } ], "model": "pc.qwen2.5:14b", "n": 1, "stream": false, "temperature": 0.7, "tools": [ { "type": "function", "function": { "name": "weather", "description": "Retrieve the weather for a city.", "parameters": { "properties": { "city": { "description": "The city to inspect current weather", "type": "string" } }, "required": ["city"], "type": "object" } } } ] }' 'http://localhost:8080/api/chat/completions' ``` Response ```json { "id":"qwen2.5:14b-a7679dcc-3f1e-48cd-9e13-8d170943b44a", "created":1739219565, "model":"qwen2.5:14b", "choices":[ { "index":0, "logprobs":null, "finish_reason":"stop", "message":{ "content":"", "role":"assistant" } } ], "object":"chat.completion", "usage":{ "response_token/s":47.56, "prompt_token/s":19500.0, "total_duration":802440073, "load_duration":10723211, "prompt_eval_count":156, "prompt_eval_duration":8000000, "eval_count":37, "eval_duration":778000000, "approximate_total":"0h0m0s" } } ``` ## Logs and Screenshots **Browser Console Logs:** not applicable **Docker Container Logs:** INFO: 10.88.5.1:0 - "POST /api/chat/completions HTTP/1.1" 200 OK **Screenshots/Screen Recordings (if applicable):** not applicable ## Additional Information Sorry to disturb again. Seems like the tools field isn't populated. This time I don't have a solution yet, but I will try to address. I let you know ;-)

GiteaMirror closed this issue 2026-05-17 23:57:11 -05:00

GiteaMirror commented 2026-05-17 23:57:12 -05:00

Author

Owner

Copy Link

Copy Source

@Seniorsimo commented on GitHub (Feb 10, 2025):

Yeee! Easier than expected!

This should solve the problem:
https://github.com/open-webui/open-webui/commit/ba66d586c7d9541b56c625e56c81d231116987f4

I've just moved the tool part outside the 2 function to not copy it.
Tested now and seems working with stream false as well as with stream true

Response with stream false

{
	"id":"qwen2.5:14b-7b5ec4b5-32c5-4f9e-ad85-f23dfd55ae6d",
	"created":1739222142,
	"model":"qwen2.5:14b",
	"choices":[
		{
			"index":0,
			"logprobs":null,
			"finish_reason":"stop",
			"message":{
				"content":"",
				"role":"assistant"
			},
			"tool_calls":[
				{
					"index":0,
					"id":"call_6fb45e1a-7999-4fde-8fa9-4a83cd8512b5",
					"type":"function",
					"function":{
						"name":"weather",
						"arguments":"{\"city\": \"New York\"}"
					}
				}
			]
		}
	],
	"object":"chat.completion",
	"usage":{
		"response_token/s":47.56,
		"prompt_token/s":19500.0,
		"total_duration":802440073,
		"load_duration":10723211,
		"prompt_eval_count":156,
		"prompt_eval_duration":8000000,
		"eval_count":37,
		"eval_duration":778000000,
		"approximate_total":"0h0m0s"
	}
}

Enought for today Great work yours!!! Open WebUI is becaming a great success!!

<!-- gh-comment-id:2649273043 --> @Seniorsimo commented on GitHub (Feb 10, 2025): Yeee! Easier than expected! This should solve the problem: https://github.com/open-webui/open-webui/commit/ba66d586c7d9541b56c625e56c81d231116987f4 I've just moved the tool part outside the 2 function to not copy it. Tested now and seems working with stream false as well as with stream true Response with stream false ```json { "id":"qwen2.5:14b-7b5ec4b5-32c5-4f9e-ad85-f23dfd55ae6d", "created":1739222142, "model":"qwen2.5:14b", "choices":[ { "index":0, "logprobs":null, "finish_reason":"stop", "message":{ "content":"", "role":"assistant" }, "tool_calls":[ { "index":0, "id":"call_6fb45e1a-7999-4fde-8fa9-4a83cd8512b5", "type":"function", "function":{ "name":"weather", "arguments":"{\"city\": \"New York\"}" } } ] } ], "object":"chat.completion", "usage":{ "response_token/s":47.56, "prompt_token/s":19500.0, "total_duration":802440073, "load_duration":10723211, "prompt_eval_count":156, "prompt_eval_duration":8000000, "eval_count":37, "eval_duration":778000000, "approximate_total":"0h0m0s" } } ``` Enought for today Great work yours!!! Open WebUI is becaming a great success!!

GiteaMirror commented 2026-05-17 23:57:12 -05:00

Author

Owner

Copy Link

Copy Source

@taylorwilsdon commented on GitHub (Feb 11, 2025):

This is interesting, I'm not seeing it necessarily tied to stream false but am seeing this behavior introduced in v0.5.10 where tools are seemingly executing but disregarding the response unless I prompt it explicitly to await RAG response (tool in question fires a retrieval call and adds the response documents as citations)

<!-- gh-comment-id:2651337457 --> @taylorwilsdon commented on GitHub (Feb 11, 2025): This is interesting, I'm not seeing it necessarily tied to stream false but am seeing this behavior introduced in v0.5.10 where tools are seemingly executing but disregarding the response unless I prompt it explicitly to await RAG response (tool in question fires a retrieval call and adds the response documents as citations)

GiteaMirror commented 2026-05-17 23:57:13 -05:00

Author

Owner

Copy Link

Copy Source

@taylorwilsdon commented on GitHub (Feb 11, 2025):

My symptoms look like:

"Await the rag response" fixes it ->

In both cases, the middleware is firing - open_webui.utils.middleware] tools={'fetch_rag_context
But in the former, the tool doesn't appear to fire the call and nothing is ever actually retrieved, the model answers immediately with the wrong information

<!-- gh-comment-id:2651344249 --> @taylorwilsdon commented on GitHub (Feb 11, 2025): My symptoms look like:  "Await the rag response" fixes it ->  In both cases, the middleware is firing - `open_webui.utils.middleware] tools={'fetch_rag_context` But in the former, the tool doesn't appear to fire the call and nothing is ever actually retrieved, the model answers immediately with the wrong information

GiteaMirror commented 2026-05-17 23:57:13 -05:00

Author

Owner

Copy Link

Copy Source

@Seniorsimo commented on GitHub (Feb 11, 2025):

Interesting indeed!
Maybe the same api code is invoked?
If you can, you can try the fix I posted and tell us if it solves also in your case.

From what i saw yesterday, for me seems simply the tools not being copied to the response

<!-- gh-comment-id:2651349294 --> @Seniorsimo commented on GitHub (Feb 11, 2025): Interesting indeed! Maybe the same api code is invoked? If you can, you can try the fix I posted and tell us if it solves also in your case. From what i saw yesterday, for me seems simply the tools not being copied to the response

GiteaMirror commented 2026-05-17 23:57:13 -05:00

Author

Owner

Copy Link

Copy Source

@taylorwilsdon commented on GitHub (Feb 11, 2025):

It seems to be introduced between v0.5.7 and v0.5.10 with changes to backend/open_webui/utils/middleware.py - seems like the native tool calling support and code interpreter logic introduced some significant changes to middleware.py that are resulting in inconsistent tool execution for me where previously the tool would always fire and fully complete before any response was generated by the underlying model:
https://github.com/open-webui/open-webui/compare/v0.5.7...v0.5.10

Edit - another breadcrumb, if I force function calling to "Native" it is now working much more reliably. Interesting... seems a little faster too, but still worth fixing because it'll be confusing to users

<!-- gh-comment-id:2651371659 --> @taylorwilsdon commented on GitHub (Feb 11, 2025): It seems to be introduced between v0.5.7 and v0.5.10 with changes to `backend/open_webui/utils/middleware.py` - seems like the native tool calling support and code interpreter logic introduced some significant changes to middleware.py that are resulting in inconsistent tool execution for me where previously the tool would always fire and fully complete before any response was generated by the underlying model: https://github.com/open-webui/open-webui/compare/v0.5.7...v0.5.10 Edit - another breadcrumb, if I force function calling to "Native" it is now working much more reliably. Interesting... seems a little faster too, but still worth fixing because it'll be confusing to users

GiteaMirror commented 2026-05-17 23:57:14 -05:00

Author

Owner

Copy Link

Copy Source

@Seniorsimo commented on GitHub (Feb 11, 2025):

I was searching the diff between v0.5.7 and v0.5.10, can't say for sure after a fast look, but for me they look like 2 separate bug.

• some logic change in the middleware.py that cause inconsistency with tool calls in the UI
• the missing tools field in response.py that cause no tools beign returned in API when invoked with stream=false

<!-- gh-comment-id:2651511103 --> @Seniorsimo commented on GitHub (Feb 11, 2025): I was searching the diff between v0.5.7 and v0.5.10, can't say for sure after a fast look, but for me they look like 2 separate bug. - some logic change in the middleware.py that cause inconsistency with tool calls in the UI - the missing `tools` field in response.py that cause no tools beign returned in API when invoked with stream=false

GiteaMirror commented 2026-05-17 23:57:15 -05:00

Author

Owner

Copy Link

Copy Source

@tjbck commented on GitHub (Feb 12, 2025):

@Seniorsimo PR welcome!

<!-- gh-comment-id:2652890798 --> @tjbck commented on GitHub (Feb 12, 2025): @Seniorsimo PR welcome!

GiteaMirror commented 2026-05-17 23:57:15 -05:00

Author

Owner

Copy Link

Copy Source

@Seniorsimo commented on GitHub (Feb 12, 2025):

Here it is

<!-- gh-comment-id:2652976381 --> @Seniorsimo commented on GitHub (Feb 12, 2025): Here it is

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dev

v0.11.0

v0.10.2

v0.10.1

v0.10.0

v0.9.6

v0.9.5

v0.9.4

v0.9.3

v0.9.2

v0.9.1

v0.9.0

v0.8.12

v0.8.11

v0.8.10

v0.8.9

v0.8.8

v0.8.7

v0.8.6

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.2

v0.7.1

v0.7.0

v0.6.43

v0.6.42

v0.6.41

v0.6.40

v0.6.39

v0.6.38

v0.6.37

v0.6.36

v0.6.35

v0.6.34

v0.6.33

v0.6.32

v0.6.31

v0.6.30

v0.6.29

v0.6.28

v0.6.27

v0.6.26

v0.6.25

v0.6.24

v0.6.23

v0.6.22

v0.6.21

v0.6.20

v0.6.19

v0.6.18

v0.6.17

v0.6.16

v0.6.15

v0.6.14

v0.6.13

v0.6.12

v0.6.11

v0.6.10

v0.6.9

v0.6.8

v0.6.7

v0.6.6

v0.6.5

v0.6.4

v0.6.3

v0.6.2

v0.6.1

v0.6.0

v0.5.20

v0.5.19

v0.5.18

v0.5.17

v0.5.16

v0.5.15

v0.5.14

v0.5.13

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.35

v0.3.34

v0.3.33

v0.3.32

v0.3.31

v0.3.30

v0.3.29

v0.3.28

v0.3.27

v0.3.26

v0.3.25

v0.3.24

v0.3.23

v0.3.22

v0.3.21

v0.3.20

v0.3.19

v0.3.18

v0.3.17

v0.3.16

v0.3.15

v0.3.14

v0.3.13

v0.3.12

v0.3.11

v0.3.10

v0.3.9

v0.3.8

v0.3.7

v0.3.6

v0.3.5

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.125

v0.1.124

v0.1.123

v0.1.122

v0.1.121

v0.1.120

v0.1.119

v0.1.118

v0.1.117

v0.1.116

v0.1.115

v0.1.114

v0.1.113

v0.1.112

v0.1.111

v0.1.110

v0.1.109

v0.1.108

v0.1.107

v0.1.106

v0.1.105

v0.1.104

v0.1.103

v0.1.102

Labels

Clear labels

bug

confirmed

confirmed issue

core

documentation

enhancement

good first issue

help wanted

non-core

pull-request

Mirrored from GitHub Pull Request

python

question

testing wanted

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

GiteaMirror ninjasurge (Kaehlen Rog)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#102635