issue: Incorrect Token Usage Calculation using Function Calling in Native Mode #4913

New Issue

2025-11-11T16:06:21-06:00

GiteaMirror commented

2025-11-11 16:06:21 -06:00

Originally created by @ChenxingLi on GitHub (Apr 20, 2025).

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.6.5

Ollama Version (if applicable)

Not related

Operating System

Ubuntu 22.04

Browser (if applicable)

Not related

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

I have not included additional browser or server logs, as I believe the provided network packet records offer more relevant insights into this issue than logs. That said, if you think additional log details might be useful, please don’t hesitate to let me know, and I would be glad to share them.

Principle

Native function calling involves multiple calls to the Completion API:

In the first call, the LLM ends with a function call instruction.
In the second call, the history (including previous messages and function call results) is sent as the prompt to determine the final response.
If multiple function calls are required, additional API calls will be made following the same process.

Expected Behavior

Each API call should include "include_usage": true in the request payload.
The usage data returned by every API call should be captured and aggregated correctly.
For the Filter tool functions, both outlet and stream interface inputs should reflect the full token usage, including all API calls, ensuring accurate reporting and cost calculation across the entire workflow.

Actual Behavior

Open WebUI does not correctly track or calculate token usage during native function calling:

During the second API call (and any subsequent calls), the request does not include "include_usage": true.
As a result, the usage data is not returned for these calls, causing the token usage from subsequent calls to be omitted.
The outlet function of the Filter Tool functions only reflects the token usage from the first API call, while the stream function inputs lack any token usage data entirely.

This behavior results in incomplete and incorrect token cost calculations for workflows involving native function calling.

Steps to Reproduce

Set Up LLM Model
- Go to Admin Panel > Settings > Connections > Open AI API
- Add an external OpenAI model link (ensure the model supports function calling, e.g., gpt-4o or claude-3.7).
Set Up a Tool Server
- Navigate to Admin Panel > Tools > Manage Tool Servers
- Add an external tool server (e.g., the simplest example of get_current_time from the tutorial).
Enable Usage Capabilities for the Model
- Go to Admin Panel > Settings > Models.
- Select the model you are using.
- Enable Capabilities > Usage for the model.
Configure Conversation Settings
- Enter the conversation interface.
- Enable the time tool
- In conversation settings, select Function Calling mode as Native.
Ask a Function-Calling-Related Question
- e.g., "Using function calling, tell me the current time in xx timezone."
Capture Network Traffic
- Use an HTTP network packet capture tool to observe the outgoing API calls from OpenWebUI.
- You will notice multiple /chat/completions calls made to the OpenAI API. The second call(s) will not include the expected token usage calculation ("include_usage": true).
Check Filter Tool Logs
- Navigate to Admin Panel > Functions.
- Add a Filter function that logs the stream and outlet handler outputs.
- Observe the logs and verify:
  - The outlet function input only includes token usage from the first API call.
  - The stream function input contains no token usage data for the second API call.

Logs & Screenshots

The screenshots below show the HTTP packets exchanged between OpenWebUI and the OpenAI API server during the aforementioned steps. It can be observed that the second communication fails to correctly request token usage.

Additional Information

I am not using the official native API but rather a third-party API. Therefore, I am unsure whether the official native API can still return token usage during the second call, even if include_usage is not requested. However, after testing multiple OpenAI-compatible APIs, I found that none of them return token usage if usage is not explicitly requested.

Originally created by @ChenxingLi on GitHub (Apr 20, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.6.5 ### Ollama Version (if applicable) Not related ### Operating System Ubuntu 22.04 ### Browser (if applicable) Not related ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. I have not included additional browser or server logs, as I believe the provided **network packet records** offer more relevant insights into this issue than logs. That said, if you think additional log details might be useful, please don’t hesitate to let me know, and I would be glad to share them. ### Principle Native function calling involves multiple calls to the Completion API: - In the first call, the LLM ends with a function call instruction. - In the second call, the history (including previous messages and function call results) is sent as the prompt to determine the final response. - If multiple function calls are required, additional API calls will be made following the same process. ### Expected Behavior 1. Each API call should include `"include_usage": true` in the request payload. 2. The `usage` data returned by every API call should be captured and aggregated correctly. 3. For the `Filter` tool functions, both `outlet` and `stream` interface inputs should reflect the full token usage, including all API calls, ensuring accurate reporting and cost calculation across the entire workflow. ### Actual Behavior Open WebUI does not correctly track or calculate token usage during native function calling: 1. During the second API call (and any subsequent calls), the request does not include `"include_usage": true`. 2. As a result, the `usage` data is not returned for these calls, causing the token usage from subsequent calls to be omitted. 3. The `outlet` function of the `Filter` Tool functions only reflects the token usage from the first API call, while the stream function inputs lack any token usage data entirely. This behavior results in incomplete and incorrect token cost calculations for workflows involving native function calling. ### Steps to Reproduce 1. **Set Up LLM Model** - Go to **Admin Panel > Settings > Connections > Open AI API** - Add an external OpenAI model link (ensure the model supports function calling, e.g., `gpt-4o` or `claude-3.7`). 2. **Set Up a Tool Server** - Navigate to **Admin Panel > Tools > Manage Tool Servers** - Add an external tool server (e.g., the simplest example of `get_current_time` from the tutorial). 3. **Enable Usage Capabilities for the Model** - Go to Admin Panel > Settings > Models. - Select the model you are using. - Enable Capabilities > Usage for the model. 4. **Configure Conversation Settings** - Enter the conversation interface. - Enable the time tool - In conversation settings, select `Function Calling` mode as **Native**. 5. **Ask a Function-Calling-Related Question** - e.g., "Using function calling, tell me the current time in xx timezone." 6. **Capture Network Traffic** - Use an HTTP network packet capture tool to observe the outgoing API calls from OpenWebUI. - You will notice multiple `/chat/completions` calls made to the OpenAI API. The second call(s) will not include the expected token usage calculation (`"include_usage": true`). 7. **Check Filter Tool Logs** - Navigate to **Admin Panel > Functions**. - Add a `Filter` function that logs the `stream` and `outlet` handler outputs. - Observe the logs and verify: - The `outlet` function input only includes token usage from the first API call. - The `stream` function input contains no token usage data for the second API call. ### Logs & Screenshots The screenshots below show the HTTP packets exchanged between OpenWebUI and the OpenAI API server during the aforementioned steps. It can be observed that the second communication fails to correctly request token usage. ![Image](https://github.com/user-attachments/assets/c95976ca-104d-4a9f-87da-799047223591) ![Image](https://github.com/user-attachments/assets/cf71232d-cf45-46d3-aa7f-f6df7eb4f1ce) ![Image](https://github.com/user-attachments/assets/7a97784b-9f33-43d7-a6b9-ce88bd6cf995) ![Image](https://github.com/user-attachments/assets/c5930ab5-9efc-48d9-837c-ab2949375778) ### Additional Information I am not using the official native API but rather a third-party API. Therefore, I am unsure whether the official native API can still return token usage during the second call, even if `include_usage` is not requested. However, after testing multiple OpenAI-compatible APIs, I found that none of them return token usage if `usage` is not explicitly requested.

GiteaMirror added the bug label 2025-11-11 16:06:21 -06:00

GiteaMirror commented

2025-11-11 16:06:24 -06:00

@rgaricano commented on GitHub (Apr 23, 2025):

On CommVersion Outlet filters are not triggered on API calls.
https://github.com/open-webui/open-webui/discussions/8722#discussioncomment-12193783

@rgaricano commented on GitHub (Apr 23, 2025): On CommVersion Outlet filters are not triggered on API calls. https://github.com/open-webui/open-webui/discussions/8722#discussioncomment-12193783

GiteaMirror referenced this issue

2026-04-19 20:23:53 -05:00

[GH-ISSUE #4913] The reaction time of chat window become slower when I use nginx with SSL as reverse proxy #13785

GiteaMirror referenced this issue

2026-04-25 03:44:09 -05:00

[GH-ISSUE #4913] The reaction time of chat window become slower when I use nginx with SSL as reverse proxy #29313

GiteaMirror referenced this issue

2026-05-05 13:32:40 -05:00

[GH-ISSUE #4913] The reaction time of chat window become slower when I use nginx with SSL as reverse proxy #52451

GiteaMirror referenced this issue

2026-05-13 00:45:24 -05:00

[GH-ISSUE #4913] The reaction time of chat window become slower when I use nginx with SSL as reverse proxy #68389