github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 06:03:26 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #24381] issue: Unable to properly use Tool Calls with Mistral 3.5 using VLLM OpenAI Endpoint #91024

New Issue
Closed
opened 2026-05-15 16:18:19 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-15 16:18:19 -05:00
Owner
Copy Link

Originally created by @Notbici on GitHub (May 5, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24381

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.9.2

Ollama Version (if applicable)

NA

Operating System

Linux

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

[THINK] and [TOOL_CALL] to properly be handled but are not.

Actual Behavior

The LLM prints [THINK] and [TOOL_CALL] tokens into plain chat.

Steps to Reproduce

  1. Setup VLLM and use this instruction here its very easy, https://huggingface.co/mistralai/Mistral-Medium-3.5-128B-EAGLE#serve-the-model
  2. Go to OpenWebUI -> Admin -> Connections -> enable OpenAI and put your VLLM url in.
  3. Make sure to add your model "mistralai/Mistral-Medium-3.5-128B-EAGLE" to the connection so you can see it when you make a chat.
  4. Since OpenWEBUI has a bunch of pre-installed tools just ask the model "Hey what time is it".

Logs & Screenshots

Image

As you can see it dumps in plain text. Its CORRECT.. just not parsing properly.

Additional Information

I tried to use the "Reasoning Tags" feature in the sidebar and that does work, I can fix the "thinking" but I cannot fix the tool calls. How can I do this with an OpenAI compatible served Model?

Originally created by @Notbici on GitHub (May 5, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24381 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.9.2 ### Ollama Version (if applicable) NA ### Operating System Linux ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior [THINK] and [TOOL_CALL] to properly be handled but are not. ### Actual Behavior The LLM prints [THINK] and [TOOL_CALL] tokens into plain chat. ### Steps to Reproduce 1. Setup VLLM and use this instruction here its very easy, https://huggingface.co/mistralai/Mistral-Medium-3.5-128B-EAGLE#serve-the-model 2. Go to OpenWebUI -> Admin -> Connections -> enable OpenAI and put your VLLM url in. 3. Make sure to add your model "mistralai/Mistral-Medium-3.5-128B-EAGLE" to the connection so you can see it when you make a chat. 4. Since OpenWEBUI has a bunch of pre-installed tools just ask the model "Hey what time is it". ### Logs & Screenshots <img width="1114" height="850" alt="Image" src="https://github.com/user-attachments/assets/12354bbf-1af6-4003-8365-591961692395" /> As you can see it dumps in plain text. Its CORRECT.. just not parsing properly. ### Additional Information I tried to use the "Reasoning Tags" feature in the sidebar and that does work, I can fix the "thinking" but I cannot fix the tool calls. How can I do this with an OpenAI compatible served Model?
GiteaMirror added the bug label 2026-05-15 16:18:19 -05:00
GiteaMirror commented 2026-05-15 16:18:21 -05:00
Author
Owner
Copy Link

@owui-terminator[bot] commented on GitHub (May 5, 2026):

🔍 Similar Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

  1. #24299 issue: Minimax tool_call not working
    by sharyuke

  2. #24291 Minimax tool_call not working
    by sharyuke

  3. #24051 issue: Mistral AI TTS raising a 400 Bad Request error
    by deathmaster9 · bug

  4. #23823 issue: Channel @mention ignores model's native function calling settings and sends no tools to LLM (v0.8.12)
    by shommey · bug

  5. #22259 issue: Tools are not sent to Ollama
    by shinobijs · bug

💡 If this is a duplicate, consider closing it and adding details to the existing issue.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

<!-- gh-comment-id:4380349889 --> @owui-terminator[bot] commented on GitHub (May 5, 2026): 🔍 **Similar Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. [#24299](https://github.com/open-webui/open-webui/issues/24299) **issue: Minimax tool_call not working** *by sharyuke* 2. [#24291](https://github.com/open-webui/open-webui/issues/24291) **Minimax tool_call not working** *by sharyuke* 3. [#24051](https://github.com/open-webui/open-webui/issues/24051) **issue: Mistral AI TTS raising a 400 Bad Request error** *by deathmaster9 · `bug`* 4. [#23823](https://github.com/open-webui/open-webui/issues/23823) **issue: Channel @mention ignores model's native function calling settings and sends no tools to LLM (v0.8.12)** *by shommey · `bug`* 5. [#22259](https://github.com/open-webui/open-webui/issues/22259) **issue: Tools are not sent to Ollama** *by shinobijs · `bug`* --- 💡 If this is a duplicate, consider closing it and adding details to the existing issue. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.
GiteaMirror commented 2026-05-15 16:18:21 -05:00
Author
Owner
Copy Link

@Notbici commented on GitHub (May 5, 2026):

Fixed fixed!

Sorry it was not OpenWebUI it was my deployment of VLLM, here's what I changed for me in the future or 1 other person who might have this happen.

vllm serve mistralai/Mistral-Medium-3.5-128B \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.9 \
  --host 0.0.0.0 \
  --port 8080 \
  --tokenizer-mode mistral \
  --config-format mistral \
  --load-format mistral \
  --enable-auto-tool-choice \
  --tool-call-parser mistral \
  --reasoning-parser mistral

this is what changed:

--tokenizer-mode mistral
--config-format mistral
--load-format mistral
--enable-auto-tool-choice
--tool-call-parser mistral
--reasoning-parser mistral
<!-- gh-comment-id:4380539596 --> @Notbici commented on GitHub (May 5, 2026): Fixed fixed! Sorry it was not OpenWebUI it was my deployment of VLLM, here's what I changed for me in the future or 1 other person who might have this happen. ``` vllm serve mistralai/Mistral-Medium-3.5-128B \ --tensor-parallel-size 2 \ --max-model-len 32768 \ --gpu-memory-utilization 0.9 \ --host 0.0.0.0 \ --port 8080 \ --tokenizer-mode mistral \ --config-format mistral \ --load-format mistral \ --enable-auto-tool-choice \ --tool-call-parser mistral \ --reasoning-parser mistral ``` this is what changed: ``` --tokenizer-mode mistral --config-format mistral --load-format mistral --enable-auto-tool-choice --tool-call-parser mistral --reasoning-parser mistral ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#91024
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?