mirror of
https://github.com/open-webui/open-webui.git
synced 2026-03-22 14:13:08 -05:00
feat: Separate Tool Model from Task model #4677
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#4677
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @weikinhuang on GitHub (Apr 2, 2025).
Check Existing Issues
Problem Description
Currently when a
Task Modelis set, when calling a tool, it will use it instead of theCurrent Model. This sometimes presents an issue where the Task model might be much less knowledgable than the main model.I opened a discussion here https://github.com/open-webui/open-webui/discussions/12245, where I'm trying to use the
Web Searchtool to automatically let the LLM decide if it needs more information from the web instead of manually enabling/disabling it from the chat ui. While I was debugging, I noticed theAutoToolsfilter, which uses the Main Model correctly determines that it does not need additional information, but the tools call is done by the task model, and will make a web request regardless of theAutoToolsfilter's response.Desired Solution you'd like
It would be great to separate the
Tool Modelsetting from theTask Modelsetting, giving users the ability to use say a0.5Bmodel for title, tags, search query generation, and having a larger model (ex.gemma:4bor theCurrent Model) for calling tools.Alternatives Considered
Creating a dedicated system prompt for the smaller Task Model.
Additional Context
Referenced issue body:
I've been trying to speed up my instance of open-webui with separate ollama servers in my homelab, where I'm hosting smaller models (gemma:4b) and larger models (mistral-small-3:24b) on separate ollama instances on different nodes.
I recently set up the (Web Search Tool)[https://openwebui.com/t/constliakos/web_search] along with the (AutoTool Filter)[https://openwebui.com/f/hub/autotool_filter]. Everything is working great when I set the Task Model in the admin to
Current Model. The main model correctly determines when it needs to use the websearch tool, and calls it appropriately in a turn. However when I set a different task model, the AutoTool Filter correctly determines that no tool needs to be called, but the task model will call it anyways.Looking through the logs with some debugging, I can correctly attribute the tool call to the task model and not the main model when set when logging
__model__in the Tool class.Attached videos show the issue.
Using
Current Modelas task model.https://github.com/user-attachments/assets/52eabdd1-7acd-47ae-b477-a97e9d92e711
Using
gemma:4bas task model.https://github.com/user-attachments/assets/60803bee-da61-4d2a-844d-463221d2c4cd
This is the setting I'm referring to:
@Classic298 commented on GitHub (Apr 2, 2025):
"Current Model for calling tools."
Set function calling to "native" for a model and you will achieve exactly this while retaining a very small model for all other tasks.
@weikinhuang commented on GitHub (Apr 2, 2025):
I tried to do so, but for using (Mistral Small)[https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF] the proper tool calling prompt isn't correctly built into the model file. And when I try to do so ollama I get:
@weikinhuang commented on GitHub (Apr 8, 2025):
Another possibility for this feature request is the have a option for different models for each of the different types of tasks and not just for tools specifically.