[PR #14108] [CLOSED] feat: Call native tools automatically within non-streaming chat-completion request. #39021

New Issue

Closed

opened 2026-04-25 11:46:00 -05:00 by GiteaMirror · 0 comments

GiteaMirror commented 2026-04-25 11:46:00 -05:00

Owner

Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/14108
Author: @diwakar-s-maurya
Created: 5/20/2025
Status: ❌ Closed

Base: dev ← Head: patch-5

---

📝 Commits (3)

• d5e0c0c improve native function calling to call multiple tools in a chain
• cee86d5 make MAX_TOOL_CALL_RETRIES equal for both streaming and non-streaming case.
• a023a88 make non-streaming mode work from curl api calls too instead of only from browser.

📊 Changes

1 file changed (+1046 additions, -774 deletions)

View changed files

📝 backend/open_webui/utils/middleware.py (+1046 -774)

📄 Description

Pull Request Checklist

Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.

Before submitting, make sure you've checked the following:

• Target branch: Please verify that the pull request targets the dev branch.
• Description: Provide a concise description of the changes made in this pull request.
• Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
• Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
• Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
• Testing: Have you written and run sufficient tests to validate the changes?
• Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
• Prefix: To clearly categorize this pull request, prefix the pull request title using one of the following:
  • BREAKING CHANGE: Significant changes that may affect compatibility
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to our continuous integration processes or workflows
  • chore: Refactor, cleanup, or other non-functional code changes
  • docs: Documentation update or addition
  • feat: Introduces a new feature or enhancement to the codebase
  • fix: Bug fix or error correction
  • i18n: Internationalization or localization changes
  • perf: Performance improvement
  • refactor: Code restructuring for better maintainability, readability, or scalability
  • style: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.)
  • test: Adding missing tests or correcting existing tests
  • WIP: Work in progress, a temporary label for incomplete or ongoing work

Changelog Entry

Description

The changes are to enhance the chat response processing system with a focus making native tool/function calling capabilities available in non-streaming requests such as a simple curl POST /api/chat/completions API call.

• At present, with function_calling set to native and stream-chat-response set to false, when a request is sent using curl to /api/chat/completions, it stops after 1 tool call instruction generation. Same thing happens when done using browser i.e. it stops after 1 call and a infinite loader keeps showing on screen. The problem does not happen on browser when stream-chat-response is turned on.
• With this PR, the tool calls are processed in a response-loop similar to how it happens for streaming requests, bringing same cabalities for both request modes.

This unlocks capability of using open-webui's model+tools from other simple web application without implementing streaming response handling logic. With the right prompts sent in a API request, open-webui can now become a universal backend for every frontend. Imagine this. You have an simple web-application having a button "Send popularity report of open-webui". The web application can do simple POST /api/chat/completions API of open-webui to leverage the model+tools and get the work done easily. We can offloaded the entire logic of calling and orchestrating API logic to open-webui (given that tools are added in open-webui).

Example:
Currently, if I send this request body to /api/chat/completions,

{
    "stream": false,
    "model": "google.gemini-2.0-flash-001",
    "messages": [
        {
            "role": "user",
            "content": "- For open-webui project on github, please tell me how many github stars this project has currently?\n- See if they are increasing?\n- Add these findings in notion\n- Finally reply back one word: true or false. Return true only if all things passed, otherwise false."
        }
    ],
    "params": {
        "function_calling": "native"
    },
    "tool_ids": [
        "my_productivity_tools"
    ]
}

open-webui returns this, which is not useful. It stopped after tool call instruction generation.

{
    "id": "chatcmpl-6d3a6e1c-c973-4a43-86c6-2bfa7b1796b2",
    "object": "chat.completion",
    "created": 1747844668,
    "model": "gemini-2.0-flash-001",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": null,
                "tool_calls": [
                    {
                        "id": "call_5c0abee0-7fde-4613-9517-94328738ebca",
                        "type": "function",
                        "function": {
                            "name": "get_github_repo_ids",
                            "arguments": "{\"reponame\": \"open-webui\"}"
                        }
                    }
                ]
            },
            "finish_reason": "tool_calls"        <--------------- stopped at tool call
        }
    ],
    "usage": {
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "total_tokens": 0
    }
}

But with this PR code, it will complete all the tools-calls and acheive the task.

{
    "id": "chatcmpl-ef0c34ff-9d53-4df8-ad85-b4aa00bf10a5",
    "object": "chat.completion",
    "created": 1747843536,
    "model": "gemini-2.0-flash-001",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "true"      <-------------it completed the task and responded as instructed in the prompt. 
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "total_tokens": 0
    }
}

Added

• Call native tools automatically within non-streaming chat-completion request.

Fixed

• Infinite loader shown on chat UI when function-calling is native and chat-response-streaming is false.

---

Screenshots or Videos

• The API request/response pairs are mentioned in the description. Kindly see that.

• On browser, without this PR, it does not call multiple functions/tools. It gets stuck when function_calling was set to native and streaming was false.
  

• After, with this feature, it does call multiple functions/tools when function_calling is native and streaming is false.
  

Here is the dummy tool used for demonstration for these screenshots and api calls.

import os
import requests
from datetime import datetime
from pydantic import BaseModel, Field
import random

class Tools:
    def __init__(self):
        self.citation = True
        pass

    def get_github_repo_ids(
        self, reponame: str = Field("Github project repository name")
    ) -> str:
        """
        Returns the github project id for the given name
        """

        repo_mapping = {
            "open-webui": "6b0e6b7e-6e2a-4849-81d9-a2ec7ca6b076",
            "vscode": "31729b7e-10c8-40a3-ac6e-73560ef5a086",
        }
        return repo_mapping.get(reponame, "Invalid repository name")

    def get_github_stars(
        self, repo_id: str = Field("Github project repository id")
    ) -> str:
        """
        Returns the current number of stars for the given github project id
        """

        return f"The number of stars for the given github project id is {random.randint(500, 1000)}"

    def get_github_start_history(
        self, repo_id: str = Field("Github project repository id")
    ) -> str:
        """
        Returns the repo star year history for the given github project id
        """

        return f"2020: 100 stars, 2021: 200 stars, 2022: 300 stars, 2023: 400 stars, 2024: 500 stars"

    def add_in_notion(self, data: str = Field("data to add in notion")) -> str:
        """
        Adds the data to a new page in notion
        """

        return f"Data added to notion"

    def send_email(self, data: str = Field("data to add in email")) -> str:
        """
        Sends the data to an email
        """

        return f"Data sent to email"

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

---

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/14108 **Author:** [@diwakar-s-maurya](https://github.com/diwakar-s-maurya) **Created:** 5/20/2025 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `patch-5` --- ### 📝 Commits (3) - [`d5e0c0c`](https://github.com/open-webui/open-webui/commit/d5e0c0c7808fed9a014f6ef7f8bf7035f303de1b) improve native function calling to call multiple tools in a chain - [`cee86d5`](https://github.com/open-webui/open-webui/commit/cee86d5f9351e2153510a93b21d68fc186370c95) make MAX_TOOL_CALL_RETRIES equal for both streaming and non-streaming case. - [`a023a88`](https://github.com/open-webui/open-webui/commit/a023a887bf8574d7cbb6acb2ed2a2f0a950765ea) make non-streaming mode work from curl api calls too instead of only from browser. ### 📊 Changes **1 file changed** (+1046 additions, -774 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/utils/middleware.py` (+1046 -774) </details> ### 📄 Description # Pull Request Checklist ### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) and describe your changes before submitting a pull request. **Before submitting, make sure you've checked the following:** - [x] **Target branch:** Please verify that the pull request targets the `dev` branch. - [x] **Description:** Provide a concise description of the changes made in this pull request. - [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description. - [ ] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources? - [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation? - [x] **Testing:** Have you written and run sufficient tests to validate the changes? - [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [x] **Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following: - **BREAKING CHANGE**: Significant changes that may affect compatibility - **build**: Changes that affect the build system or external dependencies - **ci**: Changes to our continuous integration processes or workflows - **chore**: Refactor, cleanup, or other non-functional code changes - **docs**: Documentation update or addition - **feat**: Introduces a new feature or enhancement to the codebase - **fix**: Bug fix or error correction - **i18n**: Internationalization or localization changes - **perf**: Performance improvement - **refactor**: Code restructuring for better maintainability, readability, or scalability - **style**: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.) - **test**: Adding missing tests or correcting existing tests - **WIP**: Work in progress, a temporary label for incomplete or ongoing work # Changelog Entry ### Description The changes are to enhance the chat response processing system with a focus making **native** tool/function calling capabilities available in **non-streaming** requests such as a simple curl POST /api/chat/completions API call. - At present, with function_calling set to native and stream-chat-response set to false, when a request is sent using curl to /api/chat/completions, it stops after 1 tool call instruction generation. Same thing happens when done using browser i.e. it stops after 1 call and a infinite loader keeps showing on screen. The problem does not happen on browser when stream-chat-response is turned on. - With this PR, the tool calls are processed in a response-loop similar to how it happens for streaming requests, bringing same cabalities for both request modes. This unlocks capability of using open-webui's model+tools from other simple web application without implementing streaming response handling logic. With the right prompts sent in a API request, open-webui can now become a universal backend for every frontend. Imagine this. You have an simple web-application having a button "Send popularity report of open-webui". The web application can do simple `POST /api/chat/completions` API of open-webui to leverage the model+tools and get the work done easily. We can offloaded the entire logic of calling and orchestrating API logic to open-webui (given that tools are added in open-webui). Example: Currently, if I send this request body to /api/chat/completions, ```json { "stream": false, "model": "google.gemini-2.0-flash-001", "messages": [ { "role": "user", "content": "- For open-webui project on github, please tell me how many github stars this project has currently?\n- See if they are increasing?\n- Add these findings in notion\n- Finally reply back one word: true or false. Return true only if all things passed, otherwise false." } ], "params": { "function_calling": "native" }, "tool_ids": [ "my_productivity_tools" ] } ``` open-webui returns this, which is not useful. It stopped after tool call instruction generation. ``` { "id": "chatcmpl-6d3a6e1c-c973-4a43-86c6-2bfa7b1796b2", "object": "chat.completion", "created": 1747844668, "model": "gemini-2.0-flash-001", "choices": [ { "index": 0, "message": { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_5c0abee0-7fde-4613-9517-94328738ebca", "type": "function", "function": { "name": "get_github_repo_ids", "arguments": "{\"reponame\": \"open-webui\"}" } } ] }, "finish_reason": "tool_calls" <--------------- stopped at tool call } ], "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 } } ``` But with this PR code, it will complete all the tools-calls and acheive the task. ```json { "id": "chatcmpl-ef0c34ff-9d53-4df8-ad85-b4aa00bf10a5", "object": "chat.completion", "created": 1747843536, "model": "gemini-2.0-flash-001", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "true" <-------------it completed the task and responded as instructed in the prompt. }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 } } ``` ### Added - Call native tools automatically within non-streaming chat-completion request. ### Fixed - Infinite loader shown on chat UI when function-calling is native and chat-response-streaming is false. --- ### Screenshots or Videos - The API request/response pairs are mentioned in the description. Kindly see that. - On browser, without this PR, it does not call multiple functions/tools. It gets stuck when function_calling was set to native and streaming was false.  - After, with this feature, it **does** call multiple functions/tools when function_calling is native and streaming is false.  Here is the dummy tool used for demonstration for these screenshots and api calls. ```python import os import requests from datetime import datetime from pydantic import BaseModel, Field import random class Tools: def __init__(self): self.citation = True pass def get_github_repo_ids( self, reponame: str = Field("Github project repository name") ) -> str: """ Returns the github project id for the given name """ repo_mapping = { "open-webui": "6b0e6b7e-6e2a-4849-81d9-a2ec7ca6b076", "vscode": "31729b7e-10c8-40a3-ac6e-73560ef5a086", } return repo_mapping.get(reponame, "Invalid repository name") def get_github_stars( self, repo_id: str = Field("Github project repository id") ) -> str: """ Returns the current number of stars for the given github project id """ return f"The number of stars for the given github project id is {random.randint(500, 1000)}" def get_github_start_history( self, repo_id: str = Field("Github project repository id") ) -> str: """ Returns the repo star year history for the given github project id """ return f"2020: 100 stars, 2021: 200 stars, 2022: 300 stars, 2023: 400 stars, 2024: 500 stars" def add_in_notion(self, data: str = Field("data to add in notion")) -> str: """ Adds the data to a new page in notion """ return f"Data added to notion" def send_email(self, data: str = Field("data to add in email")) -> str: """ Sends the data to an email """ return f"Data sent to email" ``` ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. --- _{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

GiteaMirror added the pull-request label 2026-04-25 11:46:00 -05:00

GiteaMirror closed this issue 2026-04-25 11:46:00 -05:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dev

v0.9.2

v0.9.1

v0.9.0

v0.8.12

v0.8.11

v0.8.10

v0.8.9

v0.8.8

v0.8.7

v0.8.6

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.2

v0.7.1

v0.7.0

v0.6.43

v0.6.42

v0.6.41

v0.6.40

v0.6.39

v0.6.38

v0.6.37

v0.6.36

v0.6.35

v0.6.34

v0.6.33

v0.6.32

v0.6.31

v0.6.30

v0.6.29

v0.6.28

v0.6.27

v0.6.26

v0.6.25

v0.6.24

v0.6.23

v0.6.22

v0.6.21

v0.6.20

v0.6.19

v0.6.18

v0.6.17

v0.6.16

v0.6.15

v0.6.14

v0.6.13

v0.6.12

v0.6.11

v0.6.10

v0.6.9

v0.6.8

v0.6.7

v0.6.6

v0.6.5

v0.6.4

v0.6.3

v0.6.2

v0.6.1

v0.6.0

v0.5.20

v0.5.19

v0.5.18

v0.5.17

v0.5.16

v0.5.15

v0.5.14

v0.5.13

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.35

v0.3.34

v0.3.33

v0.3.32

v0.3.31

v0.3.30

v0.3.29

v0.3.28

v0.3.27

v0.3.26

v0.3.25

v0.3.24

v0.3.23

v0.3.22

v0.3.21

v0.3.20

v0.3.19

v0.3.18

v0.3.17

v0.3.16

v0.3.15

v0.3.14

v0.3.13

v0.3.12

v0.3.11

v0.3.10

v0.3.9

v0.3.8

v0.3.7

v0.3.6

v0.3.5

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.125

v0.1.124

v0.1.123

v0.1.122

v0.1.121

v0.1.120

v0.1.119

v0.1.118

v0.1.117

v0.1.116

v0.1.115

v0.1.114

v0.1.113

v0.1.112

v0.1.111

v0.1.110

v0.1.109

v0.1.108

v0.1.107

v0.1.106

v0.1.105

v0.1.104

v0.1.103

v0.1.102

Labels

Clear labels

bug

confirmed

confirmed issue

core

documentation

enhancement

good first issue

help wanted

non-core

pull-request

Mirrored from GitHub Pull Request

python

question

testing wanted

No Label pull-request

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

GiteaMirror ninjasurge

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#39021