issue: OpenAPI Tool Servers Error with AWS Bedrock's Anthropic Claude 3.7 #4775

New Issue

GiteaMirror · 2025-11-11T16:02:53-06:00

GiteaMirror commented

2025-11-11 16:02:53 -06:00

Originally created by @dulicon on GitHub (Apr 9, 2025).

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.6.2

Ollama Version (if applicable)

No response

Operating System

Amazon Linux 2

Browser (if applicable)

Firefox 137.0

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

Expected Behavior

I am using AWS Bedrock's Anthropic Claude 3.7 through LiteLLM (https://docs.litellm.ai/docs/ ).
I tested Microsoft's playwright-mcp server (https://github.com/microsoft/playwright-mcp ) using mcpo and asked it to open www.google.com.
When testing with the gpt-4o model, the browser opened normally.

Actual Behavior

As shown in the image, the Tool operation occurred twice. The first Tool operation resulted in an error, while the second Tool operation continues to run indefinitely.

Steps to Reproduce

Run Microsoft's playwright-mcp server (https://github.com/microsoft/playwright-mcp ) using mcpo.
Configure AWS Bedrock's Anthropic Claude 3.7 to be usable in Open Webui through litellm.
Enter "navigate to www.google.com" in the chat window.

Logs & Screenshots

Additional Information

After debugging the source code locally, it appears to be an index issue when setting up the received tool_calls.

For the gpt-4o model, the tool_calls index comes in as 0.

    "choices": [
        {
            "index": 0,
            "delta": {
                "role": "assistant",
                "tool_calls": [
                    {
                        "function": {
                            "arguments": "url"
                        },
                        "type": "function",
                        "index": 0
                    }
                ]
            }
        }
    ]

However, for AWS Bedrock's Anthropic Claude 3.7, the tool_calls index comes in as 1.

    "choices": [
        {
            "index": 0,
            "delta": {
                "content": "",
                "role": "assistant",
                "tool_calls": [
                    {
                        "function": {
                            "arguments": "url\": \"www"
                        },
                        "type": "function",
                        "index": 1
                    }
                ]
            }
        }
    ],

When examining the middleware.py source code(Line 1655), it seems the error occurs because the index starts from 1, causing it to be entered twice.

                                    if delta_tool_calls:
                                        for delta_tool_call in delta_tool_calls:
                                            tool_call_index = delta_tool_call.get(
                                                "index"
                                            )

                                            if tool_call_index is not None:
                                                if (
                                                    len(response_tool_calls)
                                                    <= tool_call_index
                                                ):
                                                    response_tool_calls.append(
                                                        delta_tool_call
                                                    )
                                                else:
                                                    delta_name = delta_tool_call.get(
                                                        "function", {}
                                                    ).get("name")
                                                    delta_arguments = (
                                                        delta_tool_call.get(
                                                            "function", {}
                                                        ).get("arguments")
                                                    )

                                                    if delta_name:
                                                        response_tool_calls[
                                                            tool_call_index
                                                        ]["function"][
                                                            "name"
                                                        ] += delta_name

                                                    if delta_arguments:
                                                        response_tool_calls[
                                                            tool_call_index
                                                        ]["function"][
                                                            "arguments"
                                                        ] += delta_arguments

I confirmed that it works properly when I made the following modifications locally for testing:

                                    if delta_tool_calls:
                                        for delta_tool_call in delta_tool_calls:
                                            tool_call_index = delta_tool_call.get(
                                                "index"
                                            )

                                            if tool_call_index is not None:
                                                response_tool_index = None
                                                for idx, item in enumerate(response_tool_calls):
                                                    if item.get("index") == tool_call_index:
                                                        response_tool_index = idx
                                            
                                                if response_tool_index is None:
                                                    response_tool_calls.append(
                                                        delta_tool_call
                                                    )
                                                else:
                                                    delta_name = delta_tool_call.get(
                                                        "function", {}
                                                    ).get("name")
                                                    delta_arguments = (
                                                        delta_tool_call.get(
                                                            "function", {}
                                                        ).get("arguments")
                                                    )

                                                    if delta_name:
                                                        response_tool_calls[
                                                            response_tool_index
                                                        ]["function"][
                                                            "name"
                                                        ] += delta_name

                                                    if delta_arguments:
                                                        response_tool_calls[
                                                            response_tool_index
                                                        ]["function"][
                                                            "arguments"
                                                        ] += delta_arguments

Originally created by @dulicon on GitHub (Apr 9, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.6.2 ### Ollama Version (if applicable) _No response_ ### Operating System Amazon Linux 2 ### Browser (if applicable) Firefox 137.0 ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior I am using AWS Bedrock's Anthropic Claude 3.7 through LiteLLM ([https://docs.litellm.ai/docs/ ](https://docs.litellm.ai/docs/)). I tested Microsoft's playwright-mcp server ([https://github.com/microsoft/playwright-mcp ](https://github.com/microsoft/playwright-mcp)) using mcpo and asked it to open www.google.com. When testing with the gpt-4o model, the browser opened normally. ### Actual Behavior As shown in the image, the Tool operation occurred twice. The first Tool operation resulted in an error, while the second Tool operation continues to run indefinitely. ![Image](https://github.com/user-attachments/assets/71f4f97d-f87d-4c52-9550-2ca4839be2e8) ### Steps to Reproduce 1. Run Microsoft's playwright-mcp server ([https://github.com/microsoft/playwright-mcp ](https://github.com/microsoft/playwright-mcp)) using mcpo. 2. Configure AWS Bedrock's Anthropic Claude 3.7 to be usable in Open Webui through litellm. 3. Enter "navigate to www.google.com" in the chat window. ### Logs & Screenshots ![Image](https://github.com/user-attachments/assets/526c8bfd-2db9-429f-9f30-b16ba5275421) ### Additional Information After debugging the source code locally, it appears to be an index issue when setting up the received tool_calls. For the gpt-4o model, the tool_calls index comes in as 0. ```json "choices": [ { "index": 0, "delta": { "role": "assistant", "tool_calls": [ { "function": { "arguments": "url" }, "type": "function", "index": 0 } ] } } ] ``` However, for AWS Bedrock's Anthropic Claude 3.7, the tool_calls index comes in as 1. ```json "choices": [ { "index": 0, "delta": { "content": "", "role": "assistant", "tool_calls": [ { "function": { "arguments": "url\": \"www" }, "type": "function", "index": 1 } ] } } ], ``` When examining the middleware.py source code(Line 1655), it seems the error occurs because the index starts from 1, causing it to be entered twice. ```python if delta_tool_calls: for delta_tool_call in delta_tool_calls: tool_call_index = delta_tool_call.get( "index" ) if tool_call_index is not None: if ( len(response_tool_calls) <= tool_call_index ): response_tool_calls.append( delta_tool_call ) else: delta_name = delta_tool_call.get( "function", {} ).get("name") delta_arguments = ( delta_tool_call.get( "function", {} ).get("arguments") ) if delta_name: response_tool_calls[ tool_call_index ]["function"][ "name" ] += delta_name if delta_arguments: response_tool_calls[ tool_call_index ]["function"][ "arguments" ] += delta_arguments ``` I confirmed that it works properly when I made the following modifications locally for testing: ```python if delta_tool_calls: for delta_tool_call in delta_tool_calls: tool_call_index = delta_tool_call.get( "index" ) if tool_call_index is not None: response_tool_index = None for idx, item in enumerate(response_tool_calls): if item.get("index") == tool_call_index: response_tool_index = idx if response_tool_index is None: response_tool_calls.append( delta_tool_call ) else: delta_name = delta_tool_call.get( "function", {} ).get("name") delta_arguments = ( delta_tool_call.get( "function", {} ).get("arguments") ) if delta_name: response_tool_calls[ response_tool_index ]["function"][ "name" ] += delta_name if delta_arguments: response_tool_calls[ response_tool_index ]["function"][ "arguments" ] += delta_arguments ``` ![Image](https://github.com/user-attachments/assets/bd6c9ba8-4817-4ea8-a653-8145b58fc1e9)

GiteaMirror added the bug label 2025-11-11 16:02:53 -06:00

GiteaMirror closed this issue

2025-11-11 16:02:54 -06:00

GiteaMirror commented

2025-11-11 16:02:54 -06:00

@tjbck commented on GitHub (Apr 10, 2025):

PR Welcome!

@tjbck commented on GitHub (Apr 10, 2025): PR Welcome!

GiteaMirror commented

2025-11-11 16:02:54 -06:00

@tjbck commented on GitHub (Apr 11, 2025):

This should also be fixed on LiteLLM as well.