issue: Files from tool servers are ignored #5002

New Issue

2025-11-11T16:09:33-06:00

GiteaMirror commented

2025-11-11 16:09:33 -06:00

Originally created by @timsomers on GitHub (Apr 30, 2025).

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Git Clone

Open WebUI Version

0.6.5

Ollama Version (if applicable)

No response

Operating System

Debian (wsl)

Browser (if applicable)

No response

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

Expected Behavior

Binary data coming from an openapi tool server is included (as a file?) in the conversation.

Actual Behavior

The binary data gets silently dropped, only text data is used.

Steps to Reproduce

Set up an openapi server that returns a "List[EMail | str]" type. The EMail model is not relevant here as these get properly included, but mail attachments are string encoded using the "data:image/png;base64,..." url scheme. The server responds with a mix of EMails and str's, which all start with "data:". I found that the returned data gets processed in https://github.com/open-webui/open-webui/blob/main/backend/open_webui/utils/middleware.py

In line 226 had to change "tool_result" to "tool_result[:]" to work with a copy of the list, as this causes issues once items get removed on line 230. Once this is done, the original mix of mails and attachments gets properly split into mails only in tool_result, and the attachments in tool_result_files. After that, tool_result_files gets ignored completely, only the email text gets returned for use.

Logs & Screenshots

not relevant

Additional Information

This looks like a feature that's just not finished yet. Is this currently being worked on? Would it be as simple as doing the equivalent of uploading an image at https://github.com/open-webui/open-webui/blob/main/backend/open_webui/routers/images.py#L453 and then adding it to body["metadata"]["files"] ?

Originally created by @timsomers on GitHub (Apr 30, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Git Clone ### Open WebUI Version 0.6.5 ### Ollama Version (if applicable) _No response_ ### Operating System Debian (wsl) ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior Binary data coming from an openapi tool server is included (as a file?) in the conversation. ### Actual Behavior The binary data gets silently dropped, only text data is used. ### Steps to Reproduce Set up an openapi server that returns a "List[EMail | str]" type. The EMail model is not relevant here as these get properly included, but mail attachments are string encoded using the "data:image/png;base64,..." url scheme. The server responds with a mix of EMails and str's, which all start with "data:". I found that the returned data gets processed in https://github.com/open-webui/open-webui/blob/main/backend/open_webui/utils/middleware.py In line 226 had to change "tool_result" to "tool_result[:]" to work with a copy of the list, as this causes issues once items get removed on line 230. Once this is done, the original mix of mails and attachments gets properly split into mails only in tool_result, and the attachments in tool_result_files. After that, tool_result_files gets ignored completely, only the email text gets returned for use. ### Logs & Screenshots not relevant ### Additional Information This looks like a feature that's just not finished yet. Is this currently being worked on? Would it be as simple as doing the equivalent of uploading an image at https://github.com/open-webui/open-webui/blob/main/backend/open_webui/routers/images.py#L453 and then adding it to body["metadata"]["files"] ?

GiteaMirror added the bug label 2025-11-11 16:09:33 -06:00

GiteaMirror closed this issue

2025-11-11 16:09:33 -06:00

GiteaMirror commented

2025-11-11 16:09:33 -06:00