issue: task models error when base64 text content is present (images) #5156

New Issue

2025-11-11T16:13:26-06:00

GiteaMirror commented

2025-11-11 16:13:26 -06:00

Originally created by @almajo on GitHub (May 13, 2025).

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Git Clone

Open WebUI Version

v0.6.9

Ollama Version (if applicable)

No response

Operating System

linux

Browser (if applicable)

No response

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

Expected Behavior

Task models like title and tag generation should work without images or other base64 content for quick and cheap responses.

Actual Behavior

As embedded images are simply text, they use a lot of tokens and thereby surpass context lengths which leads to chats being called "New Chat".

Steps to Reproduce

Create a large image with any model so that it is present in the chat (e.g. the gpt-image-1 function)

Logs & Screenshots

Additional Information

I already added a fix in the latest release for some use cases but forgot about the embedded image case. This should be an easy add-on in the implementation.

Originally created by @almajo on GitHub (May 13, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Git Clone ### Open WebUI Version v0.6.9 ### Ollama Version (if applicable) _No response_ ### Operating System linux ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior Task models like title and tag generation should work without images or other base64 content for quick and cheap responses. ### Actual Behavior As embedded images are simply text, they use a lot of tokens and thereby surpass context lengths which leads to chats being called "New Chat". ### Steps to Reproduce 1. Create a large image with any model so that it is present in the chat (e.g. the gpt-image-1 function) ### Logs & Screenshots <img width="380" alt="Image" src="https://github.com/user-attachments/assets/5839cbc6-699a-4836-99ac-41c4fdfa45f5" /> ### Additional Information I already added a fix in the latest release for some use cases but forgot about the embedded image case. This should be an easy add-on in the implementation.

GiteaMirror added the bug label 2025-11-11 16:13:26 -06:00

GiteaMirror closed this issue

2025-11-11 16:13:26 -06:00

GiteaMirror commented

2025-11-11 16:13:27 -06:00

@tjbck commented on GitHub (May 16, 2025):

Should be addressed by 528a2cf96a, testing wanted here!

@tjbck commented on GitHub (May 16, 2025): Should be addressed by 528a2cf96a8bef32fd9f8fdc8467908b52895a3a, testing wanted here!

GiteaMirror commented

2025-11-11 16:13:27 -06:00

@almajo commented on GitHub (May 19, 2025):

Hey, as far as I see, this doesn't fix the issue.

The use case I described is especially important for Functions which create images and show them to the user via markdown. Hereby, they simply emit the picture as a string, hence the content would be as string in message['content'].

A fix would be

if isinstance(content, str):
    patterns = [r"<details\b[^>]*>.*?<\/details>", "r"!\[.*?\]\(data:image\/[a-zA-Z]+;base64,[A-Za-z0-9+/=]+\)"]
    for p in patterns:
      content = re.sub(
          p,
          "",
          content,
          flags=re.S | re.I,
      ).strip()

The base64 pattern should be verified

@almajo commented on GitHub (May 19, 2025): Hey, as far as I see, this doesn't fix the issue. The use case I described is especially important for Functions which create images and show them to the user via markdown. Hereby, they simply emit the picture as a string, hence the content would be as string in `message['content']`. A fix would be ```python if isinstance(content, str): patterns = [r"<details\b[^>]*>.*?<\/details>", "r"!\[.*?\]\(data:image\/[a-zA-Z]+;base64,[A-Za-z0-9+/=]+\)"] for p in patterns: content = re.sub( p, "", content, flags=re.S | re.I, ).strip() ``` The base64 pattern should be verified