issue: &#34;Chunk too big&#34; error when using Google Gemini 2.5 Flash with image input #6460

@yborunov commented on GitHub (Sep 26, 2025):

same issue. Any update?

@yborunov commented on GitHub (Sep 26, 2025): same issue. Any update?

GiteaMirror commented

@cj1324 commented on GitHub (Sep 26, 2025):

same issue. Any update?

@cj1324 commented on GitHub (Sep 26, 2025): same issue. Any update?

GiteaMirror commented

https://github.com/open-webui/open-webui/blob/main/backend/open_webui/routers/openai.py#L947

@ShirasawaSama commented on GitHub (Sep 26, 2025):

We previously discovered that when SSE single-line data is extremely large (likely exceeding 16KB), directly using Python's asynchronous iterator to traverse aiohttp's response.content forces the use of aiohttp's built-in buffer size, which cannot be configured.

Our solution was to replace it with our own asynchronous iterator, calling AsyncContentStream's iter_chunks() method to iterate over all bytes. We actively split each line using the \n symbol and then yield the data for each line.

This approach avoids the constraints of aiohttp's buffer size. Unfortunately, I don't have permission to publish this specific code, so I can only share the underlying solution concept.

cc @tjbck

@ShirasawaSama commented on GitHub (Sep 26, 2025): We previously discovered that when SSE single-line data is extremely large (likely exceeding 16KB), directly using Python's asynchronous iterator to traverse aiohttp's `response.content` forces the use of aiohttp's built-in buffer size, which cannot be configured. https://github.com/open-webui/open-webui/blob/main/backend/open_webui/routers/openai.py#L947 Our solution was to replace it with our own asynchronous iterator, calling AsyncContentStream's `iter_chunks()` method to iterate over all bytes. We actively split each line using the `\n` symbol and then yield the data for each line. This approach avoids the constraints of aiohttp's buffer size. Unfortunately, I don't have permission to publish this specific code, so I can only share the underlying solution concept. cc @tjbck

GiteaMirror commented

@OakerO commented on GitHub (Sep 29, 2025):

Maybe someone should publish it then?

@OakerO commented on GitHub (Sep 29, 2025): Maybe someone should publish it then?

GiteaMirror commented

@rgaricano commented on GitHub (Sep 29, 2025):

approach, for 37d1c85c99/backend/open_webui/routers/openai.py (L929-L977)

Increased buffer & replaced the default async iterator on response.content uses line-based iteration, which buffers entire lines in memory (for SSE data exceeding 16KB on a single line, this forces the StreamReader to buffer the entire line until it finds a newline character).
By using iter_chunked() or iter_any(), we avoid this line-buffering behavior and process data in smaller, manageable chunks regardless of line boundaries.
We can choose iter_any() for immediate data availability or iter_chunked() with a specific chunk size (like 8192 bytes) for more predictable memory usage.

try:
    session = aiohttp.ClientSession(
        trust_env=True, 
        timeout=aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT),
        read_bufsize=128 * 1024  # Increased buffer size
    )

    r = await session.request(
        method="POST",
        url=request_url,
        data=payload,
        headers=headers,
        cookies=cookies,
        ssl=AIOHTTP_CLIENT_SESSION_SSL,
    )

    # Check if response is SSE
    if "text/event-stream" in r.headers.get("Content-Type", ""):
        streaming = True

        # Use chunk-based iteration instead of direct content
        async def sse_generator():
            async for chunk in r.content.iter_any():  # or iter_chunked(8192)
                yield chunk

        return StreamingResponse(
            sse_generator(),
            status_code=r.status,
            headers=dict(r.headers),
            background=BackgroundTask(
                cleanup_response, response=r, session=session
            ),
        )
# ...
# rest of code

@rgaricano commented on GitHub (Sep 29, 2025): approach, for https://github.com/open-webui/open-webui/blob/37d1c85c996e1bdcd505e1e6d62b2f17acd8df23/backend/open_webui/routers/openai.py#L929-L977 - Increased buffer & replaced the default async iterator on response.content uses line-based iteration, which buffers entire lines in memory (for SSE data exceeding 16KB on a single line, this forces the StreamReader to buffer the entire line until it finds a newline character). By using iter_chunked() or iter_any(), we avoid this line-buffering behavior and process data in smaller, manageable chunks regardless of line boundaries. We can choose iter_any() for immediate data availability or iter_chunked() with a specific chunk size (like 8192 bytes) for more predictable memory usage. ``` try: session = aiohttp.ClientSession( trust_env=True, timeout=aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT), read_bufsize=128 * 1024 # Increased buffer size ) r = await session.request( method="POST", url=request_url, data=payload, headers=headers, cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, ) # Check if response is SSE if "text/event-stream" in r.headers.get("Content-Type", ""): streaming = True # Use chunk-based iteration instead of direct content async def sse_generator(): async for chunk in r.content.iter_any(): # or iter_chunked(8192) yield chunk return StreamingResponse( sse_generator(), status_code=r.status, headers=dict(r.headers), background=BackgroundTask( cleanup_response, response=r, session=session ), ) # ... # rest of code ```

GiteaMirror commented

@ShirasawaSama commented on GitHub (Sep 29, 2025):

@rgaricano

In fact, many of my MCP SSE calls exceed 500KB, and occasionally even surpass 50MB per line. Consequently, my code includes logic to skip lines exceeding a certain length.

However, I don't think this is a common scenario, so it may not necessarily need to be implemented.

Additionally, we cannot directly call iter_any or iter_chunked and then yield the chunk directly; this approach is incorrect.

Because a single chunk may contain multiple lines, or a single line may be split across multiple chunks. Therefore, we must concatenate multiple chunks and use the \n character to separate lines of text. If the current chunk's terminator is not a \n character, we can assume that the next chunk or subsequent chunks still contain the remainder of the current line.

@ShirasawaSama commented on GitHub (Sep 29, 2025): @rgaricano In fact, many of my MCP SSE calls exceed 500KB, and occasionally even surpass 50MB per line. Consequently, my code includes logic to skip lines exceeding a certain length. However, I don't think this is a common scenario, so it may not necessarily need to be implemented. Additionally, we cannot directly call `iter_any` or `iter_chunked` and then yield the chunk directly; this approach is incorrect. Because a single chunk may contain multiple lines, or a single line may be split across multiple chunks. Therefore, we must concatenate multiple chunks and use the `\n` character to separate lines of text. If the current chunk's terminator is not a `\n` character, we can assume that the next chunk or subsequent chunks still contain the remainder of the current line.

GiteaMirror commented

@rgaricano commented on GitHub (Sep 29, 2025):

then maybe something like this:

try:
    session = aiohttp.ClientSession(
        trust_env=True, 
        timeout=aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT),
        read_bufsize=128 * 1024  # Increased buffer size
    )

    r = await session.request(
        method="POST",
        url=request_url,
        data=payload,
        headers=headers,
        cookies=cookies,
        ssl=AIOHTTP_CLIENT_SESSION_SSL,
    )

    # Check if response is SSE
    if "text/event-stream" in r.headers.get("Content-Type", ""):
        streaming = True
        buffer = b""

        async def sse_line_generator():
            """Generator that handles SSE line boundaries."""

            async for chunk in r.content.iter_any():
                buffer += chunk

                # Process complete lines
                while b"\n" in buffer:
                    line, buffer = buffer.split(b"\n", 1)
                    yield line + b"\n"

            # Handle any remaining incomplete line
            if buffer:
                yield buffer

        return StreamingResponse(
            sse_line_generator(),
            status_code=r.status,
            headers=dict(r.headers),
            background=BackgroundTask(
                cleanup_response, response=r, session=session
            ),
        )
# ...
# rest of  code

@rgaricano commented on GitHub (Sep 29, 2025): then maybe something like this: ``` try: session = aiohttp.ClientSession( trust_env=True, timeout=aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT), read_bufsize=128 * 1024 # Increased buffer size ) r = await session.request( method="POST", url=request_url, data=payload, headers=headers, cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, ) # Check if response is SSE if "text/event-stream" in r.headers.get("Content-Type", ""): streaming = True buffer = b"" async def sse_line_generator(): """Generator that handles SSE line boundaries.""" async for chunk in r.content.iter_any(): buffer += chunk # Process complete lines while b"\n" in buffer: line, buffer = buffer.split(b"\n", 1) yield line + b"\n" # Handle any remaining incomplete line if buffer: yield buffer return StreamingResponse( sse_line_generator(), status_code=r.status, headers=dict(r.headers), background=BackgroundTask( cleanup_response, response=r, session=session ), ) # ... # rest of code ```

GiteaMirror commented

@ddddavid-he commented on GitHub (Oct 8, 2025):

Any update?

@ddddavid-he commented on GitHub (Oct 8, 2025): Any update?

GiteaMirror commented

@ShirasawaSama commented on GitHub (Oct 11, 2025):

Any update? 🤔

@ShirasawaSama commented on GitHub (Oct 11, 2025): Any update? 🤔

GiteaMirror commented

@COCONUTwwater commented on GitHub (Oct 17, 2025):

Any update? 🤔

@COCONUTwwater commented on GitHub (Oct 17, 2025): Any update? 🤔

GiteaMirror commented

@Classic298 commented on GitHub (Oct 17, 2025):

PR welcome afaik

@Classic298 commented on GitHub (Oct 17, 2025): PR welcome afaik

GiteaMirror commented

@GeoffreyLng commented on GitHub (Oct 21, 2025):

Same issue, any updates ?

@GeoffreyLng commented on GitHub (Oct 21, 2025): Same issue, any updates ?

GiteaMirror commented

@pixma140 commented on GitHub (Oct 21, 2025):

Having the same issue with the GPT-5 Image model. Just added OpenRouter and my API Key in my Open WebUI UI and experiencing the same issue. Sometimes I get back a little bit of text like: "Here you go! A corgi in space, in XYZ-style!" and then the red error message pops up.

@pixma140 commented on GitHub (Oct 21, 2025): Having the same issue with the _GPT-5 Image_ model. Just added OpenRouter and my API Key in my Open WebUI UI and experiencing the same issue. Sometimes I get back a little bit of text like: "_Here you go! A corgi in space, in XYZ-style!_" and then the red error message pops up.

GiteaMirror commented

@Classic298 commented on GitHub (Oct 21, 2025):

No update yet, PR welcome

@Classic298 commented on GitHub (Oct 21, 2025): No update yet, PR welcome

GiteaMirror commented

@mweichert commented on GitHub (Oct 25, 2025):

Also experiencing this issue with gemini-2.5-flash-lite.

@mweichert commented on GitHub (Oct 25, 2025): Also experiencing this issue with `gemini-2.5-flash-lite`.

GiteaMirror commented

@NopeNix commented on GitHub (Oct 30, 2025):

same here

@NopeNix commented on GitHub (Oct 30, 2025): same here

GiteaMirror commented

2025-11-11 16:55:45 -06:00

@zuogod commented on GitHub (Nov 3, 2025):

Any update? 🤔

@zuogod commented on GitHub (Nov 3, 2025): Any update? 🤔

GiteaMirror commented

@Classic298 commented on GitHub (Nov 3, 2025):

PR welcome

@Classic298 commented on GitHub (Nov 3, 2025): PR welcome

GiteaMirror commented

@ShirasawaSama commented on GitHub (Nov 3, 2025):

#18884

@ShirasawaSama commented on GitHub (Nov 3, 2025): #18884

GiteaMirror commented

@silentoplayz commented on GitHub (Nov 5, 2025):

Related - https://github.com/open-webui/open-webui/issues/16935

@silentoplayz commented on GitHub (Nov 5, 2025): Related - https://github.com/open-webui/open-webui/issues/16935

GiteaMirror commented

@Classic298 commented on GitHub (Nov 6, 2025):

should be fixed in dev

@Classic298 commented on GitHub (Nov 6, 2025): should be fixed in dev

GiteaMirror commented

2025-11-11 16:55:47 -06:00

@ShirasawaSama commented on GitHub (Nov 7, 2025):

@Classic298 Not yet resolved

@ShirasawaSama commented on GitHub (Nov 7, 2025): @Classic298 Not yet resolved <img width="1078" height="890" alt="Image" src="https://github.com/user-attachments/assets/eb3e4405-72f1-440b-8367-1022fec5a8ae" />

GiteaMirror commented

@ddddavid-he commented on GitHub (Nov 7, 2025):

Same situation on my machine

@ddddavid-he commented on GitHub (Nov 7, 2025): Same situation on my machine

GiteaMirror commented

2025-11-11 16:55:47 -06:00

@DeusMaximus commented on GitHub (Nov 7, 2025):

Still happening through LiteLLM

@DeusMaximus commented on GitHub (Nov 7, 2025): Still happening through LiteLLM <img width="833" height="273" alt="Image" src="https://github.com/user-attachments/assets/614fad83-1b22-483b-96bd-7b8904960648" />

GiteaMirror commented

@GeoffreyLng commented on GitHub (Nov 7, 2025):

Still occuring when using LiteLLM

@GeoffreyLng commented on GitHub (Nov 7, 2025): Still occuring when using LiteLLM <img width="1081" height="486" alt="Image" src="https://github.com/user-attachments/assets/ba011592-e10c-4ebc-9e9a-a05ca717238b" />

GiteaMirror commented

@ddkedr commented on GitHub (Nov 7, 2025):

@Classic298 can we reopen it? Still does not work when using LiteLLM

@ddkedr commented on GitHub (Nov 7, 2025): @Classic298 can we reopen it? Still does not work when using LiteLLM

GiteaMirror commented

@Classic298 commented on GitHub (Nov 7, 2025):

I am aware, but that is as it looks to me, an issue with LiteLLM. I raised it with them, hopefully they can address it soon.

(Add Gemini 2.5 Flash Image in the Image configs)

@Classic298 commented on GitHub (Nov 7, 2025): I am aware, but that is as it looks to me, an issue with LiteLLM. I raised it with them, hopefully they can address it soon. (Add Gemini 2.5 Flash Image in the Image configs)

GiteaMirror commented

2025-11-11 16:55:49 -06:00

@AlexZorzi commented on GitHub (Nov 7, 2025):

@Classic298 for me its not working either, im not using LiteLLM, connected directly to OpenRouter.

@AlexZorzi commented on GitHub (Nov 7, 2025): @Classic298 for me its not working either, im not using LiteLLM, connected directly to OpenRouter. <img width="1047" height="271" alt="Image" src="https://github.com/user-attachments/assets/d015d547-f452-43dc-a217-b6b29880e789" />

GiteaMirror commented

@Classic298 commented on GitHub (Nov 7, 2025):

Do not talk to the model - add it in the image config as I said. - and not sure openrouter is supported yet, but the official Gemini API is supported.

@Classic298 commented on GitHub (Nov 7, 2025): Do not talk to the model - add it in the image config as I said. - and not sure openrouter is supported yet, but the official Gemini API is supported.

GiteaMirror commented

2025-11-11 16:55:49 -06:00

@ShirasawaSama commented on GitHub (Nov 10, 2025):

Hey everyone, the “Chunk too big” issue has been fixed in the latest dev branch.

You can enable it using the following environment variable:

CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760

And the issue where models in OpenRouter cannot output images will be fixed in this pull request #19073.

@ShirasawaSama commented on GitHub (Nov 10, 2025): Hey everyone, the “Chunk too big” issue has been fixed in the latest dev branch. You can enable it using the following environment variable: ```bash CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760 ``` And the issue where models in OpenRouter cannot output images will be fixed in this pull request #19073.

GiteaMirror commented

2025-11-11 16:55:49 -06:00

@tjbck commented on GitHub (Nov 10, 2025):

@silentoplayz @Classic298 Documentation updated wanted here!

@tjbck commented on GitHub (Nov 10, 2025): @silentoplayz @Classic298 Documentation updated wanted here!

GiteaMirror commented

@silentoplayz commented on GitHub (Nov 10, 2025):

You can enable it using the following environment variable:
CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760

What is enabled when I set this as an environment variable for my Open WebUI instance? Is it necessary to have this environment variable set at all for image generation/editing to work properly using Google Gemini 2.5 Flash in Open WebUI?

@silentoplayz commented on GitHub (Nov 10, 2025): > You can enable it using the following environment variable: > `CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760` What is enabled when I set this as an environment variable for my Open WebUI instance? Is it necessary to have this environment variable set at all for image generation/editing to work properly using Google Gemini 2.5 Flash in Open WebUI?

GiteaMirror commented

@ShirasawaSama commented on GitHub (Nov 10, 2025):

Related PR #19076

@ShirasawaSama commented on GitHub (Nov 10, 2025): Related PR #19076

GiteaMirror commented

@ShirasawaSama commented on GitHub (Nov 10, 2025):

You can enable it using the following environment variable:
CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760

What is enabled when I set this as an environment variable for my Open WebUI instance? Is it necessary to have this environment variable set at all for image generation/editing to work properly using Google Gemini 2.5 Flash in Open WebUI?

This value is recommended to be greater than 5MB. As a reference, a GPT image typically outputs around 3.5MB of Base64-encoded image data. Our production environment is configured to 8MB (since we implemented sending socket.io data via SSE, which prevents massive single-line data from causing Redis cluster crashes; therefore, I do not recommend setting this value too high).

A value greater than 2.5MB must be set to display images generated by the nano banana model.

@ShirasawaSama commented on GitHub (Nov 10, 2025): > > You can enable it using the following environment variable: > > `CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=10485760` > > What is enabled when I set this as an environment variable for my Open WebUI instance? Is it necessary to have this environment variable set at all for image generation/editing to work properly using Google Gemini 2.5 Flash in Open WebUI? This value is recommended to be greater than 5MB. As a reference, a GPT image typically outputs around 3.5MB of Base64-encoded image data. Our production environment is configured to 8MB (since we implemented sending socket.io data via SSE, which prevents massive single-line data from causing Redis cluster crashes; therefore, I do not recommend setting this value too high). A value greater than 2.5MB must be set to display images generated by the nano banana model.

GiteaMirror commented