[GH-ISSUE #16722] issue: Random 500 and 502 even when open-webui is not used. #18021

New Issue

GiteaMirror · 2026-04-19T23:55:11-05:00

GiteaMirror commented

2026-04-19 23:55:11 -05:00

Originally created by @nitanmarcel on GitHub (Aug 19, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/16722

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Pip Install

Open WebUI Version

v0.6.22

Ollama Version (if applicable)

0.11.4

Operating System

Debian 12

Browser (if applicable)

Any

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided every relevant configuration, setting, and environment variable used in my setup.
I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
Start with the initial platform/version/OS and dependencies used,
Specify exact install/launch/configure commands,
List URLs visited, user input (incl. example values/emails/passwords if needed),
Describe all options and toggles enabled or changed,
Include any files or environmental changes,
Identify the expected and actual result at each stage,
Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

Open-WebUI should not randomly crash

Actual Behavior

I get random crashes even when open-webui is not used. I actually have no idea why these happen so I dumped the last 15 minutes of my journalctl to hopefully catch the actual issue in the logs, uptime robot announced the crash 8 minutes before I took the logs so it should be enough hopefully.

Steps to Reproduce

.

Logs & Screenshots

crash_log.txt

Additional Information

No response

Originally created by @nitanmarcel on GitHub (Aug 19, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/16722 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Pip Install ### Open WebUI Version v0.6.22 ### Ollama Version (if applicable) 0.11.4 ### Operating System Debian 12 ### Browser (if applicable) Any ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior Open-WebUI should not randomly crash ### Actual Behavior I get random crashes even when open-webui is not used. I actually have no idea why these happen so I dumped the last 15 minutes of my journalctl to hopefully catch the actual issue in the logs, uptime robot announced the crash 8 minutes before I took the logs so it should be enough hopefully. ### Steps to Reproduce . ### Logs & Screenshots [crash_log.txt](https://github.com/user-attachments/files/21850576/crash_log.txt) ### Additional Information _No response_

GiteaMirror added the bug label 2026-04-19 23:55:11 -05:00

GiteaMirror closed this issue

2026-04-19 23:55:11 -05:00

GiteaMirror commented

2026-04-19 23:55:12 -05:00

@rgaricano commented on GitHub (Aug 19, 2025):

did you update or using another version (e.g. dev) ?

I had this kind of errors response = await call_next(request) in different places when
running dev version (& some tabs with sessions opened in previous version).
But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server.

Try cleaning browser cache & site cookie (or close/reopen user session)

@rgaricano commented on GitHub (Aug 19, 2025): did you update or using another version (e.g. dev) ? I had this kind of errors `response = await call_next(request)` in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server. Try cleaning browser cache & site cookie (or close/reopen user session)

GiteaMirror commented

2026-04-19 23:55:14 -05:00

@nitanmarcel commented on GitHub (Aug 19, 2025):

did you update or using another version (e.g. dev) ?

I had this kind of errors response = await call_next(request) in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server.

Try cleaning browser cache & site cookie (or close/reopen user session)

Nope, I follower

did you update or using another version (e.g. dev) ?

I had this kind of errors response = await call_next(request) in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server.

Try cleaning browser cache & site cookie (or close/reopen user session)

It's installed with pip install open-webui so I'm guessing is the stable version

@nitanmarcel commented on GitHub (Aug 19, 2025): > did you update or using another version (e.g. dev) ? > > I had this kind of errors `response = await call_next(request)` in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server. > > Try cleaning browser cache & site cookie (or close/reopen user session) Nope, I follower > did you update or using another version (e.g. dev) ? > > I had this kind of errors `response = await call_next(request)` in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server. > > Try cleaning browser cache & site cookie (or close/reopen user session) It's installed with `pip install open-webui` so I'm guessing is the stable version

GiteaMirror commented

2026-04-19 23:55:15 -05:00

@nitanmarcel commented on GitHub (Aug 19, 2025):

did you update or using another version (e.g. dev) ?

I had this kind of errors response = await call_next(request) in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server.

Try cleaning browser cache & site cookie (or close/reopen user session)

I don't think it's related to the browser since no one was using the web ui at that time

@nitanmarcel commented on GitHub (Aug 19, 2025): > did you update or using another version (e.g. dev) ? > > I had this kind of errors `response = await call_next(request)` in different places when running dev version (& some tabs with sessions opened in previous version). But I didn't found what was the origin, I supose that is because some websockets connections are opening in client but closed in server. > > Try cleaning browser cache & site cookie (or close/reopen user session) I don't think it's related to the browser since no one was using the web ui at that time

GiteaMirror commented

2026-04-19 23:55:16 -05:00

@rgaricano commented on GitHub (Aug 19, 2025):

In my case it was due for validation errors uploading docs to knowledge.

for reference:
i think that it's due because when concurrent tasks are running and multiple tasks fail simultaneously, this causes ExceptionGroup errors that isn't handled, and, at lest, tasks manager should handle its.

e.g.
tasks.py

import sys
from builtins import ExceptionGroup  

...

async def redis_task_command_listener(app):  
    redis: Redis = app.state.redis  
    pubsub = redis.pubsub()  
    await pubsub.subscribe(REDIS_PUBSUB_CHANNEL)  
  
    async for message in pubsub.listen():  
        if message["type"] != "message":  
            continue  
        try:  
            command = json.loads(message["data"])  
            if command.get("action") == "stop":  
                task_id = command.get("task_id")  
                local_task = tasks.get(task_id)  
                if local_task:  
                    try:  
                        local_task.cancel()  
                        # Wait briefly for cancellation to complete  
                        await asyncio.sleep(0.1)  
                    except Exception as cancel_error:  
                        log.error(f"Error cancelling task {task_id}: {cancel_error}")  
        except ExceptionGroup as eg:  
            # Handle multiple concurrent exceptions  
            log.error(f"Multiple errors in task command processing: {len(eg.exceptions)} exceptions")  
            for i, exc in enumerate(eg.exceptions):  
                log.error(f"  Exception {i+1}: {type(exc).__name__}: {exc}")  
        except Exception as e:  
            log.exception(f"Error handling distributed task command: {e}")

...

async def cleanup_task(redis, task_id: str, id=None):  
    """  
    Remove a completed or canceled task with proper exception handling.  
    """  
    cleanup_errors = []  
      
    # Redis cleanup  
    if redis:  
        try:  
            await redis_cleanup_task(redis, task_id, id)  
        except Exception as e:  
            cleanup_errors.append(e)  
            log.error(f"Redis cleanup failed for task {task_id}: {e}")  
  
    # Local cleanup  
    try:  
        tasks.pop(task_id, None)  
        if id and task_id in item_tasks.get(id, []):  
            item_tasks[id].remove(task_id)  
            if not item_tasks[id]:  
                item_tasks.pop(id, None)  
    except Exception as e:  
        cleanup_errors.append(e)  
        log.error(f"Local cleanup failed for task {task_id}: {e}")  
      
    # If multiple errors occurred, group them  
    if len(cleanup_errors) > 1 and ExceptionGroup:  
        raise ExceptionGroup(f"Multiple cleanup errors for task {task_id}", cleanup_errors)  
    elif cleanup_errors:  
        raise cleanup_errors[0]

and for listener cancellation during shutdown
main.py 576

...
# In the lifespan shutdown  
if hasattr(app.state, "redis_task_command_listener"):  
    try:  
        app.state.redis_task_command_listener.cancel()  
        app.state.redis_task_command_listener  
    except ExceptionGroup as eg:  
        log.error(f"Multiple errors during listener shutdown: {len(eg.exceptions)} exceptions")  
        for exc in eg.exceptions:  
            log.error(f"Shutdown error: {type(exc).__name__}: {exc}")  
    except asyncio.CancelledError:  
        pass  # Expected during shutdown  
    except Exception as e:  
        log.error(f"Error during listener shutdown: {e}")
...

@rgaricano commented on GitHub (Aug 19, 2025): In my case it was due for validation errors uploading docs to knowledge. for reference: i think that it's due because when concurrent tasks are running and multiple tasks fail simultaneously, this causes ExceptionGroup errors that isn't handled, and, at lest, tasks manager should handle its. e.g. tasks.py ``` import sys from builtins import ExceptionGroup ... async def redis_task_command_listener(app): redis: Redis = app.state.redis pubsub = redis.pubsub() await pubsub.subscribe(REDIS_PUBSUB_CHANNEL) async for message in pubsub.listen(): if message["type"] != "message": continue try: command = json.loads(message["data"]) if command.get("action") == "stop": task_id = command.get("task_id") local_task = tasks.get(task_id) if local_task: try: local_task.cancel() # Wait briefly for cancellation to complete await asyncio.sleep(0.1) except Exception as cancel_error: log.error(f"Error cancelling task {task_id}: {cancel_error}") except ExceptionGroup as eg: # Handle multiple concurrent exceptions log.error(f"Multiple errors in task command processing: {len(eg.exceptions)} exceptions") for i, exc in enumerate(eg.exceptions): log.error(f" Exception {i+1}: {type(exc).__name__}: {exc}") except Exception as e: log.exception(f"Error handling distributed task command: {e}") ... async def cleanup_task(redis, task_id: str, id=None): """ Remove a completed or canceled task with proper exception handling. """ cleanup_errors = [] # Redis cleanup if redis: try: await redis_cleanup_task(redis, task_id, id) except Exception as e: cleanup_errors.append(e) log.error(f"Redis cleanup failed for task {task_id}: {e}") # Local cleanup try: tasks.pop(task_id, None) if id and task_id in item_tasks.get(id, []): item_tasks[id].remove(task_id) if not item_tasks[id]: item_tasks.pop(id, None) except Exception as e: cleanup_errors.append(e) log.error(f"Local cleanup failed for task {task_id}: {e}") # If multiple errors occurred, group them if len(cleanup_errors) > 1 and ExceptionGroup: raise ExceptionGroup(f"Multiple cleanup errors for task {task_id}", cleanup_errors) elif cleanup_errors: raise cleanup_errors[0] ``` and for listener cancellation during shutdown main.py 576 ``` ... # In the lifespan shutdown if hasattr(app.state, "redis_task_command_listener"): try: app.state.redis_task_command_listener.cancel() app.state.redis_task_command_listener except ExceptionGroup as eg: log.error(f"Multiple errors during listener shutdown: {len(eg.exceptions)} exceptions") for exc in eg.exceptions: log.error(f"Shutdown error: {type(exc).__name__}: {exc}") except asyncio.CancelledError: pass # Expected during shutdown except Exception as e: log.error(f"Error during listener shutdown: {e}") ... ```

GiteaMirror commented

2026-04-19 23:55:17 -05:00

@nitanmarcel commented on GitHub (Aug 19, 2025):

I notice just now that the logs do not show the crash reason since they are since the last time the app restarted

@nitanmarcel commented on GitHub (Aug 19, 2025): I notice just now that the logs do not show the crash reason since they are since the last time the app restarted

GiteaMirror referenced this issue

2026-04-20 05:30:53 -05:00

[PR #18021] [CLOSED] feat(connections): add support for custom HTTP headers #24642

GiteaMirror referenced this issue

2026-04-25 12:40:34 -05:00

[PR #18021] [CLOSED] feat(connections): add support for custom HTTP headers #40272

GiteaMirror referenced this issue

2026-04-29 23:01:51 -05:00

[PR #18021] [CLOSED] feat(connections): add support for custom HTTP headers #47690

GiteaMirror referenced this issue

2026-05-06 08:19:03 -05:00

[PR #18021] [CLOSED] feat(connections): add support for custom HTTP headers #63498