[GH-ISSUE #13524] issue: Exhausting Workers in Nginx Reverse Proxy in 0.6.6 #16939

Closed
opened 2026-04-19 22:45:01 -05:00 by GiteaMirror · 0 comments
Owner

Originally created by @borysow on GitHub (May 5, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/13524

Check Existing Issues

  • I have searched the existing issues and discussions.
  • I am using the latest version of Open WebUI.

Installation Method

Pip Install

Open WebUI Version

0.6.6

Ollama Version (if applicable)

No response

Operating System

Ubuntu 22.04 Container

Browser (if applicable)

Google Chrome 121.0.6167.139

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have listed steps to reproduce the bug in detail.

Expected Behavior

I have an Open WebUI instance set up via docker compose to bring up Open WebUI on port 8080 on the internal docker network, and have an nginx reverse proxy configured to establish a secure connection to users outside the host. I believe I based the nginx config on an example in the documentation.

I expect to have this configuration still work to allow end users to get a secure connection by going to https://<docker_host_name>:8080.

Everything works happily on OpenWebUI 0.6.5.

Actual Behavior

After updating to 0.6.6, I can log in briefly, but after a short amount of time my nginx server logs report that xxx "worker_connections are not enough while connecting to upstream" and I get either Error 500 or 503 (Bad Gateway).

(I know it's not the proper solution, but I increased the max_num_workers to 4096 as a test, and all it did was make it take longer to have the errors emerge).

I don't know if this is related to the changes in Websocket mentioned in the changelog.

Below are some logs of the issue as well as my docker compose file and nginx.conf.

Steps to Reproduce

With the following nginx and docker compose setup, performing any activity on the OpenWebUI server (maybe opening a few chats and wandering around the configs) very quickly causes this.

docker-compose:

services:
  # nothing weird in the build, just apt installing the same stuff in the official dockerfile, creating an openwebui user
  # and pip installing openwebui 
  openwebui:
    container_name: openwebui-server-${OPENWEBUI_VERSION}-${OPENWEBUI_ENVIRONMENT}
    image: openwebui-server:${OPENWEBUI_VERSION}
    build:
      context: ../openwebui
      dockerfile: ../docker/dockerfiles/dockerfile_openwebui
      args:
        OPENWEBUI_VERSION: ${OPENWEBUI_VERSION}
    volumes:
      - open-webui:/home/openwebui/app:z
    environment:
      HF_DATASETS_ONLINE: 1
      HF_HUB_OFFLINE: 1
      WEBUI_NAME: ${OPENWEBUI_NAME}
      WEBUI_URL: http://0.0.0.0:8080
      PORT: 8080
      GLOBAL_LOG_LEVEL: INFO

    command: 
      - "serve"
    restart: always
  
  # this is just a vanilla nginx image with .conf and certs added (this is a prototype instance they are temporary self-signed certs)
  nginx:
    container_name: nginx-openwebui-server-${OPENWEBUI_VERSION}-${OPENWEBUI_ENVIRONMENT}
    image: nginx-certs:latest
    build:
      context: ../nginx
      dockerfile: ../docker/dockerfiles/dockerfile_nginx
    ports:
      - ${OPENWEBUI_EXPOSED_PORT}:443
    depends_on:
      - openwebui
    restart: always

I believe I basically copied one of these from the OpenWebUI documentation page somewhere.
nginx.conf

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  120;

    #gzip  on;

    # needed to properly handle websockets (streaming)
    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    # Redirect all HTTP traffic to HTTPS
    server {
        listen 80;
        server_name <docker_host_name>

        return 301 https://$host$request_uri;
    }

    # Handle HTTPS traffic
    server {
        listen 443 ssl;
        server_name <docker_host_name>;

        # SSL Settings (ensure paths are correct)

        ssl_certificate /etc/ssl/certs/cert.pem;
        ssl_certificate_key /etc/ssl/private/key.pem;        
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
        ssl_prefer_server_ciphers on;

        # OCSP Stapling
        #ssl_stapling on;
        #ssl_stapling_verify on;

        # Proxy settings to your local service
        location / {
            # proxy_pass should point to your running localhost version of open-webui
            proxy_pass http://openwebui:8090;

            # Add WebSocket support (Necessary for version 0.5.0 and up)
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;

            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            # (Optional) Disable proxy buffering for better streaming response from models
            proxy_buffering off;

            # (Optional) Increase max request size for large attachments and long audio messages
            client_max_body_size 20M;
            proxy_read_timeout 10m;
        }
    }

}

Logs & Screenshots

OpenWebUI log while this is happening:

openwebui-server-0.6.6-prod             | 2025-05-05 16:46:59.625 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {}                                    
n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0"                                                                                                                                                                                                               
openwebui-server-0.6.6-prod             | 2025-05-05 16:46:59.632 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {}                                    
n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0"                                                                                                                                                                                                               
openwebui-server-0.6.6-prod             | 2025-05-05 16:46:59.698 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {}                                    
n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0"                                                                                                                                                                                                               
openwebui-server-0.6.6-prod             | 2025-05-05 16:46:59.715 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/e932c15b-8fa1-4b8a-b770-03020d7800e3 HTTP/1.1" 401 - {}                                    
n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0"                                                                                                                                                                                                               
openwebui-server-0.6.6-prod             | 2025-05-05 16:47:22.059 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.72:0 - "GET /api/config HTTP/1.1" 200 - {}       

Nginx log while this is happening:

nginx-openwebui-server-0.6.6-prod       | 10.14.106.72 - - [05/May/2025:16:47:22 +0000] "GET /api/config HTTP/1.1" 200 2720 "https://<docker_host_name>:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"      
nginx-openwebui-server-0.6.6-prod       | 2025/05/05 16:47:29 [alert] 29#29: *5946 4096 worker_connections are not enough while connecting to upstream, client: 10.14.106.72, server: <docker_host_name>, request: "GET /api/config HTTP/1.1", upstream: "h
ttp://172.22.0.3:8080/api/config", host: "<docker_host_name>:8080", referrer: "https://<docker_host_name>:8080/"                                                                                                                                                                        
nginx-openwebui-server-0.6.6-prod       | 10.14.106.72 - - [05/May/2025:16:47:29 +0000] "GET /api/config HTTP/1.1" 500 579 "https://external_host:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"       
nginx-openwebui-server-0.6.6-prod       | 10.14.106.253 - - [05/May/2025:16:47:29 +0000] "GET /ws/socket.io/?EIO=4&transport=websocket HTTP/1.1" 500 579 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 
Safari/537.36 Edg/135.0.0.0"                                                                                                                                                                                                                                    
nginx-openwebui-server-0.6.6-prod       | 2025/05/05 16:47:29 [alert] 29#29: *5948 4096 worker_connections are not enough while connecting to upstream, client: 10.14.106.253, server: <docker_host_name>, request: "GET /ws/socket.io/?EIO=4&transport=web
socket HTTP/1.1", upstream: "http://172.22.0.3:8080/ws/socket.io/?EIO=4&transport=websocket", host: "<docker_host_name>:8080"  
nginx-openwebui-server-0.6.6-prod       | 2025/05/05 16:47:35 [alert] 29#29: 4096 worker_connections are not enough

Additional Information

No response

Originally created by @borysow on GitHub (May 5, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/13524 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Pip Install ### Open WebUI Version 0.6.6 ### Ollama Version (if applicable) _No response_ ### Operating System Ubuntu 22.04 Container ### Browser (if applicable) Google Chrome 121.0.6167.139 ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior I have an Open WebUI instance set up via docker compose to bring up Open WebUI on port 8080 on the internal docker network, and have an nginx reverse proxy configured to establish a secure connection to users outside the host. I believe I based the nginx config on an example in the documentation. I expect to have this configuration still work to allow end users to get a secure connection by going to https://<docker_host_name>:8080. Everything works happily on OpenWebUI 0.6.5. ### Actual Behavior After updating to 0.6.6, I can log in briefly, but after a short amount of time my nginx server logs report that xxx "worker_connections are not enough while connecting to upstream" and I get either Error 500 or 503 (Bad Gateway). (I know it's not the proper solution, but I increased the max_num_workers to 4096 as a test, and all it did was make it take longer to have the errors emerge). I don't know if this is related to the changes in Websocket mentioned in the changelog. Below are some logs of the issue as well as my docker compose file and nginx.conf. ### Steps to Reproduce With the following nginx and docker compose setup, performing any activity on the OpenWebUI server (maybe opening a few chats and wandering around the configs) very quickly causes this. docker-compose: ``` services: # nothing weird in the build, just apt installing the same stuff in the official dockerfile, creating an openwebui user # and pip installing openwebui openwebui: container_name: openwebui-server-${OPENWEBUI_VERSION}-${OPENWEBUI_ENVIRONMENT} image: openwebui-server:${OPENWEBUI_VERSION} build: context: ../openwebui dockerfile: ../docker/dockerfiles/dockerfile_openwebui args: OPENWEBUI_VERSION: ${OPENWEBUI_VERSION} volumes: - open-webui:/home/openwebui/app:z environment: HF_DATASETS_ONLINE: 1 HF_HUB_OFFLINE: 1 WEBUI_NAME: ${OPENWEBUI_NAME} WEBUI_URL: http://0.0.0.0:8080 PORT: 8080 GLOBAL_LOG_LEVEL: INFO command: - "serve" restart: always # this is just a vanilla nginx image with .conf and certs added (this is a prototype instance they are temporary self-signed certs) nginx: container_name: nginx-openwebui-server-${OPENWEBUI_VERSION}-${OPENWEBUI_ENVIRONMENT} image: nginx-certs:latest build: context: ../nginx dockerfile: ../docker/dockerfiles/dockerfile_nginx ports: - ${OPENWEBUI_EXPOSED_PORT}:443 depends_on: - openwebui restart: always ``` I believe I basically copied one of these from the OpenWebUI documentation page somewhere. nginx.conf ``` worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; #tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 120; #gzip on; # needed to properly handle websockets (streaming) map $http_upgrade $connection_upgrade { default upgrade; '' close; } # Redirect all HTTP traffic to HTTPS server { listen 80; server_name <docker_host_name> return 301 https://$host$request_uri; } # Handle HTTPS traffic server { listen 443 ssl; server_name <docker_host_name>; # SSL Settings (ensure paths are correct) ssl_certificate /etc/ssl/certs/cert.pem; ssl_certificate_key /etc/ssl/private/key.pem; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256; ssl_prefer_server_ciphers on; # OCSP Stapling #ssl_stapling on; #ssl_stapling_verify on; # Proxy settings to your local service location / { # proxy_pass should point to your running localhost version of open-webui proxy_pass http://openwebui:8090; # Add WebSocket support (Necessary for version 0.5.0 and up) proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; # (Optional) Disable proxy buffering for better streaming response from models proxy_buffering off; # (Optional) Increase max request size for large attachments and long audio messages client_max_body_size 20M; proxy_read_timeout 10m; } } } ``` ### Logs & Screenshots OpenWebUI log while this is happening: ``` openwebui-server-0.6.6-prod | 2025-05-05 16:46:59.625 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {} n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0" openwebui-server-0.6.6-prod | 2025-05-05 16:46:59.632 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {} n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0" openwebui-server-0.6.6-prod | 2025-05-05 16:46:59.698 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/d31de0e3-db90-400a-8976-ee075eb5551f HTTP/1.1" 401 - {} n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0" openwebui-server-0.6.6-prod | 2025-05-05 16:46:59.715 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.146:0 - "GET /api/v1/chats/e932c15b-8fa1-4b8a-b770-03020d7800e3 HTTP/1.1" 401 - {} n64; x64; rv:136.0) Gecko/20100101 Firefox/136.0" openwebui-server-0.6.6-prod | 2025-05-05 16:47:22.059 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 10.14.106.72:0 - "GET /api/config HTTP/1.1" 200 - {} ``` Nginx log while this is happening: ``` nginx-openwebui-server-0.6.6-prod | 10.14.106.72 - - [05/May/2025:16:47:22 +0000] "GET /api/config HTTP/1.1" 200 2720 "https://<docker_host_name>:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" nginx-openwebui-server-0.6.6-prod | 2025/05/05 16:47:29 [alert] 29#29: *5946 4096 worker_connections are not enough while connecting to upstream, client: 10.14.106.72, server: <docker_host_name>, request: "GET /api/config HTTP/1.1", upstream: "h ttp://172.22.0.3:8080/api/config", host: "<docker_host_name>:8080", referrer: "https://<docker_host_name>:8080/" nginx-openwebui-server-0.6.6-prod | 10.14.106.72 - - [05/May/2025:16:47:29 +0000] "GET /api/config HTTP/1.1" 500 579 "https://external_host:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" nginx-openwebui-server-0.6.6-prod | 10.14.106.253 - - [05/May/2025:16:47:29 +0000] "GET /ws/socket.io/?EIO=4&transport=websocket HTTP/1.1" 500 579 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0" nginx-openwebui-server-0.6.6-prod | 2025/05/05 16:47:29 [alert] 29#29: *5948 4096 worker_connections are not enough while connecting to upstream, client: 10.14.106.253, server: <docker_host_name>, request: "GET /ws/socket.io/?EIO=4&transport=web socket HTTP/1.1", upstream: "http://172.22.0.3:8080/ws/socket.io/?EIO=4&transport=websocket", host: "<docker_host_name>:8080" nginx-openwebui-server-0.6.6-prod | 2025/05/05 16:47:35 [alert] 29#29: 4096 worker_connections are not enough ``` ### Additional Information _No response_
GiteaMirror added the bug label 2026-04-19 22:45:01 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#16939