File "/app/backend/apps/litellm/main.py", line 339, in proxy
r = requests.request(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a4ef10>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO: 192.168.2.2:58654 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
INFO: 192.168.2.2:58654 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "POST /ollama/v1/chat/completions HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "POST /api/v1/chats/15f4c736-fe20-4da0-bca3-ca9128cbeeae HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:60064 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO: 192.168.2.2:60064 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
self.endheaders()
File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
self.send(msg)
File "/usr/local/lib/python3.11/http/client.py", line 996, in send
self.connect()
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/backend/apps/litellm/main.py", line 339, in proxy
r = requests.request(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO: 192.168.2.2:60064 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
Originally created by @imkebe on GitHub (May 5, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1988
# Bug Report
## Description
**Bug Summary:**
Despite setting the ENV variables Open WebUI is using localhost as a LiteLLM hostname
**Steps to Reproduce:**
N/A
**Expected Behavior:**
Use the LITELLM_PROXY_HOST variable value
**Actual Behavior:**
Log file for details - localhost:4000 is used
## Environment
- **Open WebUI Version:**
v0.1.123
Both services are independent containers:
Open WebUI - 172.18.0.3
LiteLLM - 172.18.0.2
The Open WebUI has following enviroment variables set:
```
LITELLM_PROXY_HOST=172.18.0.2
LITELLM_PROXY_PORT=4000
```
## Logs and Screenshots
```Traceback (most recent call last):
File "/app/backend/apps/litellm/main.py", line 339, in proxy
r = requests.request(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a4ef10>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO: 192.168.2.2:58654 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
INFO: 192.168.2.2:58654 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "POST /ollama/v1/chat/completions HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "POST /api/v1/chats/15f4c736-fe20-4da0-bca3-ca9128cbeeae HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:58642 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO: 192.168.2.2:60064 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO: 192.168.2.2:60064 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
self.endheaders()
File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
self.send(msg)
File "/usr/local/lib/python3.11/http/client.py", line 996, in send
self.connect()
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/backend/apps/litellm/main.py", line 339, in proxy
r = requests.request(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4000): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f43a3bdd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO: 192.168.2.2:60064 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
```
That variable is for setting the listen address of our internal LiteLLM container, not specifying the endpoint of an external one.
<!-- gh-comment-id:2094987998 -->
@justinh-rahb commented on GitHub (May 5, 2024):
That variable is for setting the listen address of our internal LiteLLM container, not specifying the endpoint of an external one.
That changes a lot :) In general there should be some documentation about Open WebUI usage of builtin Lite LLM. I wasn't aware of that. Thought that it relies on external one.
I have managed to set proper LLLM proxy solution for using external one however there should be an option to delegate it to external LLLM one since Lite LLM is kind or routing solution used in different tools.
<!-- gh-comment-id:2095778825 -->
@imkebe commented on GitHub (May 6, 2024):
That changes a lot :) In general there should be some documentation about Open WebUI usage of builtin Lite LLM. I wasn't aware of that. Thought that it relies on external one.
I have managed to set proper LLLM proxy solution for using external one however there should be an option to delegate it to external LLLM one since Lite LLM is kind or routing solution used in different tools.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @imkebe on GitHub (May 5, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1988
Bug Report
Description
Bug Summary:
Despite setting the ENV variables Open WebUI is using localhost as a LiteLLM hostname
Steps to Reproduce:
N/A
Expected Behavior:
Use the LITELLM_PROXY_HOST variable value
Actual Behavior:
Log file for details - localhost:4000 is used
Environment
v0.1.123
Both services are independent containers:
Open WebUI - 172.18.0.3
LiteLLM - 172.18.0.2
The Open WebUI has following enviroment variables set:
Logs and Screenshots
@justinh-rahb commented on GitHub (May 5, 2024):
That variable is for setting the listen address of our internal LiteLLM container, not specifying the endpoint of an external one.
@imkebe commented on GitHub (May 6, 2024):
That changes a lot :) In general there should be some documentation about Open WebUI usage of builtin Lite LLM. I wasn't aware of that. Thought that it relies on external one.
I have managed to set proper LLLM proxy solution for using external one however there should be an option to delegate it to external LLLM one since Lite LLM is kind or routing solution used in different tools.