Originally created by @NiLon on GitHub (Mar 16, 2024).
Bug Report
Description
Bug Summary:
OPENAI_API_BASE_URLS with one api url timeouts, with two urls python crash. Model lists get loaded correctly. The webui settings show all the endpoints correctly.
Steps to Reproduce:
Just running the webui with (do assume correct values in testing) and making prompt:
For timeout:
OPENAI_API_BASE_URLS=http://apiurl/v1
OPENAI_API_KEYS=firstkey
I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
Error message for multiple urls
UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
INFO: 172.22.0.1:55850 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
| await response(scope, wrapped_receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 59, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 317, in proxy
| if r is not None:
| ^
| UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
+------------------------------------
During handling of the above exception, another exception occurred:
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
| yield
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__
| await wrap(partial(self.listen_for_disconnect, receive))
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect
| message = await receive()
| ^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive
| msg = await self.receive()
| ^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive
| await self.message_event.wait()
| File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
| await fut
| asyncio.exceptions.CancelledError: Cancelled by cancel scope 7472113b4e90
|
| During handling of the above exception, another exception occurred:
|
| Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
| await response(scope, wrapped_receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 59, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 317, in proxy
| if r is not None:
| ^
| UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
+------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
await func()
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
return await super().stream_response(send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
async for chunk in self.body_iterator:
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
await func()
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
return await super().stream_response(send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
async for chunk in self.body_iterator:
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/openai/main.py", line 59, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/openai/main.py", line 317, in proxy
if r is not None:
^
UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
Additional Information
I have verified that using OPENAI_API_BASE_URL and OPENAI_API_KEY do work correctly with used endpoints and keys.
Originally created by @NiLon on GitHub (Mar 16, 2024).
# Bug Report
## Description
**Bug Summary:**
OPENAI_API_BASE_URLS with one api url timeouts, with two urls python crash. Model lists get loaded correctly. The webui settings show all the endpoints correctly.
**Steps to Reproduce:**
Just running the webui with (do assume correct values in testing) and making prompt:
For timeout:
OPENAI_API_BASE_URLS=http://apiurl/v1
OPENAI_API_KEYS=firstkey
For crash:
OPENAI_API_BASE_URLS=http://apiurl/v1;http://apiurl2/v1
OPENAI_API_KEYS=firstkey;secondkey
## Environment
- Docker image of v0.1.112 & v0.1.111
- Host Linux
## Reproduction Details
**Confirmation:**
- [X] I have read and followed all the instructions provided in the README.md.
- [X] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [ ] I have included the Docker container logs.
## Error message for multiple urls
<details>
<summary>UnboundLocalError: cannot access local variable 'r' where it is not associated with a value</summary>
```text
INFO: 172.22.0.1:55850 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
| await response(scope, wrapped_receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 59, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 317, in proxy
| if r is not None:
| ^
| UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
+------------------------------------
During handling of the above exception, another exception occurred:
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
| yield
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__
| await wrap(partial(self.listen_for_disconnect, receive))
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect
| message = await receive()
| ^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive
| msg = await self.receive()
| ^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive
| await self.message_event.wait()
| File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
| await fut
| asyncio.exceptions.CancelledError: Cancelled by cancel scope 7472113b4e90
|
| During handling of the above exception, another exception occurred:
|
| Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
| await response(scope, wrapped_receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
| await func()
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
| return await super().stream_response(send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
| async for chunk in self.body_iterator:
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 59, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/openai/main.py", line 317, in proxy
| if r is not None:
| ^
| UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
+------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
await func()
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
return await super().stream_response(send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
async for chunk in self.body_iterator:
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
await func()
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
return await super().stream_response(send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
async for chunk in self.body_iterator:
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/openai/main.py", line 59, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/openai/main.py", line 317, in proxy
if r is not None:
^
UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
```
</details>
## Additional Information
I have verified that using OPENAI_API_BASE_URL and OPENAI_API_KEY do work correctly with used endpoints and keys.
Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
@tjbck commented on GitHub (Mar 16, 2024):
Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
Now that you said that I made additional test. In my case I had list like http://apiurl/v1;http://apiurl2/v1 the first one was offline but second one wasn't. And when I tried to use the model on second one it borked. But as I did bring up the first one, then the second started working. So I would suspect that if one of the urls in the list is down, then none of them work or something in that line.
@NiLon commented on GitHub (Mar 16, 2024):
> Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
Now that you said that I made additional test. In my case I had list like http://apiurl/v1;http://apiurl2/v1 the first one was offline but second one wasn't. And when I tried to use the model on second one it borked. But as I did bring up the first one, then the second started working. So I would suspect that if one of the urls in the list is down, then none of them work or something in that line.
@NiLon Thanks for the pointer! Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!
@tjbck commented on GitHub (Mar 18, 2024):
@NiLon Thanks for the pointer! Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @NiLon on GitHub (Mar 16, 2024).
Bug Report
Description
Bug Summary:
OPENAI_API_BASE_URLS with one api url timeouts, with two urls python crash. Model lists get loaded correctly. The webui settings show all the endpoints correctly.
Steps to Reproduce:
Just running the webui with (do assume correct values in testing) and making prompt:
For timeout:
OPENAI_API_BASE_URLS=http://apiurl/v1
OPENAI_API_KEYS=firstkey
For crash:
OPENAI_API_BASE_URLS=http://apiurl/v1;http://apiurl2/v1
OPENAI_API_KEYS=firstkey;secondkey
Environment
Reproduction Details
Confirmation:
Error message for multiple urls
UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
Additional Information
I have verified that using OPENAI_API_BASE_URL and OPENAI_API_KEY do work correctly with used endpoints and keys.
@tjbck commented on GitHub (Mar 16, 2024):
Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!
@NiLon commented on GitHub (Mar 16, 2024):
Tested the dev branch, seems like there is more underlying problem. Maybe it doesn't get proper url to try?
AttributeError: 'NoneType' object has no attribute 'status_code'
@tjbck commented on GitHub (Mar 16, 2024):
Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
@NiLon commented on GitHub (Mar 16, 2024):
Now that you said that I made additional test. In my case I had list like http://apiurl/v1;http://apiurl2/v1 the first one was offline but second one wasn't. And when I tried to use the model on second one it borked. But as I did bring up the first one, then the second started working. So I would suspect that if one of the urls in the list is down, then none of them work or something in that line.
@tjbck commented on GitHub (Mar 18, 2024):
@NiLon Thanks for the pointer! Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!