[GH-ISSUE #1184] OPENAI_API_BASE_URLS borked #27911

Closed
opened 2026-04-25 02:40:17 -05:00 by GiteaMirror · 5 comments
Owner

Originally created by @NiLon on GitHub (Mar 16, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1184

Bug Report

Description

Bug Summary:
OPENAI_API_BASE_URLS with one api url timeouts, with two urls python crash. Model lists get loaded correctly. The webui settings show all the endpoints correctly.

Steps to Reproduce:
Just running the webui with (do assume correct values in testing) and making prompt:

For timeout:
OPENAI_API_BASE_URLS=http://apiurl/v1
OPENAI_API_KEYS=firstkey

For crash:
OPENAI_API_BASE_URLS=http://apiurl/v1;http://apiurl2/v1
OPENAI_API_KEYS=firstkey;secondkey

Environment

  • Docker image of v0.1.112 & v0.1.111
  • Host Linux

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.

Error message for multiple urls

UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
INFO:     172.22.0.1:55850 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
  |     await response(scope, wrapped_receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    |     return await self.app(scope, receive, send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    |     return await super().stream_response(send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    |     await self.simple_response(scope, receive, send, request_headers=headers)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    |     return await super().stream_response(send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/openai/main.py", line 59, in check_url
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    |     await self.simple_response(scope, receive, send, request_headers=headers)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
    |     response = await func(request)
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/openai/main.py", line 317, in proxy
    |     if r is not None:
    |        ^
    | UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
    +------------------------------------

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
  |     yield
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__
    |     await wrap(partial(self.listen_for_disconnect, receive))
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect
    |     message = await receive()
    |               ^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive
    |     msg = await self.receive()
    |           ^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive
    |     await self.message_event.wait()
    |   File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
    |     await fut
    | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7472113b4e90
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Exception Group Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
    |     await response(scope, wrapped_receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
    |     async with anyio.create_task_group() as task_group:
    |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
    |     raise BaseExceptionGroup(
    | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
    +-+---------------- 1 ----------------
      | Traceback (most recent call last):
      |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
      |     result = await app(  # type: ignore[func-returns-value]
      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
      |     return await self.app(scope, receive, send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
      |     await func()
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
      |     return await super().stream_response(send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
      |     async for chunk in self.body_iterator:
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
      |     await self.simple_response(scope, receive, send, request_headers=headers)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
      |     await func()
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
      |     return await super().stream_response(send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
      |     async for chunk in self.body_iterator:
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
      |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
      |     await route.handle(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
      |     response = await self.dispatch_func(request, call_next)
      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/app/backend/apps/openai/main.py", line 59, in check_url
      |     response = await call_next(request)
      |                ^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
      |     await self.simple_response(scope, receive, send, request_headers=headers)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
      |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
      |     await route.handle(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
      |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
      |     response = await func(request)
      |                ^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
      |     raw_response = await run_endpoint_function(
      |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
      |     return await dependant.call(**values)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/app/backend/apps/openai/main.py", line 317, in proxy
      |     if r is not None:
      |        ^
      | UnboundLocalError: cannot access local variable 'r' where it is not associated with a value
      +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    await func()
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    async for chunk in self.body_iterator:
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    await func()
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    async for chunk in self.body_iterator:
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/openai/main.py", line 59, in check_url
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/openai/main.py", line 317, in proxy
    if r is not None:
       ^
UnboundLocalError: cannot access local variable 'r' where it is not associated with a value

Additional Information

I have verified that using OPENAI_API_BASE_URL and OPENAI_API_KEY do work correctly with used endpoints and keys.

Originally created by @NiLon on GitHub (Mar 16, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/1184 # Bug Report ## Description **Bug Summary:** OPENAI_API_BASE_URLS with one api url timeouts, with two urls python crash. Model lists get loaded correctly. The webui settings show all the endpoints correctly. **Steps to Reproduce:** Just running the webui with (do assume correct values in testing) and making prompt: For timeout: OPENAI_API_BASE_URLS=http://apiurl/v1 OPENAI_API_KEYS=firstkey For crash: OPENAI_API_BASE_URLS=http://apiurl/v1;http://apiurl2/v1 OPENAI_API_KEYS=firstkey;secondkey ## Environment - Docker image of v0.1.112 & v0.1.111 - Host Linux ## Reproduction Details **Confirmation:** - [X] I have read and followed all the instructions provided in the README.md. - [X] I am on the latest version of both Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. ## Error message for multiple urls <details> <summary>UnboundLocalError: cannot access local variable 'r' where it is not associated with a value</summary> ```text INFO: 172.22.0.1:55850 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application + Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__ | await response(scope, wrapped_receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi | result = await app( # type: ignore[func-returns-value] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ | return await self.app(scope, receive, send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ | response = await self.dispatch_func(request, call_next) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 59, in check_url | response = await call_next(request) | ^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app | await wrap_app_handling_exceptions(app, request)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app | response = await func(request) | ^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app | raw_response = await run_endpoint_function( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function | return await dependant.call(**values) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 317, in proxy | if r is not None: | ^ | UnboundLocalError: cannot access local variable 'r' where it is not associated with a value +------------------------------------ During handling of the above exception, another exception occurred: + Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups | yield | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__ | await wrap(partial(self.listen_for_disconnect, receive)) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect | message = await receive() | ^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive | msg = await self.receive() | ^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive | await self.message_event.wait() | File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait | await fut | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7472113b4e90 | | During handling of the above exception, another exception occurred: | | Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__ | await response(scope, wrapped_receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi | result = await app( # type: ignore[func-returns-value] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ | return await self.app(scope, receive, send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ | response = await self.dispatch_func(request, call_next) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 59, in check_url | response = await call_next(request) | ^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app | await wrap_app_handling_exceptions(app, request)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app | response = await func(request) | ^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app | raw_response = await run_endpoint_function( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function | return await dependant.call(**values) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 317, in proxy | if r is not None: | ^ | UnboundLocalError: cannot access local variable 'r' where it is not associated with a value +------------------------------------ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap await func() File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response return await super().stream_response(send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap await func() File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response return await super().stream_response(send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ response = await self.dispatch_func(request, call_next) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/openai/main.py", line 59, in check_url response = await call_next(request) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/openai/main.py", line 317, in proxy if r is not None: ^ UnboundLocalError: cannot access local variable 'r' where it is not associated with a value ``` </details> ## Additional Information I have verified that using OPENAI_API_BASE_URL and OPENAI_API_KEY do work correctly with used endpoints and keys.
Author
Owner

@tjbck commented on GitHub (Mar 16, 2024):

Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!

<!-- gh-comment-id:2001931016 --> @tjbck commented on GitHub (Mar 16, 2024): Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!
Author
Owner

@NiLon commented on GitHub (Mar 16, 2024):

Tested the dev branch, seems like there is more underlying problem. Maybe it doesn't get proper url to try?

AttributeError: 'NoneType' object has no attribute 'status_code'
INFO:     172.22.0.1:47562 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
  |     await response(scope, wrapped_receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    |     return await self.app(scope, receive, send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    |     return await super().stream_response(send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    |     await self.simple_response(scope, receive, send, request_headers=headers)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    |     return await super().stream_response(send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/openai/main.py", line 59, in check_url
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    |     await self.simple_response(scope, receive, send, request_headers=headers)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
    |     response = await func(request)
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/openai/main.py", line 331, in proxy
    |     raise HTTPException(status_code=r.status_code, detail=error_detail)
    |                                     ^^^^^^^^^^^^^
    | AttributeError: 'NoneType' object has no attribute 'status_code'
    +------------------------------------

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
  |     yield
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__
    |     await wrap(partial(self.listen_for_disconnect, receive))
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect
    |     message = await receive()
    |               ^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive
    |     msg = await self.receive()
    |           ^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive
    |     await self.message_event.wait()
    |   File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
    |     await fut
    | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7870c5b96150
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Exception Group Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__
    |     await response(scope, wrapped_receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
    |     async with anyio.create_task_group() as task_group:
    |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
    |     raise BaseExceptionGroup(
    | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
    +-+---------------- 1 ----------------
      | Traceback (most recent call last):
      |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
      |     result = await app(  # type: ignore[func-returns-value]
      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
      |     return await self.app(scope, receive, send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
      |     await func()
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
      |     return await super().stream_response(send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
      |     async for chunk in self.body_iterator:
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
      |     await self.simple_response(scope, receive, send, request_headers=headers)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
      |     await func()
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
      |     return await super().stream_response(send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
      |     async for chunk in self.body_iterator:
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
      |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
      |     await route.handle(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
      |     with collapse_excgroups():
      |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
      |     self.gen.throw(typ, value, traceback)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
      |     response = await self.dispatch_func(request, call_next)
      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/app/backend/apps/openai/main.py", line 59, in check_url
      |     response = await call_next(request)
      |                ^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
      |     raise app_exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
      |     await self.app(scope, receive_or_disconnect, send_no_error)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
      |     await self.simple_response(scope, receive, send, request_headers=headers)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
      |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
      |     await route.handle(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
      |     await self.app(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
      |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
      |     response = await func(request)
      |                ^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
      |     raw_response = await run_endpoint_function(
      |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
      |     return await dependant.call(**values)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/app/backend/apps/openai/main.py", line 331, in proxy
      |     raise HTTPException(status_code=r.status_code, detail=error_detail)
      |                                     ^^^^^^^^^^^^^
      | AttributeError: 'NoneType' object has no attribute 'status_code'
      +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    await func()
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    async for chunk in self.body_iterator:
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap
    await func()
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response
    async for chunk in self.body_iterator:
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/openai/main.py", line 59, in check_url
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/openai/main.py", line 331, in proxy
    raise HTTPException(status_code=r.status_code, detail=error_detail)
                                    ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'status_code'
<!-- gh-comment-id:2001940536 --> @NiLon commented on GitHub (Mar 16, 2024): Tested the dev branch, seems like there is more underlying problem. Maybe it doesn't get proper url to try? <details> <summary>AttributeError: 'NoneType' object has no attribute 'status_code'</summary> ``` INFO: 172.22.0.1:47562 - "POST /openai/api/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application + Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__ | await response(scope, wrapped_receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi | result = await app( # type: ignore[func-returns-value] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ | return await self.app(scope, receive, send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ | response = await self.dispatch_func(request, call_next) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 59, in check_url | response = await call_next(request) | ^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app | await wrap_app_handling_exceptions(app, request)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app | response = await func(request) | ^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app | raw_response = await run_endpoint_function( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function | return await dependant.call(**values) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 331, in proxy | raise HTTPException(status_code=r.status_code, detail=error_detail) | ^^^^^^^^^^^^^ | AttributeError: 'NoneType' object has no attribute 'status_code' +------------------------------------ During handling of the above exception, another exception occurred: + Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups | yield | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 264, in __call__ | await wrap(partial(self.listen_for_disconnect, receive)) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 237, in listen_for_disconnect | message = await receive() | ^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive | msg = await self.receive() | ^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 571, in receive | await self.message_event.wait() | File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait | await fut | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7870c5b96150 | | During handling of the above exception, another exception occurred: | | Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 192, in __call__ | await response(scope, wrapped_receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__ | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__ | raise BaseExceptionGroup( | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi | result = await app( # type: ignore[func-returns-value] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ | return await self.app(scope, receive, send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap | await func() | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response | return await super().stream_response(send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ | await super().__call__(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ | with collapse_excgroups(): | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ | self.gen.throw(typ, value, traceback) | File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ | response = await self.dispatch_func(request, call_next) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 59, in check_url | response = await call_next(request) | ^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next | raise app_exc | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro | await self.app(scope, receive_or_disconnect, send_no_error) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ | await self.simple_response(scope, receive, send, request_headers=headers) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app | await wrap_app_handling_exceptions(app, request)(scope, receive, send) | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app | raise exc | File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app | response = await func(request) | ^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app | raw_response = await run_endpoint_function( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function | return await dependant.call(**values) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/app/backend/apps/openai/main.py", line 331, in proxy | raise HTTPException(status_code=r.status_code, detail=error_detail) | ^^^^^^^^^^^^^ | AttributeError: 'NoneType' object has no attribute 'status_code' +------------------------------------ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap await func() File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response return await super().stream_response(send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 260, in wrap await func() File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response return await super().stream_response(send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 249, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 487, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__ with collapse_excgroups(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ response = await self.dispatch_func(request, call_next) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/openai/main.py", line 59, in check_url response = await call_next(request) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/openai/main.py", line 331, in proxy raise HTTPException(status_code=r.status_code, detail=error_detail) ^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'status_code' ``` </details>
Author
Owner

@tjbck commented on GitHub (Mar 16, 2024):

Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!

<!-- gh-comment-id:2001941643 --> @tjbck commented on GitHub (Mar 16, 2024): Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!
Author
Owner

@NiLon commented on GitHub (Mar 16, 2024):

Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case!

Now that you said that I made additional test. In my case I had list like http://apiurl/v1;http://apiurl2/v1 the first one was offline but second one wasn't. And when I tried to use the model on second one it borked. But as I did bring up the first one, then the second started working. So I would suspect that if one of the urls in the list is down, then none of them work or something in that line.

<!-- gh-comment-id:2001943448 --> @NiLon commented on GitHub (Mar 16, 2024): > Just added additional fix to address the attribute error, but based on the error log you provided, it seems like the backend also can't connect to the url at all? I'd suggest you check if you've provided the proper urls just in case! Now that you said that I made additional test. In my case I had list like http://apiurl/v1;http://apiurl2/v1 the first one was offline but second one wasn't. And when I tried to use the model on second one it borked. But as I did bring up the first one, then the second started working. So I would suspect that if one of the urls in the list is down, then none of them work or something in that line.
Author
Owner

@tjbck commented on GitHub (Mar 18, 2024):

@NiLon Thanks for the pointer! Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!

<!-- gh-comment-id:2003162472 --> @tjbck commented on GitHub (Mar 18, 2024): @NiLon Thanks for the pointer! Should be fixed on our dev branch now, feel free to try it out and let me know if the issue persists!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#27911