Originally created by @thiswillbeyourgithub on GitHub (Aug 21, 2024).
Bug Report
Installation Method
docker
Environment
Open WebUI Version: 0.3.14 (not present before)
Operating System: ubuntu 22
Confirmation:
[ X ] I have read and followed all the instructions provided in the README.md.
[ X ] I am on the latest version of both Open WebUI and Ollama.
[ X ] I have included the browser console logs.
[ X ] I have included the Docker container logs.
[ X ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
My pipe should work :)
Actual Behavior:
The message errors out.
Description
Bug Summary:
[Provide a brief but clear summary of the bug]
Reproduction Details
Steps to Reproduce:
I'm using openrouter via litellm
The model I'm using is openrouter/anthropic/claude-3.5-sonnet
The pipe I'm using is:
My pipe
"""
title: CostTrackingPipe
author: thiswillbeyourgithub
date: 2024-08-21
version: 3.0.0
license: MIT
description: A pipe function to track user costs and remove 'thinkin' blocks
"""fromtypingimportList,Union,Generator,Iterator,Callable,Any,OptionalfrompydanticimportBaseModel,FieldimportrequestsimportosimportreimportjsonDEFAULT_BASE_URL="http://127.0.0.1:4000"DEFAULT_CHAT_MODEL="litellm_sonnet-3.5_beta"DEFAULT_TITLE_CHAT_MODEL="litellm_gpt-4o-mini"classPipe:classValves(BaseModel):LITELLM_BASE_URL:str=DEFAULT_BASE_URLapi_keys:Optional[str]=Field(default=None,description="Dict where keys are litellm users and values are their virtual api keys (a string that will be json loaded as a dict). Leave to None if you want to load from env 'COSTTRACKINGPIPE_API_KEYS'",)classUserValves(BaseModel):enabled:bool=Field(default=True,description="True to enable price counting")chat_model:str=Field(default=DEFAULT_CHAT_MODEL,description="Chat model to use")title_chat_model:str=Field(default=DEFAULT_TITLE_CHAT_MODEL,description="Model to use to generate titles",)remove_thoughts:bool=Field(default=True,description="True to remove the thoughts block")start_thoughts:str=Field(default="^``` ?thinking",description="Start of thought block")stop_thoughts:str=Field(default="```",description="End of thought block")debug:bool=Field(default=False,description="Set to True to print more info to the docker logs, also to not remove the last emitter message.",)def__init__(self):# You can also set the pipelines that are available in this pipeline.# Set manifold to True if you want to use this pipeline as a manifold.# Manifold pipelines can have multiple pipelines.self.type="manifold"# Optionally, you can set the id and name of the pipeline.# Best practice is to not specify the id so that it can be automatically inferred from the filename, so that users can install multiple versions of the same pipeline.# The identifier must be unique across all pipelines.# The identifier must be an alphanumeric string that can include underscores or hyphens. It cannot contain spaces, special characters, slashes, or backslashes.self.id="cost_tracking_pipe"# Optionally, you can set the name of the manifold pipeline.self.name="CostTrackingPipe"# Initialize rate limitsself.valves=self.Valves()self.uvalves=self.UserValves()self.start_thought=re.compile(self.uvalves.start_thoughts)self.stop_thought=re.compile(self.uvalves.stop_thoughts)self.pattern=re.compile(self.uvalves.start_thoughts+"(.*)?"+self.uvalves.stop_thoughts,flags=re.DOTALL|re.MULTILINE,)asyncdefon_valves_updated(self):"""This function is called when the valves are updated."""# just checking the validity of the api_keysifself.valves.api_keysisNone:assert("COSTTRACKINGPIPE_API_KEYS"inos.environ),"You left the valve api_keys to None but didn't set an env variable COSTTRACKINGPIPE_API_KEYS"api_keys=os.environ["COSTTRACKINGPIPE_API_KEYS"]else:api_keys=self.valves.api_keysassertisinstance(api_keys,str),f"Expected api_keys to be a str at this point, not {type(api_keys)}"try:api_keys=json.loads(api_keys)assertisinstance(api_keys,dict),f"Expected api_keys to be a dict at this point, not {type(api_keys)}"exceptExceptionaserr:raiseException(f"Error when casting api_keys from str to dict: '{err}'")assert"default"inapi_keys,f"No 'default' key found in dict: {api_keys}"asyncdefpipe(self,body:dict,__user__:dict,__event_emitter__:Callable[[dict],Any]=None,# this is just to debug if there are breaking changes etc*args,**kwargs,)->Union[str,Generator,Iterator]:# load the api_keys as a dictifself.valves.api_keysisNone:assert("COSTTRACKINGPIPE_API_KEYS"inos.environ),"You left the valve api_keys to None but didn't set an env variable COSTTRACKINGPIPE_API_KEYS"api_keys=os.environ["COSTTRACKINGPIPE_API_KEYS"]else:api_keys=self.valves.api_keysassertisinstance(api_keys,str),f"Expected api_keys to be a str at this point, not {type(api_keys)}"try:api_keys=json.loads(api_keys)assertisinstance(api_keys,dict),f"Expected api_keys to be a dict at this point, not {type(api_keys)}"exceptExceptionaserr:raiseException(f"Error when casting api_keys from str to dict: '{err}'")assert"default"inapi_keys,f"No 'default' key found in dict: {api_keys}"# prints and emitter to show progressdefpprint(message:str)->str:print(f"CostTrackingPipe of '{__user__['name']}': "+str(message))returnmessageemitter=EventEmitter(__event_emitter__)asyncdefprog(message:str)->None:awaitemitter.progress_update(pprint(message))asyncdefsucc(message:str)->None:awaitemitter.success_update(pprint(message))asyncdeferr(message:str)->None:awaitemitter.error_update(pprint(message))# to know in the future if there are new arguments I could useifargsorkwargs:ifargs:pprint("Received args:"+str(args))ifkwargs:pprint("Received kwargs:"+str(kwargs))ifself.uvalves.debug:pprint(body.keys())pprint(body)# match the api keyheaders={}awaitprog("Start")ifnotself.uvalves.enabled:awaitprog("Disabled api key matching, will use the default key")apikey=api_keys["default"]headers["Authorization"]=f"Bearer {apikey}"else:username=__user__["name"]ifusernameinapi_keys:apikey=api_keys[username]awaitprog(f"Will use key for {username}")headers["Authorization"]=f"Bearer {apikey}"else:awaiterr(f"Username {username} not found in litellm env keys")raiseException(f"User not found: {username}")try:ifbody["stream"]:model=self.uvalves.chat_modelelse:# stream disabled is only used for the summary title creator AFAIKmodel=self.uvalves.title_chat_modelpayload={**body,"model":model,"user":username}awaitprog("Waiting for response")r=requests.post(url=f"{self.valves.LITELLM_BASE_URL}/v1/chat/completions",json=payload,headers=headers,stream=True,)r.raise_for_status()assertr.status_code==200,f"Invalid status code: {r.status_code}"ifbody["stream"]:awaitprog("Receiving response by chunk")if(notself.uvalves.remove_thoughts)or(notself.uvalves.enabled):forlineinr.iter_lines():yieldlinereturnbuffer=""thought_pattern=re.compile(r"``` ?thinking.*?```",re.DOTALL)thought_removed=Falseforlineinr.iter_lines():ifline:line=line.decode("utf-8")ifline.startswith("data: "):line=line[6:]# Remove "data: " prefixifline.strip()=="[DONE]":breaktry:parsed_line=json.loads(line)except(json.JSONDecodeError,KeyError):continuecontent=parsed_line["choices"][0]["delta"].get("content","")ifnotcontent:continueifthought_removed:yieldcontentcontinuebuffer+=contentmatch=thought_pattern.search(buffer)ifmatch:# Remove the thought blockbuffer=buffer[:match.start()]+buffer[match.end():]yieldbufferbuffer=""thought_removed=Trueawaitsucc("Removed thought block")ifnotthought_removed:# model didn't produce a thought (for example can happen for the chat title)awaitsucc("Thought block never found")yieldbufferifbuffer:# Yield any remaining content with finish_reason "stop"yieldbufferelse:# return the whole text directlyawaitprog("Returning directly")j=r.json()to_yield=j["choices"][0]["message"].get("content","")yieldto_yieldifnotself.uvalves.debug:awaitsucc("")# hides itreturnexceptExceptionase:awaiterr(f"Error: {e}")raiseclassEventEmitter:def__init__(self,event_emitter:Callable[[dict],Any]=None):self.event_emitter=event_emitterasyncdefprogress_update(self,description):awaitself.emit(description)asyncdeferror_update(self,description):awaitself.emit(description,"error",True)asyncdefsuccess_update(self,description):awaitself.emit(description,"success",True)asyncdefemit(self,description="Unknown State",status="in_progress",done=False):ifself.event_emitter:awaitself.event_emitter({"type":"status","data":{"status":status,"description":description,"done":done,},})
Ask any kind of message to my model
Logs and Screenshots
Docker Container Logs:
Docker logs
INFO: 192.168.1.254:0 - "POST /api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
| yield
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 792, in update_embedding_function
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 783, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 769, in commit_session_after_request
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 748, in dispatch
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 613, in dispatch
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 993, in generate_chat_completions
| return await generate_function_chat_completion(form_data, user=user)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
| configured_tools = get_tools(app, tool_ids, user, tools_params)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/utils/tools.py", line 37, in get_tools
| for tool_id in tool_ids:
| TypeError: 'NoneType' object is not iterable
+------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 792, in update_embedding_function
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 783, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 769, in commit_session_after_request
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 748, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 613, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 993, in generate_chat_completions
return await generate_function_chat_completion(form_data, user=user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
configured_tools = get_tools(app, tool_ids, user, tools_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/utils/tools.py", line 37, in get_tools
for tool_id in tool_ids:
TypeError: 'NoneType' object is not iterable
Along with the error:
Uh-oh! There was an issue connecting to ClaudeSonnet Beta.
Unexpected token 'I', "Internal S"... is not valid JSON
Screenshots/Screen Recordings (if applicable): ##
Additional Information
Started happening as soon as I upgraded to 0.3.14
Originally created by @thiswillbeyourgithub on GitHub (Aug 21, 2024).
# Bug Report
## Installation Method
docker
## Environment
- **Open WebUI Version:** 0.3.14 (not present before)
- **Operating System:** ubuntu 22
**Confirmation:**
- [ X ] I have read and followed all the instructions provided in the README.md.
- [ X ] I am on the latest version of both Open WebUI and Ollama.
- [ X ] I have included the browser console logs.
- [ X ] I have included the Docker container logs.
- [ X ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
My pipe should work :)
## Actual Behavior:
The message errors out.
## Description
**Bug Summary:**
[Provide a brief but clear summary of the bug]
## Reproduction Details
**Steps to Reproduce:**
- I'm using openrouter via litellm
- The model I'm using is `openrouter/anthropic/claude-3.5-sonnet`
- The pipe I'm using is:
<details><summary>My pipe</summary>
``` python
"""
title: CostTrackingPipe
author: thiswillbeyourgithub
date: 2024-08-21
version: 3.0.0
license: MIT
description: A pipe function to track user costs and remove 'thinkin' blocks
"""
from typing import List, Union, Generator, Iterator, Callable, Any, Optional
from pydantic import BaseModel, Field
import requests
import os
import re
import json
DEFAULT_BASE_URL = "http://127.0.0.1:4000"
DEFAULT_CHAT_MODEL = "litellm_sonnet-3.5_beta"
DEFAULT_TITLE_CHAT_MODEL = "litellm_gpt-4o-mini"
class Pipe:
class Valves(BaseModel):
LITELLM_BASE_URL: str = DEFAULT_BASE_URL
api_keys: Optional[str] = Field(
default=None,
description="Dict where keys are litellm users and values are their virtual api keys (a string that will be json loaded as a dict). Leave to None if you want to load from env 'COSTTRACKINGPIPE_API_KEYS'",
)
class UserValves(BaseModel):
enabled: bool = Field(default=True, description="True to enable price counting")
chat_model: str = Field(
default=DEFAULT_CHAT_MODEL, description="Chat model to use"
)
title_chat_model: str = Field(
default=DEFAULT_TITLE_CHAT_MODEL,
description="Model to use to generate titles",
)
remove_thoughts: bool = Field(
default=True, description="True to remove the thoughts block"
)
start_thoughts: str = Field(
default="^``` ?thinking", description="Start of thought block"
)
stop_thoughts: str = Field(default="```", description="End of thought block")
debug: bool = Field(
default=False,
description="Set to True to print more info to the docker logs, also to not remove the last emitter message.",
)
def __init__(self):
# You can also set the pipelines that are available in this pipeline.
# Set manifold to True if you want to use this pipeline as a manifold.
# Manifold pipelines can have multiple pipelines.
self.type = "manifold"
# Optionally, you can set the id and name of the pipeline.
# Best practice is to not specify the id so that it can be automatically inferred from the filename, so that users can install multiple versions of the same pipeline.
# The identifier must be unique across all pipelines.
# The identifier must be an alphanumeric string that can include underscores or hyphens. It cannot contain spaces, special characters, slashes, or backslashes.
self.id = "cost_tracking_pipe"
# Optionally, you can set the name of the manifold pipeline.
self.name = "CostTrackingPipe"
# Initialize rate limits
self.valves = self.Valves()
self.uvalves = self.UserValves()
self.start_thought = re.compile(self.uvalves.start_thoughts)
self.stop_thought = re.compile(self.uvalves.stop_thoughts)
self.pattern = re.compile(
self.uvalves.start_thoughts + "(.*)?" + self.uvalves.stop_thoughts,
flags=re.DOTALL | re.MULTILINE,
)
async def on_valves_updated(self):
"""This function is called when the valves are updated."""
# just checking the validity of the api_keys
if self.valves.api_keys is None:
assert (
"COSTTRACKINGPIPE_API_KEYS" in os.environ
), "You left the valve api_keys to None but didn't set an env variable COSTTRACKINGPIPE_API_KEYS"
api_keys = os.environ["COSTTRACKINGPIPE_API_KEYS"]
else:
api_keys = self.valves.api_keys
assert isinstance(
api_keys, str
), f"Expected api_keys to be a str at this point, not {type(api_keys)}"
try:
api_keys = json.loads(api_keys)
assert isinstance(
api_keys, dict
), f"Expected api_keys to be a dict at this point, not {type(api_keys)}"
except Exception as err:
raise Exception(f"Error when casting api_keys from str to dict: '{err}'")
assert "default" in api_keys, f"No 'default' key found in dict: {api_keys}"
async def pipe(
self,
body: dict,
__user__: dict,
__event_emitter__: Callable[[dict], Any] = None,
# this is just to debug if there are breaking changes etc
*args,
**kwargs,
) -> Union[str, Generator, Iterator]:
# load the api_keys as a dict
if self.valves.api_keys is None:
assert (
"COSTTRACKINGPIPE_API_KEYS" in os.environ
), "You left the valve api_keys to None but didn't set an env variable COSTTRACKINGPIPE_API_KEYS"
api_keys = os.environ["COSTTRACKINGPIPE_API_KEYS"]
else:
api_keys = self.valves.api_keys
assert isinstance(
api_keys, str
), f"Expected api_keys to be a str at this point, not {type(api_keys)}"
try:
api_keys = json.loads(api_keys)
assert isinstance(
api_keys, dict
), f"Expected api_keys to be a dict at this point, not {type(api_keys)}"
except Exception as err:
raise Exception(f"Error when casting api_keys from str to dict: '{err}'")
assert "default" in api_keys, f"No 'default' key found in dict: {api_keys}"
# prints and emitter to show progress
def pprint(message: str) -> str:
print(f"CostTrackingPipe of '{__user__['name']}': " + str(message))
return message
emitter = EventEmitter(__event_emitter__)
async def prog(message: str) -> None:
await emitter.progress_update(pprint(message))
async def succ(message: str) -> None:
await emitter.success_update(pprint(message))
async def err(message: str) -> None:
await emitter.error_update(pprint(message))
# to know in the future if there are new arguments I could use
if args or kwargs:
if args:
pprint("Received args:" + str(args))
if kwargs:
pprint("Received kwargs:" + str(kwargs))
if self.uvalves.debug:
pprint(body.keys())
pprint(body)
# match the api key
headers = {}
await prog("Start")
if not self.uvalves.enabled:
await prog("Disabled api key matching, will use the default key")
apikey = api_keys["default"]
headers["Authorization"] = f"Bearer {apikey}"
else:
username = __user__["name"]
if username in api_keys:
apikey = api_keys[username]
await prog(f"Will use key for {username}")
headers["Authorization"] = f"Bearer {apikey}"
else:
await err(f"Username {username} not found in litellm env keys")
raise Exception(f"User not found: {username}")
try:
if body["stream"]:
model = self.uvalves.chat_model
else:
# stream disabled is only used for the summary title creator AFAIK
model = self.uvalves.title_chat_model
payload = {**body, "model": model, "user": username}
await prog("Waiting for response")
r = requests.post(
url=f"{self.valves.LITELLM_BASE_URL}/v1/chat/completions",
json=payload,
headers=headers,
stream=True,
)
r.raise_for_status()
assert r.status_code == 200, f"Invalid status code: {r.status_code}"
if body["stream"]:
await prog("Receiving response by chunk")
if (not self.uvalves.remove_thoughts) or (not self.uvalves.enabled):
for line in r.iter_lines():
yield line
return
buffer = ""
thought_pattern = re.compile(r"``` ?thinking.*?```", re.DOTALL)
thought_removed = False
for line in r.iter_lines():
if line:
line = line.decode("utf-8")
if line.startswith("data: "):
line = line[6:] # Remove "data: " prefix
if line.strip() == "[DONE]":
break
try:
parsed_line = json.loads(line)
except (json.JSONDecodeError, KeyError):
continue
content = parsed_line["choices"][0]["delta"].get("content", "")
if not content:
continue
if thought_removed:
yield content
continue
buffer += content
match = thought_pattern.search(buffer)
if match:
# Remove the thought block
buffer = buffer[: match.start()] + buffer[match.end() :]
yield buffer
buffer = ""
thought_removed = True
await succ("Removed thought block")
if not thought_removed:
# model didn't produce a thought (for example can happen for the chat title)
await succ("Thought block never found")
yield buffer
if buffer: # Yield any remaining content with finish_reason "stop"
yield buffer
else: # return the whole text directly
await prog("Returning directly")
j = r.json()
to_yield = j["choices"][0]["message"].get("content", "")
yield to_yield
if not self.uvalves.debug:
await succ("") # hides it
return
except Exception as e:
await err(f"Error: {e}")
raise
class EventEmitter:
def __init__(self, event_emitter: Callable[[dict], Any] = None):
self.event_emitter = event_emitter
async def progress_update(self, description):
await self.emit(description)
async def error_update(self, description):
await self.emit(description, "error", True)
async def success_update(self, description):
await self.emit(description, "success", True)
async def emit(self, description="Unknown State", status="in_progress", done=False):
if self.event_emitter:
await self.event_emitter(
{
"type": "status",
"data": {
"status": status,
"description": description,
"done": done,
},
}
)
```
</details>
- Ask any kind of message to my model
## Logs and Screenshots
**Docker Container Logs:**
<details> <summary>Docker logs</summary>
```
INFO: 192.168.1.254:0 - "POST /api/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
+ Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
| yield
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in __call__
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 792, in update_embedding_function
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 783, in check_url
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 769, in commit_session_after_request
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
| await self.simple_response(scope, receive, send, request_headers=headers)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 748, in dispatch
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
| with collapse_excgroups():
| File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
| self.gen.throw(typ, value, traceback)
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
| response = await self.dispatch_func(request, call_next)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 613, in dispatch
| response = await call_next(request)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
| raise app_exc
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
| await self.app(scope, receive_or_disconnect, send_no_error)
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
| await route.handle(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
| await self.app(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
| response = await func(request)
| ^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
| raw_response = await run_endpoint_function(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
| return await dependant.call(**values)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/main.py", line 993, in generate_chat_completions
| return await generate_function_chat_completion(form_data, user=user)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
| configured_tools = get_tools(app, tool_ids, user, tools_params)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/app/backend/utils/tools.py", line 37, in get_tools
| for tool_id in tool_ids:
| TypeError: 'NoneType' object is not iterable
+------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 792, in update_embedding_function
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 783, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 769, in commit_session_after_request
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 748, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 613, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 993, in generate_chat_completions
return await generate_function_chat_completion(form_data, user=user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
configured_tools = get_tools(app, tool_ids, user, tools_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/utils/tools.py", line 37, in get_tools
for tool_id in tool_ids:
TypeError: 'NoneType' object is not iterable
```
</details>
Along with the error:
```
Uh-oh! There was an issue connecting to ClaudeSonnet Beta.
Unexpected token 'I', "Internal S"... is not valid JSON
```
**Screenshots/Screen Recordings (if applicable):**
##
Additional Information
Started happening as soon as I upgraded to 0.3.14
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 792, in update_embedding_function
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 783, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 769, in commit_session_after_request
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 748, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 613, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 993, in generate_chat_completions
return await generate_function_chat_completion(form_data, user=user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
configured_tools = get_tools(app, tool_ids, user, tools_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/utils/tools.py", line 37, in get_tools
for tool_id in tool_ids:
TypeError: 'NoneType' object is not iterable
generate_title
anthropic.claude-3-5-sonnet-20240620
anthropic
anthropic
@darkvertex commented on GitHub (Aug 21, 2024):
Same here with the Anthropic manifold pipe from the community:
https://openwebui.com/f/justinrahb/anthropic/
```
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 792, in update_embedding_function
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 783, in check_url
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 769, in commit_session_after_request
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 748, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 613, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/main.py", line 993, in generate_chat_completions
return await generate_function_chat_completion(form_data, user=user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/apps/webui/main.py", line 304, in generate_function_chat_completion
configured_tools = get_tools(app, tool_ids, user, tools_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/utils/tools.py", line 37, in get_tools
for tool_id in tool_ids:
TypeError: 'NoneType' object is not iterable
generate_title
anthropic.claude-3-5-sonnet-20240620
anthropic
anthropic
```
Fixed faster than Sonic makes scrambled eggs. Amazing! Thank you. 🙏
@darkvertex commented on GitHub (Aug 22, 2024):
> Fixed on dev, 0.3.15 coming soon!
Fixed faster than Sonic makes scrambled eggs. Amazing! Thank you. 🙏
You're getting a KeyError that "dolphin" is not in your models. Looking at the page above under "Base Model ID" it says it's "dolphin", meaning it depends on a model by that name existing in your ollama server.
Which dolphin model though? That is the question, as there are many. 🤔 There's not one called simply "dolphin" though, so I wonder if the script ever worked?
You may have to download one of the many dolphins such as: https://ollama.com/library/dolphin-mixtral
...and then edit your imported "model prompt" where it says "dolphin", change to the full model name like "dolphin-mixtral".
(If this doesn't solve it, open a new GitHub issue because your error is different from the one that started this one.)
@darkvertex commented on GitHub (Aug 26, 2024):
> Unfortunately I still receive this error, although I just updated to version 0.3.15.
>
> I'm running it in two docker containers, both Open-WebUI and Ollama. When I check the Ollama API in Settings, it says the connection works fine.
>
Not the same error as I and others were having. Perhaps you may wanna open a new GitHub issue for it if this reply does not solve your problem.
@MatthK From the error it appears you're trying to use this "model" prompt, yes? 👇
https://openwebui.com/m/uaquax/chatgpt-4-uncensored:latest
You're getting a KeyError that "dolphin" is not in your models. Looking at the page above under "Base Model ID" it says it's "dolphin", meaning it depends on a model by that name existing in your ollama server.
Which dolphin model though? That is the question, as there are many. 🤔 There's not one called simply "dolphin" though, so I wonder if the script ever worked?
You may have to download one of the many dolphins such as:
https://ollama.com/library/dolphin-mixtral
...and then edit your imported "model prompt" where it says "dolphin", change to the full model name like "dolphin-mixtral".
(If this doesn't solve it, open a new GitHub issue because your error is different from the one that started this one.)
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @thiswillbeyourgithub on GitHub (Aug 21, 2024).
Bug Report
Installation Method
docker
Environment
Open WebUI Version: 0.3.14 (not present before)
Operating System: ubuntu 22
Confirmation:
Expected Behavior:
My pipe should work :)
Actual Behavior:
The message errors out.
Description
Bug Summary:
[Provide a brief but clear summary of the bug]
Reproduction Details
Steps to Reproduce:
openrouter/anthropic/claude-3.5-sonnetMy pipe
Logs and Screenshots
Docker Container Logs:
Docker logs
Along with the error:
Screenshots/Screen Recordings (if applicable):
##
Additional Information
Started happening as soon as I upgraded to 0.3.14
@darkvertex commented on GitHub (Aug 21, 2024):
Same here with the Anthropic manifold pipe from the community:
https://openwebui.com/f/justinrahb/anthropic/
@justinh-rahb commented on GitHub (Aug 21, 2024):
Looking into it!
@tjbck commented on GitHub (Aug 21, 2024):
Fixed on dev, 0.3.15 coming soon!
@darkvertex commented on GitHub (Aug 22, 2024):
Fixed faster than Sonic makes scrambled eggs. Amazing! Thank you. 🙏
@thiswillbeyourgithub commented on GitHub (Aug 22, 2024):
Confirmed fixed for me too, you people rock :)
@MatthK commented on GitHub (Aug 23, 2024):
Unfortunately I still receive this error, although I just updated to version 0.3.15.
I'm running it in two docker containers, both Open-WebUI and Ollama. When I check the Ollama API in Settings, it says the connection works fine.
@darkvertex commented on GitHub (Aug 26, 2024):
Not the same error as I and others were having. Perhaps you may wanna open a new GitHub issue for it if this reply does not solve your problem.
@MatthK From the error it appears you're trying to use this "model" prompt, yes? 👇
https://openwebui.com/m/uaquax/chatgpt-4-uncensored:latest
You're getting a KeyError that "dolphin" is not in your models. Looking at the page above under "Base Model ID" it says it's "dolphin", meaning it depends on a model by that name existing in your ollama server.
Which dolphin model though? That is the question, as there are many. 🤔 There's not one called simply "dolphin" though, so I wonder if the script ever worked?
You may have to download one of the many dolphins such as:
https://ollama.com/library/dolphin-mixtral
...and then edit your imported "model prompt" where it says "dolphin", change to the full model name like "dolphin-mixtral".
(If this doesn't solve it, open a new GitHub issue because your error is different from the one that started this one.)