I have more than 80 models and UI is very slow, every call to a completion it takes more time than usual compared to another interfaces.
I have realized that every call to "get_all_models()" is very slow, and this is why (at least in my case):
open_webui/main.py: 1078
formodelinmodels:action_ids=[action_idforaction_idinlist(set(model.pop("action_ids",[])+global_action_ids))ifaction_idinenabled_action_ids]model["actions"]=[]log.debug("loop-2-int-1")log.debug(f"action_ids: {action_ids}")foraction_idinaction_ids:log.debug("Functions.get_function_by_id")action_function=Functions.get_function_by_id(action_id)ifaction_functionisNone:raiseException(f"Action not found: {action_id}")log.debug("get_function_module_by_id")function_module=get_function_module_by_id(action_id)log.debug("get_action_items_from_module")model["actions"].extend(get_action_items_from_module(action_function,function_module)
I'm using global_action_ids and, for each model is making the repeated calls to "Functions.get_function_by_id(action_id)" and "get_function_module_by_id(action_id)". The time to process this for 83 models is: 3.225s on my server.
Refactoring that function to prefetch global_action_ids data improves the function:
# Pre-fetch global actionsglobal_action_functions={}ifglobal_action_ids:foraction_idinglobal_action_ids:ifaction_idinenabled_action_ids:action_function=Functions.get_function_by_id(action_id)ifaction_functionisNone:raiseException(f"Action not found: {action_id}")function_module=get_function_module_by_id(action_id)global_action_functions[action_id]=(action_function,function_module)formodelinmodels:model_action_ids=model.pop("action_ids",[])action_ids=list(set(model_action_ids+global_action_ids))model["actions"]=[]log.debug("loop-2-int-1")log.debug(f"action_ids: {action_ids}")foraction_idinaction_ids:ifaction_idnotinenabled_action_ids:continueifaction_idinglobal_action_functions:# Use pre-fetched global actionaction_function,function_module=global_action_functions[action_id]else:# Fetch model-specific actionlog.debug("Functions.get_function_by_id")action_function=Functions.get_function_by_id(action_id)ifaction_functionisNone:raiseException(f"Action not found: {action_id}")log.debug("get_function_module_by_id")function_module=get_function_module_by_id(action_id)log.debug("get_action_items_from_module")model["actions"].extend(get_action_items_from_module(action_function,function_module))
With this changes, that loop takes from: 3.225 to: 0.233s
I have seen significant improvement in the UI with this change. Any of you can reproduce it?
Originally created by @juancarlosm on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7766
I have more than 80 models and UI is very slow, every call to a completion it takes more time than usual compared to another interfaces.
I have realized that every call to "get_all_models()" is very slow, and this is why (at least in my case):
open_webui/main.py: 1078
```python
for model in models:
action_ids = [
action_id
for action_id in list(set(model.pop("action_ids", []) + global_action_ids))
if action_id in enabled_action_ids
]
model["actions"] = []
log.debug("loop-2-int-1")
log.debug(f"action_ids: {action_ids}")
for action_id in action_ids:
log.debug("Functions.get_function_by_id")
action_function = Functions.get_function_by_id(action_id)
if action_function is None:
raise Exception(f"Action not found: {action_id}")
log.debug("get_function_module_by_id")
function_module = get_function_module_by_id(action_id)
log.debug("get_action_items_from_module")
model["actions"].extend(
get_action_items_from_module(action_function, function_module)
```
I'm using global_action_ids and, for each model is making the repeated calls to "Functions.get_function_by_id(action_id)" and "get_function_module_by_id(action_id)". The time to process this for 83 models is: 3.225s on my server.
Refactoring that function to prefetch global_action_ids data improves the function:
```python
# Pre-fetch global actions
global_action_functions = {}
if global_action_ids:
for action_id in global_action_ids:
if action_id in enabled_action_ids:
action_function = Functions.get_function_by_id(action_id)
if action_function is None:
raise Exception(f"Action not found: {action_id}")
function_module = get_function_module_by_id(action_id)
global_action_functions[action_id] = (action_function, function_module)
for model in models:
model_action_ids = model.pop("action_ids", [])
action_ids = list(set(model_action_ids + global_action_ids))
model["actions"] = []
log.debug("loop-2-int-1")
log.debug(f"action_ids: {action_ids}")
for action_id in action_ids:
if action_id not in enabled_action_ids:
continue
if action_id in global_action_functions:
# Use pre-fetched global action
action_function, function_module = global_action_functions[action_id]
else:
# Fetch model-specific action
log.debug("Functions.get_function_by_id")
action_function = Functions.get_function_by_id(action_id)
if action_function is None:
raise Exception(f"Action not found: {action_id}")
log.debug("get_function_module_by_id")
function_module = get_function_module_by_id(action_id)
log.debug("get_action_items_from_module")
model["actions"].extend(
get_action_items_from_module(action_function, function_module)
)
```
With this changes, that loop takes from: 3.225 to: 0.233s
I have seen significant improvement in the UI with this change. Any of you can reproduce it?
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @juancarlosm on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7766
I have more than 80 models and UI is very slow, every call to a completion it takes more time than usual compared to another interfaces.
I have realized that every call to "get_all_models()" is very slow, and this is why (at least in my case):
open_webui/main.py: 1078
I'm using global_action_ids and, for each model is making the repeated calls to "Functions.get_function_by_id(action_id)" and "get_function_module_by_id(action_id)". The time to process this for 83 models is: 3.225s on my server.
Refactoring that function to prefetch global_action_ids data improves the function:
With this changes, that loop takes from: 3.225 to: 0.233s
I have seen significant improvement in the UI with this change. Any of you can reproduce it?