mirror of
https://github.com/open-webui/open-webui.git
synced 2026-05-06 19:08:59 -05:00
[GH-ISSUE #1785] feat: ollama server naming support #12636
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#12636
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @francip on GitHub (Apr 26, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1785
Originally assigned to: @tjbck on GitHub.
Is your feature request related to a problem? Please describe.
When Open WebUI is configured with several connections to different Ollama servers running the same model (eg llama3:latest), it is impossible to determine which connection a model is running on in the model selection dropdown box on the chat. Similarly, it's impossible to know which connection particular response comes from.
Describe the solution you'd like
Allow nicknaming the connections (with pregenerated nickname for "local" connections, ie in the same docker container, or on the same machine"), and then show the nickname every time the model name is shown.
Describe alternatives you've considered
Alternatively, show the full connection string in the (i) tooltip.
Additional context
Here's an example of list of models from two different Ollama servers. There should be two entries for llama3:latest model from each connection. It is important to support this, because the two machines have wildly different capabilities (lapotop with Nvidia 3070 w/ 8G vs desktop with Nvidia 4090 w/ 24G)
@mjtechguy commented on GitHub (May 3, 2024):
This please!
@befocken commented on GitHub (Jun 14, 2024):
Is there already a plan for implementing this? I can envision two broad use cases: first, transparent load-balancing, as it currently stands (with the possible additional feature to see which connection the response actually came from). Secondly, when one has multiple, very different capable systems. For example, a local set of models that is always available and a remote set of models that is only sometimes available.
I believe both use cases are realistic, although the latter might occur more often in hobbyist settings.
The decision would, of course, influence how this is implemented then.
@jeremiahsb commented on GitHub (Aug 23, 2024):
An alternative solution would just be to have the models arranged under the servers they are on in the list or the ability to filter by server.
@bkev commented on GitHub (Sep 21, 2024):
Has this been implemented? There seems to be several requests and some have been closed but I don't know how to do it if it's been added.
Basically, I think it would be really useful to have a preference for a certain node when it's online if that's possible? For example, openweb UI running on a low powered server with Ollama, but a higher powered node, when powered up and availabl, is preferred for to better performance.
@tjbck commented on GitHub (Sep 21, 2024):
You should be able to implement this as a Function, Here's an example of OpenAI pipe function: https://openwebui.com/f/hub/openai_manifold
With slight modification it should be able to support Ollama instances as well.