mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-16 14:39:31 -05:00
Multiple External APIs #317
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#317
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jacob-hansen on GitHub (Feb 18, 2024).
Is your feature request related to a problem? Please describe.
I use a vLLM backend to host model inference. This provides a more scalable deployment, but each vLLM server can only host one model. I can connect Open WebUI to the vLLM backend using the External API endpoint selection.
I'd like to both reference multiple vLLM backends, as well as the OpenAI api.
Describe the solution you'd like
The External API selection needs to be multi-select.
Describe alternatives you've considered
Right now, I've just considered launching multiple instances of Open WebUI.
Additional context
I'd be willing to help out. This would be an extremely valuable feature for me. Anyone have pointers?
@justinh-rahb commented on GitHub (Feb 18, 2024):
You can have a look at some past discussions we've had about configurations for LiteLLM that will be helpful to your situation in the meantime. Multiple inference backends is on the roadmap, stay tuned!
@jacob-hansen commented on GitHub (Feb 18, 2024):
Oh, fantastic! LiteLLM would work great for now.