github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-08-30 17:25:30 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

Multiple External APIs #317

New Issue
Closed
opened 2025-11-11 14:16:45 -06:00 by GiteaMirror · 2 comments
GiteaMirror commented 2025-11-11 14:16:45 -06:00
Owner
Copy Link
Copy Source

Originally created by @jacob-hansen on GitHub (Feb 18, 2024).

Is your feature request related to a problem? Please describe.
I use a vLLM backend to host model inference. This provides a more scalable deployment, but each vLLM server can only host one model. I can connect Open WebUI to the vLLM backend using the External API endpoint selection.

I'd like to both reference multiple vLLM backends, as well as the OpenAI api.

Describe the solution you'd like
The External API selection needs to be multi-select.

Describe alternatives you've considered
Right now, I've just considered launching multiple instances of Open WebUI.

Additional context
I'd be willing to help out. This would be an extremely valuable feature for me. Anyone have pointers?

Originally created by @jacob-hansen on GitHub (Feb 18, 2024). **Is your feature request related to a problem? Please describe.** I use a vLLM backend to host model inference. This provides a more scalable deployment, but each vLLM server can only host one model. I can connect Open WebUI to the vLLM backend using the External API endpoint selection. I'd like to both reference multiple vLLM backends, as well as the OpenAI api. **Describe the solution you'd like** The External API selection needs to be multi-select. **Describe alternatives you've considered** Right now, I've just considered launching multiple instances of Open WebUI. **Additional context** I'd be willing to help out. This would be an extremely valuable feature for me. Anyone have pointers?
GiteaMirror commented 2025-11-11 14:16:47 -06:00
Author
Owner
Copy Link
Copy Source

@justinh-rahb commented on GitHub (Feb 18, 2024):

You can have a look at some past discussions we've had about configurations for LiteLLM that will be helpful to your situation in the meantime. Multiple inference backends is on the roadmap, stay tuned!

@justinh-rahb commented on GitHub (Feb 18, 2024): You can have a look at some past discussions we've had about configurations for [LiteLLM](https://github.com/BerriAI/litellm) that will be helpful to your situation in the meantime. Multiple inference backends is on the roadmap, stay tuned! - https://github.com/open-webui/open-webui/discussions?discussions_q=is%3Aopen+litellm
GiteaMirror commented 2025-11-11 14:16:47 -06:00
Author
Owner
Copy Link
Copy Source

@jacob-hansen commented on GitHub (Feb 18, 2024):

Oh, fantastic! LiteLLM would work great for now.

@jacob-hansen commented on GitHub (Feb 18, 2024): Oh, fantastic! LiteLLM would work great for now.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#317