[GH-ISSUE #1527] Whitelist Connections, or Whitelist Models on specific Connection #28066

New Issue

GiteaMirror · 2026-04-25T02:48:18-05:00

GiteaMirror commented

2026-04-25 02:48:18 -05:00

Originally created by @lewismacnow on GitHub (Apr 12, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1527

Is your feature request related to a problem? Please describe.
Background:

I have multiple connections pointed at a single host.
Connection 1 is a load balancer rotating through 5 Ollama instance.
Connection 2 is pointed directly at 1 Ollama instance.

Problem:
I need to force users to utilise the load balancer on Connection 1 AND restrict model usage.
Because Connection 1 and 2 both use the same Model library (all instances are on the same host), white listing models does not restrict connections so the result we may end up with is user roating through Ollama instances, unloading the model in memory from the previous interaction so they can use the one they have selected.

Load balancing across multiple connections 'can' be useful, but can also cause unintended and irritating delays in the current way it is implemented.

The loading/unloading process adds seconds to the response so if users are activiely using multiple models which are shared bewteen connections, we will potentially add time to responses instead of benefiting from load balancing over multiple instances.

The solution here I believe is to prevent unloading the model uncessesarily so not to impact the user experience, by allowing restrictions on models per connection.

Describe the solution you'd like
One of:

Where models exist over multiple connections, allow global restriction for connections to use only specific models.
Whitelist connections: Force users (not just users but Admins too!) to forward API requests to one or more Connection
Whitelist models on a per-connection basis (For Admins too!), for example:

The aim here is to prevent unloading the model uncessesarily so not to impact the user experience.

Another possibility, perhaps a more flexible approach, a model is restricted to one connection:
In this example, although Mistral is available on Connection 2, the user will never use it and therefore not unload Command-R from Ollama 1. This is beneficial for Users of Command-R, as they will

Another possible use case:
3 Connections to 3 different Ollama Instanced, but all which own the same models.
1 Connection is used exclusively for Model A
The other 2 Connections Load balance between the other Ollama instanced for Model B

Describe alternatives you've considered
Whitelisting models as it currently is while effective is limited in this example.

Additional context
Nothing to add.

Originally created by @lewismacnow on GitHub (Apr 12, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/1527 **Is your feature request related to a problem? Please describe.** Background: I have multiple connections pointed at a single host. Connection 1 is a load balancer rotating through 5 Ollama instance. Connection 2 is pointed directly at 1 Ollama instance. ![image](https://github.com/open-webui/open-webui/assets/93997957/ac2fe9e8-9143-4999-8634-ed9280eed05d) Problem: I need to force users to utilise the load balancer on Connection 1 AND restrict model usage. Because Connection 1 and 2 both use the same Model library (all instances are on the same host), white listing models does not restrict connections so the result we may end up with is user roating through Ollama instances, unloading the model in memory from the previous interaction so they can use the one they have selected. Load balancing across multiple connections 'can' be useful, but can also cause unintended and irritating delays in the current way it is implemented. The loading/unloading process adds seconds to the response so if users are activiely using multiple models which are shared bewteen connections, we will potentially **_add time to responses_** instead of benefiting from load balancing over multiple instances. The solution here I believe is to prevent unloading the model uncessesarily so not to impact the user experience, by allowing restrictions on models per connection. **Describe the solution you'd like** One of: 1. Where models exist over multiple connections, allow global restriction for connections to use only specific models. 2. Whitelist connections: Force users (not just users but Admins too!) to forward API requests to one or more Connection 3. Whitelist models on a per-connection basis (For Admins too!), for example: ![image](https://github.com/open-webui/open-webui/assets/93997957/c77a0a56-612f-4e38-bda9-cbcbf40d6c19) The aim here is to prevent unloading the model uncessesarily so not to impact the user experience. ![image](https://github.com/open-webui/open-webui/assets/93997957/ef737edd-8134-4f38-80db-98d9c4f73f60) Another possibility, perhaps a more flexible approach, a model is restricted to one connection: In this example, although Mistral is available on Connection 2, the user will never use it and therefore not unload Command-R from Ollama 1. This is beneficial for Users of Command-R, as they will ![image](https://github.com/open-webui/open-webui/assets/93997957/31f9f974-9144-4599-89d3-caeef3f2682c) Another possible use case: 3 Connections to 3 different Ollama Instanced, but all which own the same models. 1 Connection is used exclusively for Model A The other 2 Connections Load balance between the other Ollama instanced for Model B ![image](https://github.com/open-webui/open-webui/assets/93997957/c23fa01f-a3ed-4c3c-a39f-c1b9dad52b9b) **Describe alternatives you've considered** Whitelisting models as it currently is while effective is limited in this example. **Additional context** Nothing to add.

GiteaMirror closed this issue

2026-04-25 02:48:19 -05:00

GiteaMirror commented

2026-04-25 02:48:20 -05:00

@tjbck commented on GitHub (Apr 12, 2024):

Related #1081, Let's continue our discussion there.

@tjbck commented on GitHub (Apr 12, 2024): Related #1081, Let's continue our discussion there.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#28066