fix(settings): ensure settings sync between multi replicas #3979

Closed
opened 2025-11-11 15:43:40 -06:00 by GiteaMirror · 13 comments
Owner

Originally created by @U8F69 on GitHub (Feb 19, 2025).

Originally assigned to: @tjbck on GitHub.

Bug Report

Installation Method

Pod in K8S Cluster

Environment

  • Open WebUI Version: v0.5.14
  • Operating System: MacOS 15
  • Browser: Edge 133.0

Description

Bug Summary:

When using open-webui for multi-replica deployment, configuration changes do not take effect across different replicas, possibly due to the settings being loaded into memory. For example, after enabling internet search, querying /api/v1/retrieval/config occasionally still shows web.search.enable as false. The issue is resolved after restarting all the Pods.

Reproduction Details

Steps to Reproduce:

  • Scale the open-webui deployment to 2 or more Pods.
  • Change some configuration settings and save them.
  • Use the API (e.g., /api/v1/retrieval/config) to check if the settings have been applied.
  • Refresh and repeat the API call multiple times to ensure requests are routed to different Pods.
Originally created by @U8F69 on GitHub (Feb 19, 2025). Originally assigned to: @tjbck on GitHub. # Bug Report ## Installation Method Pod in K8S Cluster ## Environment - **Open WebUI Version:** v0.5.14 - **Operating System:** MacOS 15 - **Browser:** Edge 133.0 ## Description **Bug Summary:** When using open-webui for multi-replica deployment, configuration changes do not take effect across different replicas, possibly due to the settings being loaded into memory. For example, after enabling internet search, querying `/api/v1/retrieval/config` occasionally still shows `web.search.enable` as false. The issue is resolved after restarting all the Pods. ## Reproduction Details **Steps to Reproduce:** - Scale the open-webui deployment to 2 or more Pods. - Change some configuration settings and save them. - Use the API (e.g., /api/v1/retrieval/config) to check if the settings have been applied. - Refresh and repeat the API call multiple times to ensure requests are routed to different Pods.
Author
Owner

@tjbck commented on GitHub (Feb 19, 2025):

Taking a look!

@tjbck commented on GitHub (Feb 19, 2025): Taking a look!
Author
Owner

@a517363523 commented on GitHub (Feb 27, 2025):

Has it been fixed? I also encountered this problem

@a517363523 commented on GitHub (Feb 27, 2025): Has it been fixed? I also encountered this problem
Author
Owner

@tkg61 commented on GitHub (Feb 28, 2025):

Has it been fixed? I also encountered this problem

Not yet, still have it in 5.18

@tkg61 commented on GitHub (Feb 28, 2025): > Has it been fixed? I also encountered this problem Not yet, still have it in 5.18
Author
Owner

@tjbck commented on GitHub (Mar 6, 2025):

Allocating more resources into resolving this issue, aiming for 0.5.21

@tjbck commented on GitHub (Mar 6, 2025): Allocating more resources into resolving this issue, aiming for 0.5.21
Author
Owner

@tjbck commented on GitHub (Mar 8, 2025):

Addressed with d40e696468, Proper redis configuration is required.

@tjbck commented on GitHub (Mar 8, 2025): Addressed with d40e69646823f31711f6ad6c269456d2435c0d01, Proper redis configuration is required.
Author
Owner

@tkg61 commented on GitHub (Apr 2, 2025):

@tjbck Still having issues with this in 0.6.0 with redis configured. Settings are not synced between instances. i can just refresh the page and see different banners appear between the instances

@tkg61 commented on GitHub (Apr 2, 2025): @tjbck Still having issues with this in 0.6.0 with redis configured. Settings are not synced between instances. i can just refresh the page and see different banners appear between the instances
Author
Owner

@tkg61 commented on GitHub (Apr 7, 2025):

@tjbck Still having issues in .6.2 with a fresh install via the helm chart. Not sure what else to try at the moment. Let me know if i should open up another ticket

Image

Image

@tkg61 commented on GitHub (Apr 7, 2025): @tjbck Still having issues in .6.2 with a fresh install via the helm chart. Not sure what else to try at the moment. Let me know if i should open up another ticket ![Image](https://github.com/user-attachments/assets/cc3241db-2aa2-4a97-8fb3-d817c49c40d4) ![Image](https://github.com/user-attachments/assets/2a66378f-8463-4ab6-8a27-b555e97fef6a)
Author
Owner

@sFritsch09 commented on GitHub (Apr 7, 2025):

@tjbck Still having issues in .6.2 with a fresh install via the helm chart. Not sure what else to try at the moment. Let me know if i should open up another ticket

Image

Image

Can you share a minimal values file without exposing personals to check if you have websockets turned on in your ingress for example

@sFritsch09 commented on GitHub (Apr 7, 2025): > @tjbck Still having issues in .6.2 with a fresh install via the helm chart. Not sure what else to try at the moment. Let me know if i should open up another ticket > > ![Image](https://github.com/user-attachments/assets/cc3241db-2aa2-4a97-8fb3-d817c49c40d4) > > ![Image](https://github.com/user-attachments/assets/2a66378f-8463-4ab6-8a27-b555e97fef6a) Can you share a minimal values file without exposing personals to check if you have websockets turned on in your ingress for example
Author
Owner

@Classic298 commented on GitHub (Apr 7, 2025):

@tkg61 did you also set the REDIS_URL environment variable or only the WEBHOOK_REDIS_URL ?

make sure to configure both, then it should work afaik

@Classic298 commented on GitHub (Apr 7, 2025): @tkg61 did you also set the REDIS_URL environment variable or only the WEBHOOK_REDIS_URL ? make sure to configure both, then it should work afaik
Author
Owner

@tkg61 commented on GitHub (Apr 7, 2025):

@tkg61 did you also set the REDIS_URL environment variable or only the WEBHOOK_REDIS_URL ?

make sure to configure both, then it should work afaik

Ah, that might be it, i don't see that in the configured envs. Thanks for the clarification. Will make a ticket in the helm repo to update as well.

@tkg61 commented on GitHub (Apr 7, 2025): > [@tkg61](https://github.com/tkg61) did you also set the REDIS_URL environment variable or only the WEBHOOK_REDIS_URL ? > > make sure to configure both, then it should work afaik Ah, that might be it, i don't see that in the configured envs. Thanks for the clarification. Will make a ticket in the helm repo to update as well.
Author
Owner

@tkg61 commented on GitHub (Apr 7, 2025):

It works! Not sure why we have 2 URLs but it works!

Thanks for the help everyone

@tkg61 commented on GitHub (Apr 7, 2025): It works! Not sure why we have 2 URLs but it works! Thanks for the help everyone
Author
Owner

@Classic298 commented on GitHub (Apr 7, 2025):

Wonderful! Enjoy

@Classic298 commented on GitHub (Apr 7, 2025): Wonderful! Enjoy
Author
Owner

@Classic298 commented on GitHub (Apr 8, 2025):

PR to improve docs to inform users that all these variables need to be set: https://github.com/open-webui/docs/pull/481

@Classic298 commented on GitHub (Apr 8, 2025): PR to improve docs to inform users that all these variables need to be set: https://github.com/open-webui/docs/pull/481
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#3979