[GH-ISSUE #12314] issue: when pod with openwebui restarts, dependencies installed on runtime are lost #55215

New Issue

GiteaMirror · 2026-05-05T17:19:23-05:00

GiteaMirror commented

2026-05-05 17:19:23 -05:00

Originally created by @Lutherwaves on GitHub (Apr 1, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12314

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Other

Open WebUI Version

0.6.0

Ollama Version (if applicable)

No response

Operating System

Ubuntu 22.04

Browser (if applicable)

No response

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

Expected Behavior

When openwebui pod restarts, dependencies installed during runtime (when creating tools) are persisted via pvc and are available upon pod restart. These include

previously setup embedding/re-ranking models when using non-transformers models
previously setup tool additional dependencies are already present

Actual Behavior

When a pod restarts (after upgrade and/or manual restart)

playwright if enabled is scheduled to be installed and causes delays in restarting openwebuo
tools that have additional dependencies do not work and need to be edited and saved
if using re-ranking model with ollama/openai, it tries to re-install the model, but since it is not a known dependency available for transformers library, it fails and required embedding -> re-ranking to be resetup

Steps to Reproduce

Install openwebui in kubernetes using helm
Install a tool with external library dependency https://openwebui.com/t/lutherwaves/todoist
Enable RAG hybrid search and select mx-embed-large as the model with ollama configured as the host
Delete open-webui-0 pod and watch the restart
Upon successful restart, test chatting with a model and a tool that has external dependencies, you get "library not found" since its' installation is lost
In the logs of the pod, you can see it fails to pull mx-embed-large into transformers; if you go to Admin Settings -> Documents -> Reset Hybrid search and Save, it will again point to ollama and be fixed

Logs & Screenshots

Restarting pod and showing how playwright reinstall is scheduled (I had overwritten the open-webui name to prism)

➜ ~ kubectl get pods NAME READY STATUS RESTARTS AGE open-webui-pipelines-869fd55cc7-rqlbs 1/1 Running 0 2d21h prism-0 1/1 Running 0 15h prism-ollama-684b78fb7f-cpqvx 1/1 Running 0 15h prism-open-webui-tika-7995fb7dcb-v5tmj 1/1 Running 0 29h prism-redis-6d77f64b5d-6gmbx 1/1 Running 0 15h searxng-787f984b98-sfsrb 1/1 Running 0 15h tika-58b567c6c4-8wnnf 1/1 Running 0 30h ➜ ~ kubectl delete pod prism-0 pod "prism-0" deleted ➜ ~ kubectl get pods NAME READY STATUS RESTARTS AGE open-webui-pipelines-869fd55cc7-rqlbs 1/1 Running 0 2d21h prism-0 1/1 Running 0 4s prism-ollama-684b78fb7f-cpqvx 1/1 Running 0 15h prism-open-webui-tika-7995fb7dcb-v5tmj 1/1 Running 0 29h prism-redis-6d77f64b5d-6gmbx 1/1 Running 0 15h searxng-787f984b98-sfsrb 1/1 Running 0 15h tika-58b567c6c4-8wnnf 1/1 Running 0 30h ➜ ~ kubectl logs -f prism-0 Defaulted container "open-webui" out of: open-webui, copy-app-data (init) Installing Playwright browsers... Downloading Chromium 131.0.6778.33 (playwright build v1148) from https://playwright.azureedge.net/builds/chromium/1148/chromium-linux.zip 161.3 MiB [] 100% 0.0s Chromium 131.0.6778.33 (playwright build v1148) downloaded to /root/.cache/ms-playwright/chromium-1148 Downloading FFMPEG playwright build v1010 from https://playwright.azureedge.net/builds/ffmpeg/1010/ffmpeg-linux.zip 2.3 MiB [] 100% 0.0s FFMPEG playwright build v1010 downloaded to /root/.cache/ms-playwright/ffmpeg-1010 Downloading Chromium Headless Shell 131.0.6778.33 (playwright build v1148)

mx-embed-large error

Repository Not Found for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main. Please make sure you specified the correct repo_idandrepo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authentication
Invalid username or password.
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 409, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main

The above exception was the direct cause of the following exception:

`Traceback (most recent call last):
File "/app/backend/open_webui/retrieval/utils.py", line 596, in get_model_path
model_repo_path = snapshot_download(**snapshot_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 232, in snapshot_download
raise api_call_error
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 155, in snapshot_download
repo_info = api.repo_info(repo_id=repo_id, repo_type=repo_type, revision=revision, token=token)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2807, in repo_info
return method(
^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2592, in model_info
hf_raise_for_status(r)
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 459, in hf_raise_for_status
raise _format(RepositoryNotFoundError, message, response) from e
huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-67ec4cb1-2129a38565f0c3ce45e450d4;ba617702-393a-4d6b-a106-780fe5894784)

Repository Not Found for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main.
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authentication
Invalid username or password.
ERROR [open_webui.routers.retrieval] CrossEncoder error
ERROR [open_webui.main] Error updating models: [ERROR: CrossEncoder error]
INFO: Started server process [1]
INFO: Waiting for application startup.`

Also error when initating a chat with Todoist tool

Also missing valves for todoist

Additional Information

I am running inside a k3s cluster on Ubuntu.

Originally created by @Lutherwaves on GitHub (Apr 1, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/12314 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Other ### Open WebUI Version 0.6.0 ### Ollama Version (if applicable) _No response_ ### Operating System Ubuntu 22.04 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior When openwebui pod restarts, dependencies installed during runtime (when creating tools) are persisted via pvc and are available upon pod restart. These include - previously setup embedding/re-ranking models when using non-transformers models - previously setup tool additional dependencies are already present ### Actual Behavior When a pod restarts (after upgrade and/or manual restart) - playwright if enabled is scheduled to be installed and causes delays in restarting openwebuo - tools that have additional dependencies do not work and need to be edited and saved - if using re-ranking model with ollama/openai, it tries to re-install the model, but since it is not a known dependency available for transformers library, it fails and required embedding -> re-ranking to be resetup ### Steps to Reproduce 1. Install openwebui in kubernetes using helm 2. Install a tool with external library dependency https://openwebui.com/t/lutherwaves/todoist 3. Enable RAG hybrid search and select mx-embed-large as the model with ollama configured as the host 4. Delete open-webui-0 pod and watch the restart 5. Upon successful restart, test chatting with a model and a tool that has external dependencies, you get "library not found" since its' installation is lost 6. In the logs of the pod, you can see it fails to pull mx-embed-large into transformers; if you go to Admin Settings -> Documents -> Reset Hybrid search and Save, it will again point to ollama and be fixed ### Logs & Screenshots Restarting pod and showing how playwright reinstall is scheduled (I had overwritten the open-webui name to prism) `➜ ~ kubectl get pods NAME READY STATUS RESTARTS AGE open-webui-pipelines-869fd55cc7-rqlbs 1/1 Running 0 2d21h prism-0 1/1 Running 0 15h prism-ollama-684b78fb7f-cpqvx 1/1 Running 0 15h prism-open-webui-tika-7995fb7dcb-v5tmj 1/1 Running 0 29h prism-redis-6d77f64b5d-6gmbx 1/1 Running 0 15h searxng-787f984b98-sfsrb 1/1 Running 0 15h tika-58b567c6c4-8wnnf 1/1 Running 0 30h ➜ ~ kubectl delete pod prism-0 pod "prism-0" deleted ➜ ~ kubectl get pods NAME READY STATUS RESTARTS AGE open-webui-pipelines-869fd55cc7-rqlbs 1/1 Running 0 2d21h prism-0 1/1 Running 0 4s prism-ollama-684b78fb7f-cpqvx 1/1 Running 0 15h prism-open-webui-tika-7995fb7dcb-v5tmj 1/1 Running 0 29h prism-redis-6d77f64b5d-6gmbx 1/1 Running 0 15h searxng-787f984b98-sfsrb 1/1 Running 0 15h tika-58b567c6c4-8wnnf 1/1 Running 0 30h ➜ ~ kubectl logs -f prism-0 Defaulted container "open-webui" out of: open-webui, copy-app-data (init) Installing Playwright browsers... Downloading Chromium 131.0.6778.33 (playwright build v1148) from https://playwright.azureedge.net/builds/chromium/1148/chromium-linux.zip 161.3 MiB [] 100% 0.0s Chromium 131.0.6778.33 (playwright build v1148) downloaded to /root/.cache/ms-playwright/chromium-1148 Downloading FFMPEG playwright build v1010 from https://playwright.azureedge.net/builds/ffmpeg/1010/ffmpeg-linux.zip 2.3 MiB [] 100% 0.0s FFMPEG playwright build v1010 downloaded to /root/.cache/ms-playwright/ffmpeg-1010 Downloading Chromium Headless Shell 131.0.6778.33 (playwright build v1148)` mx-embed-large error `Repository Not Found for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main. Please make sure you specified the correct `repo_id` and `repo_type`. If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authentication Invalid username or password. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 409, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main The above exception was the direct cause of the following exception: `Traceback (most recent call last): File "/app/backend/open_webui/retrieval/utils.py", line 596, in get_model_path model_repo_path = snapshot_download(**snapshot_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 232, in snapshot_download raise api_call_error File "/usr/local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 155, in snapshot_download repo_info = api.repo_info(repo_id=repo_id, repo_type=repo_type, revision=revision, token=token) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2807, in repo_info return method( ^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2592, in model_info hf_raise_for_status(r) File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 459, in hf_raise_for_status raise _format(RepositoryNotFoundError, message, response) from e huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-67ec4cb1-2129a38565f0c3ce45e450d4;ba617702-393a-4d6b-a106-780fe5894784) Repository Not Found for url: https://huggingface.co/api/models/sentence-transformers/mxbai-embed-large/revision/main. Please make sure you specified the correct `repo_id` and `repo_type`. If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authentication Invalid username or password. ERROR [open_webui.routers.retrieval] CrossEncoder error ERROR [open_webui.main] Error updating models: [ERROR: CrossEncoder error] INFO: Started server process [1] INFO: Waiting for application startup.` Also error when initating a chat with Todoist tool <img width="1172" alt="Image" src="https://github.com/user-attachments/assets/643fe350-e6f7-496b-a1f9-dea4d36d7c69" /> Also missing valves for todoist <img width="1172" alt="Image" src="https://github.com/user-attachments/assets/c09daa1b-579d-4e28-8935-d63763b1006c" /> ### Additional Information I am running inside a k3s cluster on Ubuntu.

GiteaMirror added the bug label 2026-05-05 17:19:23 -05:00

GiteaMirror closed this issue

2026-05-05 17:19:24 -05:00

GiteaMirror commented

2026-05-05 17:19:26 -05:00

@tjbck commented on GitHub (Apr 1, 2025):

Please check for existing issues before creating one. Duplicate.

@tjbck commented on GitHub (Apr 1, 2025): Please check for existing issues before creating one. Duplicate.

GiteaMirror commented

2026-05-05 17:19:28 -05:00

@Lutherwaves commented on GitHub (Apr 1, 2025):

@tjbck I did check, can you tell me which issue to track that this duplicates?

@Lutherwaves commented on GitHub (Apr 1, 2025): @tjbck I did check, can you tell me which issue to track that this duplicates?

GiteaMirror commented

2026-05-05 17:19:30 -05:00

@tjbck commented on GitHub (Apr 1, 2025):

#12121

@tjbck commented on GitHub (Apr 1, 2025): #12121

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#55215