issue: Very long restart time after update to version 0.6.3x – takes several hours, no visible errors in logs #6880

Closed
opened 2025-11-11 17:08:30 -06:00 by GiteaMirror · 9 comments
Owner

Originally created by @Schwenn2002 on GitHub (Nov 10, 2025).

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.6.36

Ollama Version (if applicable)

No response

Operating System

Ubuntu 24.04

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

Open WebUI starts without significant delay after an update of the Docker container. The log should reveal which steps actually take so much time.

Actual Behavior

After an update, Open WebUI initially starts in an unhealthy state and is processing files (fetching files). It takes several hours (> 18 hours) before Open WebUI is available. There are no error messages in the log (debug log level). This behavior occurs with multiple versions. System utilization is consistently idle (htop, iftop).

Steps to Reproduce

docker-compose up -d open-webui

Logs & Screenshots

No Logs...

Additional Information

Description:
After updating Open WebUI to version 0.6.3x, the restart process takes an unusually long time — several hours in some cases. During startup, the logs show no explicit errors, but it appears to get stuck for a long period while fetching files.

Observed behavior:

  • Restart process can take hours to complete.
  • The log output seems normal at first, but the fetching step is extremely slow.
  • System monitoring tools (htop, iftop) show no abnormal CPU, memory, or network activity.

Relevant log excerpt:

Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00,  7.09s/it]
Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00,  2.00s/it]

Expected behavior:
Restart should complete within a reasonable time (a few minutes at most), as in previous versions.

Environment:

  • Open WebUI version: 0.6.3x
  • Installation method: docker-compose
  • Operating system: Ubuntu 24.04

Additional context:
The issue started immediately after upgrading to version 0.6.3x. No configuration changes were made otherwise.

Originally created by @Schwenn2002 on GitHub (Nov 10, 2025). ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.6.36 ### Ollama Version (if applicable) _No response_ ### Operating System Ubuntu 24.04 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior Open WebUI starts without significant delay after an update of the Docker container. The log should reveal which steps actually take so much time. ### Actual Behavior After an update, Open WebUI initially starts in an unhealthy state and is processing files (fetching files). It takes several hours (> 18 hours) before Open WebUI is available. There are no error messages in the log (debug log level). This behavior occurs with multiple versions. System utilization is consistently idle (htop, iftop). ### Steps to Reproduce docker-compose up -d open-webui ### Logs & Screenshots No Logs... ### Additional Information **Description:** After updating Open WebUI to version **0.6.3x**, the restart process takes an unusually long time — several hours in some cases. During startup, the logs show no explicit errors, but it appears to get stuck for a long period while fetching files. **Observed behavior:** * Restart process can take hours to complete. * The log output seems normal at first, but the fetching step is extremely slow. * System monitoring tools (`htop`, `iftop`) show no abnormal CPU, memory, or network activity. **Relevant log excerpt:** ``` Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00, 7.09s/it] Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00, 2.00s/it] ``` **Expected behavior:** Restart should complete within a reasonable time (a few minutes at most), as in previous versions. **Environment:** * Open WebUI version: 0.6.3x * Installation method: docker-compose * Operating system: Ubuntu 24.04 **Additional context:** The issue started immediately after upgrading to version 0.6.3x. No configuration changes were made otherwise.
GiteaMirror added the bug label 2025-11-11 17:08:30 -06:00
Author
Owner

@nohalloween commented on GitHub (Nov 10, 2025):

similar problems:
I was handling with the openwebui-pipeline problem so I had to restart docker open-webui several times and after restarting for the last time, the log remains:

INFO [open_webui.env] Embedding model set: sentence-transformers/all-MiniLM-L6-v2
WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests.

██████╗ ██████╗ ███████╗███╗ ██╗ ██╗ ██╗███████╗██████╗ ██╗ ██╗██╗
██╔═══██╗██╔══██╗██╔════╝████╗ ██║ ██║ ██║██╔════╝██╔══██╗██║ ██║██║
██║ ██║██████╔╝█████╗ ██╔██╗ ██║ ██║ █╗ ██║█████╗ ██████╔╝██║ ██║██║
██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║ ██║███╗██║██╔══╝ ██╔══██╗██║ ██║██║
╚██████╔╝██║ ███████╗██║ ╚████║ ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝ ╚══╝╚══╝ ╚══════╝╚═════╝ ╚═════╝ ╚═╝

v0.6.33 - building the best AI user interface.

https://github.com/open-webui/open-webui

nothing else appears

@nohalloween commented on GitHub (Nov 10, 2025): similar problems: I was handling with the openwebui-pipeline problem so I had to restart docker open-webui several times and after restarting for the last time, the log remains: INFO [open_webui.env] Embedding model set: sentence-transformers/all-MiniLM-L6-v2 WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests. ██████╗ ██████╗ ███████╗███╗ ██╗ ██╗ ██╗███████╗██████╗ ██╗ ██╗██╗ ██╔═══██╗██╔══██╗██╔════╝████╗ ██║ ██║ ██║██╔════╝██╔══██╗██║ ██║██║ ██║ ██║██████╔╝█████╗ ██╔██╗ ██║ ██║ █╗ ██║█████╗ ██████╔╝██║ ██║██║ ██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║ ██║███╗██║██╔══╝ ██╔══██╗██║ ██║██║ ╚██████╔╝██║ ███████╗██║ ╚████║ ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║ ╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝ ╚══╝╚══╝ ╚══════╝╚═════╝ ╚═════╝ ╚═╝ v0.6.33 - building the best AI user interface. https://github.com/open-webui/open-webui nothing else appears
Author
Owner

@Schwenn2002 commented on GitHub (Nov 10, 2025):

My complete log after 14 hours looks like this:

Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.
Generating WEBUI_SECRET_KEY
Loading WEBUI_SECRET_KEY from .webui_secret_key
CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests.

 ██████╗ ██████╗ ███████╗███╗   ██╗    ██╗    ██╗███████╗██████╗ ██╗   ██╗██╗
██╔═══██╗██╔══██╗██╔════╝████╗  ██║    ██║    ██║██╔════╝██╔══██╗██║   ██║██║
██║   ██║██████╔╝█████╗  ██╔██╗ ██║    ██║ █╗ ██║█████╗  ██████╔╝██║   ██║██║
██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║    ██║███╗██║██╔══╝  ██╔══██╗██║   ██║██║
╚██████╔╝██║     ███████╗██║ ╚████║    ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
 ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝     ╚══╝╚══╝ ╚══════╝╚═════╝  ╚═════╝ ╚═╝


v0.6.36 - building the best AI user interface.

https://github.com/open-webui/open-webui

Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00,  7.09s/it]
Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00,  2.00s/it]

Last time I waited about 48 hours, then the service was suddenly available again.

@Schwenn2002 commented on GitHub (Nov 10, 2025): My complete log after 14 hours looks like this: ``` Loading WEBUI_SECRET_KEY from file, not provided as an environment variable. Generating WEBUI_SECRET_KEY Loading WEBUI_SECRET_KEY from .webui_secret_key CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries. INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests. ██████╗ ██████╗ ███████╗███╗ ██╗ ██╗ ██╗███████╗██████╗ ██╗ ██╗██╗ ██╔═══██╗██╔══██╗██╔════╝████╗ ██║ ██║ ██║██╔════╝██╔══██╗██║ ██║██║ ██║ ██║██████╔╝█████╗ ██╔██╗ ██║ ██║ █╗ ██║█████╗ ██████╔╝██║ ██║██║ ██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║ ██║███╗██║██╔══╝ ██╔══██╗██║ ██║██║ ╚██████╔╝██║ ███████╗██║ ╚████║ ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║ ╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝ ╚══╝╚══╝ ╚══════╝╚═════╝ ╚═════╝ ╚═╝ v0.6.36 - building the best AI user interface. https://github.com/open-webui/open-webui Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00, 7.09s/it] Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00, 2.00s/it] ``` Last time I waited about 48 hours, then the service was suddenly available again.
Author
Owner

@nohalloween commented on GitHub (Nov 10, 2025):

me too lol

My complete log after 14 hours looks like this:

Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.
Generating WEBUI_SECRET_KEY
Loading WEBUI_SECRET_KEY from .webui_secret_key
CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests.

 ██████╗ ██████╗ ███████╗███╗   ██╗    ██╗    ██╗███████╗██████╗ ██╗   ██╗██╗
██╔═══██╗██╔══██╗██╔════╝████╗  ██║    ██║    ██║██╔════╝██╔══██╗██║   ██║██║
██║   ██║██████╔╝█████╗  ██╔██╗ ██║    ██║ █╗ ██║█████╗  ██████╔╝██║   ██║██║
██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║    ██║███╗██║██╔══╝  ██╔══██╗██║   ██║██║
╚██████╔╝██║     ███████╗██║ ╚████║    ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
 ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝     ╚══╝╚══╝ ╚══════╝╚═════╝  ╚═════╝ ╚═╝


v0.6.36 - building the best AI user interface.

https://github.com/open-webui/open-webui

Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00,  7.09s/it]
Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00,  2.00s/it]

Last time I waited about 48 hours, then the service was suddenly available again.

@nohalloween commented on GitHub (Nov 10, 2025): me too lol > My complete log after 14 hours looks like this: > > ``` > Loading WEBUI_SECRET_KEY from file, not provided as an environment variable. > Generating WEBUI_SECRET_KEY > Loading WEBUI_SECRET_KEY from .webui_secret_key > CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries. > INFO [alembic.runtime.migration] Context impl SQLiteImpl. > INFO [alembic.runtime.migration] Will assume non-transactional DDL. > WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests. > > ██████╗ ██████╗ ███████╗███╗ ██╗ ██╗ ██╗███████╗██████╗ ██╗ ██╗██╗ > ██╔═══██╗██╔══██╗██╔════╝████╗ ██║ ██║ ██║██╔════╝██╔══██╗██║ ██║██║ > ██║ ██║██████╔╝█████╗ ██╔██╗ ██║ ██║ █╗ ██║█████╗ ██████╔╝██║ ██║██║ > ██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║ ██║███╗██║██╔══╝ ██╔══██╗██║ ██║██║ > ╚██████╔╝██║ ███████╗██║ ╚████║ ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║ > ╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝ ╚══╝╚══╝ ╚══════╝╚═════╝ ╚═════╝ ╚═╝ > > > v0.6.36 - building the best AI user interface. > > https://github.com/open-webui/open-webui > > Fetching 22 files: 100%|██████████| 22/22 [02:35<00:00, 7.09s/it] > Fetching 13 files: 100%|██████████| 13/13 [00:25<00:00, 2.00s/it] > ``` > > Last time I waited about 48 hours, then the service was suddenly available again.
Author
Owner

@Classic298 commented on GitHub (Nov 10, 2025):

do you see anything if you set it to debug?

@Classic298 commented on GitHub (Nov 10, 2025): do you see anything if you set it to debug?
Author
Owner

@Schwenn2002 commented on GitHub (Nov 10, 2025):

No, unfortunately no further information appears in the debug log level either.

@Schwenn2002 commented on GitHub (Nov 10, 2025): No, unfortunately no further information appears in the debug log level either.
Author
Owner

@rgaricano commented on GitHub (Nov 10, 2025):

I think that it's because it's updating the embedd or rerank model/s, it can be slow for the traffic/system or by the hf auth (I had similar problems with the hugginface login client token, huggingface-cli login),

If you already have it downloaded and working you can bypass it setting env var RAG_EMBEDDING_MODEL_AUTO_UPDATE="false" or env var OFFLINE_MODE="true"
(There are other env var for that: HF_HUB_OFFLINE but not sure if it is used now, I can find it in the code)

try it & if needed enable it in other moment.

@rgaricano commented on GitHub (Nov 10, 2025): I think that it's because it's updating the embedd or rerank model/s, it can be slow for the traffic/system or by the hf auth (I had similar problems with the hugginface login client token, `huggingface-cli login`), If you already have it downloaded and working you can bypass it setting env var `RAG_EMBEDDING_MODEL_AUTO_UPDATE="false"` or env var `OFFLINE_MODE="true"` (There are other env var for that: `HF_HUB_OFFLINE` but not sure if it is used now, I can find it in the code) try it & if needed enable it in other moment.
Author
Owner

@Schwenn2002 commented on GitHub (Nov 10, 2025):

Yes, those could be the files being downloaded (fetching files). But according to the log, that was already done successfully. What happens during the subsequent startup? It's been stuck there for 17 hours now, and there's no further log entry.

@Schwenn2002 commented on GitHub (Nov 10, 2025): Yes, those could be the files being downloaded (fetching files). But according to the log, that was already done successfully. What happens during the subsequent startup? It's been stuck there for 17 hours now, and there's no further log entry.
Author
Owner

@rgaricano commented on GitHub (Nov 10, 2025):

If auto update enabled, on start the HuggingFace model loading functions checks if a newer version exists on their servers (comparing commit hashes in the model repository to determine if an update is needed, if the remote commit is newer than the local cache, it downloads the updated files)

For Ollama models is similar, but manual & it's compared the model digest (SHA256 hash), if the digests differ the updated layers are downloaded.

@rgaricano commented on GitHub (Nov 10, 2025): If auto update enabled, on start the HuggingFace model loading functions checks if a newer version exists on their servers (comparing commit hashes in the model repository to determine if an update is needed, if the remote commit is newer than the local cache, it downloads the updated files) For Ollama models is similar, but manual & it's compared the model digest (SHA256 hash), if the digests differ the updated layers are downloaded.
Author
Owner

@Schwenn2002 commented on GitHub (Nov 10, 2025):

Issue: System freeze during CUDA initialization test with PyTorch

I tested whether CUDA was initialized correctly using the following command in the docker:

python -c "import torch; x = torch.rand(1000, 1000).cuda(); print('GPU computation ok:', (x @ x).mean())"

However, the system completely froze during the CUDA initialization step — there was no output, and I had to force a reboot.

After restarting Ubuntu, the issue was resolved and the same command executed successfully.

It seems that the system hang was related to CUDA initialization or driver state prior to reboot.

@Schwenn2002 commented on GitHub (Nov 10, 2025): **Issue:** System freeze during CUDA initialization test with PyTorch I tested whether CUDA was initialized correctly using the following command in the docker: ```docker exec -it open-webui bash python -c "import torch; x = torch.rand(1000, 1000).cuda(); print('GPU computation ok:', (x @ x).mean())" ``` However, the system completely froze during the CUDA initialization step — there was no output, and I had to force a reboot. After restarting Ubuntu, the issue was resolved and the same command executed successfully. It seems that the system hang was related to CUDA initialization or driver state prior to reboot.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#6880