github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-05-06 19:08:59 -05:00
Code Issues 1.3k Packages Projects Releases 126 Wiki Activity

[GH-ISSUE #12175] issue: Failing SSL verification on search retrieval blocks OpenWebUI until (very long) timeouts happen #32023

New Issue
Closed
opened 2026-04-25 05:54:40 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-25 05:54:40 -05:00
Owner
Copy Link

Originally created by @Ithanil on GitHub (Mar 29, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12175

Check Existing Issues

  • I have searched the existing issues and discussions.
  • I am using the latest version of Open WebUI.

Installation Method

Other

Open WebUI Version

dev (few commits off from current)

Ollama Version (if applicable)

No response

Operating System

.

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have listed steps to reproduce the bug in detail.

Expected Behavior

  1. OpenWebUI should not be blocked by something like SSL verification attempts
  2. The bypass SSL verification setting should do what it says and not the opposite

Actual Behavior

At least if using Tavily Search AND extract (but I think it doesn't matter) and having the OpenWebUI in a private network with restrictive web proxy, but having bypass SSL verification enabled in the UI, the web loader will synchronously attempt to verify the SSL certificates of returned websites and wait for timeout on every single page. This completely blocks OpenWebUI for two minutes per result page.

So, there are multiple issues here:

  1. The SSL verification happens synchronously and completely blocks OpenWebUI
  2. The 2 minute timeout on SSL verification is excessive (I think it's just using some default)
  3. The "bypass SSL verification" setting in the UI is mislabeled and should be named "enable SSL verification".

Steps to Reproduce

  1. enable "bypass SSL verification" (i.e. enable SSL verification)
  2. make sure it won't work, e.g. by using tavily extract and only allowing tavily API in the proxy
  3. perform web search

Logs & Screenshots

2025-03-29 09:32:00.893 | DEBUG    | open_webui.retrieval.web.utils:get_web_loader:628 - Using RAG_WEB_LOADER_ENGINE SafeTavilyLoader for 5 URLs - {}
2025-03-29 09:34:11.146 | WARNING  | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.usatoday.com/: [Errno 110] Connection timed out - {}
2025-03-29 09:34:11.174 | WARNING  | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.usatoday.com/: SSL certificate verification failed for https://www.usatoday.com/ - {}
2025-03-29 09:36:22.168 | WARNING  | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.foxnews.com/: [Errno 110] Connection timed out - {}
2025-03-29 09:36:22.172 | WARNING  | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.foxnews.com/: SSL certificate verification failed for https://www.foxnews.com/ - {}
2025-03-29 09:38:33.240 | WARNING  | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.nbcnews.com/: [Errno 110] Connection timed out - {}
2025-03-29 09:38:33.242 | WARNING  | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.nbcnews.com/: SSL certificate verification failed for https://www.nbcnews.com/ - {}
2025-03-29 09:40:44.312 | WARNING  | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.cnn.com/: [Errno 110] Connection timed out - {}
2025-03-29 09:40:44.314 | WARNING  | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.cnn.com/: SSL certificate verification failed for https://www.cnn.com/ - {}
2025-03-29 09:42:55.384 | WARNING  | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://apnews.com/: [Errno 110] Connection timed out - {}
2025-03-29 09:42:55.387 | WARNING  | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://apnews.com/: SSL certificate verification failed for https://apnews.com/ - {}
2025-03-29 09:42:55.387 | WARNING  | open_webui.retrieval.web.utils:alazy_load:348 - No valid URLs to process after SSL verification - {}
2025-03-29 09:42:55.423 | ERROR    | open_webui.socket.main:periodic_usage_pool_cleanup:96 - Unable to renew cleanup lock. Exiting usage pool cleanup. - {}
2025-03-29 09:42:55.426 | ERROR    | asyncio.runners:run:118 - Task exception was never retrieved
future: <Task finished name='Task-3' coro=<periodic_usage_pool_cleanup() done, defined at /app/backend/open_webui/socket/main.py:88> exception=Exception('Unable to renew usage pool cleanup lock.')> - {}
Traceback (most recent call last):

> File "/app/backend/open_webui/socket/main.py", line 97, in periodic_usage_pool_cleanup
    raise Exception("Unable to renew usage pool cleanup lock.")

Exception: Unable to renew usage pool cleanup lock.

Additional Information

Regarding the bypass setting, see current UI code:
See the current UI code:

                                        <div class="  mb-2.5 flex w-full justify-between">
                                                <div class=" self-center text-xs font-medium">
                                                        {$i18n.t('Bypass SSL verification for Websites')}
                                                </div>
                                                <div class="flex items-center relative">
                                                        <Switch bind:state={webConfig.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION} />
                                                </div>
                                        </div>
Originally created by @Ithanil on GitHub (Mar 29, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/12175 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Other ### Open WebUI Version dev (few commits off from current) ### Ollama Version (if applicable) _No response_ ### Operating System . ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior 1) OpenWebUI should not be blocked by something like SSL verification attempts 2) The bypass SSL verification setting should do what it says and not the opposite ### Actual Behavior At least if using Tavily Search AND extract (but I think it doesn't matter) and having the OpenWebUI in a private network with restrictive web proxy, but having **bypass** SSL verification enabled in the UI, the web loader *will* synchronously attempt to verify the SSL certificates of returned websites and wait for timeout on every single page. This completely blocks OpenWebUI for two minutes per result page. So, there are multiple issues here: 1) The SSL verification happens synchronously and completely blocks OpenWebUI 2) The 2 minute timeout on SSL verification is excessive (I think it's just using some default) 3) The "bypass SSL verification" setting in the UI is mislabeled and should be named "enable SSL verification". ### Steps to Reproduce 1) enable "bypass SSL verification" (i.e. enable SSL verification) 2) make sure it won't work, e.g. by using tavily extract and only allowing tavily API in the proxy 3) perform web search ### Logs & Screenshots ``` 2025-03-29 09:32:00.893 | DEBUG | open_webui.retrieval.web.utils:get_web_loader:628 - Using RAG_WEB_LOADER_ENGINE SafeTavilyLoader for 5 URLs - {} 2025-03-29 09:34:11.146 | WARNING | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.usatoday.com/: [Errno 110] Connection timed out - {} 2025-03-29 09:34:11.174 | WARNING | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.usatoday.com/: SSL certificate verification failed for https://www.usatoday.com/ - {} 2025-03-29 09:36:22.168 | WARNING | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.foxnews.com/: [Errno 110] Connection timed out - {} 2025-03-29 09:36:22.172 | WARNING | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.foxnews.com/: SSL certificate verification failed for https://www.foxnews.com/ - {} 2025-03-29 09:38:33.240 | WARNING | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.nbcnews.com/: [Errno 110] Connection timed out - {} 2025-03-29 09:38:33.242 | WARNING | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.nbcnews.com/: SSL certificate verification failed for https://www.nbcnews.com/ - {} 2025-03-29 09:40:44.312 | WARNING | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://www.cnn.com/: [Errno 110] Connection timed out - {} 2025-03-29 09:40:44.314 | WARNING | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://www.cnn.com/: SSL certificate verification failed for https://www.cnn.com/ - {} 2025-03-29 09:42:55.384 | WARNING | open_webui.retrieval.web.utils:verify_ssl_cert:116 - SSL verification failed for https://apnews.com/: [Errno 110] Connection timed out - {} 2025-03-29 09:42:55.387 | WARNING | open_webui.retrieval.web.utils:alazy_load:342 - SSL verification failed for https://apnews.com/: SSL certificate verification failed for https://apnews.com/ - {} 2025-03-29 09:42:55.387 | WARNING | open_webui.retrieval.web.utils:alazy_load:348 - No valid URLs to process after SSL verification - {} 2025-03-29 09:42:55.423 | ERROR | open_webui.socket.main:periodic_usage_pool_cleanup:96 - Unable to renew cleanup lock. Exiting usage pool cleanup. - {} 2025-03-29 09:42:55.426 | ERROR | asyncio.runners:run:118 - Task exception was never retrieved future: <Task finished name='Task-3' coro=<periodic_usage_pool_cleanup() done, defined at /app/backend/open_webui/socket/main.py:88> exception=Exception('Unable to renew usage pool cleanup lock.')> - {} Traceback (most recent call last): > File "/app/backend/open_webui/socket/main.py", line 97, in periodic_usage_pool_cleanup raise Exception("Unable to renew usage pool cleanup lock.") Exception: Unable to renew usage pool cleanup lock. ``` ### Additional Information Regarding the bypass setting, see current UI code: See the current UI code: ``` <div class=" mb-2.5 flex w-full justify-between"> <div class=" self-center text-xs font-medium"> {$i18n.t('Bypass SSL verification for Websites')} </div> <div class="flex items-center relative"> <Switch bind:state={webConfig.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION} /> </div> </div> ```
GiteaMirror added the bug label 2026-04-25 05:54:40 -05:00
GiteaMirror commented 2026-04-25 05:54:42 -05:00
Author
Owner
Copy Link

@rgaricano commented on GitHub (Mar 29, 2025):

@@ -28,6 +28,7 @@ from open_webui.retrieval.loaders.tavily import TavilyLoader
from open_webui.constants import ERROR_MESSAGES
from open_webui.config import (
    ENABLE_RAG_LOCAL_WEB_FETCH,
    ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
    PLAYWRIGHT_WS_URI,
    PLAYWRIGHT_TIMEOUT,
    RAG_WEB_LOADER_ENGINE,
@@ -101,7 +102,7 @@ def extract_metadata(soup, url):

def verify_ssl_cert(url: str) -> bool:
    """Verify SSL certificate for the given URL."""
    if not url.startswith("https://"):
    if not url.startswith("https://") or not ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION:
        return True

    try:
<!-- gh-comment-id:2763368508 --> @rgaricano commented on GitHub (Mar 29, 2025): ``` @@ -28,6 +28,7 @@ from open_webui.retrieval.loaders.tavily import TavilyLoader from open_webui.constants import ERROR_MESSAGES from open_webui.config import ( ENABLE_RAG_LOCAL_WEB_FETCH, ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, PLAYWRIGHT_WS_URI, PLAYWRIGHT_TIMEOUT, RAG_WEB_LOADER_ENGINE, @@ -101,7 +102,7 @@ def extract_metadata(soup, url): def verify_ssl_cert(url: str) -> bool: """Verify SSL certificate for the given URL.""" if not url.startswith("https://"): if not url.startswith("https://") or not ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION: return True try: ```
GiteaMirror commented 2026-04-25 05:54:43 -05:00
Author
Owner
Copy Link

@rgaricano commented on GitHub (Mar 29, 2025):

if you want try this fix, i upload to my fork for test before PR: 9ad8182f7d & 629ea4d0eb (lost comma)

(I also had periodic_usage_pool_cleanup error in other circumstances, i think it is not directly related but I would like to know when this error occurs.)

<!-- gh-comment-id:2763379084 --> @rgaricano commented on GitHub (Mar 29, 2025): if you want try this fix, i upload to my fork for test before PR: https://github.com/rgaricano/open-webui/commit/9ad8182f7dedd76a611d75a0a320911fcd6969a1 & https://github.com/rgaricano/open-webui/commit/629ea4d0eb9b43993a1a239c6d60aa639a6154c6 (lost comma) (I also had `periodic_usage_pool_cleanup` error in other circumstances, i think it is not directly related but I would like to know when this error occurs.)
GiteaMirror commented 2026-04-25 05:54:43 -05:00
Author
Owner
Copy Link

@Ithanil commented on GitHub (Mar 29, 2025):

if you want try this fix, i upload to my fork for test before PR: rgaricano@9ad8182

(I also had periodic_usage_pool_cleanup error in other circumstances, i think it is not directly related but I would like to know when this error occurs.)

Appreciate your immediate effort, but I won't be able to test it today. But could tomorrow, if still needed.

I would assume the cleanup thing is actually caused by the long blockage. I don't see it in my prod/staging logs otherwise, except for situations where Redis was inaccessible. But I don't have enough understanding of the mechanism to really know.

<!-- gh-comment-id:2763387894 --> @Ithanil commented on GitHub (Mar 29, 2025): > if you want try this fix, i upload to my fork for test before PR: [rgaricano@9ad8182](https://github.com/rgaricano/open-webui/commit/9ad8182f7dedd76a611d75a0a320911fcd6969a1) > > (I also had `periodic_usage_pool_cleanup` error in other circumstances, i think it is not directly related but I would like to know when this error occurs.) Appreciate your immediate effort, but I won't be able to test it today. But could tomorrow, if still needed. I would assume the cleanup thing is actually caused by the long blockage. I don't see it in my prod/staging logs otherwise, except for situations where Redis was inaccessible. But I don't have enough understanding of the mechanism to really know.
GiteaMirror commented 2026-04-25 05:54:44 -05:00
Author
Owner
Copy Link

@rgaricano commented on GitHub (Mar 29, 2025):

( I lost a comma on commit i recommit & updated message )

<!-- gh-comment-id:2764177671 --> @rgaricano commented on GitHub (Mar 29, 2025): ( I lost a comma on commit i recommit & updated message )
GiteaMirror commented 2026-04-25 05:54:45 -05:00
Author
Owner
Copy Link

@Ithanil commented on GitHub (Mar 29, 2025):

OK, I looked at your commits and I'm afraid this seems not helpful. ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION already comes into play at this point: 08ff1d8d88/backend/open_webui/routers/retrieval.py (L1194) and has the desired effect, just the label on the corresponding UI switch is inverted ("bypass" instead of "enable").

Besides the UI problem the actual main issues are:

  1. The SSL verification happens synchronously and completely blocks OpenWebUI (that's the big one)
  2. The 2 minute timeout on SSL verification
<!-- gh-comment-id:2764264579 --> @Ithanil commented on GitHub (Mar 29, 2025): OK, I looked at your commits and I'm afraid this seems not helpful. ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION already comes into play at this point: https://github.com/open-webui/open-webui/blob/08ff1d8d8883fdb43142b6c86b1b01ca675bd9be/backend/open_webui/routers/retrieval.py#L1194 and has the desired effect, just the label on the corresponding UI switch is inverted ("bypass" instead of "enable"). Besides the UI problem the actual main issues are: 1) The SSL verification happens synchronously and completely blocks OpenWebUI (that's the big one) 2) The 2 minute timeout on SSL verification
GiteaMirror commented 2026-04-25 05:54:46 -05:00
Author
Owner
Copy Link

@Lawlietr commented on GitHub (Mar 30, 2025):

Same issues here, i don't know why this issue happened today,
Could this problem be related to Ollama? Because I noticed that Ollama had an update yesterday.

<!-- gh-comment-id:2764427101 --> @Lawlietr commented on GitHub (Mar 30, 2025): Same issues here, i don't know why this issue happened today, Could this problem be related to Ollama? Because I noticed that Ollama had an update yesterday.
GiteaMirror commented 2026-04-25 05:54:48 -05:00
Author
Owner
Copy Link

@Ithanil commented on GitHub (Mar 30, 2025):

No, this is not related to Ollama.

<!-- gh-comment-id:2764434162 --> @Ithanil commented on GitHub (Mar 30, 2025): No, this is not related to Ollama.
GiteaMirror commented 2026-04-25 05:54:49 -05:00
Author
Owner
Copy Link

@Xi-Gong commented on GitHub (Apr 4, 2025):

Same issues with tavily used, temporary closed "Bypass SSL verification for website".

<!-- gh-comment-id:2777326121 --> @Xi-Gong commented on GitHub (Apr 4, 2025): Same issues with tavily used, temporary closed "Bypass SSL verification for website".
GiteaMirror commented 2026-04-25 05:54:50 -05:00
Author
Owner
Copy Link

@Ithanil commented on GitHub (Apr 13, 2025):

Same issues with tavily used, temporary closed "Bypass SSL verification for website".

Just FYI, "Bypass SSL verification" meant the opposite of what it was doing before some recent changes.

Now, you need to first change web loader to "default" in the UI, disable verify SSL there, and then switch back to your desired loader (e.g. tavily).

<!-- gh-comment-id:2799857557 --> @Ithanil commented on GitHub (Apr 13, 2025): > Same issues with tavily used, temporary closed "Bypass SSL verification for website". Just FYI, "Bypass SSL verification" meant the opposite of what it was doing before some recent changes. Now, you need to first change web loader to "default" in the UI, disable verify SSL there, and then switch back to your desired loader (e.g. tavily).
GiteaMirror commented 2026-04-25 05:54:50 -05:00
Author
Owner
Copy Link

@Xi-Gong commented on GitHub (Apr 13, 2025):

Thanks, It's good to see serveral web loader choice, I'm currently using tavily extract to complete solve extract problems.

<!-- gh-comment-id:2799861458 --> @Xi-Gong commented on GitHub (Apr 13, 2025): Thanks, It's good to see serveral web loader choice, I'm currently using tavily extract to complete solve extract problems.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#32023
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?