github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-15 21:19:39 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #9752] Upgrade duckduckgo-search to enable REQUESTS_CA_BUNDLE cert handling when using custom root CAs #135358

New Issue
Closed
opened 2026-05-25 02:08:53 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-25 02:08:53 -05:00
Owner
Copy Link

Originally created by @btiernay on GitHub (Feb 10, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9752

Feature Request

Upgrade to the latest version duckduckgo-search to avoid issues with certificates described in:
https://github.com/deedy5/duckduckgo_search/issues/257:

webui-1                |   File "/usr/local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 255, in text
webui-1                |     raise DuckDuckGoSearchException(err)
webui-1                | duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html RuntimeError: error sending request for url (https://html.duckduckgo.com/html): client error (Connect)
webui-1                |
webui-1                | Caused by:
webui-1                |     0: client error (Connect)
webui-1                |     1: TLS handshake failed: cert verification failed - self signed certificate in certificate chain [CERTIFICATE_VERIFY_FAILED]
webui-1                |     2: [CERTIFICATE_VERIFY_FAILED]
webui-1                |     3: [CERTIFICATE_VERIFY_FAILED]
webui-1                |
webui-1                | During handling of the above exception, another exception occurred:
webui-1                |
webui-1                | Traceback (most recent call last):
webui-1                |   File "/app/backend/open_webui/utils/middleware.py", line 429, in chat_web_search_handler
webui-1                |     results = await loop.run_in_executor(
webui-1                |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
webui-1                |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
webui-1                |     result = self.fn(*self.args, **self.kwargs)
webui-1                |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
webui-1                |   File "/app/backend/open_webui/utils/middleware.py", line 431, in <lambda>
webui-1                |     lambda: process_web_search(
webui-1                |             ^^^^^^^^^^^^^^^^^^^
webui-1                |   File "/app/backend/open_webui/routers/retrieval.py", line 1274, in process_web_search
webui-1                |     raise HTTPException(
webui-1                | fastapi.exceptions.HTTPException: 400: https://html.duckduckgo.com/html RuntimeError: error sending request for url (https://html.duckduckgo.com/html): client error (Connect)
webui-1                |
webui-1                | Caused by:
webui-1                |     0: client error (Connect)
webui-1                |     1: TLS handshake failed: cert verification failed - self signed certificate in certificate chain [CERTIFICATE_VERIFY_FAILED]
webui-1                |     2: [CERTIFICATE_VERIFY_FAILED]
webui-1                |     3: [CERTIFICATE_VERIFY_FAILED]

Currently we have 7.3.0:
https://github.com/open-webui/open-webui/blob/main/pyproject.toml#L97

This should be updated to 7.3.2 or higher. This will then allow the standard REQUESTS_CA_BUNDLE configuration to propagate to this library.

Version tested:

ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206

Info:

 docker run -it --rm ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206 cat requirements.txt | grep duckduckgo
duckduckgo-search~=7.3.0

docker run -it --rm ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206 cat /usr/local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py | sed -n '235,275p'
            TimeoutException: Inherits from DuckDuckGoSearchException, raised for API request timeouts.
        """
        if backend in ("api", "ecosia"):
            warnings.warn(f"{backend=} is deprecated, using backend='auto'", stacklevel=2)
            backend = "auto"
        backends = ["html", "lite"] if backend == "auto" else [backend]
        shuffle(backends)

        results, err = [], None
        for b in backends:
            try:
                if b == "html":
                    results = self._text_html(keywords, region, timelimit, max_results)
                elif b == "lite":
                    results = self._text_lite(keywords, region, timelimit, max_results)
                return results
            except Exception as ex:
                logger.info(f"Error to search using {b} backend: {ex}")
                err = ex

        raise DuckDuckGoSearchException(err)

    def _text_html(
        self,
        keywords: str,
        region: str = "wt-wt",
        timelimit: str | None = None,
        max_results: int | None = None,
    ) -> list[dict[str, str]]:
        assert keywords, "keywords is mandatory"

        payload = {
            "q": keywords,
            "s": "0",
            "o": "json",
            "api": "d.js",
            "vqd": "",
            "kl": region,
            "bing_market": region,
        }
        if timelimit:

Note, this line is not present:
https://github.com/deedy5/duckduckgo_search/pull/278/files#diff-4abf7bf906b175e10dee218af6b116e6967c91a6798a9b519d91175e4834abfbR73

Originally created by @btiernay on GitHub (Feb 10, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/9752 # Feature Request Upgrade to the latest version `duckduckgo-search` to avoid issues with certificates described in: https://github.com/deedy5/duckduckgo_search/issues/257: ``` webui-1 | File "/usr/local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 255, in text webui-1 | raise DuckDuckGoSearchException(err) webui-1 | duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html RuntimeError: error sending request for url (https://html.duckduckgo.com/html): client error (Connect) webui-1 | webui-1 | Caused by: webui-1 | 0: client error (Connect) webui-1 | 1: TLS handshake failed: cert verification failed - self signed certificate in certificate chain [CERTIFICATE_VERIFY_FAILED] webui-1 | 2: [CERTIFICATE_VERIFY_FAILED] webui-1 | 3: [CERTIFICATE_VERIFY_FAILED] webui-1 | webui-1 | During handling of the above exception, another exception occurred: webui-1 | webui-1 | Traceback (most recent call last): webui-1 | File "/app/backend/open_webui/utils/middleware.py", line 429, in chat_web_search_handler webui-1 | results = await loop.run_in_executor( webui-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ webui-1 | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run webui-1 | result = self.fn(*self.args, **self.kwargs) webui-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ webui-1 | File "/app/backend/open_webui/utils/middleware.py", line 431, in <lambda> webui-1 | lambda: process_web_search( webui-1 | ^^^^^^^^^^^^^^^^^^^ webui-1 | File "/app/backend/open_webui/routers/retrieval.py", line 1274, in process_web_search webui-1 | raise HTTPException( webui-1 | fastapi.exceptions.HTTPException: 400: https://html.duckduckgo.com/html RuntimeError: error sending request for url (https://html.duckduckgo.com/html): client error (Connect) webui-1 | webui-1 | Caused by: webui-1 | 0: client error (Connect) webui-1 | 1: TLS handshake failed: cert verification failed - self signed certificate in certificate chain [CERTIFICATE_VERIFY_FAILED] webui-1 | 2: [CERTIFICATE_VERIFY_FAILED] webui-1 | 3: [CERTIFICATE_VERIFY_FAILED] ``` Currently we have 7.3.0: https://github.com/open-webui/open-webui/blob/main/pyproject.toml#L97 This should be updated to 7.3.2 or higher. This will then allow the standard `REQUESTS_CA_BUNDLE` configuration to propagate to this library. Version tested: ``` ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206 ``` Info: ``` docker run -it --rm ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206 cat requirements.txt | grep duckduckgo duckduckgo-search~=7.3.0 ``` ``` docker run -it --rm ghcr.io/open-webui/open-webui@sha256:fbe4e8219e0598d05a6c67340b573a9bbb7d6de493baecc5d8d1f2d75b43f206 cat /usr/local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py | sed -n '235,275p' TimeoutException: Inherits from DuckDuckGoSearchException, raised for API request timeouts. """ if backend in ("api", "ecosia"): warnings.warn(f"{backend=} is deprecated, using backend='auto'", stacklevel=2) backend = "auto" backends = ["html", "lite"] if backend == "auto" else [backend] shuffle(backends) results, err = [], None for b in backends: try: if b == "html": results = self._text_html(keywords, region, timelimit, max_results) elif b == "lite": results = self._text_lite(keywords, region, timelimit, max_results) return results except Exception as ex: logger.info(f"Error to search using {b} backend: {ex}") err = ex raise DuckDuckGoSearchException(err) def _text_html( self, keywords: str, region: str = "wt-wt", timelimit: str | None = None, max_results: int | None = None, ) -> list[dict[str, str]]: assert keywords, "keywords is mandatory" payload = { "q": keywords, "s": "0", "o": "json", "api": "d.js", "vqd": "", "kl": region, "bing_market": region, } if timelimit: ``` Note, this line is not present: https://github.com/deedy5/duckduckgo_search/pull/278/files#diff-4abf7bf906b175e10dee218af6b116e6967c91a6798a9b519d91175e4834abfbR73
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#135358
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?