github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 14:39:31 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #11370] issue: langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error when websearch parses PDFs (for Default and Tika Content Extraction Engines) #31733

New Issue
Closed
opened 2026-04-25 05:38:34 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-25 05:38:34 -05:00
Owner
Copy Link

Originally created by @MikeNatC on GitHub (Mar 7, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/11370

Check Existing Issues

  • I have searched the existing issues and discussions.

Installation Method

Docker

Open WebUI Version

v0.5.20

Ollama Version (if applicable)

v0.5.13

Operating System

Unraid

Browser (if applicable)

Chrome Version 133.0.6943.142

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have checked the browser console logs.
  • I have checked the Docker container logs.
  • I have listed steps to reproduce the bug in detail.

Expected Behavior

When running a query with a web search using SearXNG, Open WebUI should be able to parse information from PDFs found online.

Actual Behavior

When running a query with a web search using SearXNG, the Open WebUI logs show errors such as:

2025-03-07 08:01:01.233 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, skipping due to continue_on_failure=True - {}

Steps to Reproduce

  1. Go to Admin Panel -> Settings -> Documents
  2. Change the Content Extraction Engine to Tika and input the relevant IP of the Apache Tika container.
  3. Change the Embedding Model Engine to Ollama, nomic-embed-text.
  4. Open up the log for Open WebUI.
  5. Open a new chat, toggle the websearch capability and run the query 'What is Section 25A of the Legal Profession Act of Singapore about?' This should invariably involve a websearch that will include PDF files.
  6. Look in the Open WebUI log for the errors starting with 2025-03-07 08:01:01.233 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, skipping due to continue_on_failure=True - {}
  7. Chat completion should say something along the lines of being unable to determine specific details of the query due to a lack of context, which should not be the case if it were able to scrape the data from the PDFs returned from the SearXNG search.

Logs & Screenshots

2025-03-07 08:01:02.369 | WARNING  | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, skipping due to continue_on_failure=True - {}
2025-03-07 08:01:02.671 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, 
        Legal Profession Act 1966 - Singapore Statutes Online
    , Just a moment... web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}
2025-03-07 08:01:02.683 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:854 - collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 already exists - {}
2025-03-07 08:01:02.751 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:858 - deleting existing collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}
2025-03-07 08:01:02.751 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:865 - adding to collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}

2025-03-07 08:01:02.369 | WARNING  | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, skipping due to continue_on_failure=True - {}
2025-03-07 08:01:02.671 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, 
        Legal Profession Act 1966 - Singapore Statutes Online
    , Just a moment... web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}
2025-03-07 08:01:02.683 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:854 - collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 already exists - {}
2025-03-07 08:01:02.751 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:858 - deleting existing collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}
2025-03-07 08:01:02.751 | INFO     | open_webui.routers.retrieval:save_docs_to_vector_db:865 - adding to collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {}

Image

X

Additional Information

Other discussions that appear relevant:

#11171
#4458
#10034

Originally created by @MikeNatC on GitHub (Mar 7, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/11370 ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Installation Method Docker ### Open WebUI Version v0.5.20 ### Ollama Version (if applicable) v0.5.13 ### Operating System Unraid ### Browser (if applicable) Chrome Version 133.0.6943.142 ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have checked the browser console logs. - [x] I have checked the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior When running a query with a web search using SearXNG, Open WebUI should be able to parse information from PDFs found online. ### Actual Behavior When running a query with a web search using SearXNG, the Open WebUI logs show errors such as: `2025-03-07 08:01:01.233 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, skipping due to continue_on_failure=True - {}` ### Steps to Reproduce 1. Go to Admin Panel -> Settings -> Documents 2. Change the Content Extraction Engine to Tika and input the relevant IP of the Apache Tika container. 3. Change the Embedding Model Engine to Ollama, nomic-embed-text. 4. Open up the log for Open WebUI. 5. Open a new chat, toggle the websearch capability and run the query 'What is Section 25A of the Legal Profession Act of Singapore about?' This should invariably involve a websearch that will include PDF files. 6. Look in the Open WebUI log for the errors starting with `2025-03-07 08:01:01.233 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, skipping due to continue_on_failure=True - {}` 7. Chat completion should say something along the lines of being unable to determine specific details of the query due to a lack of context, which should not be the case if it were able to scrape the data from the PDFs returned from the SearXNG search. ### Logs & Screenshots ``` 2025-03-07 08:01:02.369 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, skipping due to continue_on_failure=True - {} 2025-03-07 08:01:02.671 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, Legal Profession Act 1966 - Singapore Statutes Online , Just a moment... web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} 2025-03-07 08:01:02.683 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:854 - collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 already exists - {} 2025-03-07 08:01:02.751 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:858 - deleting existing collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} 2025-03-07 08:01:02.751 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:865 - adding to collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} ``` ``` 2025-03-07 08:01:02.369 | WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, skipping due to continue_on_failure=True - {} 2025-03-07 08:01:02.671 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document https://www.aseanlawassociation.org/wp-content/uploads/2019/11/ALA-SG-legal-system-Part-4.pdf, https://www.sile.edu.sg/pdf/Legal%20Profession%20Act.pdf, Legal Profession Act 1966 - Singapore Statutes Online , Just a moment... web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} 2025-03-07 08:01:02.683 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:854 - collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 already exists - {} 2025-03-07 08:01:02.751 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:858 - deleting existing collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} 2025-03-07 08:01:02.751 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:865 - adding to collection web-search-7844dd3927281fc7524d0df94fd9b672f50e154ad64d4b2adf18 - {} ``` ![Image](https://github.com/user-attachments/assets/fc807fba-e8f9-4275-be8c-e0d263588428) X ### Additional Information Other discussions that appear relevant: [#11171 ](https://github.com/open-webui/open-webui/discussions/11171 ) [#4458 ](https://github.com/open-webui/open-webui/discussions/4458) [#10034 ](https://github.com/open-webui/open-webui/discussions/10034)
GiteaMirror added the bug label 2026-04-25 05:38:34 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#31733
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?