I have searched for any existing and/or related issues.
I have searched for any existing and/or related discussions.
I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
I am using the latest version of Open WebUI.
Installation Method
Docker
Open WebUI Version
v0.8.12
Ollama Version (if applicable)
No response
Operating System
Fedora 43
Browser (if applicable)
No response
Confirmation
I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided every relevant configuration, setting, and environment variable used in my setup.
I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
Start with the initial platform/version/OS and dependencies used,
Specify exact install/launch/configure commands,
List URLs visited, user input (incl. example values/emails/passwords if needed),
Describe all options and toggles enabled or changed,
Include any files or environmental changes,
Identify the expected and actual result at each stage,
Ensure any reasonably skilled user can follow and hit the same issue.
Expected Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and correctly pointed towards a local firecrawl instance, is expected to return the scraped contents of web pages it is asked to fetch. While timeouts would not be impossible, they should not happen frequently and on web pages that are simple enough that a curl call to firecrawl can scrape them in under a second.
Actual Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and pointed towards a local firecrawl instance, will succeed in fetching the contents of some sites, like https://example.com. However, for other sites, like https://hub.docker.com/r/rocm/vllm-dev, causes no response to be provided after 3+ seconds, and the tool call simply ends with an empty result. This is unexpected because a curl call to scrape the same site using firecrawl takes less than 1 second.
Steps to Reproduce
Hardware is a Ryzen AI Max+ 395 system with 128 GB shared memory. OpenWebUI v.0.8.12 installed using docker.
which is able to fetch the page correctly in under 1 second.
Using latest llama.cpp and Qwen3.5-35B-A3B-UD-Q6_K_XL from unsloth, native tool calling and web search enabled. Built in firecrawl tool configured with URL as http://localhost:3002/v1/scrape, api key set to firecrawl-key, concurrent requests left default at 5 (also tested at 1).
Asking the model "I suspect fetch_url is not working right now. Try fetching https://hub.docker.com/r/rocm/vllm-dev to check." causes the model to call the fetch_url tool for the specified page. The tool call then returns nothing and after a few seconds, it times out and provides the model with an empty response.
Logs & Screenshots
There is no activity in the browser console logs wyhen the issue occurs.
Relevant docker logs section:
2026-04-05 02:30:09.973 | ERROR | open_webui.retrieval.web.utils:lazy_load:273 - Error extracting content from URLs: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
Traceback (most recent call last):
File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0x7fb88d898ae0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
│ └ <function Thread.run at 0x7fb88d8987c0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ │ │ └ (<weakref at 0x7fb6eaf907c0; to 'ThreadPoolExecutor' at 0x7fb7090fc950>, <_queue.SimpleQueue object at 0x7fb709133a60>, None,...
│ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ └ <function _worker at 0x7fb88c972e80>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
│ └ <function _WorkItem.run at 0x7fb88c972fc0>
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ │ │ └ ()
│ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ └ functools.partial(<built-in method run of _contextvars.Context object at 0x7fb708586d40>, <function get_content_from_url at 0...
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/app/backend/open_webui/retrieval/utils.py", line 87, in get_content_from_url
docs = loader.load()
│ └ <function BaseLoader.load at 0x7fb7af65fe20>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
File "/usr/local/lib/python3.11/site-packages/langchain_core/document_loaders/base.py", line 43, in load
return list(self.lazy_load())
│ └ <function SafeFireCrawlLoader.lazy_load at 0x7fb7a41879c0>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
File "/app/backend/open_webui/retrieval/web/utils.py", line 250, in lazy_load
result = firecrawl.batch_scrape(
│ └ <bound method FirecrawlClient.batch_scrape of <firecrawl.v2.client.FirecrawlClient object at 0x7fb6eafedc50>>
└ <firecrawl.client.Firecrawl object at 0x7fb6eafee290>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/client.py", line 1146, in batch_scrape
return batch_module.batch_scrape(
│ └ <function batch_scrape at 0x7fb70875df80>
└ <module 'firecrawl.v2.methods.batch' from '/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py'>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 382, in batch_scrape
return wait_for_batch_completion(
└ <function wait_for_batch_completion at 0x7fb70875dee0>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 327, in wait_for_batch_completion
raise TimeoutError(f"Batch scrape job {job_id} did not complete within {timeout} seconds")
TimeoutError: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
Additional Information
It seems like this PR would have resolved this issue: https://github.com/open-webui/open-webui/pull/23140, specifically the "Replaced inefficient single-URL batch scraping with direct scrape calls while preserving batch processing for true multi-URL workloads..." part.
Originally created by @DanielHe212 on GitHub (Apr 5, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/23411
### Check Existing Issues
- [x] I have searched for any existing and/or related issues.
- [x] I have searched for any existing and/or related discussions.
- [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
- [x] I am using the latest version of Open WebUI.
### Installation Method
Docker
### Open WebUI Version
v0.8.12
### Ollama Version (if applicable)
_No response_
### Operating System
Fedora 43
### Browser (if applicable)
_No response_
### Confirmation
- [x] I have read and followed all instructions in `README.md`.
- [x] I am using the latest version of **both** Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
- [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.**
- [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
- [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps:
- Start with the initial platform/version/OS and dependencies used,
- Specify exact install/launch/configure commands,
- List URLs visited, user input (incl. example values/emails/passwords if needed),
- Describe all options and toggles enabled or changed,
- Include any files or environmental changes,
- Identify the expected and actual result at each stage,
- Ensure any reasonably skilled user can follow and hit the same issue.
### Expected Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and correctly pointed towards a local firecrawl instance, is expected to return the scraped contents of web pages it is asked to fetch. While timeouts would not be impossible, they should not happen frequently and on web pages that are simple enough that a curl call to firecrawl can scrape them in under a second.
### Actual Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and pointed towards a local firecrawl instance, will succeed in fetching the contents of some sites, like https://example.com. However, for other sites, like https://hub.docker.com/r/rocm/vllm-dev, causes no response to be provided after 3+ seconds, and the tool call simply ends with an empty result. This is unexpected because a curl call to scrape the same site using firecrawl takes less than 1 second.
### Steps to Reproduce
Hardware is a Ryzen AI Max+ 395 system with 128 GB shared memory. OpenWebUI v.0.8.12 installed using docker.
I have local firecrawl set up in accordance with the instructions from https://docs.firecrawl.dev/contributing/self-host, key set as "firecrawl-key" for testing, confirmed to be working by using:
curl -X POST http://localhost:3002/v1/scrape \
-H "Content-Type: application/json" \
-H "Authorization: Bearer firecrawl-key" \
-d '{"url": "https://hub.docker.com/r/rocm/vllm-dev"}'
which is able to fetch the page correctly in under 1 second.
Using latest llama.cpp and Qwen3.5-35B-A3B-UD-Q6_K_XL from unsloth, native tool calling and web search enabled. Built in firecrawl tool configured with URL as http://localhost:3002/v1/scrape, api key set to firecrawl-key, concurrent requests left default at 5 (also tested at 1).
Asking the model "I suspect fetch_url is not working right now. Try fetching https://hub.docker.com/r/rocm/vllm-dev to check." causes the model to call the fetch_url tool for the specified page. The tool call then returns nothing and after a few seconds, it times out and provides the model with an empty response.
### Logs & Screenshots
There is no activity in the browser console logs wyhen the issue occurs.
Relevant docker logs section:
2026-04-05 02:30:09.973 | ERROR | open_webui.retrieval.web.utils:lazy_load:273 - Error extracting content from URLs: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
Traceback (most recent call last):
File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0x7fb88d898ae0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
│ └ <function Thread.run at 0x7fb88d8987c0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ │ │ └ (<weakref at 0x7fb6eaf907c0; to 'ThreadPoolExecutor' at 0x7fb7090fc950>, <_queue.SimpleQueue object at 0x7fb709133a60>, None,...
│ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ └ <function _worker at 0x7fb88c972e80>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
│ └ <function _WorkItem.run at 0x7fb88c972fc0>
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ │ │ └ ()
│ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ └ functools.partial(<built-in method run of _contextvars.Context object at 0x7fb708586d40>, <function get_content_from_url at 0...
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/app/backend/open_webui/retrieval/utils.py", line 87, in get_content_from_url
docs = loader.load()
│ └ <function BaseLoader.load at 0x7fb7af65fe20>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
File "/usr/local/lib/python3.11/site-packages/langchain_core/document_loaders/base.py", line 43, in load
return list(self.lazy_load())
│ └ <function SafeFireCrawlLoader.lazy_load at 0x7fb7a41879c0>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
> File "/app/backend/open_webui/retrieval/web/utils.py", line 250, in lazy_load
result = firecrawl.batch_scrape(
│ └ <bound method FirecrawlClient.batch_scrape of <firecrawl.v2.client.FirecrawlClient object at 0x7fb6eafedc50>>
└ <firecrawl.client.Firecrawl object at 0x7fb6eafee290>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/client.py", line 1146, in batch_scrape
return batch_module.batch_scrape(
│ └ <function batch_scrape at 0x7fb70875df80>
└ <module 'firecrawl.v2.methods.batch' from '/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py'>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 382, in batch_scrape
return wait_for_batch_completion(
└ <function wait_for_batch_completion at 0x7fb70875dee0>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 327, in wait_for_batch_completion
raise TimeoutError(f"Batch scrape job {job_id} did not complete within {timeout} seconds")
TimeoutError: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
<img width="1746" height="903" alt="Image" src="https://github.com/user-attachments/assets/1ed43c83-6c40-4a9d-9202-3423a956de44" />
### Additional Information
It seems like this PR would have resolved this issue: https://github.com/open-webui/open-webui/pull/23140, specifically the "Replaced inefficient single-URL batch scraping with direct scrape calls while preserving batch processing for true multi-URL workloads..." part.
GiteaMirror
added the bug label 2026-05-05 23:37:16 -05:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @DanielHe212 on GitHub (Apr 5, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/23411
Check Existing Issues
Installation Method
Docker
Open WebUI Version
v0.8.12
Ollama Version (if applicable)
No response
Operating System
Fedora 43
Browser (if applicable)
No response
Confirmation
README.md.Expected Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and correctly pointed towards a local firecrawl instance, is expected to return the scraped contents of web pages it is asked to fetch. While timeouts would not be impossible, they should not happen frequently and on web pages that are simple enough that a curl call to firecrawl can scrape them in under a second.
Actual Behavior
The fetch_url tool, when the Web Loader Engine is configured as firecrawl and pointed towards a local firecrawl instance, will succeed in fetching the contents of some sites, like https://example.com. However, for other sites, like https://hub.docker.com/r/rocm/vllm-dev, causes no response to be provided after 3+ seconds, and the tool call simply ends with an empty result. This is unexpected because a curl call to scrape the same site using firecrawl takes less than 1 second.
Steps to Reproduce
Hardware is a Ryzen AI Max+ 395 system with 128 GB shared memory. OpenWebUI v.0.8.12 installed using docker.
I have local firecrawl set up in accordance with the instructions from https://docs.firecrawl.dev/contributing/self-host, key set as "firecrawl-key" for testing, confirmed to be working by using:
curl -X POST http://localhost:3002/v1/scrape
-H "Content-Type: application/json"
-H "Authorization: Bearer firecrawl-key"
-d '{"url": "https://hub.docker.com/r/rocm/vllm-dev"}'
which is able to fetch the page correctly in under 1 second.
Using latest llama.cpp and Qwen3.5-35B-A3B-UD-Q6_K_XL from unsloth, native tool calling and web search enabled. Built in firecrawl tool configured with URL as http://localhost:3002/v1/scrape, api key set to firecrawl-key, concurrent requests left default at 5 (also tested at 1).
Asking the model "I suspect fetch_url is not working right now. Try fetching https://hub.docker.com/r/rocm/vllm-dev to check." causes the model to call the fetch_url tool for the specified page. The tool call then returns nothing and after a few seconds, it times out and provides the model with an empty response.
Logs & Screenshots
There is no activity in the browser console logs wyhen the issue occurs.
Relevant docker logs section:
2026-04-05 02:30:09.973 | ERROR | open_webui.retrieval.web.utils:lazy_load:273 - Error extracting content from URLs: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
Traceback (most recent call last):
File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0x7fb88d898ae0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
│ └ <function Thread.run at 0x7fb88d8987c0>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ │ │ └ (<weakref at 0x7fb6eaf907c0; to 'ThreadPoolExecutor' at 0x7fb7090fc950>, <_queue.SimpleQueue object at 0x7fb709133a60>, None,...
│ │ └ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
│ └ <function _worker at 0x7fb88c972e80>
└ <Thread(ThreadPoolExecutor-5_4, started 140423334975168)>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
│ └ <function _WorkItem.run at 0x7fb88c972fc0>
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ │ │ └ ()
│ │ └ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
│ └ functools.partial(<built-in method run of _contextvars.Context object at 0x7fb708586d40>, <function get_content_from_url at 0...
└ <concurrent.futures.thread._WorkItem object at 0x7fb6eafee490>
File "/app/backend/open_webui/retrieval/utils.py", line 87, in get_content_from_url
docs = loader.load()
│ └ <function BaseLoader.load at 0x7fb7af65fe20>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
File "/usr/local/lib/python3.11/site-packages/langchain_core/document_loaders/base.py", line 43, in load
return list(self.lazy_load())
│ └ <function SafeFireCrawlLoader.lazy_load at 0x7fb7a41879c0>
└ <open_webui.retrieval.web.utils.SafeFireCrawlLoader object at 0x7fb6eaf6db10>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/client.py", line 1146, in batch_scrape
return batch_module.batch_scrape(
│ └ <function batch_scrape at 0x7fb70875df80>
└ <module 'firecrawl.v2.methods.batch' from '/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py'>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 382, in batch_scrape
return wait_for_batch_completion(
└ <function wait_for_batch_completion at 0x7fb70875dee0>
File "/usr/local/lib/python3.11/site-packages/firecrawl/v2/methods/batch.py", line 327, in wait_for_batch_completion
raise TimeoutError(f"Batch scrape job {job_id} did not complete within {timeout} seconds")
TimeoutError: Batch scrape job 019d5b79-cdfb-764b-8c17-39df4295ea5c did not complete within 3 seconds
Additional Information
It seems like this PR would have resolved this issue: https://github.com/open-webui/open-webui/pull/23140, specifically the "Replaced inefficient single-URL batch scraping with direct scrape calls while preserving batch processing for true multi-URL workloads..." part.
@tjbck commented on GitHub (Apr 13, 2026):
Should be addressed in dev.