I can't reproduce yet, is the collection empty? @DrZoidberg09 Please provide the log earlier, the error does not match the code in my opinion.
There we have an issue @tjbck This is the output when querying a collection without any documents in it. I think this be handled early, maybe before even calling query_doc_with_hybrid_search()?
open-webui | 2025-04-03 07:16:43.989 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
open-webui | 2025-04-03 07:16:46.120 | ERROR | open_webui.retrieval.utils:query_collection_with_hybrid_search:299 - Failed to fetch collection 26268380-14ee-4c1a-bd25-c790c8fd3e16: Collection 26268380-14ee-4c1a-bd25-c790c8fd3e16 does not exist. - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | │ │ │ └ (<weakref at 0x76700dde31a0; to 'ThreadPoolExecutor' at 0x76700dd206d0>, <_queue.SimpleQueue object at 0x76700f041df0>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | │ │ │ └ ()
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | │ └ <function chat_completion_files_handler.<locals>.<lambda> at 0x76700e76e2a0>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui |
open-webui | File "/app/backend/open_webui/utils/middleware.py", line 594, in <lambda>
open-webui | lambda: get_sources_from_files(
open-webui | └ <function get_sources_from_files at 0x7670838236a0>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 501, in get_sources_from_files
open-webui | context = query_collection_with_hybrid_search(
open-webui | └ <function query_collection_with_hybrid_search at 0x767083822ac0>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 295, in query_collection_with_hybrid_search
open-webui | collection_results[collection_name] = VECTOR_DB_CLIENT.get(
open-webui | │ │ │ └ <function ChromaClient.get at 0x76708dfd8ea0>
open-webui | │ │ └ <open_webui.retrieval.vector.dbs.chroma.ChromaClient object at 0x76708dfce950>
open-webui | │ └ '26268380-14ee-4c1a-bd25-c790c8fd3e16'
open-webui | └ {}
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/vector/dbs/chroma.py", line 121, in get
open-webui | collection = self.client.get_collection(name=collection_name)
open-webui | │ │ │ └ '26268380-14ee-4c1a-bd25-c790c8fd3e16'
open-webui | │ │ └ <function Client.get_collection at 0x76708ebac2c0>
open-webui | │ └ <chromadb.api.client.Client object at 0x76708df6c750>
open-webui | └ <open_webui.retrieval.vector.dbs.chroma.ChromaClient object at 0x76708dfce950>
open-webui |
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/client.py", line 171, in get_collection
open-webui | model = self._server.get_collection(
open-webui | │ │ └ <function SegmentAPI.get_collection at 0x76708dbb40e0>
open-webui | │ └ <chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>
open-webui | └ <chromadb.api.client.Client object at 0x76708df6c750>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 150, in wrapper
open-webui | return f(*args, **kwargs)
open-webui | │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | └ <function SegmentAPI.get_collection at 0x76708dbb4040>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/segment.py", line 103, in wrapper
open-webui | return self._rate_limit_enforcer.rate_limit(func)(*args, **kwargs)
open-webui | │ │ │ │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ │ │ │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | │ │ │ └ <function SegmentAPI.get_collection at 0x76708dba3f60>
open-webui | │ │ └ <function SimpleRateLimitEnforcer.rate_limit at 0x76708d99d300>
open-webui | │ └ <chromadb.rate_limit.simple_rate_limit.SimpleRateLimitEnforcer object at 0x76708e0ee310>
open-webui | └ <chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/rate_limit/simple_rate_limit/__init__.py", line 23, in wrapper
open-webui | return func(*args, **kwargs)
open-webui | │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | └ <function SegmentAPI.get_collection at 0x76708dba3f60>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/segment.py", line 300, in get_collection
open-webui | raise InvalidCollectionException(f"Collection {name} does not exist.")
open-webui | └ <class 'chromadb.errors.InvalidCollectionException'>
open-webui |
open-webui | chromadb.errors.InvalidCollectionException: Collection 26268380-14ee-4c1a-bd25-c790c8fd3e16 does not exist.
open-webui | 2025-04-03 07:16:46.123 | INFO | open_webui.retrieval.utils:query_collection_with_hybrid_search:302 - Starting hybrid search for 3 queries in 1 collections... - {}
open-webui | 2025-04-03 07:16:46.124 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Was steht drin?')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
open-webui | 2025-04-03 07:16:46.125 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Aktuelle Informationen zu Was steht drin?')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
open-webui | 2025-04-03 07:16:46.126 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Was steht drin? 2025')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
Batches: 100%|██████████| 1/1 [00:00<00:00, 77.42it/s]
Batches: 100%|██████████| 1/1 [00:00<00:00, 77.52it/s]
Batches: 100%|██████████| 1/1 [00:00<00:00, 92.58it/s]
open-webui | 2025-04-03 07:16:46.187 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "POST /api/chat/completions HTTP/1.1" 200 - {}
open-webui | 2025-04-03 07:16:46.226 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
<!-- gh-comment-id:2774723614 -->
@Phlogi commented on GitHub (Apr 3, 2025):
I can't reproduce yet, is the collection empty? @DrZoidberg09 Please provide the log earlier, the error does not match the code in my opinion.
There we have an issue @tjbck This is the output when querying a collection without any documents in it. I think this be handled early, maybe before even calling query_doc_with_hybrid_search()?
```
open-webui | 2025-04-03 07:16:43.989 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
open-webui | 2025-04-03 07:16:46.120 | ERROR | open_webui.retrieval.utils:query_collection_with_hybrid_search:299 - Failed to fetch collection 26268380-14ee-4c1a-bd25-c790c8fd3e16: Collection 26268380-14ee-4c1a-bd25-c790c8fd3e16 does not exist. - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | │ │ │ └ (<weakref at 0x76700dde31a0; to 'ThreadPoolExecutor' at 0x76700dd206d0>, <_queue.SimpleQueue object at 0x76700f041df0>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-5_0, started 130223286781632)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | │ │ │ └ ()
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui | │ └ <function chat_completion_files_handler.<locals>.<lambda> at 0x76700e76e2a0>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e745bd0>
open-webui |
open-webui | File "/app/backend/open_webui/utils/middleware.py", line 594, in <lambda>
open-webui | lambda: get_sources_from_files(
open-webui | └ <function get_sources_from_files at 0x7670838236a0>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 501, in get_sources_from_files
open-webui | context = query_collection_with_hybrid_search(
open-webui | └ <function query_collection_with_hybrid_search at 0x767083822ac0>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 295, in query_collection_with_hybrid_search
open-webui | collection_results[collection_name] = VECTOR_DB_CLIENT.get(
open-webui | │ │ │ └ <function ChromaClient.get at 0x76708dfd8ea0>
open-webui | │ │ └ <open_webui.retrieval.vector.dbs.chroma.ChromaClient object at 0x76708dfce950>
open-webui | │ └ '26268380-14ee-4c1a-bd25-c790c8fd3e16'
open-webui | └ {}
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/vector/dbs/chroma.py", line 121, in get
open-webui | collection = self.client.get_collection(name=collection_name)
open-webui | │ │ │ └ '26268380-14ee-4c1a-bd25-c790c8fd3e16'
open-webui | │ │ └ <function Client.get_collection at 0x76708ebac2c0>
open-webui | │ └ <chromadb.api.client.Client object at 0x76708df6c750>
open-webui | └ <open_webui.retrieval.vector.dbs.chroma.ChromaClient object at 0x76708dfce950>
open-webui |
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/client.py", line 171, in get_collection
open-webui | model = self._server.get_collection(
open-webui | │ │ └ <function SegmentAPI.get_collection at 0x76708dbb40e0>
open-webui | │ └ <chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>
open-webui | └ <chromadb.api.client.Client object at 0x76708df6c750>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 150, in wrapper
open-webui | return f(*args, **kwargs)
open-webui | │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | └ <function SegmentAPI.get_collection at 0x76708dbb4040>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/segment.py", line 103, in wrapper
open-webui | return self._rate_limit_enforcer.rate_limit(func)(*args, **kwargs)
open-webui | │ │ │ │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ │ │ │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | │ │ │ └ <function SegmentAPI.get_collection at 0x76708dba3f60>
open-webui | │ │ └ <function SimpleRateLimitEnforcer.rate_limit at 0x76708d99d300>
open-webui | │ └ <chromadb.rate_limit.simple_rate_limit.SimpleRateLimitEnforcer object at 0x76708e0ee310>
open-webui | └ <chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/rate_limit/simple_rate_limit/__init__.py", line 23, in wrapper
open-webui | return func(*args, **kwargs)
open-webui | │ │ └ {'name': '26268380-14ee-4c1a-bd25-c790c8fd3e16', 'tenant': 'default_tenant', 'database': 'default_database'}
open-webui | │ └ (<chromadb.api.segment.SegmentAPI object at 0x76708dfcf110>,)
open-webui | └ <function SegmentAPI.get_collection at 0x76708dba3f60>
open-webui | File "/usr/local/lib/python3.11/site-packages/chromadb/api/segment.py", line 300, in get_collection
open-webui | raise InvalidCollectionException(f"Collection {name} does not exist.")
open-webui | └ <class 'chromadb.errors.InvalidCollectionException'>
open-webui |
open-webui | chromadb.errors.InvalidCollectionException: Collection 26268380-14ee-4c1a-bd25-c790c8fd3e16 does not exist.
open-webui | 2025-04-03 07:16:46.123 | INFO | open_webui.retrieval.utils:query_collection_with_hybrid_search:302 - Starting hybrid search for 3 queries in 1 collections... - {}
open-webui | 2025-04-03 07:16:46.124 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Was steht drin?')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700e7455d0>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
open-webui | 2025-04-03 07:16:46.125 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Aktuelle Informationen zu Was steht drin?')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4e8490>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
open-webui | 2025-04-03 07:16:46.126 | ERROR | open_webui.retrieval.utils:process_query:317 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'documents' - {}
open-webui | Traceback (most recent call last):
open-webui |
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui | self._bootstrap_inner()
open-webui | │ └ <function Thread._bootstrap_inner at 0x767101794860>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui | self.run()
open-webui | │ └ <function Thread.run at 0x767101794540>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui | self._target(*self._args, **self._kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ │ │ └ (<weakref at 0x76700e7b31a0; to 'ThreadPoolExecutor' at 0x76700dd75090>, <_queue.SimpleQueue object at 0x76700fbe7290>, None,...
open-webui | │ │ └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | │ └ <function _worker at 0x7671006b00e0>
open-webui | └ <Thread(ThreadPoolExecutor-6_0, started 130222502446784)>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui | work_item.run()
open-webui | │ └ <function _WorkItem.run at 0x7671006b0220>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui | result = self.fn(*self.args, **self.kwargs)
open-webui | │ │ │ │ │ └ {}
open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | │ │ │ └ ('26268380-14ee-4c1a-bd25-c790c8fd3e16', 'Was steht drin? 2025')
open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7670fdbe4400>
open-webui | └ <concurrent.futures.thread._WorkItem object at 0x76700c4eaa50>
open-webui |
open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 305, in process_query
open-webui | result = query_doc_with_hybrid_search(
open-webui | └ <function query_doc_with_hybrid_search at 0x767083823880>
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui | raise e
open-webui |
open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 120, in query_doc_with_hybrid_search
open-webui | texts=collection_result.documents[0],
open-webui | └ None
open-webui |
open-webui | AttributeError: 'NoneType' object has no attribute 'documents'
Batches: 100%|██████████| 1/1 [00:00<00:00, 77.42it/s]
Batches: 100%|██████████| 1/1 [00:00<00:00, 77.52it/s]
Batches: 100%|██████████| 1/1 [00:00<00:00, 92.58it/s]
open-webui | 2025-04-03 07:16:46.187 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "POST /api/chat/completions HTTP/1.1" 200 - {}
open-webui | 2025-04-03 07:16:46.226 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - 213.200.228.112:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
```
Please also share exactly how run the image and what tag etc. you're using. Also, what are you doing in the app exactly to get this error? I assume searching through a collection, however then some default logging output is missing. So it seems you're not using the exact 0.6.0 version or doing something else that triggers hybrid search.
Coud you try to reproduce it without ROCm, using standard image and CPU only?
<!-- gh-comment-id:2775395275 -->
@Phlogi commented on GitHub (Apr 3, 2025):
Please also share exactly how run the image and what tag etc. you're using. Also, what are you doing in the app exactly to get this error? I assume searching through a collection, however then some default logging output is missing. So it seems you're not using the exact 0.6.0 version or doing something else that triggers hybrid search.
Coud you try to reproduce it without ROCm, using standard image and CPU only?
I just had a look into the Dockerfile. And it seems that sentencetransformer is installed separately (dont know why). I dont have access right now to check, but I think it will install sentencetransformer 4, which heavily refactored the reranking. I will bake a new Docker image without that and will let you know. In best case this will solve the issue and just be a heads up that there need to be adaptions for sentencetransformer 4, if you plan to update it.
<!-- gh-comment-id:2775768643 -->
@DrZoidberg09 commented on GitHub (Apr 3, 2025):
I just had a look into the Dockerfile. And it seems that sentencetransformer is installed separately (dont know why). I dont have access right now to check, but I think it will install sentencetransformer 4, which heavily refactored the reranking. I will bake a new Docker image without that and will let you know. In best case this will solve the issue and just be a heads up that there need to be adaptions for sentencetransformer 4, if you plan to update it.
Hi, for me too with pip local installation, since version 0.6, the hybrid search option switch doesn't even appear anymore. I can't specify a reranker model, the text input file no longer appears.
<!-- gh-comment-id:2775959439 -->
@dromeuf commented on GitHub (Apr 3, 2025):
Hi, for me too with pip local installation, since version 0.6, the hybrid search option switch doesn't even appear anymore. I can't specify a reranker model, the text input file no longer appears.
I think it's because XLM Roberta model that is used,
If i'm not wrong, Roberta work with subwords embeddings and tensor output it's different than expected.
(the output is a tensor with a shape type [1, 4, 505], because LM will output the probability distribution across all the words in the vocabulary, and we expect another shape, like [1,68])
If this is it, I read that there are ways to deal doing token embeddings averages of the subwords to generate an embedding for the whole word....
<!-- gh-comment-id:2776133987 -->
@rgaricano commented on GitHub (Apr 3, 2025):
I think it's because XLM Roberta model that is used,
If i'm not wrong, Roberta work with subwords embeddings and tensor output it's different than expected.
(the output is a tensor with a shape type [1, 4, 505], because LM will output the probability distribution across all the words in the vocabulary, and we expect another shape, like [1,68])
If this is it, I read that there are ways to deal doing token embeddings averages of the subwords to generate an embedding for the whole word....
You error is on flash attn using rocm, dunno if AMD is properly supported:
File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/flash_attn-2.7.4.post1-py3.12.egg/flash_attn/ops/triton/rotary.py", line 166, in apply_rotary
open-webui-rocm | assert sin.shape == cos.shape
<!-- gh-comment-id:2779169471 -->
@bet0x commented on GitHub (Apr 4, 2025):
You error is on flash attn using rocm, dunno if AMD is properly supported:
```
File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/flash_attn-2.7.4.post1-py3.12.egg/flash_attn/ops/triton/rotary.py", line 166, in apply_rotary
open-webui-rocm | assert sin.shape == cos.shape
```
I did use Triton ROCm implementation of flash attention without any issue before the refactoring of the hybrid search (using JinaAI based on XLM Roberta). And all embedding/reranking with models not using flash attention still work fine. Hence, the refactoring somehow has broken something.
<!-- gh-comment-id:2785622787 -->
@DrZoidberg09 commented on GitHub (Apr 8, 2025):
I did use Triton ROCm implementation of flash attention without any issue before the refactoring of the hybrid search (using JinaAI based on XLM Roberta). And all embedding/reranking with models not using flash attention still work fine. Hence, the refactoring somehow has broken something.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @DrZoidberg09 on GitHub (Apr 2, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12347
Check Existing Issues
Installation Method
Docker
Open WebUI Version
0.6.0
Ollama Version (if applicable)
No response
Operating System
Ubuntu 24.04
Browser (if applicable)
No response
Confirmation
README.md.Expected Behavior
I use Openweb-ui for hybrid RAG with ROCm and Flash-Attention based on Triton. This worked until the newest version. Please see docker log.
Actual Behavior
Hybrid search does not work anymore
Steps to Reproduce
Logs & Screenshots
open-webui-rocm | 2025-04-02 13:41:29.465 | ERROR | open_webui.retrieval.utils:process_query:320 - Error when querying the collection with hybrid_search: 'NoneType' object has no attribute 'shape' - {}
open-webui-rocm | Traceback (most recent call last):
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/threading.py", line 1032, in _bootstrap
open-webui-rocm | self._bootstrap_inner()
open-webui-rocm | │ └ <function Thread._bootstrap_inner at 0x7f009fd8d580>
open-webui-rocm | └ <Thread(ThreadPoolExecutor-9_0, started 139616139085504)>
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
open-webui-rocm | self.run()
open-webui-rocm | │ └ <function Thread.run at 0x7f009fd8d260>
open-webui-rocm | └ <Thread(ThreadPoolExecutor-9_0, started 139616139085504)>
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/threading.py", line 1012, in run
open-webui-rocm | self._target(*self._args, **self._kwargs)
open-webui-rocm | │ │ │ │ │ └ {}
open-webui-rocm | │ │ │ │ └ <Thread(ThreadPoolExecutor-9_0, started 139616139085504)>
open-webui-rocm | │ │ │ └ (<weakref at 0x7efc42c29f80; to 'ThreadPoolExecutor' at 0x7efd5bc89e80>, <_queue.SimpleQueue object at 0x7efc42c29a80>, None,...
open-webui-rocm | │ │ └ <Thread(ThreadPoolExecutor-9_0, started 139616139085504)>
open-webui-rocm | │ └ <function _worker at 0x7f009ee21300>
open-webui-rocm | └ <Thread(ThreadPoolExecutor-9_0, started 139616139085504)>
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/concurrent/futures/thread.py", line 93, in _worker
open-webui-rocm | work_item.run()
open-webui-rocm | │ └ <function _WorkItem.run at 0x7f009ee21440>
open-webui-rocm | └ <concurrent.futures.thread._WorkItem object at 0x7efd2dcacd10>
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/concurrent/futures/thread.py", line 59, in run
open-webui-rocm | result = self.fn(*self.args, **self.kwargs)
open-webui-rocm | │ │ │ │ │ └ {}
open-webui-rocm | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x7efd2dcacd10>
open-webui-rocm | │ │ │ └ ('ab37fb86-7213-422c-b37d-01c83b58e6e5', 'nonwoven projects')
open-webui-rocm | │ │ └ <concurrent.futures.thread._WorkItem object at 0x7efd2dcacd10>
open-webui-rocm | │ └ <function query_collection_with_hybrid_search..process_query at 0x7efc9c5b6c00>
open-webui-rocm | └ <concurrent.futures.thread._WorkItem object at 0x7efd2dcacd10>
open-webui-rocm |
open-webui-rocm | > File "/app/backend/open_webui/retrieval/utils.py", line 308, in process_query
open-webui-rocm | result = query_doc_with_hybrid_search(
open-webui-rocm | └ <function query_doc_with_hybrid_search at 0x7efd5c75c040>
open-webui-rocm |
open-webui-rocm | File "/app/backend/open_webui/retrieval/utils.py", line 171, in query_doc_with_hybrid_search
open-webui-rocm | raise e
open-webui-rocm |
open-webui-rocm | File "/app/backend/open_webui/retrieval/utils.py", line 145, in query_doc_with_hybrid_search
open-webui-rocm | result = compression_retriever.invoke(query)
open-webui-rocm | │ │ └ 'nonwoven projects'
open-webui-rocm | │ └ <function BaseRetriever.invoke at 0x7efd5cc97b00>
open-webui-rocm | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain_core/retrievers.py", line 259, in invoke
open-webui-rocm | result = self._get_relevant_documents(
open-webui-rocm | │ └ <function ContextualCompressionRetriever._get_relevant_documents at 0x7efd5cc97740>
open-webui-rocm | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain/retrievers/contextual_compression.py", line 44, in _get_relevant_documents
open-webui-rocm | docs = self.base_retriever.invoke(
open-webui-rocm | │ │ └ <function EnsembleRetriever.invoke at 0x7efd5c9a80e0>
open-webui-rocm | │ └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7efd2e580500>, k=30), VectorSearchRet...
open-webui-rocm | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain/retrievers/ensemble.py", line 118, in invoke
open-webui-rocm | raise e
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain/retrievers/ensemble.py", line 115, in invoke
open-webui-rocm | result = self.rank_fusion(input, run_manager=run_manager, config=config)
open-webui-rocm | │ │ │ │ └ {'tags': [], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.CallbackManager object at 0x7efd2d0875c0>, 'recur...
open-webui-rocm | │ │ │ └ <langchain_core.callbacks.manager.CallbackManagerForRetrieverRun object at 0x7f009ed8c500>
open-webui-rocm | │ │ └ 'nonwoven projects'
open-webui-rocm | │ └ <function EnsembleRetriever.rank_fusion at 0x7efd5c9a8400>
open-webui-rocm | └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7efd2e580500>, k=30), VectorSearchRet...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain/retrievers/ensemble.py", line 223, in rank_fusion
open-webui-rocm | retriever.invoke(
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/langchain_core/retrievers.py", line 259, in invoke
open-webui-rocm | result = self._get_relevant_documents(
open-webui-rocm | │ └ <function VectorSearchRetriever._get_relevant_documents at 0x7efd5c78bec0>
open-webui-rocm | └ VectorSearchRetriever(collection_name='ab37fb86-7213-422c-b37d-01c83b58e6e5', embedding_function=<function chat_completion_fi...
open-webui-rocm |
open-webui-rocm | File "/app/backend/open_webui/retrieval/utils.py", line 57, in _get_relevant_documents
open-webui-rocm | vectors=[self.embedding_function(query, RAG_EMBEDDING_QUERY_PREFIX)],
open-webui-rocm | │ │ │ └ None
open-webui-rocm | │ │ └ 'nonwoven projects'
open-webui-rocm | │ └ <function chat_completion_files_handler.... at 0x7efc169f9260>
open-webui-rocm | └ VectorSearchRetriever(collection_name='ab37fb86-7213-422c-b37d-01c83b58e6e5', embedding_function=<function chat_completion_fi...
open-webui-rocm |
open-webui-rocm | File "/app/backend/open_webui/utils/middleware.py", line 598, in
open-webui-rocm | embedding_function=lambda query, prefix: request.app.state.EMBEDDING_FUNCTION(
open-webui-rocm | │ │ │ └ <property object at 0x7f009c5791c0>
open-webui-rocm | │ │ └ <starlette.requests.Request object at 0x7efc753e1880>
open-webui-rocm | │ └ None
open-webui-rocm | └ 'nonwoven projects'
open-webui-rocm |
open-webui-rocm | File "/app/backend/open_webui/retrieval/utils.py", line 356, in
open-webui-rocm | return lambda query, prefix=None, user=None: embedding_function.encode(
open-webui-rocm | │ │ └ <function SentenceTransformer.encode at 0x7efd2fb54d60>
open-webui-rocm | │ └ SentenceTransformer(
open-webui-rocm | │ (transformer): Transformer(
open-webui-rocm | │ (auto_model): XLMRobertaLoRA(
open-webui-rocm | │ (roberta): XLMRobertaModel(
open-webui-rocm | │ ...
open-webui-rocm | └ 'nonwoven projects'
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/sentence_transformers/SentenceTransformer.py", line 557, in encode
open-webui-rocm | out_features = self.forward(features, **kwargs)
open-webui-rocm | │ │ │ └ {}
open-webui-rocm | │ │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | │ └ <function SentenceTransformer.forward at 0x7efd2fb54e00>
open-webui-rocm | └ SentenceTransformer(
open-webui-rocm | (transformer): Transformer(
open-webui-rocm | (auto_model): XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | ...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/sentence_transformers/SentenceTransformer.py", line 624, in forward
open-webui-rocm | input = module(input, **module_kwargs)
open-webui-rocm | │ │ └ {}
open-webui-rocm | │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | └ Transformer(
open-webui-rocm | (auto_model): XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | ...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
open-webui-rocm | return self._call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {}
open-webui-rocm | │ │ └ ({'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, ...
open-webui-rocm | │ └ <function Module._call_impl at 0x7efe70110f40>
open-webui-rocm | └ Transformer(
open-webui-rocm | (auto_model): XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | ...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {}
open-webui-rocm | │ └ ({'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, ...
open-webui-rocm | └ <bound method Transformer.forward of Transformer(
open-webui-rocm | (auto_model): XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | (embe...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/f1944de8402dcd5f2b03f822a4bc22a7f2de2eb9/custom_st.py", line 143, in forward
open-webui-rocm | output_states = self.auto_model.forward(**features, **lora_arguments, return_dict=False)
open-webui-rocm | │ │ └ {}
open-webui-rocm | │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | └ Transformer(
open-webui-rocm | (auto_model): XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | ...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/modeling_lora.py", line 372, in forward
open-webui-rocm | return self.roberta(*args, **kwargs)
open-webui-rocm | │ │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | │ └ ()
open-webui-rocm | └ XLMRobertaLoRA(
open-webui-rocm | (roberta): XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | (word_embeddings): ParametrizedEm...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
open-webui-rocm | return self._call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | │ │ └ ()
open-webui-rocm | │ └ <function Module._call_impl at 0x7efe70110f40>
open-webui-rocm | └ XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | (word_embeddings): ParametrizedEmbedding(
open-webui-rocm | 250002, 1024, padd...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {'input_ids': tensor( 0, 351, 3613, 1353, 77635, 2, device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1...
open-webui-rocm | │ └ ()
open-webui-rocm | └ <bound method XLMRobertaModel.forward of XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | (word_embeddings): Parame...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/modeling_xlm_roberta.py", line 734, in forward
open-webui-rocm | sequence_output = self.encoder(
open-webui-rocm | └ XLMRobertaModel(
open-webui-rocm | (embeddings): XLMRobertaEmbeddings(
open-webui-rocm | (word_embeddings): ParametrizedEmbedding(
open-webui-rocm | 250002, 1024, padd...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
open-webui-rocm | return self._call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {'key_padding_mask': tensor(1, 1, 1, 1, 1, 1, device='cuda:0'), 'subset_mask': None, 'adapter_mask': None, 'output_hidden...
open-webui-rocm | │ │ └ (tensor([[[-0.0913, -0.1147, -0.3457, ..., 0.1426, -0.3750, -0.5352],
open-webui-rocm | │ │ [ 0.1177, 0.1758, 0.6406, ..., 0.3984, ...
open-webui-rocm | │ └ <function Module._call_impl at 0x7efe70110f40>
open-webui-rocm | └ XLMRobertaEncoder(
open-webui-rocm | (layers): ModuleList(
open-webui-rocm | (0-23): 24 x Block(
open-webui-rocm | (mixer): MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding(...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {'key_padding_mask': tensor(1, 1, 1, 1, 1, 1, device='cuda:0'), 'subset_mask': None, 'adapter_mask': None, 'output_hidden...
open-webui-rocm | │ └ (tensor([[[-0.0913, -0.1147, -0.3457, ..., 0.1426, -0.3750, -0.5352],
open-webui-rocm | │ [ 0.1177, 0.1758, 0.6406, ..., 0.3984, ...
open-webui-rocm | └ <bound method XLMRobertaEncoder.forward of XLMRobertaEncoder(
open-webui-rocm | (layers): ModuleList(
open-webui-rocm | (0-23): 24 x Block(
open-webui-rocm | (mixer): ...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/modeling_xlm_roberta.py", line 258, in forward
open-webui-rocm | hidden_states = layer(hidden_states, mixer_kwargs=mixer_kwargs)
open-webui-rocm | │ │ └ {'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6, 'adapter_mask': None}
open-webui-rocm | │ └ tensor([[ 2.2583e-02, 8.3618e-03, -3.0670e-03, ..., 3.9062e-02,
open-webui-rocm | │ 8.1177e-03, -5.4932e-02],
open-webui-rocm | │ [ 2.1094e-01, ...
open-webui-rocm | └ Block(
open-webui-rocm | (mixer): MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=1024, ou...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
open-webui-rocm | return self._call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {'mixer_kwargs': {'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6, 'adapter_mask': None}}
open-webui-rocm | │ │ └ (tensor([[ 2.2583e-02, 8.3618e-03, -3.0670e-03, ..., 3.9062e-02,
open-webui-rocm | │ │ 8.1177e-03, -5.4932e-02],
open-webui-rocm | │ │ [ 2.1094e-01,...
open-webui-rocm | │ └ <function Module._call_impl at 0x7efe70110f40>
open-webui-rocm | └ Block(
open-webui-rocm | (mixer): MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=1024, ou...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {'mixer_kwargs': {'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6, 'adapter_mask': None}}
open-webui-rocm | │ └ (tensor([[ 2.2583e-02, 8.3618e-03, -3.0670e-03, ..., 3.9062e-02,
open-webui-rocm | │ 8.1177e-03, -5.4932e-02],
open-webui-rocm | │ [ 2.1094e-01,...
open-webui-rocm | └ <bound method Block.forward of Block(
open-webui-rocm | (mixer): MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResid...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/block.py", line 201, in forward
open-webui-rocm | mixer_out = self.mixer(
open-webui-rocm | └ Block(
open-webui-rocm | (mixer): MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=1024, ou...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
open-webui-rocm | return self._call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6, 'adapter_mask': None}
open-webui-rocm | │ │ └ (tensor([[ 2.2583e-02, 8.3618e-03, -3.0670e-03, ..., 3.9062e-02,
open-webui-rocm | │ │ 8.1177e-03, -5.4932e-02],
open-webui-rocm | │ │ [ 2.1094e-01,...
open-webui-rocm | │ └ <function Module._call_impl at 0x7efe70110f40>
open-webui-rocm | └ MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=1024, out_features=3072, bias=Tr...
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6, 'adapter_mask': None}
open-webui-rocm | │ └ (tensor([[ 2.2583e-02, 8.3618e-03, -3.0670e-03, ..., 3.9062e-02,
open-webui-rocm | │ 8.1177e-03, -5.4932e-02],
open-webui-rocm | │ [ 2.1094e-01,...
open-webui-rocm | └ <bound method MHA.forward of MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=102...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/mha.py", line 732, in forward
open-webui-rocm | qkv = self.rotary_emb(
open-webui-rocm | └ MHA(
open-webui-rocm | (rotary_emb): RotaryEmbedding()
open-webui-rocm | (Wqkv): ParametrizedLinearResidual(
open-webui-rocm | in_features=1024, out_features=3072, bias=Tr...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in wrapped_call_impl
open-webui-rocm | return self.call_impl(*args, **kwargs)
open-webui-rocm | │ │ │ └ {'seqlen_offset': 0, 'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6}
open-webui-rocm | │ │ └ (tensor([[[[ 1.8555e-01, 1.9531e-01, 2.7344e-01, ..., -4.1992e-01,
open-webui-rocm | │ │ 6.8750e-01, 9.4141e-01],
open-webui-rocm | │ │ [ 1.845...
open-webui-rocm | │ └ <function Module.call_impl at 0x7efe70110f40>
open-webui-rocm | └ RotaryEmbedding()
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in call_impl
open-webui-rocm | return forward_call(*args, **kwargs)
open-webui-rocm | │ │ └ {'seqlen_offset': 0, 'cu_seqlens': tensor([0, 6], device='cuda:0', dtype=torch.int32), 'max_seqlen': 6}
open-webui-rocm | │ └ (tensor([[[[ 1.8555e-01, 1.9531e-01, 2.7344e-01, ..., -4.1992e-01,
open-webui-rocm | │ 6.8750e-01, 9.4141e-01],
open-webui-rocm | │ [ 1.845...
open-webui-rocm | └ <bound method RotaryEmbedding.forward of RotaryEmbedding()>
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/rotary.py", line 604, in forward
open-webui-rocm | return apply_rotary_emb_qkv(
open-webui-rocm | └ <function apply_rotary_emb_qkv at 0x7efd2e5d23e0>
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/rotary.py", line 327, in apply_rotary_emb_qkv
open-webui-rocm | return ApplyRotaryEmbQKV.apply(
open-webui-rocm | │ └ <classmethod(<function Function.apply at 0x7efe6fe8d760>)>
open-webui-rocm | └ <class 'transformers_modules.jinaai.xlm-roberta-flash-implementation.2b6bc3f30750b3a9648fe9b63448c09920efe9be.rotary.ApplyRot...
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/autograd/function.py", line 574, in apply
open-webui-rocm | return super().apply(*args, **kwargs) # type: ignore[misc]
open-webui-rocm | │ └ {}
open-webui-rocm | └ (tensor([[[[ 1.8555e-01, 1.9531e-01, 2.7344e-01, ..., -4.1992e-01,
open-webui-rocm | 6.8750e-01, 9.4141e-01],
open-webui-rocm | [ 1.845...
open-webui-rocm |
open-webui-rocm | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/2b6bc3f30750b3a9648fe9b63448c09920efe9be/rotary.py", line 186, in forward
open-webui-rocm | apply_rotary(
open-webui-rocm | └ <function apply_rotary at 0x7efd2e5d1ee0>
open-webui-rocm |
open-webui-rocm | File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/flash_attn-2.7.4.post1-py3.12.egg/flash_attn/ops/triton/rotary.py", line 166, in apply_rotary
open-webui-rocm | assert sin.shape == cos.shape
open-webui-rocm | │ │ └ <attribute 'shape' of 'torch._C.TensorBase' objects>
open-webui-rocm | │ └ tensor([[ 1.0000, 1.0000, 1.0000, 1.0000, 1.0000, 1.0000, 1.0000, 1.0000,
open-webui-rocm | │ 1.0000, 1.0000, 1.0000, 1.0000,...
open-webui-rocm | └ None
Additional Information
No response
@tjbck commented on GitHub (Apr 3, 2025):
@Phlogi
@Phlogi commented on GitHub (Apr 3, 2025):
I can't reproduce yet, is the collection empty? @DrZoidberg09 Please provide the log earlier, the error does not match the code in my opinion.
There we have an issue @tjbck This is the output when querying a collection without any documents in it. I think this be handled early, maybe before even calling query_doc_with_hybrid_search()?
@DrZoidberg09 commented on GitHub (Apr 3, 2025):
The collection is totally fine. It works perfectly fine without reranking / hybrid RAG.
Here are the complete logs, but I don't think they give any extra information:
@Phlogi commented on GitHub (Apr 3, 2025):
Please also share exactly how run the image and what tag etc. you're using. Also, what are you doing in the app exactly to get this error? I assume searching through a collection, however then some default logging output is missing. So it seems you're not using the exact 0.6.0 version or doing something else that triggers hybrid search.
Coud you try to reproduce it without ROCm, using standard image and CPU only?
@DrZoidberg09 commented on GitHub (Apr 3, 2025):
I just had a look into the Dockerfile. And it seems that sentencetransformer is installed separately (dont know why). I dont have access right now to check, but I think it will install sentencetransformer 4, which heavily refactored the reranking. I will bake a new Docker image without that and will let you know. In best case this will solve the issue and just be a heads up that there need to be adaptions for sentencetransformer 4, if you plan to update it.
@dromeuf commented on GitHub (Apr 3, 2025):
Hi, for me too with pip local installation, since version 0.6, the hybrid search option switch doesn't even appear anymore. I can't specify a reranker model, the text input file no longer appears.
@rgaricano commented on GitHub (Apr 3, 2025):
I think it's because XLM Roberta model that is used,
If i'm not wrong, Roberta work with subwords embeddings and tensor output it's different than expected.
(the output is a tensor with a shape type [1, 4, 505], because LM will output the probability distribution across all the words in the vocabulary, and we expect another shape, like [1,68])
If this is it, I read that there are ways to deal doing token embeddings averages of the subwords to generate an embedding for the whole word....
@bet0x commented on GitHub (Apr 4, 2025):
You error is on flash attn using rocm, dunno if AMD is properly supported:
@DrZoidberg09 commented on GitHub (Apr 8, 2025):
I did use Triton ROCm implementation of flash attention without any issue before the refactoring of the hybrid search (using JinaAI based on XLM Roberta). And all embedding/reranking with models not using flash attention still work fine. Hence, the refactoring somehow has broken something.