issue: Can't use reranker(size of tensor a (13) must match size of tensor be (12) at non-singleton dimension 1 #5984

New Issue

Closed

opened 2025-11-11 16:41:31 -06:00 by GiteaMirror · 4 comments

GiteaMirror commented 2025-11-11 16:41:31 -06:00

Owner

Copy Link

Copy Source

Originally created by @frenzybiscuit on GitHub (Aug 7, 2025).

Check Existing Issues

• I have searched the existing issues and discussions.
• I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.6.18

Ollama Version (if applicable)

No response

Operating System

Debian 12

Browser (if applicable)

No response

Confirmation

• I have read and followed all instructions in README.md.
• I am using the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.
• I have provided every relevant configuration, setting, and environment variable used in my setup.
• I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
• I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
• Start with the initial platform/version/OS and dependencies used,
• Specify exact install/launch/configure commands,
• List URLs visited, user input (incl. example values/emails/passwords if needed),
• Describe all options and toggles enabled or changed,
• Include any files or environmental changes,
• Identify the expected and actual result at each stage,
• Ensure any reasonably skilled user can follow and hit the same issue.

Steps to Reproduce

Install on Docker with the cuda image

Use a nvidia GPU (2080ti)

Set chunk size 1000, chunk overlap 100

Set embedding model nomic-ai/nomic-embed-text-v1

Set hybrid search reranking engine to BAAI/bge-reranker-v2-m3

Set top k to 8

Set top k reranker to 3

set relevance threshold to 0.3

set weight of BM25 to 0.5

Logs & Screenshots

see below

Additional Information

I have tried to reset the vector/knowledge storage already, along with reindex.

Expected Behavior

Works as expected

Actual Behavior

Errors out with the following:

open-webui  | Traceback (most recent call last):
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
open-webui  |     self._bootstrap_inner()
open-webui  |     │    └ <function Thread._bootstrap_inner at 0x7f7f69ed49a0>
open-webui  |     └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)>
open-webui  |   File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
open-webui  |     self.run()
open-webui  |     │    └ <function Thread.run at 0x7f7f69ed4680>
open-webui  |     └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)>
open-webui  |   File "/usr/local/lib/python3.11/threading.py", line 982, in run
open-webui  |     self._target(*self._args, **self._kwargs)
open-webui  |     │    │        │    │        │    └ {}
open-webui  |     │    │        │    │        └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)>
open-webui  |     │    │        │    └ (<weakref at 0x7f7d99147c90; to 'ThreadPoolExecutor' at 0x7f7d30a65b90>, <_queue.SimpleQueue object at 0x7f7d99147650>, None,...
open-webui  |     │    │        └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)>
open-webui  |     │    └ <function _worker at 0x7f7f68e13d80>
open-webui  |     └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)>
open-webui  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker
open-webui  |     work_item.run()
open-webui  |     │         └ <function _WorkItem.run at 0x7f7f68e13ec0>
open-webui  |     └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090>
open-webui  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
open-webui  |     result = self.fn(*self.args, **self.kwargs)
open-webui  |              │    │   │    │       │    └ {}
open-webui  |              │    │   │    │       └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090>
open-webui  |              │    │   │    └ ('15743a95-b071-4010-98cb-515244f67c2b', 'What characters got their powers from a cauldron?')
open-webui  |              │    │   └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090>
open-webui  |              │    └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7f7d21b78540>
open-webui  |              └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090>
open-webui  |
open-webui  | > File "/app/backend/open_webui/retrieval/utils.py", line 358, in process_query
open-webui  |     result = query_doc_with_hybrid_search(
open-webui  |              └ <function query_doc_with_hybrid_search at 0x7f7e0d42a700>
open-webui  |
open-webui  |   File "/app/backend/open_webui/retrieval/utils.py", line 192, in query_doc_with_hybrid_search
open-webui  |     raise e
open-webui  |
open-webui  |   File "/app/backend/open_webui/retrieval/utils.py", line 165, in query_doc_with_hybrid_search
open-webui  |     result = compression_retriever.invoke(query)
open-webui  |              │                     │      └ 'What characters got their powers from a cauldron?'
open-webui  |              │                     └ <function BaseRetriever.invoke at 0x7f7e0d5ee5c0>
open-webui  |              └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain_core/retrievers.py", line 259, in invoke
open-webui  |     result = self._get_relevant_documents(
open-webui  |              │    └ <function ContextualCompressionRetriever._get_relevant_documents at 0x7f7e0d5ede40>
open-webui  |              └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/contextual_compression.py", line 40, in _get_relevant_documents
open-webui  |     docs = self.base_retriever.invoke(
open-webui  |            │    │              └ <function EnsembleRetriever.invoke at 0x7f7e0d5ef240>
open-webui  |            │    └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7f7d28709b50>, k=8), VectorSearchRetr...
open-webui  |            └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 114, in invoke
open-webui  |     raise e
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 111, in invoke
open-webui  |     result = self.rank_fusion(input, run_manager=run_manager, config=config)
open-webui  |              │    │           │                  │                   └ {'tags': [], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.CallbackManager object at 0x7f7d3dd38150>, 'recur...
open-webui  |              │    │           │                  └ <langchain_core.callbacks.manager.CallbackManagerForRetrieverRun object at 0x7f7d3dd38190>
open-webui  |              │    │           └ 'What characters got their powers from a cauldron?'
open-webui  |              │    └ <function EnsembleRetriever.rank_fusion at 0x7f7e0d5ef560>
open-webui  |              └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7f7d28709b50>, k=8), VectorSearchRetr...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 218, in rank_fusion
open-webui  |     retriever_docs = [
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 219, in <listcomp>
open-webui  |     retriever.invoke(
open-webui  |     │         └ <function BaseRetriever.invoke at 0x7f7e0d5ee5c0>
open-webui  |     └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/langchain_core/retrievers.py", line 259, in invoke
open-webui  |     result = self._get_relevant_documents(
open-webui  |              │    └ <function VectorSearchRetriever._get_relevant_documents at 0x7f7e0d42b380>
open-webui  |              └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi...
open-webui  |
open-webui  |   File "/app/backend/open_webui/retrieval/utils.py", line 62, in _get_relevant_documents
open-webui  |     vectors=[self.embedding_function(query, RAG_EMBEDDING_QUERY_PREFIX)],
open-webui  |              │    │                  │      └ None
open-webui  |              │    │                  └ 'What characters got their powers from a cauldron?'
open-webui  |              │    └ <function chat_completion_files_handler.<locals>.<lambda>.<locals>.<lambda> at 0x7f7d30b3d300>
open-webui  |              └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi...
open-webui  |
open-webui  |   File "/app/backend/open_webui/utils/middleware.py", line 651, in <lambda>
open-webui  |     embedding_function=lambda query, prefix: request.app.state.EMBEDDING_FUNCTION(
open-webui  |                               │      │       │       └ <property object at 0x7f7f663e6610>
open-webui  |                               │      │       └ <starlette.requests.Request object at 0x7f7d21949810>
open-webui  |                               │      └ None
open-webui  |                               └ 'What characters got their powers from a cauldron?'
open-webui  |
open-webui  |   File "/app/backend/open_webui/retrieval/utils.py", line 411, in <lambda>
open-webui  |     return lambda query, prefix=None, user=None: embedding_function.encode(
open-webui  |                   │                              │                  └ <function SentenceTransformer.encode at 0x7f7d9a1ffd80>
open-webui  |                   │                              └ SentenceTransformer(
open-webui  |                   │                                  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertMo...
open-webui  |                   └ 'What characters got their powers from a cauldron?'
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 685, in encode
open-webui  |     out_features = self.forward(features, **kwargs)
open-webui  |                    │    │       │           └ {}
open-webui  |                    │    │       └ {'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |                    │    │                   4948,  1029,   102]], ...
open-webui  |                    │    └ <function SentenceTransformer.forward at 0x7f7d9a1ffe20>
open-webui  |                    └ SentenceTransformer(
open-webui  |                        (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertMo...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 758, in forward
open-webui  |     input = module(input, **module_kwargs)
open-webui  |             │      │        └ {}
open-webui  |             │      └ {'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |             │                  4948,  1029,   102]], ...
open-webui  |             └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {}
open-webui  |            │    │           └ ({'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |            │    │                       4948,  1029,   102]],...
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {}
open-webui  |            │             └ ({'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |            │                         4948,  1029,   102]],...
open-webui  |            └ <bound method Transformer.forward of Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: No...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 442, in forward
open-webui  |     outputs = self.auto_model(**trans_features, **kwargs, return_dict=True)
open-webui  |               │                 │                 └ {}
open-webui  |               │                 └ {'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |               │                             4948,  1029,   102]], ...
open-webui  |               └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |            │    │           │                   4948,  1029,   102]], ...
open-webui  |            │    │           └ ()
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ NomicBertModel(
open-webui  |                (embeddings): NomicBertEmbeddings(
open-webui  |                  (word_embeddings): Embedding(30528, 768)
open-webui  |                  (token_type_embeddings)...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {'input_ids': tensor([[  101,  2054,  3494,  2288,  2037,  4204,  2013,  1037,  6187, 21285,
open-webui  |            │             │                   4948,  1029,   102]], ...
open-webui  |            │             └ ()
open-webui  |            └ <bound method NomicBertModel.forward of NomicBertModel(
open-webui  |                (embeddings): NomicBertEmbeddings(
open-webui  |                  (word_embeddings): Embedding...
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1915, in forward
open-webui  |     sequence_output = self.encoder(hidden_states, attention_mask=attention_mask, return_dict=return_dict)
open-webui  |                       │            │                             │                           └ True
open-webui  |                       │            │                             └ tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]],
open-webui  |                       │            │                                      device='cuda:0')
open-webui  |                       │            └ tensor([[[-0.1587,  0.1036, -0.9116,  ..., -0.4400, -0.1227, -0.2432],
open-webui  |                       │                       [-0.1781,  0.4852, -0.6123,  ..., -0.2573,  0...
open-webui  |                       └ NomicBertModel(
open-webui  |                           (embeddings): NomicBertEmbeddings(
open-webui  |                             (word_embeddings): Embedding(30528, 768)
open-webui  |                             (token_type_embeddings)...
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]],
open-webui  |            │    │           │                device='cuda:0'), '...
open-webui  |            │    │           └ (tensor([[[-0.1587,  0.1036, -0.9116,  ..., -0.4400, -0.1227, -0.2432],
open-webui  |            │    │                      [-0.1781,  0.4852, -0.6123,  ..., -0.2573,  ...
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ NomicBertEncoder(
open-webui  |                (layers): ModuleList(
open-webui  |                  (0-11): 12 x NomicBertBlock(
open-webui  |                    (attn): NomicBertAttention(
open-webui  |                      (rotary_...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]],
open-webui  |            │             │                device='cuda:0'), '...
open-webui  |            │             └ (tensor([[[-0.1587,  0.1036, -0.9116,  ..., -0.4400, -0.1227, -0.2432],
open-webui  |            │                        [-0.1781,  0.4852, -0.6123,  ..., -0.2573,  ...
open-webui  |            └ <bound method NomicBertEncoder.forward of NomicBertEncoder(
open-webui  |                (layers): ModuleList(
open-webui  |                  (0-11): 12 x NomicBertBlock(
open-webui  |                    (a...
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1794, in forward
open-webui  |     hidden_states, hidden_states2, residual = layer(
open-webui  |     │              │                          └ NomicBertBlock(
open-webui  |     │              │                              (attn): NomicBertAttention(
open-webui  |     │              │                                (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |     │              │                                (Wqkv): Linear(in_fe...
open-webui  |     │              └ None
open-webui  |     └ tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00,  ...,  1.8714e-01,
open-webui  |                 -2.9426e-01, -2.6183e-01],
open-webui  |                [-2.0184e-0...
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {'rope': None}
open-webui  |            │    │           └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00,  ...,  1.8714e-01,
open-webui  |            │    │                       -2.9426e-01, -2.6183e-01],
open-webui  |            │    │                      [-2.0184e-...
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ NomicBertBlock(
open-webui  |                (attn): NomicBertAttention(
open-webui  |                  (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |                  (Wqkv): Linear(in_fe...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {'rope': None}
open-webui  |            │             └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00,  ...,  1.8714e-01,
open-webui  |            │                         -2.9426e-01, -2.6183e-01],
open-webui  |            │                        [-2.0184e-...
open-webui  |            └ <bound method NomicBertBlock.forward of NomicBertBlock(
open-webui  |                (attn): NomicBertAttention(
open-webui  |                  (rotary_emb): NomicBertDynamicNTKRo...
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1713, in forward
open-webui  |     attn_outputs = self.attn(
open-webui  |                    └ NomicBertBlock(
open-webui  |                        (attn): NomicBertAttention(
open-webui  |                          (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |                          (Wqkv): Linear(in_fe...
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]],
open-webui  |            │    │           │                device='cuda:0'), '...
open-webui  |            │    │           └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00,  ...,  1.8714e-01,
open-webui  |            │    │                       -2.9426e-01, -2.6183e-01],
open-webui  |            │    │                      [-2.0184e-...
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ NomicBertAttention(
open-webui  |                (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |                (Wqkv): Linear(in_features=768, out_features=2304,...
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]],
open-webui  |            │             │                device='cuda:0'), '...
open-webui  |            │             └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00,  ...,  1.8714e-01,
open-webui  |            │                         -2.9426e-01, -2.6183e-01],
open-webui  |            │                        [-2.0184e-...
open-webui  |            └ <bound method NomicBertAttention.forward of NomicBertAttention(
open-webui  |                (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |                (Wqkv)...
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1558, in forward
open-webui  |     qkv = self.rotary_emb(qkv, seqlen_offset=past_len)
open-webui  |           │               │                  └ 0
open-webui  |           │               └ tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01,  ...,  1.0232e-01,
open-webui  |           │                              2.8146e-01, -2.1487e+00],
open-webui  |           │                            [ 1.7...
open-webui  |           └ NomicBertAttention(
open-webui  |               (rotary_emb): NomicBertDynamicNTKRotaryEmbedding()
open-webui  |               (Wqkv): Linear(in_features=768, out_features=2304,...
open-webui  |
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
open-webui  |     return self._call_impl(*args, **kwargs)
open-webui  |            │    │           │       └ {'seqlen_offset': 0}
open-webui  |            │    │           └ (tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01,  ...,  1.0232e-01,
open-webui  |            │    │                          2.8146e-01, -2.1487e+00],
open-webui  |            │    │                        [ 1....
open-webui  |            │    └ <function Module._call_impl at 0x7f7e214fb380>
open-webui  |            └ NomicBertDynamicNTKRotaryEmbedding()
open-webui  |   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
open-webui  |     return forward_call(*args, **kwargs)
open-webui  |            │             │       └ {'seqlen_offset': 0}
open-webui  |            │             └ (tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01,  ...,  1.0232e-01,
open-webui  |            │                            2.8146e-01, -2.1487e+00],
open-webui  |            │                          [ 1....
open-webui  |            └ <bound method NomicBertRotaryEmbedding.forward of NomicBertDynamicNTKRotaryEmbedding()>
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1394, in forward
open-webui  |     k_rot = apply_rotary_emb(qkv[:, :, 1], self._cos_cached, self._sin_cached, seqlen_offset, self.interleaved)
open-webui  |             │                │             │    │            │    │            │              │    └ False
open-webui  |             │                │             │    │            │    │            │              └ NomicBertDynamicNTKRotaryEmbedding()
open-webui  |             │                │             │    │            │    │            └ 0
open-webui  |             │                │             │    │            │    └ tensor([[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,
open-webui  |             │                │             │    │            │                0.0000,  0.0000,  0.0000,  0.0000,...
open-webui  |             │                │             │    │            └ NomicBertDynamicNTKRotaryEmbedding()
open-webui  |             │                │             │    └ tensor([[ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00,
open-webui  |             │                │             │                1.0000e+00,  1.0000e+00,  1.0000e+00,  1....
open-webui  |             │                │             └ NomicBertDynamicNTKRotaryEmbedding()
open-webui  |             │                └ tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01,  ...,  1.0232e-01,
open-webui  |             │                               2.8146e-01, -2.1487e+00],
open-webui  |             │                             [ 1.7...
open-webui  |             └ <function apply_rotary_emb at 0x7f7d991062a0>
open-webui  |
open-webui  |   File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1280, in apply_rotary_emb
open-webui  |     [x[..., :ro_dim] * cos + rotate_half(x[..., :ro_dim], interleaved) * sin, x[..., ro_dim:]],
open-webui  |      │       │         │     │           │       │        │              │    │      └ 64
open-webui  |      │       │         │     │           │       │        │              │    └ tensor([[[[ 1.0443e-02, -2.7018e-01,  2.8168e-01,  ..., -1.1565e+00,
open-webui  |      │       │         │     │           │       │        │              │                  6.9990e-01, -2.2852e+00],
open-webui  |      │       │         │     │           │       │        │              │                [ 6.8054...
open-webui  |      │       │         │     │           │       │        │              └ tensor([[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,
open-webui  |      │       │         │     │           │       │        │                           0.0000,  0.0000,  0.0000,  0.0000,  0.000...
open-webui  |      │       │         │     │           │       │        └ False
open-webui  |      │       │         │     │           │       └ 64
open-webui  |      │       │         │     │           └ tensor([[[[ 1.0443e-02, -2.7018e-01,  2.8168e-01,  ..., -1.1565e+00,
open-webui  |      │       │         │     │                         6.9990e-01, -2.2852e+00],
open-webui  |      │       │         │     │                       [ 6.8054...
open-webui  |      │       │         │     └ <function rotate_half at 0x7f7d991054e0>
open-webui  |      │       │         └ tensor([[[ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00,
open-webui  |      │       │                      1.0000e+00,  1.0000e+00,  1.0000e+00,  ...
open-webui  |      │       └ 64
open-webui  |      └ tensor([[[[ 1.0443e-02, -2.7018e-01,  2.8168e-01,  ..., -1.1565e+00,
open-webui  |                    6.9990e-01, -2.2852e+00],
open-webui  |                  [ 6.8054...
open-webui  |
open-webui  | RuntimeError: The size of tensor a (13) must match the size of tensor b (12) at non-singleton dimension 1 ```

Originally created by @frenzybiscuit on GitHub (Aug 7, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.6.18 ### Ollama Version (if applicable) _No response_ ### Operating System Debian 12 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Steps to Reproduce Install on Docker with the cuda image Use a nvidia GPU (2080ti) Set chunk size 1000, chunk overlap 100 Set embedding model nomic-ai/nomic-embed-text-v1 Set hybrid search reranking engine to BAAI/bge-reranker-v2-m3 Set top k to 8 Set top k reranker to 3 set relevance threshold to 0.3 set weight of BM25 to 0.5 ### Logs & Screenshots see below ### Additional Information **I have tried to reset the vector/knowledge storage already, along with reindex.** ### Expected Behavior Works as expected ### Actual Behavior Errors out with the following: ```open-webui | 2025-08-07 18:32:07.943 | ERROR | open_webui.retrieval.utils:process_query:371 - Error when querying the collection with hybrid_search: The size of tensor a (13) must match the size of tensor b (12) at non-singleton dimension 1 - {} open-webui | Traceback (most recent call last): open-webui | open-webui | File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap open-webui | self._bootstrap_inner() open-webui | │ └ <function Thread._bootstrap_inner at 0x7f7f69ed49a0> open-webui | └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)> open-webui | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner open-webui | self.run() open-webui | │ └ <function Thread.run at 0x7f7f69ed4680> open-webui | └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)> open-webui | File "/usr/local/lib/python3.11/threading.py", line 982, in run open-webui | self._target(*self._args, **self._kwargs) open-webui | │ │ │ │ │ └ {} open-webui | │ │ │ │ └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)> open-webui | │ │ │ └ (<weakref at 0x7f7d99147c90; to 'ThreadPoolExecutor' at 0x7f7d30a65b90>, <_queue.SimpleQueue object at 0x7f7d99147650>, None,... open-webui | │ │ └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)> open-webui | │ └ <function _worker at 0x7f7f68e13d80> open-webui | └ <Thread(ThreadPoolExecutor-15_0, started 140174864406208)> open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83, in _worker open-webui | work_item.run() open-webui | │ └ <function _WorkItem.run at 0x7f7f68e13ec0> open-webui | └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090> open-webui | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run open-webui | result = self.fn(*self.args, **self.kwargs) open-webui | │ │ │ │ │ └ {} open-webui | │ │ │ │ └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090> open-webui | │ │ │ └ ('15743a95-b071-4010-98cb-515244f67c2b', 'What characters got their powers from a cauldron?') open-webui | │ │ └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090> open-webui | │ └ <function query_collection_with_hybrid_search.<locals>.process_query at 0x7f7d21b78540> open-webui | └ <concurrent.futures.thread._WorkItem object at 0x7f7d30925090> open-webui | open-webui | > File "/app/backend/open_webui/retrieval/utils.py", line 358, in process_query open-webui | result = query_doc_with_hybrid_search( open-webui | └ <function query_doc_with_hybrid_search at 0x7f7e0d42a700> open-webui | open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 192, in query_doc_with_hybrid_search open-webui | raise e open-webui | open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 165, in query_doc_with_hybrid_search open-webui | result = compression_retriever.invoke(query) open-webui | │ │ └ 'What characters got their powers from a cauldron?' open-webui | │ └ <function BaseRetriever.invoke at 0x7f7e0d5ee5c0> open-webui | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l... open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/langchain_core/retrievers.py", line 259, in invoke open-webui | result = self._get_relevant_documents( open-webui | │ └ <function ContextualCompressionRetriever._get_relevant_documents at 0x7f7e0d5ede40> open-webui | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l... open-webui | File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/contextual_compression.py", line 40, in _get_relevant_documents open-webui | docs = self.base_retriever.invoke( open-webui | │ │ └ <function EnsembleRetriever.invoke at 0x7f7e0d5ef240> open-webui | │ └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7f7d28709b50>, k=8), VectorSearchRetr... open-webui | └ ContextualCompressionRetriever(base_compressor=RerankCompressor(embedding_function=<function chat_completion_files_handler.<l... open-webui | File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 114, in invoke open-webui | raise e open-webui | File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 111, in invoke open-webui | result = self.rank_fusion(input, run_manager=run_manager, config=config) open-webui | │ │ │ │ └ {'tags': [], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.CallbackManager object at 0x7f7d3dd38150>, 'recur... open-webui | │ │ │ └ <langchain_core.callbacks.manager.CallbackManagerForRetrieverRun object at 0x7f7d3dd38190> open-webui | │ │ └ 'What characters got their powers from a cauldron?' open-webui | │ └ <function EnsembleRetriever.rank_fusion at 0x7f7e0d5ef560> open-webui | └ EnsembleRetriever(retrievers=[BM25Retriever(vectorizer=<rank_bm25.BM25Okapi object at 0x7f7d28709b50>, k=8), VectorSearchRetr... open-webui | File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 218, in rank_fusion open-webui | retriever_docs = [ open-webui | File "/usr/local/lib/python3.11/site-packages/langchain/retrievers/ensemble.py", line 219, in <listcomp> open-webui | retriever.invoke( open-webui | │ └ <function BaseRetriever.invoke at 0x7f7e0d5ee5c0> open-webui | └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi... open-webui | File "/usr/local/lib/python3.11/site-packages/langchain_core/retrievers.py", line 259, in invoke open-webui | result = self._get_relevant_documents( open-webui | │ └ <function VectorSearchRetriever._get_relevant_documents at 0x7f7e0d42b380> open-webui | └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi... open-webui | open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 62, in _get_relevant_documents open-webui | vectors=[self.embedding_function(query, RAG_EMBEDDING_QUERY_PREFIX)], open-webui | │ │ │ └ None open-webui | │ │ └ 'What characters got their powers from a cauldron?' open-webui | │ └ <function chat_completion_files_handler.<locals>.<lambda>.<locals>.<lambda> at 0x7f7d30b3d300> open-webui | └ VectorSearchRetriever(collection_name='15743a95-b071-4010-98cb-515244f67c2b', embedding_function=<function chat_completion_fi... open-webui | open-webui | File "/app/backend/open_webui/utils/middleware.py", line 651, in <lambda> open-webui | embedding_function=lambda query, prefix: request.app.state.EMBEDDING_FUNCTION( open-webui | │ │ │ └ <property object at 0x7f7f663e6610> open-webui | │ │ └ <starlette.requests.Request object at 0x7f7d21949810> open-webui | │ └ None open-webui | └ 'What characters got their powers from a cauldron?' open-webui | open-webui | File "/app/backend/open_webui/retrieval/utils.py", line 411, in <lambda> open-webui | return lambda query, prefix=None, user=None: embedding_function.encode( open-webui | │ │ └ <function SentenceTransformer.encode at 0x7f7d9a1ffd80> open-webui | │ └ SentenceTransformer( open-webui | │ (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertMo... open-webui | └ 'What characters got their powers from a cauldron?' open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 685, in encode open-webui | out_features = self.forward(features, **kwargs) open-webui | │ │ │ └ {} open-webui | │ │ └ {'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ │ 4948, 1029, 102]], ... open-webui | │ └ <function SentenceTransformer.forward at 0x7f7d9a1ffe20> open-webui | └ SentenceTransformer( open-webui | (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertMo... open-webui | File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 758, in forward open-webui | input = module(input, **module_kwargs) open-webui | │ │ └ {} open-webui | │ └ {'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ 4948, 1029, 102]], ... open-webui | └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {} open-webui | │ │ └ ({'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ │ 4948, 1029, 102]],... open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {} open-webui | │ └ ({'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ 4948, 1029, 102]],... open-webui | └ <bound method Transformer.forward of Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: No... open-webui | File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 442, in forward open-webui | outputs = self.auto_model(**trans_features, **kwargs, return_dict=True) open-webui | │ │ └ {} open-webui | │ └ {'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ 4948, 1029, 102]], ... open-webui | └ Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ │ │ 4948, 1029, 102]], ... open-webui | │ │ └ () open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ NomicBertModel( open-webui | (embeddings): NomicBertEmbeddings( open-webui | (word_embeddings): Embedding(30528, 768) open-webui | (token_type_embeddings)... open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {'input_ids': tensor([[ 101, 2054, 3494, 2288, 2037, 4204, 2013, 1037, 6187, 21285, open-webui | │ │ 4948, 1029, 102]], ... open-webui | │ └ () open-webui | └ <bound method NomicBertModel.forward of NomicBertModel( open-webui | (embeddings): NomicBertEmbeddings( open-webui | (word_embeddings): Embedding... open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1915, in forward open-webui | sequence_output = self.encoder(hidden_states, attention_mask=attention_mask, return_dict=return_dict) open-webui | │ │ │ └ True open-webui | │ │ └ tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]], open-webui | │ │ device='cuda:0') open-webui | │ └ tensor([[[-0.1587, 0.1036, -0.9116, ..., -0.4400, -0.1227, -0.2432], open-webui | │ [-0.1781, 0.4852, -0.6123, ..., -0.2573, 0... open-webui | └ NomicBertModel( open-webui | (embeddings): NomicBertEmbeddings( open-webui | (word_embeddings): Embedding(30528, 768) open-webui | (token_type_embeddings)... open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]], open-webui | │ │ │ device='cuda:0'), '... open-webui | │ │ └ (tensor([[[-0.1587, 0.1036, -0.9116, ..., -0.4400, -0.1227, -0.2432], open-webui | │ │ [-0.1781, 0.4852, -0.6123, ..., -0.2573, ... open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ NomicBertEncoder( open-webui | (layers): ModuleList( open-webui | (0-11): 12 x NomicBertBlock( open-webui | (attn): NomicBertAttention( open-webui | (rotary_... open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]], open-webui | │ │ device='cuda:0'), '... open-webui | │ └ (tensor([[[-0.1587, 0.1036, -0.9116, ..., -0.4400, -0.1227, -0.2432], open-webui | │ [-0.1781, 0.4852, -0.6123, ..., -0.2573, ... open-webui | └ <bound method NomicBertEncoder.forward of NomicBertEncoder( open-webui | (layers): ModuleList( open-webui | (0-11): 12 x NomicBertBlock( open-webui | (a... open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1794, in forward open-webui | hidden_states, hidden_states2, residual = layer( open-webui | │ │ └ NomicBertBlock( open-webui | │ │ (attn): NomicBertAttention( open-webui | │ │ (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | │ │ (Wqkv): Linear(in_fe... open-webui | │ └ None open-webui | └ tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00, ..., 1.8714e-01, open-webui | -2.9426e-01, -2.6183e-01], open-webui | [-2.0184e-0... open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {'rope': None} open-webui | │ │ └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00, ..., 1.8714e-01, open-webui | │ │ -2.9426e-01, -2.6183e-01], open-webui | │ │ [-2.0184e-... open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ NomicBertBlock( open-webui | (attn): NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | (Wqkv): Linear(in_fe... open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {'rope': None} open-webui | │ └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00, ..., 1.8714e-01, open-webui | │ -2.9426e-01, -2.6183e-01], open-webui | │ [-2.0184e-... open-webui | └ <bound method NomicBertBlock.forward of NomicBertBlock( open-webui | (attn): NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRo... open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1713, in forward open-webui | attn_outputs = self.attn( open-webui | └ NomicBertBlock( open-webui | (attn): NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | (Wqkv): Linear(in_fe... open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]], open-webui | │ │ │ device='cuda:0'), '... open-webui | │ │ └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00, ..., 1.8714e-01, open-webui | │ │ -2.9426e-01, -2.6183e-01], open-webui | │ │ [-2.0184e-... open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | (Wqkv): Linear(in_features=768, out_features=2304,... open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {'attention_mask': tensor([[[[-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.]]]], open-webui | │ │ device='cuda:0'), '... open-webui | │ └ (tensor([[[-2.1849e-02, -1.7754e-01, -9.2316e+00, ..., 1.8714e-01, open-webui | │ -2.9426e-01, -2.6183e-01], open-webui | │ [-2.0184e-... open-webui | └ <bound method NomicBertAttention.forward of NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | (Wqkv)... open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1558, in forward open-webui | qkv = self.rotary_emb(qkv, seqlen_offset=past_len) open-webui | │ │ └ 0 open-webui | │ └ tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01, ..., 1.0232e-01, open-webui | │ 2.8146e-01, -2.1487e+00], open-webui | │ [ 1.7... open-webui | └ NomicBertAttention( open-webui | (rotary_emb): NomicBertDynamicNTKRotaryEmbedding() open-webui | (Wqkv): Linear(in_features=768, out_features=2304,... open-webui | open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl open-webui | return self._call_impl(*args, **kwargs) open-webui | │ │ │ └ {'seqlen_offset': 0} open-webui | │ │ └ (tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01, ..., 1.0232e-01, open-webui | │ │ 2.8146e-01, -2.1487e+00], open-webui | │ │ [ 1.... open-webui | │ └ <function Module._call_impl at 0x7f7e214fb380> open-webui | └ NomicBertDynamicNTKRotaryEmbedding() open-webui | File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl open-webui | return forward_call(*args, **kwargs) open-webui | │ │ └ {'seqlen_offset': 0} open-webui | │ └ (tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01, ..., 1.0232e-01, open-webui | │ 2.8146e-01, -2.1487e+00], open-webui | │ [ 1.... open-webui | └ <bound method NomicBertRotaryEmbedding.forward of NomicBertDynamicNTKRotaryEmbedding()> open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1394, in forward open-webui | k_rot = apply_rotary_emb(qkv[:, :, 1], self._cos_cached, self._sin_cached, seqlen_offset, self.interleaved) open-webui | │ │ │ │ │ │ │ │ └ False open-webui | │ │ │ │ │ │ │ └ NomicBertDynamicNTKRotaryEmbedding() open-webui | │ │ │ │ │ │ └ 0 open-webui | │ │ │ │ │ └ tensor([[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, open-webui | │ │ │ │ │ 0.0000, 0.0000, 0.0000, 0.0000,... open-webui | │ │ │ │ └ NomicBertDynamicNTKRotaryEmbedding() open-webui | │ │ │ └ tensor([[ 1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00, open-webui | │ │ │ 1.0000e+00, 1.0000e+00, 1.0000e+00, 1.... open-webui | │ │ └ NomicBertDynamicNTKRotaryEmbedding() open-webui | │ └ tensor([[[[[-1.2684e+00, -1.1195e+00, -7.7451e-01, ..., 1.0232e-01, open-webui | │ 2.8146e-01, -2.1487e+00], open-webui | │ [ 1.7... open-webui | └ <function apply_rotary_emb at 0x7f7d991062a0> open-webui | open-webui | File "/app/backend/data/cache/embedding/models/modules/transformers_modules/nomic-ai/nomic-bert-2048/7710840340a098cfb869c4f65e87cf2b1b70caca/modeling_hf_nomic_bert.py", line 1280, in apply_rotary_emb open-webui | [x[..., :ro_dim] * cos + rotate_half(x[..., :ro_dim], interleaved) * sin, x[..., ro_dim:]], open-webui | │ │ │ │ │ │ │ │ │ └ 64 open-webui | │ │ │ │ │ │ │ │ └ tensor([[[[ 1.0443e-02, -2.7018e-01, 2.8168e-01, ..., -1.1565e+00, open-webui | │ │ │ │ │ │ │ │ 6.9990e-01, -2.2852e+00], open-webui | │ │ │ │ │ │ │ │ [ 6.8054... open-webui | │ │ │ │ │ │ │ └ tensor([[[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, open-webui | │ │ │ │ │ │ │ 0.0000, 0.0000, 0.0000, 0.0000, 0.000... open-webui | │ │ │ │ │ │ └ False open-webui | │ │ │ │ │ └ 64 open-webui | │ │ │ │ └ tensor([[[[ 1.0443e-02, -2.7018e-01, 2.8168e-01, ..., -1.1565e+00, open-webui | │ │ │ │ 6.9990e-01, -2.2852e+00], open-webui | │ │ │ │ [ 6.8054... open-webui | │ │ │ └ <function rotate_half at 0x7f7d991054e0> open-webui | │ │ └ tensor([[[ 1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00, open-webui | │ │ 1.0000e+00, 1.0000e+00, 1.0000e+00, ... open-webui | │ └ 64 open-webui | └ tensor([[[[ 1.0443e-02, -2.7018e-01, 2.8168e-01, ..., -1.1565e+00, open-webui | 6.9990e-01, -2.2852e+00], open-webui | [ 6.8054... open-webui | open-webui | RuntimeError: The size of tensor a (13) must match the size of tensor b (12) at non-singleton dimension 1 ```

GiteaMirror added the bug label 2025-11-11 16:41:31 -06:00

GiteaMirror closed this issue 2025-11-11 16:41:31 -06:00

GiteaMirror commented 2025-11-11 16:41:32 -06:00

Author

Owner

Copy Link

Copy Source

@rgaricano commented on GitHub (Aug 7, 2025):

This problem probably stems from Nomic's embedding model. Some issues have been reported (in different places, here are some when loading multiple files https://github.com/open-webui/open-webui/discussions/14502 ), and it seems to be related to a certain “difficulty” some Nomic models have in managing encoding when using multithreading.
As when uploadig multiple files or hybrid search, both run multi threads.

@rgaricano commented on GitHub (Aug 7, 2025): This problem probably stems from Nomic's embedding model. Some issues have been reported (in different places, here are some when loading multiple files https://github.com/open-webui/open-webui/discussions/14502 ), and it seems to be related to a certain “difficulty” some Nomic models have in managing encoding when using multithreading. As when uploadig multiple files or hybrid search, both run multi threads.

GiteaMirror commented 2025-11-11 16:41:32 -06:00

Author

Owner

Copy Link

Copy Source

@frenzybiscuit commented on GitHub (Aug 8, 2025):

I also cant use Qwen3 (either reranker or embedding model) with sentencetransformers.

I managed to find models that do work, but I'd prefer something larger / more recent.

Is there a list of compatible models somewhere?

@frenzybiscuit commented on GitHub (Aug 8, 2025): I also cant use Qwen3 (either reranker or embedding model) with sentencetransformers. I managed to find models that do work, but I'd prefer something larger / more recent. Is there a list of compatible models somewhere?

GiteaMirror commented 2025-11-11 16:41:32 -06:00

Author

Owner

Copy Link

Copy Source

@tjbck commented on GitHub (Aug 8, 2025):

Model implementation issue.

@tjbck commented on GitHub (Aug 8, 2025): Model implementation issue.

GiteaMirror commented 2025-11-11 16:41:32 -06:00

Author

Owner

Copy Link

Copy Source

@rgaricano commented on GitHub (Aug 8, 2025):

embebb that I'm using, working fine: Alibaba-NLP/gte-multilingual-base
rerank: BAAI/bge-reranker-v2-m3

@rgaricano commented on GitHub (Aug 8, 2025): embebb that I'm using, working fine: `Alibaba-NLP/gte-multilingual-base` rerank: `BAAI/bge-reranker-v2-m3`

GiteaMirror referenced this issue 2026-04-19 20:38:17 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #14195

GiteaMirror referenced this issue 2026-04-25 04:09:12 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #29723

GiteaMirror referenced this issue 2026-05-05 14:02:51 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #52861

GiteaMirror referenced this issue 2026-05-13 01:13:44 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #68799

GiteaMirror referenced this issue 2026-05-15 09:46:39 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #84935

GiteaMirror referenced this issue 2026-05-17 22:49:55 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #101190

GiteaMirror referenced this issue 2026-05-20 19:07:34 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #117506

GiteaMirror referenced this issue 2026-05-24 23:30:41 -05:00

[GH-ISSUE #5984] [FEAT] : Integration with OpenAI’s Assistants #133922

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dev

v0.13.3

v0.13.2

v0.13.1

v0.13.0

v0.12.5

v0.12.4

v0.12.3

v0.12.2

v0.12.1

v0.12.0

v0.11.4

v0.11.3

v0.11.2

v0.11.1

v0.11.0

v0.10.2

v0.10.1

v0.10.0

v0.9.6

v0.9.5

v0.9.4

v0.9.3

v0.9.2

v0.9.1

v0.9.0

v0.8.12

v0.8.11

v0.8.10

v0.8.9

v0.8.8

v0.8.7

v0.8.6

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.2

v0.7.1

v0.7.0

v0.6.43

v0.6.42

v0.6.41

v0.6.40

v0.6.39

v0.6.38

v0.6.37

v0.6.36

v0.6.35

v0.6.34

v0.6.33

v0.6.32

v0.6.31

v0.6.30

v0.6.29

v0.6.28

v0.6.27

v0.6.26

v0.6.25

v0.6.24

v0.6.23

v0.6.22

v0.6.21

v0.6.20

v0.6.19

v0.6.18

v0.6.17

v0.6.16

v0.6.15

v0.6.14

v0.6.13

v0.6.12

v0.6.11

v0.6.10

v0.6.9

v0.6.8

v0.6.7

v0.6.6

v0.6.5

v0.6.4

v0.6.3

v0.6.2

v0.6.1

v0.6.0

v0.5.20

v0.5.19

v0.5.18

v0.5.17

v0.5.16

v0.5.15

v0.5.14

v0.5.13

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.35

v0.3.34

v0.3.33

v0.3.32

v0.3.31

v0.3.30

v0.3.29

v0.3.28

v0.3.27

v0.3.26

v0.3.25

v0.3.24

v0.3.23

v0.3.22

v0.3.21

v0.3.20

v0.3.19

v0.3.18

v0.3.17

v0.3.16

v0.3.15

v0.3.14

v0.3.13

v0.3.12

v0.3.11

v0.3.10

v0.3.9

v0.3.8

v0.3.7

v0.3.6

v0.3.5

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.125

v0.1.124

v0.1.123

v0.1.122

v0.1.121

v0.1.120

v0.1.119

v0.1.118

v0.1.117

v0.1.116

v0.1.115

v0.1.114

v0.1.113

v0.1.112

v0.1.111

v0.1.110

v0.1.109

v0.1.108

v0.1.107

v0.1.106

v0.1.105

v0.1.104

v0.1.103

v0.1.102

Labels

Clear labels

bug

confirmed

confirmed issue

core

documentation

enhancement

good first issue

help wanted

non-core

pull-request

Mirrored from GitHub Pull Request

python

question

testing wanted

No labels bug

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

GiteaMirror ninjasurge (Kaehlen Rog)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#5984