mirror of
https://github.com/open-webui/open-webui.git
synced 2026-11-17 03:03:27 -06:00
[GH-ISSUE #23390] issue: Errors in embedding function with some models & transformers>=5.0 #123278
Reference in New Issue
Block a user
Originally created by @rgaricano on GitHub (Apr 3, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/23390
Check Existing Issues
Installation Method
Git Clone
Open WebUI Version
0.8.12
Ollama Version (if applicable)
No response
Operating System
ubuntu 24.04
Browser (if applicable)
No response
Confirmation
README.md.Expected Behavior
.
Actual Behavior
.
Steps to Reproduce
.
Logs & Screenshots
.
Additional Information
More than an issue this is an advise of incompatibility of some embedd models and
transformers>=5.0.In my case the error arise using 'Alibaba-NLP/gte-multilingual-base` and hybrid search, giving me this kind of errors:
also when trying to update the embedd model (clicking in the download arrow):
This happend because Transformers v5 now creates models on the meta device (shape-only, no real memory) before loading checkpoint weights. Non-persistent buffers like
position_idsaren't in the checkpoint, so they never get real values, ends up containing uninitialized garbage data, causing IndexError crashes or silently wrong embeddings.Models that don't re-initialize those buffers in the _init_weights method script have that problem (this wasn't needed before v5).
In my case, instead of simply changing the model, I patched it because, right now, I won't change the embedd model and have to re-embedd the vector database.
FIX:
In
backend/open_webui/routers/retrieval.pyIn
backend/open_webui/main.py(where the embedding function is initialized)Note:
I have a couple of possible generic solutions but as this it's not a Open-WebUI bug and the solution can be problematic in some scenaries maybe is better just to know and only patch it on own development if neccessary.
I hope this can be helpful if you have similar issues.
@Classic298 commented on GitHub (Apr 15, 2026):
Hey @rgaricano
The dependency was updated on dev (transformers 5.5.4, sentence-transformers 5.4.0), but no model-specific workaround was added to Open WebUI itself. The get_ef function still does a straightforward SentenceTransformer load without any position_ids patching.
This means users of Alibaba-NLP/gte-multilingual-base (or any other model that registers position_ids as a non-persistent buffer without re-initializing it in _init_weights) will still hit this issue with transformers >= 5.0.
That said, this is ultimately a model-level bug. The model's own modeling.py needs to properly handle meta-device initialization. Adding hardcoded model-specific patches in Open WebUI wouldn't be a sustainable path forward. The proper fix should come from the model authors on HuggingFace - until then that model is not compatible and deprecated since it is not compatible with newer versions of the transformers library.
In the meantime, if you're stuck on this model and can't re-embed, pinning transformers < 5.0 in your environment is probably the most reliable workaround.