github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-03-22 14:13:08 -05:00
Code Issues 308 Packages Projects Releases 100 Wiki Activity

enh: Colbert v2 support #2079

New Issue
Closed
opened 2025-11-11 14:59:58 -06:00 by GiteaMirror · 2 comments
GiteaMirror commented 2025-11-11 14:59:58 -06:00
Owner
Copy Link

Originally created by @rjmalagon on GitHub (Sep 13, 2024).

Originally assigned to: @tjbck on GitHub.

I use Open-Webui via the official Docker image.

Some colbert models have been successfully used as a reranker on Open-Webui, but Jina Colbert v2 needs some extra python dependencies for sentence transformer to support it. https://huggingface.co/jinaai/jina-colbert-v2

This particular colbert model boasts multilingual support, which is a significant advancement over the sole English language version of Jina Colbert v1 [https://huggingface.co/jinaai/jina-colbert-v1-en] (https://huggingface.co/jinaai/jina-colbert-v1-en). Jina Colbert v1 works as a reranker on Open-Webui.

Given my current proficiency level, I am unable to evaluate whether the requisite Python dependencies would pose a significant obstacle to providing support for Open-Webui.

I cite the repository.

jina-colbert-v2 is trained with flash attention and therefore requires einops and flash_attn to be
installed.

To use the model, you could either use the Standford ColBERT library or use the pylate/ragatouille package that we provide.

pip install -U einops flash_attn
pip install -U ragatouille # or
pip install -U colbert-ai # or
pip install -U pylate

This error pops up when Jina Colbert v2 is loaded on current Open-Webui

Something went wrong :/ Unrecognized configuration class <class 'transformers_modules.jinaai.xlm-roberta-flash-implementation.b20a61116af69b71d3d5f9ed9ddb2042c82b2eb1.configuration_xlm_roberta.XLMRobertaFlashConfig'> for this kind of AutoModel: AutoModelForSequenceClassification. Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BloomConfig, CamembertConfig, CanineConfig, LlamaConfig, ConvBertConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FalconConfig, FlaubertConfig, FNetConfig, FunnelConfig, GemmaConfig, Gemma2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTJConfig, IBertConfig, JambaConfig, JetMoeConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LukeConfig, MarkupLMConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MobileBertConfig, MPNetConfig, MptConfig, MraConfig, MT5Config, MvpConfig, NemotronConfig, NezhaConfig, NystromformerConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PerceiverConfig, PersimmonConfig, PhiConfig, Phi3Config, PLBartConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, StableLmConfig, Starcoder2Config, T5Config, TapasConfig, TransfoXLConfig, UMT5Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.

Originally created by @rjmalagon on GitHub (Sep 13, 2024). Originally assigned to: @tjbck on GitHub. I use Open-Webui via the official Docker image. Some colbert models have been successfully used as a reranker on Open-Webui, but Jina Colbert v2 needs some extra python dependencies for sentence transformer to support it. [https://huggingface.co/jinaai/jina-colbert-v2](https://huggingface.co/jinaai/jina-colbert-v2) This particular colbert model boasts multilingual support, which is a significant advancement over the sole English language version of Jina Colbert v1 [https://huggingface.co/jinaai/jina-colbert-v1-en] (https://huggingface.co/jinaai/jina-colbert-v1-en). Jina Colbert v1 works as a reranker on Open-Webui. Given my current proficiency level, I am unable to evaluate whether the requisite Python dependencies would pose a significant obstacle to providing support for Open-Webui. I cite the repository. > jina-colbert-v2 is trained with flash attention and therefore requires einops and flash_attn to be > installed. > > To use the model, you could either use the Standford ColBERT library or use the pylate/ragatouille package that we provide. > > pip install -U einops flash_attn > pip install -U ragatouille # or > pip install -U colbert-ai # or > pip install -U pylate > This error pops up when Jina Colbert v2 is loaded on current Open-Webui `Something went wrong :/ Unrecognized configuration class <class 'transformers_modules.jinaai.xlm-roberta-flash-implementation.b20a61116af69b71d3d5f9ed9ddb2042c82b2eb1.configuration_xlm_roberta.XLMRobertaFlashConfig'> for this kind of AutoModel: AutoModelForSequenceClassification. Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BloomConfig, CamembertConfig, CanineConfig, LlamaConfig, ConvBertConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FalconConfig, FlaubertConfig, FNetConfig, FunnelConfig, GemmaConfig, Gemma2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTJConfig, IBertConfig, JambaConfig, JetMoeConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LukeConfig, MarkupLMConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MobileBertConfig, MPNetConfig, MptConfig, MraConfig, MT5Config, MvpConfig, NemotronConfig, NezhaConfig, NystromformerConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PerceiverConfig, PersimmonConfig, PhiConfig, Phi3Config, PLBartConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, StableLmConfig, Starcoder2Config, T5Config, TapasConfig, TransfoXLConfig, UMT5Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.`
GiteaMirror commented 2025-11-11 14:59:59 -06:00
Author
Owner
Copy Link

@tjbck commented on GitHub (Sep 13, 2024):

Seems like it's not supported by sentence_transformers, major refac will be required here. Will personally take a look!

@tjbck commented on GitHub (Sep 13, 2024): Seems like it's not supported by `sentence_transformers`, major refac will be required here. Will personally take a look!
GiteaMirror commented 2025-11-11 14:59:59 -06:00
Author
Owner
Copy Link

@tjbck commented on GitHub (Sep 16, 2024):

Basic support has been added to dev, testing wanted here!

@tjbck commented on GitHub (Sep 16, 2024): Basic support has been added to dev, testing wanted here!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#2079
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?