enh: Colbert v2 support #2079

New Issue

2025-11-11T14:59:58-06:00

GiteaMirror commented

2025-11-11 14:59:58 -06:00

Originally created by @rjmalagon on GitHub (Sep 13, 2024).

Originally assigned to: @tjbck on GitHub.

I use Open-Webui via the official Docker image.

Some colbert models have been successfully used as a reranker on Open-Webui, but Jina Colbert v2 needs some extra python dependencies for sentence transformer to support it. https://huggingface.co/jinaai/jina-colbert-v2

This particular colbert model boasts multilingual support, which is a significant advancement over the sole English language version of Jina Colbert v1 [https://huggingface.co/jinaai/jina-colbert-v1-en] (https://huggingface.co/jinaai/jina-colbert-v1-en). Jina Colbert v1 works as a reranker on Open-Webui.

Given my current proficiency level, I am unable to evaluate whether the requisite Python dependencies would pose a significant obstacle to providing support for Open-Webui.

I cite the repository.

jina-colbert-v2 is trained with flash attention and therefore requires einops and flash_attn to be
installed.

To use the model, you could either use the Standford ColBERT library or use the pylate/ragatouille package that we provide.

pip install -U einops flash_attn
pip install -U ragatouille # or
pip install -U colbert-ai # or
pip install -U pylate

This error pops up when Jina Colbert v2 is loaded on current Open-Webui

Something went wrong :/ Unrecognized configuration class <class 'transformers_modules.jinaai.xlm-roberta-flash-implementation.b20a61116af69b71d3d5f9ed9ddb2042c82b2eb1.configuration_xlm_roberta.XLMRobertaFlashConfig'> for this kind of AutoModel: AutoModelForSequenceClassification. Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BloomConfig, CamembertConfig, CanineConfig, LlamaConfig, ConvBertConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FalconConfig, FlaubertConfig, FNetConfig, FunnelConfig, GemmaConfig, Gemma2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTJConfig, IBertConfig, JambaConfig, JetMoeConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LukeConfig, MarkupLMConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MobileBertConfig, MPNetConfig, MptConfig, MraConfig, MT5Config, MvpConfig, NemotronConfig, NezhaConfig, NystromformerConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PerceiverConfig, PersimmonConfig, PhiConfig, Phi3Config, PLBartConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, StableLmConfig, Starcoder2Config, T5Config, TapasConfig, TransfoXLConfig, UMT5Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.

Originally created by @rjmalagon on GitHub (Sep 13, 2024). Originally assigned to: @tjbck on GitHub. I use Open-Webui via the official Docker image. Some colbert models have been successfully used as a reranker on Open-Webui, but Jina Colbert v2 needs some extra python dependencies for sentence transformer to support it. [https://huggingface.co/jinaai/jina-colbert-v2](https://huggingface.co/jinaai/jina-colbert-v2) This particular colbert model boasts multilingual support, which is a significant advancement over the sole English language version of Jina Colbert v1 [https://huggingface.co/jinaai/jina-colbert-v1-en] (https://huggingface.co/jinaai/jina-colbert-v1-en). Jina Colbert v1 works as a reranker on Open-Webui. Given my current proficiency level, I am unable to evaluate whether the requisite Python dependencies would pose a significant obstacle to providing support for Open-Webui. I cite the repository. > jina-colbert-v2 is trained with flash attention and therefore requires einops and flash_attn to be > installed. > > To use the model, you could either use the Standford ColBERT library or use the pylate/ragatouille package that we provide. > > pip install -U einops flash_attn > pip install -U ragatouille # or > pip install -U colbert-ai # or > pip install -U pylate > This error pops up when Jina Colbert v2 is loaded on current Open-Webui `Something went wrong :/ Unrecognized configuration class <class 'transformers_modules.jinaai.xlm-roberta-flash-implementation.b20a61116af69b71d3d5f9ed9ddb2042c82b2eb1.configuration_xlm_roberta.XLMRobertaFlashConfig'> for this kind of AutoModel: AutoModelForSequenceClassification. Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BloomConfig, CamembertConfig, CanineConfig, LlamaConfig, ConvBertConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FalconConfig, FlaubertConfig, FNetConfig, FunnelConfig, GemmaConfig, Gemma2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTJConfig, IBertConfig, JambaConfig, JetMoeConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LukeConfig, MarkupLMConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MobileBertConfig, MPNetConfig, MptConfig, MraConfig, MT5Config, MvpConfig, NemotronConfig, NezhaConfig, NystromformerConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PerceiverConfig, PersimmonConfig, PhiConfig, Phi3Config, PLBartConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, StableLmConfig, Starcoder2Config, T5Config, TapasConfig, TransfoXLConfig, UMT5Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.`

GiteaMirror closed this issue

2025-11-11 14:59:58 -06:00

GiteaMirror commented

2025-11-11 14:59:59 -06:00

@tjbck commented on GitHub (Sep 13, 2024):

Seems like it's not supported by sentence_transformers, major refac will be required here. Will personally take a look!

@tjbck commented on GitHub (Sep 13, 2024): Seems like it's not supported by `sentence_transformers`, major refac will be required here. Will personally take a look!

GiteaMirror commented

2025-11-11 14:59:59 -06:00