Adds support for Sentence Transformer Instructor Models and Custom Ollama and OpenAI Compatible APIs that host an instruction embedding model. This will allow for possible external support to models such as NV-Embed-v2. This feature has been already requested in discussions such as #6623
Added
Added three new environment variables: RAG_EMBEDDING_QUERY_PREFIX, RAG_EMBEDDING_PASSAGE_PREFIX and RAG_EMBEDDING_PREFIX_FIELD_NAME that default to None.
RAG_EMBEDDING_QUERY_PREFIX is used for prefixing questions. RAG_EMBEDDING_PASSAGE_PREFIX is used for prefixing documents. RAG_EMBEDDING_PREFIX_FIELD_NAME is to determine the json data key name to be sent for the prefix value or if the text being embedded should be prefixed Open WebUI server side.
If using OpenAI or Ollama API
If RAG_EMBEDDING_PREFIX_FIELD_NAME is None and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the text embeddings are prefixed beforehand with there respective prompts before being sent to the APIs.
If RAG_EMBEDDING_PREFIX_FIELD_NAME is set and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the respective prompt values are sent with the RAG_EMBEDDING_PREFIX_FIELD_NAME key to the APIs within the json_data.
If SentenceTransformer is being used with a supported instruct model, then prefix is passed as a value for prompt as specified in the SentenceTransformer INSTRUCTOR models Documentation. RAG_EMBEDDING_PREFIX_FIELD_NAME is ignored.
If RAG_EMBEDDING_PASSAGE_PREFIX is None, then no prefix is used for embedding documents, and if RAG_EMBEDDING_QUERY_PREFIX is None, then no prefix is used for embedding queries.
Changed
embedding function now has two inputs: input and prefix
If ollama and openai api now send {RAG_EMBEDDING_PREFIX_FIELD_NAME: prefix} in json_data if environment variables are set.
Deprecated
Na
Removed
Na
Fixed
Na
Security
Na
Breaking Changes
Changing the way embedding requests are sent if new environment variables are set. However, I believe this should not affect compatibility.
Possible adding new environment variables could break the system. However, I am not quite sure how the PersistentConfig Class works so would appreciate some insight.
Additional Information
I believe it is fine to send extra fields to the openAI api if environment variables are accidentally set, but testing on this would be appreciated.
Screenshots or Videos
Not yet
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/8594
**Author:** [@jayteaftw](https://github.com/jayteaftw)
**Created:** 1/16/2025
**Status:** ✅ Merged
**Merged:** 3/31/2025
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `main`
---
### 📝 Commits (6)
- [`47b8412`](https://github.com/open-webui/open-webui/commit/47b8412695d0bc07705bda54ad07cd9145d09d66) Initialize support for prefixing embeddings
- [`7b8e5d4`](https://github.com/open-webui/open-webui/commit/7b8e5d4e7cb03d79ee832dc1107b8d74a405ae2e) Fixed errors and added more support
- [`81102f4`](https://github.com/open-webui/open-webui/commit/81102f4be210f7b1a1f76744515e07077ced16be) Merge branch 'open-webui:main' into main
- [`6d2f87e`](https://github.com/open-webui/open-webui/commit/6d2f87e9044800320656c98a501302f2f6a3f56a) Added server side Prefixing
- [`157c781`](https://github.com/open-webui/open-webui/commit/157c781b0af435dab15dbf98ae1c15b7f3b01400) Merge branch 'main' of https://github.com/jayteaftw/open-webui
- [`40dea3f`](https://github.com/open-webui/open-webui/commit/40dea3fbe142cd1d7b0cb3b5ecca3c7544cac20f) Merge branch 'dev' into main
### 📊 Changes
**3 files changed** (+72 additions, -32 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/config.py` (+12 -0)
📝 `backend/open_webui/retrieval/utils.py` (+54 -28)
📝 `backend/open_webui/routers/retrieval.py` (+6 -4)
</details>
### 📄 Description
# Changelog Entry
### Description
- Adds support for [Sentence Transformer Instructor Models](https://sbert.net/docs/sentence_transformer/pretrained_models.html#instructor-models) and Custom Ollama and OpenAI Compatible APIs that host an instruction embedding model. This will allow for possible external support to models such as [NV-Embed-v2](https://huggingface.co/nvidia/NV-Embed-v2). This feature has been already requested in discussions such as [#6623](https://github.com/ollama/ollama/issues/6623)
### Added
- Added three new environment variables: RAG_EMBEDDING_QUERY_PREFIX, RAG_EMBEDDING_PASSAGE_PREFIX and RAG_EMBEDDING_PREFIX_FIELD_NAME that default to None.
- RAG_EMBEDDING_QUERY_PREFIX is used for prefixing questions. RAG_EMBEDDING_PASSAGE_PREFIX is used for prefixing documents. RAG_EMBEDDING_PREFIX_FIELD_NAME is to determine the json data key name to be sent for the prefix value or if the text being embedded should be prefixed Open WebUI server side.
- If using OpenAI or Ollama API
- If RAG_EMBEDDING_PREFIX_FIELD_NAME is None and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the text embeddings are prefixed beforehand with there respective prompts before being sent to the APIs.
- If RAG_EMBEDDING_PREFIX_FIELD_NAME is set and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the respective prompt values are sent with the RAG_EMBEDDING_PREFIX_FIELD_NAME key to the APIs within the json_data.
- If SentenceTransformer is being used with a supported instruct model, then prefix is passed as a value for prompt as specified in the [SentenceTransformer INSTRUCTOR models Documentation](https://sbert.net/docs/sentence_transformer/pretrained_models.html#instructor-models). RAG_EMBEDDING_PREFIX_FIELD_NAME is ignored.
- If RAG_EMBEDDING_PASSAGE_PREFIX is None, then no prefix is used for embedding documents, and if RAG_EMBEDDING_QUERY_PREFIX is None, then no prefix is used for embedding queries.
### Changed
- embedding function now has two inputs: input and prefix
- If ollama and openai api now send {RAG_EMBEDDING_PREFIX_FIELD_NAME: prefix} in json_data if environment variables are set.
### Deprecated
- Na
### Removed
- Na
### Fixed
- Na
### Security
- Na
### Breaking Changes
- Changing the way embedding requests are sent if new environment variables are set. However, I believe this should not affect compatibility.
- Possible adding new environment variables could break the system. However, I am not quite sure how the PersistentConfig Class works so would appreciate some insight.
---
### Additional Information
- I believe it is fine to send extra fields to the openAI api if environment variables are accidentally set, but testing on this would be appreciated.
### Screenshots or Videos
- Not yet
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/8594
Author: @jayteaftw
Created: 1/16/2025
Status: ✅ Merged
Merged: 3/31/2025
Merged by: @tjbck
Base:
dev← Head:main📝 Commits (6)
47b8412Initialize support for prefixing embeddings7b8e5d4Fixed errors and added more support81102f4Merge branch 'open-webui:main' into main6d2f87eAdded server side Prefixing157c781Merge branch 'main' of https://github.com/jayteaftw/open-webui40dea3fMerge branch 'dev' into main📊 Changes
3 files changed (+72 additions, -32 deletions)
View changed files
📝
backend/open_webui/config.py(+12 -0)📝
backend/open_webui/retrieval/utils.py(+54 -28)📝
backend/open_webui/routers/retrieval.py(+6 -4)📄 Description
Changelog Entry
Description
Added
Changed
Deprecated
Removed
Fixed
Security
Breaking Changes
Additional Information
Screenshots or Videos
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.