github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 23:21:44 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[PR #8594] [MERGED] feat: Support for instruct/prefixing embeddings #93292

New Issue
Closed
opened 2026-05-15 19:02:13 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-15 19:02:13 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/8594
Author: @jayteaftw
Created: 1/16/2025
Status: ✅ Merged
Merged: 3/31/2025
Merged by: @tjbck

Base: dev ← Head: main

📝 Commits (6)

📊 Changes

3 files changed (+72 additions, -32 deletions)

View changed files

📝 backend/open_webui/config.py (+12 -0)
📝 backend/open_webui/retrieval/utils.py (+54 -28)
📝 backend/open_webui/routers/retrieval.py (+6 -4)

📄 Description

Changelog Entry

Description

  • Adds support for Sentence Transformer Instructor Models and Custom Ollama and OpenAI Compatible APIs that host an instruction embedding model. This will allow for possible external support to models such as NV-Embed-v2. This feature has been already requested in discussions such as #6623

Added

  • Added three new environment variables: RAG_EMBEDDING_QUERY_PREFIX, RAG_EMBEDDING_PASSAGE_PREFIX and RAG_EMBEDDING_PREFIX_FIELD_NAME that default to None.
  • RAG_EMBEDDING_QUERY_PREFIX is used for prefixing questions. RAG_EMBEDDING_PASSAGE_PREFIX is used for prefixing documents. RAG_EMBEDDING_PREFIX_FIELD_NAME is to determine the json data key name to be sent for the prefix value or if the text being embedded should be prefixed Open WebUI server side.
  • If using OpenAI or Ollama API
    • If RAG_EMBEDDING_PREFIX_FIELD_NAME is None and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the text embeddings are prefixed beforehand with there respective prompts before being sent to the APIs.
    • If RAG_EMBEDDING_PREFIX_FIELD_NAME is set and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the respective prompt values are sent with the RAG_EMBEDDING_PREFIX_FIELD_NAME key to the APIs within the json_data.
  • If SentenceTransformer is being used with a supported instruct model, then prefix is passed as a value for prompt as specified in the SentenceTransformer INSTRUCTOR models Documentation. RAG_EMBEDDING_PREFIX_FIELD_NAME is ignored.
  • If RAG_EMBEDDING_PASSAGE_PREFIX is None, then no prefix is used for embedding documents, and if RAG_EMBEDDING_QUERY_PREFIX is None, then no prefix is used for embedding queries.

Changed

  • embedding function now has two inputs: input and prefix
  • If ollama and openai api now send {RAG_EMBEDDING_PREFIX_FIELD_NAME: prefix} in json_data if environment variables are set.

Deprecated

  • Na

Removed

  • Na

Fixed

  • Na

Security

  • Na

Breaking Changes

  • Changing the way embedding requests are sent if new environment variables are set. However, I believe this should not affect compatibility.
  • Possible adding new environment variables could break the system. However, I am not quite sure how the PersistentConfig Class works so would appreciate some insight.

Additional Information

  • I believe it is fine to send extra fields to the openAI api if environment variables are accidentally set, but testing on this would be appreciated.

Screenshots or Videos

  • Not yet

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/8594 **Author:** [@jayteaftw](https://github.com/jayteaftw) **Created:** 1/16/2025 **Status:** ✅ Merged **Merged:** 3/31/2025 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `main` --- ### 📝 Commits (6) - [`47b8412`](https://github.com/open-webui/open-webui/commit/47b8412695d0bc07705bda54ad07cd9145d09d66) Initialize support for prefixing embeddings - [`7b8e5d4`](https://github.com/open-webui/open-webui/commit/7b8e5d4e7cb03d79ee832dc1107b8d74a405ae2e) Fixed errors and added more support - [`81102f4`](https://github.com/open-webui/open-webui/commit/81102f4be210f7b1a1f76744515e07077ced16be) Merge branch 'open-webui:main' into main - [`6d2f87e`](https://github.com/open-webui/open-webui/commit/6d2f87e9044800320656c98a501302f2f6a3f56a) Added server side Prefixing - [`157c781`](https://github.com/open-webui/open-webui/commit/157c781b0af435dab15dbf98ae1c15b7f3b01400) Merge branch 'main' of https://github.com/jayteaftw/open-webui - [`40dea3f`](https://github.com/open-webui/open-webui/commit/40dea3fbe142cd1d7b0cb3b5ecca3c7544cac20f) Merge branch 'dev' into main ### 📊 Changes **3 files changed** (+72 additions, -32 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/config.py` (+12 -0) 📝 `backend/open_webui/retrieval/utils.py` (+54 -28) 📝 `backend/open_webui/routers/retrieval.py` (+6 -4) </details> ### 📄 Description # Changelog Entry ### Description - Adds support for [Sentence Transformer Instructor Models](https://sbert.net/docs/sentence_transformer/pretrained_models.html#instructor-models) and Custom Ollama and OpenAI Compatible APIs that host an instruction embedding model. This will allow for possible external support to models such as [NV-Embed-v2](https://huggingface.co/nvidia/NV-Embed-v2). This feature has been already requested in discussions such as [#6623](https://github.com/ollama/ollama/issues/6623) ### Added - Added three new environment variables: RAG_EMBEDDING_QUERY_PREFIX, RAG_EMBEDDING_PASSAGE_PREFIX and RAG_EMBEDDING_PREFIX_FIELD_NAME that default to None. - RAG_EMBEDDING_QUERY_PREFIX is used for prefixing questions. RAG_EMBEDDING_PASSAGE_PREFIX is used for prefixing documents. RAG_EMBEDDING_PREFIX_FIELD_NAME is to determine the json data key name to be sent for the prefix value or if the text being embedded should be prefixed Open WebUI server side. - If using OpenAI or Ollama API - If RAG_EMBEDDING_PREFIX_FIELD_NAME is None and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the text embeddings are prefixed beforehand with there respective prompts before being sent to the APIs. - If RAG_EMBEDDING_PREFIX_FIELD_NAME is set and RAG_EMBEDDING_QUERY_PREFIX or RAG_EMBEDDING_PASSAGE_PREFIX are set, then the respective prompt values are sent with the RAG_EMBEDDING_PREFIX_FIELD_NAME key to the APIs within the json_data. - If SentenceTransformer is being used with a supported instruct model, then prefix is passed as a value for prompt as specified in the [SentenceTransformer INSTRUCTOR models Documentation](https://sbert.net/docs/sentence_transformer/pretrained_models.html#instructor-models). RAG_EMBEDDING_PREFIX_FIELD_NAME is ignored. - If RAG_EMBEDDING_PASSAGE_PREFIX is None, then no prefix is used for embedding documents, and if RAG_EMBEDDING_QUERY_PREFIX is None, then no prefix is used for embedding queries. ### Changed - embedding function now has two inputs: input and prefix - If ollama and openai api now send {RAG_EMBEDDING_PREFIX_FIELD_NAME: prefix} in json_data if environment variables are set. ### Deprecated - Na ### Removed - Na ### Fixed - Na ### Security - Na ### Breaking Changes - Changing the way embedding requests are sent if new environment variables are set. However, I believe this should not affect compatibility. - Possible adding new environment variables could break the system. However, I am not quite sure how the PersistentConfig Class works so would appreciate some insight. --- ### Additional Information - I believe it is fine to send extra fields to the openAI api if environment variables are accidentally set, but testing on this would be appreciated. ### Screenshots or Videos - Not yet --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-15 19:02:13 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#93292
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?