This is an update to my previous PR which adds a toggle for hybrid searching. Once toggled, it will unlock the reranking model field as well as the relevance score threshold. Even if you turn it on, you do not have to use reranking. It's only used if the reranking model is updated and set.
Also fixed a small issue where querying a collection would result in the scores out of order. Embeddings use a distance while reranking scores (0-1). This ensures the collections are ranked in reverse order when using hybrid + reranking, so they appear .99, .98. .97...; while embeddings (no reranking) are 300, 301, 302... respectively (this has not changed even with this PR).
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/1756
**Author:** [@buroa](https://github.com/buroa)
**Created:** 4/25/2024
**Status:** ✅ Merged
**Merged:** 4/26/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `buroa/toggle-hybrid`
---
### 📝 Commits (3)
- [`9755cd5`](https://github.com/open-webui/open-webui/commit/9755cd5baa367620f6b1f08ef0565498c505e10b) feat: toggle hybrid search
- [`69822e4`](https://github.com/open-webui/open-webui/commit/69822e4c25f038e7aace0a1f029c40009836c267) fix: sort ranking hybrid
- [`cebf733`](https://github.com/open-webui/open-webui/commit/cebf733b9d0e188e1fd903707a2342678008f4ff) refac: naming convention
### 📊 Changes
**5 files changed** (+155 additions, -102 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/apps/rag/main.py` (+26 -1)
📝 `backend/apps/rag/utils.py` (+18 -18)
📝 `backend/config.py` (+4 -0)
📝 `backend/main.py` (+1 -0)
📝 `src/lib/components/documents/Settings/General.svelte` (+106 -83)
</details>
### 📄 Description
This is an update to my previous PR which adds a toggle for hybrid searching. Once toggled, it will unlock the reranking model field as well as the relevance score threshold. Even if you turn it on, you do not have to use reranking. It's only used if the reranking model is updated and set.
Also fixed a small issue where querying a collection would result in the scores out of order. Embeddings use a distance while reranking scores (0-1). This ensures the collections are ranked in reverse order when using hybrid + reranking, so they appear .99, .98. .97...; while embeddings (no reranking) are 300, 301, 302... respectively (this *has not changed* even with this PR).
<img width="733" alt="Screenshot 2024-04-25 at 5 32 02 PM" src="https://github.com/open-webui/open-webui/assets/36205263/e3e903fe-a490-4f17-8410-6f8cb2aba7cd">
Ref: #1693
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/1756
Author: @buroa
Created: 4/25/2024
Status: ✅ Merged
Merged: 4/26/2024
Merged by: @tjbck
Base:
dev← Head:buroa/toggle-hybrid📝 Commits (3)
9755cd5feat: toggle hybrid search69822e4fix: sort ranking hybridcebf733refac: naming convention📊 Changes
5 files changed (+155 additions, -102 deletions)
View changed files
📝
backend/apps/rag/main.py(+26 -1)📝
backend/apps/rag/utils.py(+18 -18)📝
backend/config.py(+4 -0)📝
backend/main.py(+1 -0)📝
src/lib/components/documents/Settings/General.svelte(+106 -83)📄 Description
This is an update to my previous PR which adds a toggle for hybrid searching. Once toggled, it will unlock the reranking model field as well as the relevance score threshold. Even if you turn it on, you do not have to use reranking. It's only used if the reranking model is updated and set.
Also fixed a small issue where querying a collection would result in the scores out of order. Embeddings use a distance while reranking scores (0-1). This ensures the collections are ranked in reverse order when using hybrid + reranking, so they appear .99, .98. .97...; while embeddings (no reranking) are 300, 301, 302... respectively (this has not changed even with this PR).
Ref: #1693
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.