[PR #24769] feat: add support for Valkey vector database #115140

New Issue

Open

opened 2026-05-18 16:06:00 -05:00 by GiteaMirror · 0 comments

GiteaMirror commented 2026-05-18 16:06:00 -05:00

Owner

Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24769
Author: @rileydes-improving
Created: 5/15/2026
Status: 🔄 Open

Base: dev ← Head: feat/add-valkey-vector-database

---

📝 Commits (1)

• 20252b5 feat: add support for Valkey vector database

📊 Changes

7 files changed (+744 additions, -1 deletions)

View changed files

📝 .env.example (+10 -1)
📝 backend/open_webui/config.py (+9 -0)
➕ backend/open_webui/retrieval/vector/dbs/valkey.py (+718 -0)
📝 backend/open_webui/retrieval/vector/factory.py (+4 -0)
📝 backend/open_webui/retrieval/vector/type.py (+1 -0)
📝 backend/requirements.txt (+1 -0)
📝 pyproject.toml (+1 -0)

📄 Description

Pull Request Checklist

Note to first-time contributors: Please open a discussion post in Discussions to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.

This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.

Issue -> https://github.com/open-webui/open-webui/issues/24623
Discussion -> https://github.com/open-webui/open-webui/discussions/24624

Pull Request Checklist

Before submitting, make sure you've checked the following:

• Linked Issue/Discussion: Closes #24623, relates to discussion #24624.
• Target branch: This PR targets dev.
• Description: See below.
• Changelog: Entry added below.
• Documentation: A companion docs PR will be opened in open-webui/docs covering VECTOR_DB=valkey, the new VALKEY_* environment variables, and module/version requirements.
• Dependencies: Adds valkey-glide-sync==2.3.1 (the official Valkey GLIDE synchronous Python client). Added to both backend/requirements.txt and pyproject.toml. The client is only imported lazily from the factory when VECTOR_DB=valkey, so existing deployments are unaffected. The code paths that use the client have been exercised end-to-end against valkey-bundle:9.1.0-rc2.
• Testing: 66/66 integration tests and 43/43 end-to-end application tests pass against valkey-bundle:9.1.0-rc2 (Valkey core 9.1.0 + valkey-search 1.2.0) and a custom image with (Valkey core 9.0.1 + valkey-search 1.2.0). Manually verified document ingestion, KNN search, filtered KNN, metadata queries, collection reset, and deletion by id/filter.
• Agentic AI Code: AI Assistance was used while writing this feature. Every line has been human-reviewed and manually tested against a running Valkey instance.
• Code review: Self-reviewed and internally reviewed by my team. Matches the pattern used by other backends in backend/open_webui/retrieval/vector/dbs/.
• Design & Architecture: No new user-facing settings — operators opt in by setting VECTOR_DB=valkey and VALKEY_URL. Defaults mirror the other vector backends.
• Git Hygiene: Single logical change on top of dev.
• Title Prefix: feat:

Changelog Entry

Description

Adds Valkey as a vector database backend for RAG, selectable via VECTOR_DB=valkey. The implementation uses the valkey-search module's native FT.CREATE / FT.SEARCH primitives for HNSW/FLAT vector indexing, KNN queries, and tag-filtered metadata queries, no client-side vector math. Requires valkey-search 1.2.0+ running on Valkey core 9.0.1+ (either a stable core with libsearch.so loaded via --loadmodule or the valkey-bundle:9.1.0-rc2+ image).

Motivation: Valkey is the BSD-3-licensed, community-governed fork of Redis after the Redis 7.4 license change. Valkey's search module is now feature-complete enough to back Open WebUI's RAG layer.

Added

• New vector backend valkey in backend/open_webui/retrieval/vector/dbs/valkey.py implementing the VectorDBBase interface (insert, upsert, search, query, get, delete, reset, has_collection, delete_collection).
• VectorType.VALKEY enum entry and factory wiring in backend/open_webui/retrieval/vector/{type,factory}.py.
• New environment variables in backend/open_webui/config.py and .env.example:
  • VALKEY_URL
  • VALKEY_COLLECTION_PREFIX (default open_webui)
  • VALKEY_INDEX_TYPE (HNSW | FLAT, default HNSW)
  • VALKEY_DISTANCE_METRIC (COSINE | L2 | IP, default COSINE)
  • VALKEY_HNSW_M (default 16)
  • VALKEY_HNSW_EF_CONSTRUCTION (default 200)
  • VALKEY_HNSW_EF_RUNTIME (default 10)
• Startup validation that inspects MODULE LIST and fails fast if search is missing or older than 1.2.0.
• New runtime dependency: valkey-glide-sync==2.3.1 (added to backend/requirements.txt and pyproject.toml). This is the official Valkey GLIDE synchronous Python client, which provides native support for valkey-search module commands (ft.create, ft.search, ft.info, etc.) with proper response parsing.

Changed

• None.

Deprecated

• None.

Removed

• None.

Fixed

• None.

Security

• No new attack surface. Connection URL follows the same pattern as existing Redis configuration (valkey://host:port/db). No credentials are logged.

Breaking Changes

• None. Existing VECTOR_DB values and behavior are unchanged. Valkey is purely opt-in.

---

Additional Information

Module version requirement. valkey-search 1.2.0 is the binding dependency. It's what adds TEXT fields, TAG expressions, and filter-only FT.SEARCH. Operators have two supported deployment paths:

• Option A: any stable Valkey core 9.0.1+ with valkey-search 1.2.0+ loaded via --loadmodule libsearch.so.
• Option B: valkey/valkey-bundle:9.1.0-rc2 or later (single image).

The runtime check validates either path at startup.

Known workarounds (both upstream-tracked, negligible runtime impact):

1. FT.SEARCH idx "*" wildcard is not yet in a tagged valkey-search release, get() falls back to SCAN + HGETALL. Tracked in valkey-search#957, fix merged in PR #960, expected in 1.2.1/1.3.0. get() is not on the hot search path.
2. The computed __vector_score field isn't returned when a RETURN clause is used, and can't be used in SORTBY. The search() method omits RETURN and sorts client-side. Tracked as an enhancement in valkey-search#989. Extra payload is ~1.5KB per result; client-side sort of ≤10 items is microseconds.

Both workarounds are one-line changes once upstream ships the fixes.

Testing (not committed, local to my working tree):

• 66 integration tests covering schema creation, HNSW/FLAT, all three distance metrics, insert/upsert/delete, KNN, tag-filtered KNN, filter-only queries, reset, and error paths.
• 43 end-to-end tests exercising the real Open WebUI RAG flow against a containerized valkey 9.0.1+ and valkey-search 1.2.0.

Contributor License Agreement

• By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

---

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24769 **Author:** [@rileydes-improving](https://github.com/rileydes-improving) **Created:** 5/15/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `feat/add-valkey-vector-database` --- ### 📝 Commits (1) - [`20252b5`](https://github.com/open-webui/open-webui/commit/20252b5c9b390875fe4723a5c6bf177c5173bc4f) feat: add support for Valkey vector database ### 📊 Changes **7 files changed** (+744 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `.env.example` (+10 -1) 📝 `backend/open_webui/config.py` (+9 -0) ➕ `backend/open_webui/retrieval/vector/dbs/valkey.py` (+718 -0) 📝 `backend/open_webui/retrieval/vector/factory.py` (+4 -0) 📝 `backend/open_webui/retrieval/vector/type.py` (+1 -0) 📝 `backend/requirements.txt` (+1 -0) 📝 `pyproject.toml` (+1 -0) </details> ### 📄 Description <!-- ⚠️ CRITICAL CHECKS FOR CONTRIBUTORS (READ, DON'T DELETE) ⚠️ 1. Target the `dev` branch. PRs targeting `main` will be automatically closed. 2. Do NOT delete the CLA section at the bottom. It is required for the bot to accept your PR. --> # Pull Request Checklist ### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request. This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR. <!-- ### ⚠️ Important: Your PR is a contribution, not a guarantee of merge. The most impactful way to contribute to Open WebUI is through well-written bug reports, detailed feature discussions, and thoughtful ideas. These directly shape the project. If you do open a pull request, please know that Open WebUI is held to the highest standard of code quality, consistency, and architectural coherence, and every line merged becomes something the core team must own, maintain, and support indefinitely. Submitted code may be refactored, rewritten, or used as inspiration for a different implementation. This is not a reflection of your work's quality. It is how we ensure that a small team can deeply understand and evolve every part of the codebase. --> Issue -> https://github.com/open-webui/open-webui/issues/24623 Discussion -> https://github.com/open-webui/open-webui/discussions/24624 # Pull Request Checklist **Before submitting, make sure you've checked the following:** - [x] **Linked Issue/Discussion:** Closes #24623, relates to discussion #24624. - [x] **Target branch:** This PR targets `dev`. - [x] **Description:** See below. - [x] **Changelog:** Entry added below. - [x] **Documentation:** A companion docs PR will be opened in `open-webui/docs` covering `VECTOR_DB=valkey`, the new `VALKEY_*` environment variables, and module/version requirements. - [x] **Dependencies:** Adds `valkey-glide-sync==2.3.1` (the official Valkey GLIDE synchronous Python client). Added to both `backend/requirements.txt` and `pyproject.toml`. The client is only imported lazily from the factory when `VECTOR_DB=valkey`, so existing deployments are unaffected. The code paths that use the client have been exercised end-to-end against `valkey-bundle:9.1.0-rc2`. - [x] **Testing:** 66/66 integration tests and 43/43 end-to-end application tests pass against `valkey-bundle:9.1.0-rc2` (Valkey core 9.1.0 + valkey-search 1.2.0) and a custom image with (Valkey core 9.0.1 + valkey-search 1.2.0). Manually verified document ingestion, KNN search, filtered KNN, metadata queries, collection reset, and deletion by id/filter. - [x] **Agentic AI Code:** AI Assistance was used while writing this feature. Every line has been human-reviewed and manually tested against a running Valkey instance. - [x] **Code review:** Self-reviewed and internally reviewed by my team. Matches the pattern used by other backends in `backend/open_webui/retrieval/vector/dbs/`. - [x] **Design & Architecture:** No new user-facing settings — operators opt in by setting `VECTOR_DB=valkey` and `VALKEY_URL`. Defaults mirror the other vector backends. - [x] **Git Hygiene:** Single logical change on top of `dev`. - [x] **Title Prefix:** `feat:` # Changelog Entry ### Description Adds Valkey as a vector database backend for RAG, selectable via `VECTOR_DB=valkey`. The implementation uses the [valkey-search](https://github.com/valkey-io/valkey-search) module's native `FT.CREATE` / `FT.SEARCH` primitives for HNSW/FLAT vector indexing, KNN queries, and tag-filtered metadata queries, no client-side vector math. Requires **valkey-search 1.2.0+** running on **Valkey core 9.0.1+** (either a stable core with `libsearch.so` loaded via `--loadmodule` or the `valkey-bundle:9.1.0-rc2+` image). Motivation: Valkey is the BSD-3-licensed, community-governed fork of Redis after the Redis 7.4 license change. Valkey's search module is now feature-complete enough to back Open WebUI's RAG layer. ### Added - New vector backend `valkey` in `backend/open_webui/retrieval/vector/dbs/valkey.py` implementing the `VectorDBBase` interface (`insert`, `upsert`, `search`, `query`, `get`, `delete`, `reset`, `has_collection`, `delete_collection`). - `VectorType.VALKEY` enum entry and factory wiring in `backend/open_webui/retrieval/vector/{type,factory}.py`. - New environment variables in `backend/open_webui/config.py` and `.env.example`: - `VALKEY_URL` - `VALKEY_COLLECTION_PREFIX` (default `open_webui`) - `VALKEY_INDEX_TYPE` (`HNSW` | `FLAT`, default `HNSW`) - `VALKEY_DISTANCE_METRIC` (`COSINE` | `L2` | `IP`, default `COSINE`) - `VALKEY_HNSW_M` (default `16`) - `VALKEY_HNSW_EF_CONSTRUCTION` (default `200`) - `VALKEY_HNSW_EF_RUNTIME` (default `10`) - Startup validation that inspects `MODULE LIST` and fails fast if `search` is missing or older than 1.2.0. - New runtime dependency: `valkey-glide-sync==2.3.1` (added to `backend/requirements.txt` and `pyproject.toml`). This is the official Valkey GLIDE synchronous Python client, which provides native support for valkey-search module commands (`ft.create`, `ft.search`, `ft.info`, etc.) with proper response parsing. ### Changed - None. ### Deprecated - None. ### Removed - None. ### Fixed - None. ### Security - No new attack surface. Connection URL follows the same pattern as existing Redis configuration (`valkey://host:port/db`). No credentials are logged. ### Breaking Changes - None. Existing `VECTOR_DB` values and behavior are unchanged. Valkey is purely opt-in. --- ### Additional Information **Module version requirement.** valkey-search 1.2.0 is the binding dependency. It's what adds TEXT fields, TAG expressions, and filter-only `FT.SEARCH`. Operators have two supported deployment paths: - **Option A:** any stable Valkey core 9.0.1+ with valkey-search 1.2.0+ loaded via `--loadmodule libsearch.so`. - **Option B:** `valkey/valkey-bundle:9.1.0-rc2` or later (single image). The runtime check validates either path at startup. **Known workarounds (both upstream-tracked, negligible runtime impact):** 1. `FT.SEARCH idx "*"` wildcard is not yet in a tagged valkey-search release, `get()` falls back to `SCAN` + `HGETALL`. Tracked in [valkey-search#957](https://github.com/valkey-io/valkey-search/issues/957), fix merged in [PR #960](https://github.com/valkey-io/valkey-search/pull/960), expected in 1.2.1/1.3.0. `get()` is not on the hot search path. 2. The computed `__vector_score` field isn't returned when a `RETURN` clause is used, and can't be used in `SORTBY`. The `search()` method omits `RETURN` and sorts client-side. Tracked as an enhancement in [valkey-search#989](https://github.com/valkey-io/valkey-search/issues/989). Extra payload is ~1.5KB per result; client-side sort of ≤10 items is microseconds. Both workarounds are one-line changes once upstream ships the fixes. **Testing** (not committed, local to my working tree): - 66 integration tests covering schema creation, HNSW/FLAT, all three distance metrics, insert/upsert/delete, KNN, tag-filtered KNN, filter-only queries, reset, and error paths. - 43 end-to-end tests exercising the real Open WebUI RAG flow against a containerized valkey 9.0.1+ and valkey-search 1.2.0. ### Contributor License Agreement <!-- 🚨 DO NOT DELETE THE TEXT BELOW 🚨 Keep the "Contributor License Agreement" confirmation text intact. Deleting it will trigger the CLA-Bot to INVALIDATE your PR. Your PR will NOT be reviewed or merged until you check the box below confirming that you have read and agree to the terms of the CLA. --> - [x] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- _{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

GiteaMirror added the pull-request label 2026-05-18 16:06:00 -05:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dev

v0.10.2

v0.10.1

v0.10.0

v0.9.6

v0.9.5

v0.9.4

v0.9.3

v0.9.2

v0.9.1

v0.9.0

v0.8.12

v0.8.11

v0.8.10

v0.8.9

v0.8.8

v0.8.7

v0.8.6

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.2

v0.7.1

v0.7.0

v0.6.43

v0.6.42

v0.6.41

v0.6.40

v0.6.39

v0.6.38

v0.6.37

v0.6.36

v0.6.35

v0.6.34

v0.6.33

v0.6.32

v0.6.31

v0.6.30

v0.6.29

v0.6.28

v0.6.27

v0.6.26

v0.6.25

v0.6.24

v0.6.23

v0.6.22

v0.6.21

v0.6.20

v0.6.19

v0.6.18

v0.6.17

v0.6.16

v0.6.15

v0.6.14

v0.6.13

v0.6.12

v0.6.11

v0.6.10

v0.6.9

v0.6.8

v0.6.7

v0.6.6

v0.6.5

v0.6.4

v0.6.3

v0.6.2

v0.6.1

v0.6.0

v0.5.20

v0.5.19

v0.5.18

v0.5.17

v0.5.16

v0.5.15

v0.5.14

v0.5.13

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.35

v0.3.34

v0.3.33

v0.3.32

v0.3.31

v0.3.30

v0.3.29

v0.3.28

v0.3.27

v0.3.26

v0.3.25

v0.3.24

v0.3.23

v0.3.22

v0.3.21

v0.3.20

v0.3.19

v0.3.18

v0.3.17

v0.3.16

v0.3.15

v0.3.14

v0.3.13

v0.3.12

v0.3.11

v0.3.10

v0.3.9

v0.3.8

v0.3.7

v0.3.6

v0.3.5

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.125

v0.1.124

v0.1.123

v0.1.122

v0.1.121

v0.1.120

v0.1.119

v0.1.118

v0.1.117

v0.1.116

v0.1.115

v0.1.114

v0.1.113

v0.1.112

v0.1.111

v0.1.110

v0.1.109

v0.1.108

v0.1.107

v0.1.106

v0.1.105

v0.1.104

v0.1.103

v0.1.102

Labels

Clear labels

bug

confirmed

confirmed issue

core

documentation

enhancement

good first issue

help wanted

non-core

pull-request

Mirrored from GitHub Pull Request

python

question

testing wanted

No Label pull-request

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

GiteaMirror ninjasurge

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#115140