Refactored OpenSearchClient indexing strategy to match ElasticSearchClient behavior by creating indexes based on vector dimensions instead of individual files. This addresses scalability limitations that index-per-file approaches cause in OpenSearch clusters, resulting in shard exhaustion.
e.g. RequestError(400, 'illegal_argument_exception', 'Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [2999]/[3000] maximum shards open;')
Unified index management logic shared between OpenSearch and ElasticSearch clients
Breaking Changes
BREAKING CHANGE:
Requires migration of existing OpenSearch data to the new index structure. Currently, the migration is only possible by reindexing the whole vector database. If needed, a "smarter" script can be created to migrate the database without reembedding all the files.
Additional Information
The changes have been tested and deployed to our production OpenWebUI instance.
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/20685
**Author:** [@KrKOo](https://github.com/KrKOo)
**Created:** 1/15/2026
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `opensearch-optimization`
---
### 📝 Commits (1)
- [`74ea0e5`](https://github.com/open-webui/open-webui/commit/74ea0e533097c37c5da71888e1ac6fb1fb048727) enh: optimize opensearch index count
### 📊 Changes
**1 file changed** (+83 additions, -82 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/retrieval/vector/dbs/opensearch.py` (+83 -82)
</details>
### 📄 Description
# Changelog Entry
### Description
Refactored `OpenSearchClient` indexing strategy to match `ElasticSearchClient` behavior by creating indexes based on vector dimensions instead of individual files. This addresses scalability limitations that index-per-file approaches cause in OpenSearch clusters, resulting in shard exhaustion.
e.g.
`RequestError(400, 'illegal_argument_exception', 'Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [2999]/[3000] maximum shards open;')`
Discussions: https://github.com/open-webui/open-webui/discussions/19814, https://github.com/open-webui/open-webui/discussions/15507
### Changed
- Unified index management logic shared between OpenSearch and ElasticSearch clients
### Breaking Changes
- **BREAKING CHANGE**:
Requires migration of existing OpenSearch data to the new index structure. Currently, the migration is only possible by reindexing the whole vector database. If needed, a "smarter" script can be created to migrate the database without reembedding all the files.
---
### Additional Information
The changes have been tested and deployed to our production OpenWebUI instance.
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/20685
Author: @KrKOo
Created: 1/15/2026
Status: ❌ Closed
Base:
dev← Head:opensearch-optimization📝 Commits (1)
74ea0e5enh: optimize opensearch index count📊 Changes
1 file changed (+83 additions, -82 deletions)
View changed files
📝
backend/open_webui/retrieval/vector/dbs/opensearch.py(+83 -82)📄 Description
Changelog Entry
Description
Refactored
OpenSearchClientindexing strategy to matchElasticSearchClientbehavior by creating indexes based on vector dimensions instead of individual files. This addresses scalability limitations that index-per-file approaches cause in OpenSearch clusters, resulting in shard exhaustion.e.g.
RequestError(400, 'illegal_argument_exception', 'Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [2999]/[3000] maximum shards open;')Discussions: https://github.com/open-webui/open-webui/discussions/19814, https://github.com/open-webui/open-webui/discussions/15507
Changed
Breaking Changes
Requires migration of existing OpenSearch data to the new index structure. Currently, the migration is only possible by reindexing the whole vector database. If needed, a "smarter" script can be created to migrate the database without reembedding all the files.
Additional Information
The changes have been tested and deployed to our production OpenWebUI instance.
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.