This pull request introduces batch processing functionality for OpenAI embeddings, significantly improving the handling of large text inputs. The new implementation allows for more efficient processing of text data exceeding 500,000 characters.
Changelog Entry
Added
New file backend/apps/rag/batch_utils.py implementing batch processing for OpenAI embeddings
Async processing for large text inputs in OpenAI embeddings generation
Changed
Updated backend/apps/rag/utils.py to incorporate the new batch processing functionality
Modified the generate_openai_batch_embeddings function to use either sync or async processing based on input size
Fixed
Improved handling of large text inputs for OpenAI embeddings, addressing potential rate limiting issues
Pull Request Checklist
Target branch: This pull request targets the dev branch.
Description: A concise description of the changes is provided above.
Changelog: A changelog entry following the format of Keep a Changelog has been added.
Documentation: No additional documentation needed, since functionality remains the same from user's side.
Dependencies: No new dependencies were added. Currently supports both openai and tenacity packages.
Testing: A test function test_openai_batch_api_embedding has been included in backend/test/apps/rag/test_batch_utils.py.
Code review: A self-review of the code has been performed, addressing coding standards and project guidelines.
Prefix: The pull request title is prefixed with "FEAT" to indicate a new feature.
Additional Information
This implementation significantly enhances our ability to process large volumes of text for embedding generation. It automatically switches between synchronous and asynchronous processing based on the input size, optimizing for both small and large text inputs. Currently, switching threshold is set to 500,000 chars.
Screenshots or Videos
N/A for this backend implementation.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/4698
**Author:** [@c0sogi](https://github.com/c0sogi)
**Created:** 8/18/2024
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `dev`
---
### 📝 Commits (5)
- [`2002989`](https://github.com/open-webui/open-webui/commit/200298907175ebf0721547aeb80c9a36ccc5f831) feat: added batch api for openai embedding
- [`484c48d`](https://github.com/open-webui/open-webui/commit/484c48d359c666af59771478118b24e312e02a93) Merge branch 'open-webui:dev' into dev
- [`6d5b511`](https://github.com/open-webui/open-webui/commit/6d5b51149a93f252578fe0e740c2ac0c2517cf0d) Merge branch 'open-webui:dev' into dev
- [`e59c018`](https://github.com/open-webui/open-webui/commit/e59c01820255d72112df0784ab780f80e0d41965) refac: remove hardcoded parameters in batch_utils
- [`abbd606`](https://github.com/open-webui/open-webui/commit/abbd6064934d11b48de4361a8d24bb81b68342e2) test: batch_utils
### 📊 Changes
**3 files changed** (+470 additions, -23 deletions)
<details>
<summary>View changed files</summary>
➕ `backend/apps/rag/batch_utils.py` (+258 -0)
📝 `backend/apps/rag/utils.py` (+1 -23)
➕ `backend/test/apps/rag/test_batch_utils.py` (+211 -0)
</details>
### 📄 Description
## Description
This pull request introduces batch processing functionality for OpenAI embeddings, significantly improving the handling of large text inputs. The new implementation allows for more efficient processing of text data exceeding 500,000 characters.
## Changelog Entry
### Added
- New file `backend/apps/rag/batch_utils.py` implementing batch processing for OpenAI embeddings
- Async processing for large text inputs in OpenAI embeddings generation
### Changed
- Updated `backend/apps/rag/utils.py` to incorporate the new batch processing functionality
- Modified the `generate_openai_batch_embeddings` function to use either sync or async processing based on input size
### Fixed
- Improved handling of large text inputs for OpenAI embeddings, addressing potential rate limiting issues
## Pull Request Checklist
- [x] **Target branch:** This pull request targets the `dev` branch.
- [x] **Description:** A concise description of the changes is provided above.
- [x] **Changelog:** A changelog entry following the format of Keep a Changelog has been added.
- [x] **Documentation:** No additional documentation needed, since functionality remains the same from user's side.
- [x] **Dependencies:** No new dependencies were added. Currently supports both `openai` and `tenacity` packages.
- [x] **Testing:** A test function `test_openai_batch_api_embedding` has been included in `backend/test/apps/rag/test_batch_utils.py`.
- [x] **Code review:** A self-review of the code has been performed, addressing coding standards and project guidelines.
- [x] **Prefix:** The pull request title is prefixed with "FEAT" to indicate a new feature.
## Additional Information
This implementation significantly enhances our ability to process large volumes of text for embedding generation. It automatically switches between synchronous and asynchronous processing based on the input size, optimizing for both small and large text inputs. Currently, switching threshold is set to `500,000` chars.
## Screenshots or Videos
N/A for this backend implementation.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/4698
Author: @c0sogi
Created: 8/18/2024
Status: ❌ Closed
Base:
dev← Head:dev📝 Commits (5)
2002989feat: added batch api for openai embedding484c48dMerge branch 'open-webui:dev' into dev6d5b511Merge branch 'open-webui:dev' into deve59c018refac: remove hardcoded parameters in batch_utilsabbd606test: batch_utils📊 Changes
3 files changed (+470 additions, -23 deletions)
View changed files
➕
backend/apps/rag/batch_utils.py(+258 -0)📝
backend/apps/rag/utils.py(+1 -23)➕
backend/test/apps/rag/test_batch_utils.py(+211 -0)📄 Description
Description
This pull request introduces batch processing functionality for OpenAI embeddings, significantly improving the handling of large text inputs. The new implementation allows for more efficient processing of text data exceeding 500,000 characters.
Changelog Entry
Added
backend/apps/rag/batch_utils.pyimplementing batch processing for OpenAI embeddingsChanged
backend/apps/rag/utils.pyto incorporate the new batch processing functionalitygenerate_openai_batch_embeddingsfunction to use either sync or async processing based on input sizeFixed
Pull Request Checklist
devbranch.openaiandtenacitypackages.test_openai_batch_api_embeddinghas been included inbackend/test/apps/rag/test_batch_utils.py.Additional Information
This implementation significantly enhances our ability to process large volumes of text for embedding generation. It automatically switches between synchronous and asynchronous processing based on the input size, optimizing for both small and large text inputs. Currently, switching threshold is set to
500,000chars.Screenshots or Videos
N/A for this backend implementation.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.