[PR #4698] [CLOSED] FEAT: Implement batch processing for OpenAI embeddings #21538

New Issue

GiteaMirror · 2026-04-20T03:34:35-05:00

GiteaMirror commented

2026-04-20 03:34:35 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/4698
Author: @c0sogi
Created: 8/18/2024
Status: ❌ Closed

Base: dev ← Head: dev

📝 Commits (5)

2002989 feat: added batch api for openai embedding
484c48d Merge branch 'open-webui:dev' into dev
6d5b511 Merge branch 'open-webui:dev' into dev
e59c018 refac: remove hardcoded parameters in batch_utils
abbd606 test: batch_utils

📊 Changes

3 files changed (+470 additions, -23 deletions)

View changed files

➕ backend/apps/rag/batch_utils.py (+258 -0)
📝 backend/apps/rag/utils.py (+1 -23)
➕ backend/test/apps/rag/test_batch_utils.py (+211 -0)

📄 Description

Description

This pull request introduces batch processing functionality for OpenAI embeddings, significantly improving the handling of large text inputs. The new implementation allows for more efficient processing of text data exceeding 500,000 characters.

Changelog Entry

Added

New file backend/apps/rag/batch_utils.py implementing batch processing for OpenAI embeddings
Async processing for large text inputs in OpenAI embeddings generation

Changed

Updated backend/apps/rag/utils.py to incorporate the new batch processing functionality
Modified the generate_openai_batch_embeddings function to use either sync or async processing based on input size

Fixed

Improved handling of large text inputs for OpenAI embeddings, addressing potential rate limiting issues

Pull Request Checklist

Target branch: This pull request targets the dev branch.
Description: A concise description of the changes is provided above.
Changelog: A changelog entry following the format of Keep a Changelog has been added.
Documentation: No additional documentation needed, since functionality remains the same from user's side.
Dependencies: No new dependencies were added. Currently supports both openai and tenacity packages.
Testing: A test function test_openai_batch_api_embedding has been included in backend/test/apps/rag/test_batch_utils.py.
Code review: A self-review of the code has been performed, addressing coding standards and project guidelines.
Prefix: The pull request title is prefixed with "FEAT" to indicate a new feature.

Additional Information

This implementation significantly enhances our ability to process large volumes of text for embedding generation. It automatically switches between synchronous and asynchronous processing based on the input size, optimizing for both small and large text inputs. Currently, switching threshold is set to 500,000 chars.

Screenshots or Videos

N/A for this backend implementation.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/4698 **Author:** [@c0sogi](https://github.com/c0sogi) **Created:** 8/18/2024 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `dev` --- ### 📝 Commits (5) - [`2002989`](https://github.com/open-webui/open-webui/commit/200298907175ebf0721547aeb80c9a36ccc5f831) feat: added batch api for openai embedding - [`484c48d`](https://github.com/open-webui/open-webui/commit/484c48d359c666af59771478118b24e312e02a93) Merge branch 'open-webui:dev' into dev - [`6d5b511`](https://github.com/open-webui/open-webui/commit/6d5b51149a93f252578fe0e740c2ac0c2517cf0d) Merge branch 'open-webui:dev' into dev - [`e59c018`](https://github.com/open-webui/open-webui/commit/e59c01820255d72112df0784ab780f80e0d41965) refac: remove hardcoded parameters in batch_utils - [`abbd606`](https://github.com/open-webui/open-webui/commit/abbd6064934d11b48de4361a8d24bb81b68342e2) test: batch_utils ### 📊 Changes **3 files changed** (+470 additions, -23 deletions) <details> <summary>View changed files</summary> ➕ `backend/apps/rag/batch_utils.py` (+258 -0) 📝 `backend/apps/rag/utils.py` (+1 -23) ➕ `backend/test/apps/rag/test_batch_utils.py` (+211 -0) </details> ### 📄 Description ## Description This pull request introduces batch processing functionality for OpenAI embeddings, significantly improving the handling of large text inputs. The new implementation allows for more efficient processing of text data exceeding 500,000 characters. ## Changelog Entry ### Added - New file `backend/apps/rag/batch_utils.py` implementing batch processing for OpenAI embeddings - Async processing for large text inputs in OpenAI embeddings generation ### Changed - Updated `backend/apps/rag/utils.py` to incorporate the new batch processing functionality - Modified the `generate_openai_batch_embeddings` function to use either sync or async processing based on input size ### Fixed - Improved handling of large text inputs for OpenAI embeddings, addressing potential rate limiting issues ## Pull Request Checklist - [x] **Target branch:** This pull request targets the `dev` branch. - [x] **Description:** A concise description of the changes is provided above. - [x] **Changelog:** A changelog entry following the format of Keep a Changelog has been added. - [x] **Documentation:** No additional documentation needed, since functionality remains the same from user's side. - [x] **Dependencies:** No new dependencies were added. Currently supports both `openai` and `tenacity` packages. - [x] **Testing:** A test function `test_openai_batch_api_embedding` has been included in `backend/test/apps/rag/test_batch_utils.py`. - [x] **Code review:** A self-review of the code has been performed, addressing coding standards and project guidelines. - [x] **Prefix:** The pull request title is prefixed with "FEAT" to indicate a new feature. ## Additional Information This implementation significantly enhances our ability to process large volumes of text for embedding generation. It automatically switches between synchronous and asynchronous processing based on the input size, optimizing for both small and large text inputs. Currently, switching threshold is set to `500,000` chars. ## Screenshots or Videos N/A for this backend implementation. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 03:34:35 -05:00

GiteaMirror closed this issue

2026-04-20 03:34:36 -05:00

GiteaMirror referenced this issue

2026-04-20 06:20:59 -05:00

[PR #21538] [MERGED] i18n: Update fi-FI translation #26130

GiteaMirror referenced this issue

2026-04-25 13:54:36 -05:00

[PR #21538] [MERGED] i18n: Update fi-FI translation #41760

GiteaMirror referenced this issue

2026-04-30 01:29:59 -05:00

[PR #21538] [MERGED] i18n: Update fi-FI translation #49178