Implement granular session management in process_file() to prevent database connection pool exhaustion during slow embedding operations.
Previously, the database session was acquired at request start and held until the entire file processing completed, including external embedding API calls (5-60+ seconds). Under concurrent file processing, this exhausted the connection pool, causing QueuePool timeout errors for other operations.
Changes:
Extract primitive values (file_id, file_filename, etc.) before any session commits
Commit and release the session before save_docs_to_vector_db() which makes slow external embedding API calls
Use get_db() context manager for final status updates after embedding completes
Use get_db() context manager for error status updates in exception handler
This fix applies to 6 endpoints that rely on process_file():
POST /api/v1/files/ (file upload with processing)
POST /api/v1/files/{id}/data/content/update
POST /api/v1/knowledge/{id}/file/add
POST /api/v1/knowledge/{id}/file/update
POST /api/v1/knowledge/reindex
POST /api/v1/retrieval/process/file
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
Note
Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/20573
**Author:** [@Classic298](https://github.com/Classic298)
**Created:** 1/11/2026
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `fix/db-pool-process-file`
---
### 📝 Commits (1)
- [`1038bb0`](https://github.com/open-webui/open-webui/commit/1038bb080a20bc506a849b49597544143fa44afb) fix(db): release connection before embedding in process_file
### 📊 Changes
**1 file changed** (+99 additions, -74 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/routers/retrieval.py` (+99 -74)
</details>
### 📄 Description
Implement granular session management in process_file() to prevent database connection pool exhaustion during slow embedding operations.
Previously, the database session was acquired at request start and held until the entire file processing completed, including external embedding API calls (5-60+ seconds). Under concurrent file processing, this exhausted the connection pool, causing QueuePool timeout errors for other operations.
Changes:
- Extract primitive values (file_id, file_filename, etc.) before any session commits
- Commit and release the session before save_docs_to_vector_db() which makes slow external embedding API calls
- Use get_db() context manager for final status updates after embedding completes
- Use get_db() context manager for error status updates in exception handler
This fix applies to 6 endpoints that rely on process_file():
- POST /api/v1/files/ (file upload with processing)
- POST /api/v1/files/{id}/data/content/update
- POST /api/v1/knowledge/{id}/file/add
- POST /api/v1/knowledge/{id}/file/update
- POST /api/v1/knowledge/reindex
- POST /api/v1/retrieval/process/file
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/20573
Author: @Classic298
Created: 1/11/2026
Status: ❌ Closed
Base:
dev← Head:fix/db-pool-process-file📝 Commits (1)
1038bb0fix(db): release connection before embedding in process_file📊 Changes
1 file changed (+99 additions, -74 deletions)
View changed files
📝
backend/open_webui/routers/retrieval.py(+99 -74)📄 Description
Implement granular session management in process_file() to prevent database connection pool exhaustion during slow embedding operations.
Previously, the database session was acquired at request start and held until the entire file processing completed, including external embedding API calls (5-60+ seconds). Under concurrent file processing, this exhausted the connection pool, causing QueuePool timeout errors for other operations.
Changes:
Extract primitive values (file_id, file_filename, etc.) before any session commits
Commit and release the session before save_docs_to_vector_db() which makes slow external embedding API calls
Use get_db() context manager for final status updates after embedding completes
Use get_db() context manager for error status updates in exception handler
This fix applies to 6 endpoints that rely on process_file():
POST /api/v1/files/ (file upload with processing)
POST /api/v1/files/{id}/data/content/update
POST /api/v1/knowledge/{id}/file/add
POST /api/v1/knowledge/{id}/file/update
POST /api/v1/knowledge/reindex
POST /api/v1/retrieval/process/file
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.