[PR #20547] [MERGED] fix(files): prevent connection pool exhaustion in file status streaming #129323

New Issue

GiteaMirror · 2026-05-21T12:30:21-05:00

GiteaMirror commented

2026-05-21 12:30:21 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/20547
Author: @Classic298
Created: 1/10/2026
Status: ✅ Merged
Merged: 1/10/2026
Merged by: @tjbck

Base: dev ← Head: db-perf3

📝 Commits (1)

f988f65 fix(files): prevent connection pool exhaustion in file status streaming

📊 Changes

1 file changed (+25 additions, -22 deletions)

View changed files

📝 backend/open_webui/routers/files.py (+25 -22)

📄 Description

Refactored the file processing status streaming endpoint to avoid holding a database connection for the entire stream duration (up to 2 hours). Changes:

Each status poll now creates its own short-lived database session instead of capturing the request's session in the generator closure
Increased poll interval from 0.5s to 1s, halving database queries with negligible UX impact This prevents a single file status stream from blocking a connection pool slot for hours, which could contribute to pool exhaustion under load.

The old code also passed the entire file object but only ever used file.id from it. Passing just the ID is:

Cleaner - explicitly shows we only need the ID
Avoids keeping a reference to the full file object in the closure
Functionally identical - same behavior

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/20547 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 1/10/2026 **Status:** ✅ Merged **Merged:** 1/10/2026 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `db-perf3` --- ### 📝 Commits (1) - [`f988f65`](https://github.com/open-webui/open-webui/commit/f988f65cc4f8bf8a253092f0a0f5fd90a91d0b72) fix(files): prevent connection pool exhaustion in file status streaming ### 📊 Changes **1 file changed** (+25 additions, -22 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/routers/files.py` (+25 -22) </details> ### 📄 Description Refactored the file processing status streaming endpoint to avoid holding a database connection for the entire stream duration (up to 2 hours). Changes: - Each status poll now creates its own short-lived database session instead of capturing the request's session in the generator closure - Increased poll interval from 0.5s to 1s, halving database queries with negligible UX impact This prevents a single file status stream from blocking a connection pool slot for hours, which could contribute to pool exhaustion under load. The old code also passed the entire file object but only ever used file.id from it. Passing just the ID is: - Cleaner - explicitly shows we only need the ID - Avoids keeping a reference to the full file object in the closure - Functionally identical - same behavior ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-21 12:30:21 -05:00

GiteaMirror closed this issue

2026-05-21 12:30:25 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#129323