8893857 Merge branch 'dev' into refactor/serialize-output-on
📊 Changes
1 file changed (+52 additions, -33 deletions)
View changed files
📝backend/open_webui/utils/middleware.py (+52 -33)
📄 Description
Summary
It's a server-side CPU improvement during token streaming. Each streamed chunk calls serialize_output to build the content string that gets emitted via the event emitter to the frontend. With the old code, every concatenation allocated and copied a new string — that's pure CPU waste on the Python backend.
serialize_output builds a display string by iterating output items. The old implementation used repeated string concatenation (content += ...), which copies the entire accumulated string on every append — O(n²) total work. This replaces it with list.append + a single join at the end — O(n).
Changes
Replace string accumulator with a parts list in serialize_output
Same treatment for inner accumulators (reasoning content, result text)
Replace split_content_and_whitespace / is_opening_code_block helpers with content_endswith_newline / ensure_trailing_newline operating on the parts list
Fix fence-stripping for code_interpreter: use regex so language-tagged fences (e.g. triple-backtick python) are correctly stripped — the old rstrip approach silently failed on these
Benefit
The gain scales with output list size × content length × call frequency. The biggest wins come from:
The worst case: a streaming response after several tool-call rounds where the tools returned large results. The output list is big, the accumulated content is big, and serialize_output is called hundreds of times — each time re-concatenating everything.
The best case (minimal gain): a simple text-only response with no tool calls, no reasoning — just one message item, one small concatenation, called a few times. Effectively no difference.
The practical effect is lower per-chunk processing latency on the server, which means:
Less CPU time spent per streaming event
Smoother token delivery to the frontend (less jitter between chunks)
Better scalability under concurrent users (less CPU contention per active stream)
Added co author of original PR
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
Note
Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/23388
**Author:** [@Classic298](https://github.com/Classic298)
**Created:** 4/3/2026
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `refactor/serialize-output-on`
---
### 📝 Commits (5)
- [`64ff29d`](https://github.com/open-webui/open-webui/commit/64ff29df4d4fc1172765fe43cbfa1857578625f5) refactor(middleware): optimize fence-stripping O(n²) → O(n)
- [`7088b88`](https://github.com/open-webui/open-webui/commit/7088b8817bb5c6bbebe4ed7085250241ff67b240) chore: convert serialize_output helpers to async
- [`ece8312`](https://github.com/open-webui/open-webui/commit/ece8312b7f01db906bfc712803404dc2e755a449) perf: keep in-memory helpers sync to avoid coroutine overhead
- [`0c4b697`](https://github.com/open-webui/open-webui/commit/0c4b6973d67e5657ddd51942f51db116cce74e90) revert: keep serialize_output synchronous
- [`8893857`](https://github.com/open-webui/open-webui/commit/8893857461b640910d22c0b03cc93c2e5351fb33) Merge branch 'dev' into refactor/serialize-output-on
### 📊 Changes
**1 file changed** (+52 additions, -33 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/utils/middleware.py` (+52 -33)
</details>
### 📄 Description
### Summary
It's a **server-side CPU improvement** during token streaming. Each streamed chunk calls serialize_output to build the content string that gets emitted via the event emitter to the frontend. With the old code, every concatenation allocated and copied a new string — that's pure CPU waste on the Python backend.
serialize_output builds a display string by iterating output items. The old implementation used repeated string concatenation (content += ...), which copies the entire accumulated string on every append — O(n²) total work. This replaces it with list.append + a single join at the end — O(n).
### Changes
- Replace string accumulator with a parts list in serialize_output
- Same treatment for inner accumulators (reasoning content, result text)
- Replace split_content_and_whitespace / is_opening_code_block helpers with content_endswith_newline / ensure_trailing_newline operating on the parts list
- Fix fence-stripping for code_interpreter: use regex so language-tagged fences (e.g. triple-backtick python) are correctly stripped — the old rstrip approach silently failed on these
# Benefit
The gain scales with **output list size × content length × call frequency**. The biggest wins come from:
**The worst case:** a streaming response after several tool-call rounds where the tools returned large results. The output list is big, the accumulated content is big, and serialize_output is called hundreds of times — each time re-concatenating everything.
**The best case (minimal gain):** a simple text-only response with no tool calls, no reasoning — just one message item, one small concatenation, called a few times. Effectively no difference.
The practical effect is **lower per-chunk processing latency** on the server, which means:
- Less CPU time spent per streaming event
- Smoother token delivery to the frontend (less jitter between chunks)
- Better scalability under concurrent users (less CPU contention per active stream)
Added co author of original PR
### Contributor License Agreement
<!--
🚨 DO NOT DELETE THE TEXT BELOW 🚨
Keep the "Contributor License Agreement" confirmation text intact.
Deleting it will trigger the CLA-Bot to INVALIDATE your PR.
Your PR will NOT be reviewed or merged until you check the box below confirming that you have read and agree to the terms of the CLA.
-->
- [X] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/23388
Author: @Classic298
Created: 4/3/2026
Status: ❌ Closed
Base:
dev← Head:refactor/serialize-output-on📝 Commits (5)
64ff29drefactor(middleware): optimize fence-stripping O(n²) → O(n)7088b88chore: convert serialize_output helpers to asyncece8312perf: keep in-memory helpers sync to avoid coroutine overhead0c4b697revert: keep serialize_output synchronous8893857Merge branch 'dev' into refactor/serialize-output-on📊 Changes
1 file changed (+52 additions, -33 deletions)
View changed files
📝
backend/open_webui/utils/middleware.py(+52 -33)📄 Description
Summary
It's a server-side CPU improvement during token streaming. Each streamed chunk calls serialize_output to build the content string that gets emitted via the event emitter to the frontend. With the old code, every concatenation allocated and copied a new string — that's pure CPU waste on the Python backend.
serialize_output builds a display string by iterating output items. The old implementation used repeated string concatenation (content += ...), which copies the entire accumulated string on every append — O(n²) total work. This replaces it with list.append + a single join at the end — O(n).
Changes
Benefit
The gain scales with output list size × content length × call frequency. The biggest wins come from:
The worst case: a streaming response after several tool-call rounds where the tools returned large results. The output list is big, the accumulated content is big, and serialize_output is called hundreds of times — each time re-concatenating everything.
The best case (minimal gain): a simple text-only response with no tool calls, no reasoning — just one message item, one small concatenation, called a few times. Effectively no difference.
The practical effect is lower per-chunk processing latency on the server, which means:
Added co author of original PR
Contributor License Agreement
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.