mirror of
https://github.com/open-webui/open-webui.git
synced 2026-08-24 14:23:59 -05:00
Classic298andGitHub
484fb61743
The streaming handler's content accumulator is a closure cell (declared nonlocal in stream_body_handler), and CPython's in-place string append optimization only applies to plain local variables (STORE_FAST), never to cell variables (STORE_DEREF). The content += value form introduced in #27231 therefore still allocates and copies the full accumulated string on every delta, exactly like the f-string it replaced; whether that copy is cheap or expensive is up to the allocator, and measurements swing accordingly (6 to 83 ms of pure copying for a 400 KB response on Python 3.12, against 0.3 ms for this patch). Accumulate the deltas in a list instead and join once at the single read site (publish_chat_finished_event at stream end). List append is amortized O(1) with no dependence on reference counts, bytecode specialization or allocator behaviour, so accumulation is O(n) by construction. The non-str fallback keeps the previous f-string coercion semantics. Verified end to end against a mock SSE upstream: a streamed chat with a think-tag block plus 40 content deltas produces output items, message text, reasoning text and usage identical to current dev, with no errors in the server log.