github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-05-06 02:48:13 -05:00
Code Issues 1.1k Packages Projects Releases 126 Wiki Activity

[PR #21542] [MERGED] fix: migration streaming/batching #41761

New Issue
Closed
opened 2026-04-25 13:54:38 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-25 13:54:38 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/21542
Author: @Classic298
Created: 2/17/2026
Status: ✅ Merged
Merged: 3/8/2026
Merged by: @tjbck

Base: dev ← Head: fix-analytics-usage-v2

📝 Commits (4)

  • caed925 fix: normalize usage tokens + migration streaming/batching
  • bdccf02 fix: restore defensive timestamp conversion in migration
  • 64520d2 Merge branch 'dev' into fix-analytics-usage-v2
  • fb65a25 revert: remove changes to chat_messages.py

📊 Changes

1 file changed (+78 additions, -38 deletions)

View changed files

📝 backend/open_webui/migrations/versions/8452d01d26d7_add_chat_message_table.py (+78 -38)

📄 Description

  • Migration: replace .fetchall() with yield_per streaming, replace per-message INSERT+SAVEPOINT with batched inserts (5k/batch) with fallback to row-by-row on error, add progress logging

  • Write path: call normalize_usage() in upsert_message() before saving to ensure input_tokens/output_tokens always present

  • Read path: analytics queries now COALESCE across input_tokens/prompt_tokens and output_tokens/completion_tokens so historical data with OpenAI-format keys is visible

https://github.com/open-webui/open-webui/issues/21347

https://github.com/open-webui/open-webui/discussions/21539

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/21542 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 2/17/2026 **Status:** ✅ Merged **Merged:** 3/8/2026 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `fix-analytics-usage-v2` --- ### 📝 Commits (4) - [`caed925`](https://github.com/open-webui/open-webui/commit/caed925f431162955645abcec781a2973b78bc68) fix: normalize usage tokens + migration streaming/batching - [`bdccf02`](https://github.com/open-webui/open-webui/commit/bdccf02f392c87ff88d62e88a578d2ae1e6056c4) fix: restore defensive timestamp conversion in migration - [`64520d2`](https://github.com/open-webui/open-webui/commit/64520d2b6421c951a3fe71ff410688162119dac9) Merge branch 'dev' into fix-analytics-usage-v2 - [`fb65a25`](https://github.com/open-webui/open-webui/commit/fb65a25c294b3c5f7b51182a66a58ca4cd2c1438) revert: remove changes to chat_messages.py ### 📊 Changes **1 file changed** (+78 additions, -38 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/migrations/versions/8452d01d26d7_add_chat_message_table.py` (+78 -38) </details> ### 📄 Description - Migration: replace .fetchall() with yield_per streaming, replace per-message INSERT+SAVEPOINT with batched inserts (5k/batch) with fallback to row-by-row on error, add progress logging - Write path: call normalize_usage() in upsert_message() before saving to ensure input_tokens/output_tokens always present - Read path: analytics queries now COALESCE across input_tokens/prompt_tokens and output_tokens/completion_tokens so historical data with OpenAI-format keys is visible https://github.com/open-webui/open-webui/issues/21347 https://github.com/open-webui/open-webui/discussions/21539 ### Contributor License Agreement <!-- 🚨 DO NOT DELETE THE TEXT BELOW 🚨 Keep the "Contributor License Agreement" confirmation text intact. Deleting it will trigger the CLA-Bot to INVALIDATE your PR. --> By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-25 13:54:38 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#41761
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?