PR #1 of the post-v0.9.5 codebase review. Each change addresses a real
runtime bug or silently-broken code path that surfaced after the
upstream sync→async migration. All edits are mechanical — no behavior
choices that need product input. Tier 1 (spend ledger, enforcement
gaps) and Tier 2 (storage cache, loaders, migration safety) are
follow-up PRs.
Tier 0 — fixed in this PR
Missing await on async APIs
utils/task.py + routers/tasks.py — chapter_generation_template
was sync but called the now-async prompt_template without await,
so the LLM received a coroutine repr interpolated into the prompt
string. Wrapper made async; await added at definition and call site.
routers/audio.py (get_audio_segment, get_video_segment), routers/tasks.py (generate_chapters) — Files.get_file_by_id
is async post-merge; the sync calls returned coroutines that bypassed
the existence guard and crashed on .user_id.
utils/gmail_sender.py — same missing-await pattern on OAuthSessions.get_session_by_provider_and_user_id as the google_drive_client.py fix in 1cbfa1be0.
utils/knowledge_drive_sync.py — two missed awaits on Knowledges.get_knowledge_by_id and add_file_to_knowledge_by_id.
The latter meant Drive files uploaded to storage were silently never
linked into the target KB.
Pinecone namespace correctness
get_namespace_for_collection (in both retrieval/utils.py and routers/retrieval.py) — the helper made a sync call to the now-async Knowledges.get_knowledge_by_id; the swallowed AttributeError on .name meant the documented "{name}-{id}" prefix was never produced.
Dropped the lookup (matching what's actually been written to Pinecone
since the merge) and added a parent_collection_name parameter so
existing call sites in routers/knowledge.py stop raising the
swallowed TypeError that was leaving file-level namespaces orphaned
on KB delete/reset.
pinecone.pysearch() — caller filter was silently dropped.
Now safe-merges (mirror of query()).
pinecone.pydelete() — dict.update(filter) allowed a
caller-supplied collection_name to overwrite the enforced prefix.
Now safe-merges.
async_client.py — every method now accepts and forwards a namespace kwarg. Without this, VectorSearchRetriever (hybrid path)
and the admin vector-delete endpoint always TypeErrord.
Gmail data-deletion targeting
routers/gmail.pydelete_gmail_data and the force-full-sync
branch in utils/gmail_auto_sync.py — both were calling VECTOR_DB_CLIENT.delete with collection/namespace tuples that did
not match what SimplePineconeManager.schedule_upsert actually
writes. Aligned to the indexer's real layout (collection="gmail" + namespace="email-{user_id}" for Pinecone, collection="email-{user_id}"
for namespace-agnostic backends). "Delete my Gmail data" now actually
deletes vectors.
Form schema fix
routers/retrieval.pyQueryDocForm — added the hybrid_bm25_weight field the handler was already reading, and
dropped the unsupported user=user kwarg on the query_doc_with_hybrid_search call. Without these, every /query/doc
hybrid request raised AttributeError/TypeError.
Test plan
Re-deploy and verify chat completions still work end-to-end
Trigger chapter generation on a file with a transcript; confirm
chapters come back populated (no empty/coroutine content)
Stream an audio segment and a video segment via the segment
endpoints; confirm they download for the file owner and 403 for
non-owners
Send an email via the Gmail sender API; confirm token resolution
no longer fails with coroutine .id error
Trigger a Drive sync that adds a new file; confirm the file is
linked into the target knowledge base (visible in the KB UI)
Run a /query/doc hybrid query (ENABLE_RAG_HYBRID_SEARCH=true);
confirm no AttributeError: hybrid_bm25_weight
On a multi-file KB, confirm semantic search results no longer
bleed across files when a file_id filter is supplied
Disable Gmail sync from the UI ("delete my Gmail data") and
confirm vectors actually disappear from the Pinecone namespace
Delete a knowledge base and confirm both the parent collection and every per-file namespace are removed (no Pinecone orphans)
Out of scope (planned follow-ups)
PR #2 — fix/spend-ledger-and-enforcement: cost computation in normalize_usage, recording on non-stream paths, enforcement on /api/v1/tasks/*, fail-closed policy.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/24619
**Author:** [@PVBLIC-F](https://github.com/PVBLIC-F)
**Created:** 5/12/2026
**Status:** ❌ Closed
**Base:** `main` ← **Head:** `fix/post-merge-await-and-namespaces`
---
### 📝 Commits (10+)
- [`1adcd55`](https://github.com/open-webui/open-webui/commit/1adcd5533e243ec53fa8382ecbe750e311e5529b) fix: correct indentation in query retry loop
- [`a219c22`](https://github.com/open-webui/open-webui/commit/a219c2295d3289fcad8ba343f52f374866159c20) feat: increase citation modal size for better video display
- [`d7c1db6`](https://github.com/open-webui/open-webui/commit/d7c1db6c799065c2ba20081b90d7f2a9c715b49f) feat: comprehensive stopword filtering for topic extraction
- [`9776b2a`](https://github.com/open-webui/open-webui/commit/9776b2a6164cf5dfd53374a8d7ee26d7d1920a37) feat: filter headers, footers, and OCR noise from PDF/text extraction
- [`34a42fa`](https://github.com/open-webui/open-webui/commit/34a42fab6328fb9f64bb22074623017807ec270d) fix: remove OCR artifacts and noise from within text chunks
- [`959894f`](https://github.com/open-webui/open-webui/commit/959894fc17e11b779c95ab711f4d56a23812bca9) feat: upgrade cleaning level to 'standard' for better text quality
- [`1b3dae3`](https://github.com/open-webui/open-webui/commit/1b3dae357e2fe9b5f63a3ac41f1d601e6a74f5b8) feat: upgrade to hi_res strategy for superior PDF element classification
- [`37e7029`](https://github.com/open-webui/open-webui/commit/37e7029a4f558ac176684e7c32cc426170b768e4) feat: enhanced artifact removal for trailing noise and truncated text
- [`4a85722`](https://github.com/open-webui/open-webui/commit/4a85722d50b163b47414c8d408121906d00d2610) Fix video file cross-pollination in knowledge base collections
- [`63b49d8`](https://github.com/open-webui/open-webui/commit/63b49d83ea2efb6bdf9f8db524b0b86ef9275de4) Fix collection overwriting issue when uploading multiple files simultaneously
### 📊 Changes
**147 files changed** (+32484 additions, -1266 deletions)
<details>
<summary>View changed files</summary>
➕ `.cursorrules` (+1162 -0)
📝 `.github/workflows/build-release.disabled` (+0 -0)
📝 `.github/workflows/docker-build.disabled` (+0 -0)
📝 `.github/workflows/release-pypi.disabled` (+0 -0)
📝 `.gitignore` (+8 -0)
📝 `CHANGELOG.md` (+27 -0)
➕ `CODE_REVIEW_KNOWLEDGE_CLEANUP.md` (+349 -0)
➕ `CODE_REVIEW_VIDEO_STREAMING.md` (+402 -0)
➕ `CURSORRULES_REVIEW.md` (+363 -0)
📝 `Dockerfile` (+3 -0)
➕ `FEATURE_OVERVIEW_JSONB_OPTIMIZATIONS.md` (+960 -0)
➕ `NAMESPACE_FLOW_VERIFICATION.md` (+298 -0)
➕ `NAMESPACE_IMPLEMENTATION_REVIEW.md` (+567 -0)
➕ `PG17_PERFORMANCE_OPTIMIZATIONS.md` (+576 -0)
➕ `PHASE_5_6_REVIEW.md` (+422 -0)
➕ `PINECONE_NAMESPACE_ISOLATION.md` (+198 -0)
➕ `PINECONE_REFACTOR_SUMMARY.md` (+141 -0)
➕ `VIDEO_STREAMING_OPTIMIZATION.md` (+337 -0)
📝 `backend/open_webui/config.py` (+570 -39)
📝 `backend/open_webui/env.py` (+26 -3)
_...and 80 more files_
</details>
### 📄 Description
## Summary
PR #1 of the post-v0.9.5 codebase review. Each change addresses a real
runtime bug or silently-broken code path that surfaced after the
upstream sync→async migration. All edits are mechanical — no behavior
choices that need product input. Tier 1 (spend ledger, enforcement
gaps) and Tier 2 (storage cache, loaders, migration safety) are
follow-up PRs.
## Tier 0 — fixed in this PR
### Missing `await` on async APIs
- **`utils/task.py` + `routers/tasks.py`** — `chapter_generation_template`
was sync but called the now-async `prompt_template` without `await`,
so the LLM received a coroutine `repr` interpolated into the prompt
string. Wrapper made async; `await` added at definition and call site.
- **`routers/audio.py`** (`get_audio_segment`, `get_video_segment`),
**`routers/tasks.py`** (`generate_chapters`) — `Files.get_file_by_id`
is async post-merge; the sync calls returned coroutines that bypassed
the existence guard and crashed on `.user_id`.
- **`utils/gmail_sender.py`** — same missing-`await` pattern on
`OAuthSessions.get_session_by_provider_and_user_id` as the
`google_drive_client.py` fix in 1cbfa1be0.
- **`utils/knowledge_drive_sync.py`** — two missed awaits on
`Knowledges.get_knowledge_by_id` and `add_file_to_knowledge_by_id`.
The latter meant Drive files uploaded to storage were silently never
linked into the target KB.
### Pinecone namespace correctness
- **`get_namespace_for_collection`** (in both `retrieval/utils.py` and
`routers/retrieval.py`) — the helper made a sync call to the now-async
`Knowledges.get_knowledge_by_id`; the swallowed `AttributeError` on
`.name` meant the documented `"{name}-{id}"` prefix was never produced.
Dropped the lookup (matching what's actually been written to Pinecone
since the merge) and added a `parent_collection_name` parameter so
existing call sites in `routers/knowledge.py` stop raising the
swallowed `TypeError` that was leaving file-level namespaces orphaned
on KB delete/reset.
- **`pinecone.py` `search()`** — caller `filter` was silently dropped.
Now safe-merges (mirror of `query()`).
- **`pinecone.py` `delete()`** — `dict.update(filter)` allowed a
caller-supplied `collection_name` to overwrite the enforced prefix.
Now safe-merges.
- **`async_client.py`** — every method now accepts and forwards a
`namespace` kwarg. Without this, `VectorSearchRetriever` (hybrid path)
and the admin vector-delete endpoint always `TypeError`d.
### Gmail data-deletion targeting
- **`routers/gmail.py` `delete_gmail_data`** and the **force-full-sync
branch** in `utils/gmail_auto_sync.py` — both were calling
`VECTOR_DB_CLIENT.delete` with collection/namespace tuples that did
not match what `SimplePineconeManager.schedule_upsert` actually
writes. Aligned to the indexer's real layout (`collection="gmail"` +
`namespace="email-{user_id}"` for Pinecone, `collection="email-{user_id}"`
for namespace-agnostic backends). "Delete my Gmail data" now actually
deletes vectors.
### Form schema fix
- **`routers/retrieval.py` `QueryDocForm`** — added the
`hybrid_bm25_weight` field the handler was already reading, and
dropped the unsupported `user=user` kwarg on the
`query_doc_with_hybrid_search` call. Without these, every `/query/doc`
hybrid request raised `AttributeError`/`TypeError`.
## Test plan
- [ ] Re-deploy and verify chat completions still work end-to-end
- [ ] Trigger chapter generation on a file with a transcript; confirm
chapters come back populated (no empty/coroutine content)
- [ ] Stream an audio segment and a video segment via the segment
endpoints; confirm they download for the file owner and 403 for
non-owners
- [ ] Send an email via the Gmail sender API; confirm token resolution
no longer fails with coroutine `.id` error
- [ ] Trigger a Drive sync that adds a new file; confirm the file is
linked into the target knowledge base (visible in the KB UI)
- [ ] Run a `/query/doc` hybrid query (`ENABLE_RAG_HYBRID_SEARCH=true`);
confirm no `AttributeError: hybrid_bm25_weight`
- [ ] On a multi-file KB, confirm semantic search results no longer
bleed across files when a `file_id` filter is supplied
- [ ] Disable Gmail sync from the UI ("delete my Gmail data") and
confirm vectors actually disappear from the Pinecone namespace
- [ ] Delete a knowledge base and confirm both the parent collection
*and* every per-file namespace are removed (no Pinecone orphans)
## Out of scope (planned follow-ups)
- **PR #2 — `fix/spend-ledger-and-enforcement`**: cost computation in
`normalize_usage`, recording on non-stream paths, enforcement on
`/api/v1/tasks/*`, fail-closed policy.
- **PR #3 — `fix/migrations-storage-cache-loaders`**: storage cache
basename collisions, loader extension parsing, default Unstructured
strategy, `CONCURRENTLY` migration safety, header/`down_revision`
mismatches.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/24619
Author: @PVBLIC-F
Created: 5/12/2026
Status: ❌ Closed
Base:
main← Head:fix/post-merge-await-and-namespaces📝 Commits (10+)
1adcd55fix: correct indentation in query retry loopa219c22feat: increase citation modal size for better video displayd7c1db6feat: comprehensive stopword filtering for topic extraction9776b2afeat: filter headers, footers, and OCR noise from PDF/text extraction34a42fafix: remove OCR artifacts and noise from within text chunks959894ffeat: upgrade cleaning level to 'standard' for better text quality1b3dae3feat: upgrade to hi_res strategy for superior PDF element classification37e7029feat: enhanced artifact removal for trailing noise and truncated text4a85722Fix video file cross-pollination in knowledge base collections63b49d8Fix collection overwriting issue when uploading multiple files simultaneously📊 Changes
147 files changed (+32484 additions, -1266 deletions)
View changed files
➕
.cursorrules(+1162 -0)📝
.github/workflows/build-release.disabled(+0 -0)📝
.github/workflows/docker-build.disabled(+0 -0)📝
.github/workflows/release-pypi.disabled(+0 -0)📝
.gitignore(+8 -0)📝
CHANGELOG.md(+27 -0)➕
CODE_REVIEW_KNOWLEDGE_CLEANUP.md(+349 -0)➕
CODE_REVIEW_VIDEO_STREAMING.md(+402 -0)➕
CURSORRULES_REVIEW.md(+363 -0)📝
Dockerfile(+3 -0)➕
FEATURE_OVERVIEW_JSONB_OPTIMIZATIONS.md(+960 -0)➕
NAMESPACE_FLOW_VERIFICATION.md(+298 -0)➕
NAMESPACE_IMPLEMENTATION_REVIEW.md(+567 -0)➕
PG17_PERFORMANCE_OPTIMIZATIONS.md(+576 -0)➕
PHASE_5_6_REVIEW.md(+422 -0)➕
PINECONE_NAMESPACE_ISOLATION.md(+198 -0)➕
PINECONE_REFACTOR_SUMMARY.md(+141 -0)➕
VIDEO_STREAMING_OPTIMIZATION.md(+337 -0)📝
backend/open_webui/config.py(+570 -39)📝
backend/open_webui/env.py(+26 -3)...and 80 more files
📄 Description
Summary
PR #1 of the post-v0.9.5 codebase review. Each change addresses a real
runtime bug or silently-broken code path that surfaced after the
upstream sync→async migration. All edits are mechanical — no behavior
choices that need product input. Tier 1 (spend ledger, enforcement
gaps) and Tier 2 (storage cache, loaders, migration safety) are
follow-up PRs.
Tier 0 — fixed in this PR
Missing
awaiton async APIsutils/task.py+routers/tasks.py—chapter_generation_templatewas sync but called the now-async
prompt_templatewithoutawait,so the LLM received a coroutine
reprinterpolated into the promptstring. Wrapper made async;
awaitadded at definition and call site.routers/audio.py(get_audio_segment,get_video_segment),routers/tasks.py(generate_chapters) —Files.get_file_by_idis async post-merge; the sync calls returned coroutines that bypassed
the existence guard and crashed on
.user_id.utils/gmail_sender.py— same missing-awaitpattern onOAuthSessions.get_session_by_provider_and_user_idas thegoogle_drive_client.pyfix in1cbfa1be0.utils/knowledge_drive_sync.py— two missed awaits onKnowledges.get_knowledge_by_idandadd_file_to_knowledge_by_id.The latter meant Drive files uploaded to storage were silently never
linked into the target KB.
Pinecone namespace correctness
get_namespace_for_collection(in bothretrieval/utils.pyandrouters/retrieval.py) — the helper made a sync call to the now-asyncKnowledges.get_knowledge_by_id; the swallowedAttributeErroron.namemeant the documented"{name}-{id}"prefix was never produced.Dropped the lookup (matching what's actually been written to Pinecone
since the merge) and added a
parent_collection_nameparameter soexisting call sites in
routers/knowledge.pystop raising theswallowed
TypeErrorthat was leaving file-level namespaces orphanedon KB delete/reset.
pinecone.pysearch()— callerfilterwas silently dropped.Now safe-merges (mirror of
query()).pinecone.pydelete()—dict.update(filter)allowed acaller-supplied
collection_nameto overwrite the enforced prefix.Now safe-merges.
async_client.py— every method now accepts and forwards anamespacekwarg. Without this,VectorSearchRetriever(hybrid path)and the admin vector-delete endpoint always
TypeErrord.Gmail data-deletion targeting
routers/gmail.pydelete_gmail_dataand the force-full-syncbranch in
utils/gmail_auto_sync.py— both were callingVECTOR_DB_CLIENT.deletewith collection/namespace tuples that didnot match what
SimplePineconeManager.schedule_upsertactuallywrites. Aligned to the indexer's real layout (
collection="gmail"+namespace="email-{user_id}"for Pinecone,collection="email-{user_id}"for namespace-agnostic backends). "Delete my Gmail data" now actually
deletes vectors.
Form schema fix
routers/retrieval.pyQueryDocForm— added thehybrid_bm25_weightfield the handler was already reading, anddropped the unsupported
user=userkwarg on thequery_doc_with_hybrid_searchcall. Without these, every/query/dochybrid request raised
AttributeError/TypeError.Test plan
chapters come back populated (no empty/coroutine content)
endpoints; confirm they download for the file owner and 403 for
non-owners
no longer fails with coroutine
.iderrorlinked into the target knowledge base (visible in the KB UI)
/query/dochybrid query (ENABLE_RAG_HYBRID_SEARCH=true);confirm no
AttributeError: hybrid_bm25_weightbleed across files when a
file_idfilter is suppliedconfirm vectors actually disappear from the Pinecone namespace
and every per-file namespace are removed (no Pinecone orphans)
Out of scope (planned follow-ups)
fix/spend-ledger-and-enforcement: cost computation innormalize_usage, recording on non-stream paths, enforcement on/api/v1/tasks/*, fail-closed policy.fix/migrations-storage-cache-loaders: storage cachebasename collisions, loader extension parsing, default Unstructured
strategy,
CONCURRENTLYmigration safety, header/down_revisionmismatches.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.