[PR #17723] [CLOSED] Fix: Sanitize chat and other user content to prevent PostgreSQL errors #24527

New Issue

GiteaMirror · 2026-04-20T05:26:41-05:00

GiteaMirror commented

2026-04-20 05:26:41 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/17723
Author: @Classic298
Created: 9/24/2025
Status: ❌ Closed

Base: dev ← Head: fix-postgres-null-byte-search-error

📝 Commits (2)

c104b9d Fix: Sanitize chat and other user content to prevent PostgreSQL errors
51c5275 Merge branch 'dev' into fix-postgres-null-byte-search-error

📊 Changes

5 files changed (+77 additions, -59 deletions)

View changed files

📝 backend/open_webui/models/chats.py (+35 -27)
📝 backend/open_webui/models/prompts.py (+6 -4)
📝 backend/open_webui/routers/files.py (+2 -1)
📝 backend/open_webui/routers/retrieval.py (+16 -27)
📝 backend/open_webui/utils/misc.py (+18 -0)

📄 Description

This commit introduces a comprehensive sanitization mechanism to prevent null bytes (\u0000) and other C0 control characters from being stored in the database. These characters were causing UntranslatableCharacter errors during search operations in PostgreSQL deployments.

The solution consists of a new centralized function, sanitize_json_content, which recursively traverses data structures and removes problematic characters from strings.

This sanitization is now applied at all relevant data entry points, including:

Chat creation and updates
Prompt creation and updates
File upload metadata
Content extracted from documents for retrieval

This proactive approach ensures data integrity and prevents future occurrences of the search error, providing a robust and long-term fix for the issue.

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/17723 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 9/24/2025 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `fix-postgres-null-byte-search-error` --- ### 📝 Commits (2) - [`c104b9d`](https://github.com/open-webui/open-webui/commit/c104b9dc5ada0daa8830b345a7831cbd07ed89e2) Fix: Sanitize chat and other user content to prevent PostgreSQL errors - [`51c5275`](https://github.com/open-webui/open-webui/commit/51c5275353fd90efb9d482c2bb9a80c17fc6ede5) Merge branch 'dev' into fix-postgres-null-byte-search-error ### 📊 Changes **5 files changed** (+77 additions, -59 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/models/chats.py` (+35 -27) 📝 `backend/open_webui/models/prompts.py` (+6 -4) 📝 `backend/open_webui/routers/files.py` (+2 -1) 📝 `backend/open_webui/routers/retrieval.py` (+16 -27) 📝 `backend/open_webui/utils/misc.py` (+18 -0) </details> ### 📄 Description This commit introduces a comprehensive sanitization mechanism to prevent null bytes (\u0000) and other C0 control characters from being stored in the database. These characters were causing `UntranslatableCharacter` errors during search operations in PostgreSQL deployments. The solution consists of a new centralized function, `sanitize_json_content`, which recursively traverses data structures and removes problematic characters from strings. This sanitization is now applied at all relevant data entry points, including: - Chat creation and updates - Prompt creation and updates - File upload metadata - Content extracted from documents for retrieval This proactive approach ensures data integrity and prevents future occurrences of the search error, providing a robust and long-term fix for the issue. ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 05:26:41 -05:00

GiteaMirror closed this issue

2026-04-20 05:26:42 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#24527