mirror of
https://github.com/open-webui/open-webui.git
synced 2026-11-15 21:41:05 -06:00
[PR #18854] [CLOSED] fix(retrieval): preserve Chinese punctuation in uploaded documents #63857
Reference in New Issue
Block a user
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/18854
Author: @fsmw
Created: 11/1/2025
Status: ❌ Closed
Base:
dev← Head:fix/17087-chinese-punctuation-conversion📝 Commits (1)
9c9092ffix(retrieval): preserve Chinese punctuation in uploaded documents📊 Changes
1 file changed (+1 additions, -2 deletions)
View changed files
📝
backend/open_webui/retrieval/loaders/main.py(+1 -2)📄 Description
Summary
Remove ftfy.fix_text() processing that converts Chinese punctuation marks to English equivalents, preserving cultural content integrity.
Issue
Fixes #17087 - The Chinese punctuation in the uploaded text file will be automatically converted to English punctuation marks.
Changes Made
Testing
Additional Information
This fix addresses cultural content preservation issues where Chinese punctuation was being automatically converted to English equivalents during file upload.
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
💘 Generated with Crush
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.