mirror of
https://github.com/open-webui/open-webui.git
synced 2026-11-15 21:41:05 -06:00
[PR #18851] [CLOSED] fix(retrieval): preserve Chinese punctuation in uploaded documents #24998
Reference in New Issue
Block a user
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/18851
Author: @fsmw
Created: 11/1/2025
Status: ❌ Closed
Base:
main← Head:fix/17087-chinese-punctuation-conversion📝 Commits (1)
9c9092ffix(retrieval): preserve Chinese punctuation in uploaded documents📊 Changes
1 file changed (+1 additions, -2 deletions)
View changed files
📝
backend/open_webui/retrieval/loaders/main.py(+1 -2)📄 Description
Summary
Remove ftfy.fix_text() processing that converts Chinese punctuation marks to English equivalents, preserving cultural content integrity.
Issue
Fixes #17087 - The Chinese punctuation in the uploaded text file will be automatically converted to English punctuation marks.
Changes Made
Testing
Additional Information
This fix addresses cultural content preservation issues where Chinese punctuation was being automatically converted to English equivalents during file upload.
I agree to the terms of the Contributor License Agreement (CLA) and grant the necessary rights for my contributions to be included in this open-source project.
💘 Generated with Crush
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.