Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Changelog Entry
Description
Sometimes, I want to filter messagess by keywords so that we can better use the service. So that, I white this.
1. I create 5 environment variables
Variable Name
Description
Function
ENABLE_MESSAGE_FILTER
Enables message filtering functionality
When set to true, the system filters all incoming and outgoing messages to ensure they do not contain any prohibited content.
CHAT_FILTER_WORDS_FILE
File containing the list of words to filter
The system reads the words to be filtered from the badwords.txt file and checks them against message content.
CHAT_FILTER_WORDS
List of specific words to be filtered, separated by commas
The system checks message content against these words and filters them if they are found.
ENABLE_REPLACE_FILTER_WORDS
Enables the replacement functionality
When set to true, the system replaces filtered words with a preset replacement word. When set to false, it does not replace and send a error.
REPLACE_FILTER_WORDS
Content to replace filtered words with
If ENABLE_REPLACE_FILTER_WORDS is set to true, filtered words will be replaced with the content specified by this variable.
2. and I add the page in admin panel so that we can manage it very easy
3. we can manage its switches and add all the bad words we need to desable or replace
4. Global message check for bad words replacement
5. Efficient inspection efficiency
Passed the self-test
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/3627
**Author:** [@Yanyutin753](https://github.com/Yanyutin753)
**Created:** 7/4/2024
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `new_filter`
---
### 📝 Commits (7)
- [`0a4901d`](https://github.com/open-webui/open-webui/commit/0a4901db82bf5c30b4c12e325b054a37ca7cfbdf) 👀 Supports bad words filtering of message
- [`b62762f`](https://github.com/open-webui/open-webui/commit/b62762fded76c43e4c03df86fb8149c0c0fb86f5) 🤯 feat global message check for bad words replacement
- [`b363863`](https://github.com/open-webui/open-webui/commit/b3638639f30f4ca140652d3310277544aaeb8d39) 🤖 fix format and test
- [`9753dd7`](https://github.com/open-webui/open-webui/commit/9753dd7b40a7d35cf47a7fa05dd0c6a778d52074) 🤖 fix format
- [`e8a159c`](https://github.com/open-webui/open-webui/commit/e8a159cd96d0d9947310beca313687135b431f42) 🤖 fix format and test
- [`f431b21`](https://github.com/open-webui/open-webui/commit/f431b21bfb531f0d6f9cb337f5ae67f2f6c97c23) 🤖 fix format
- [`b6c1e76`](https://github.com/open-webui/open-webui/commit/b6c1e76b274858b4386983eeaf092bb7c625c1d0) 🤖 fix format
### 📊 Changes
**50 files changed** (+1024 additions, -43 deletions)
<details>
<summary>View changed files</summary>
➕ `backend/apps/filter/hashNode.py` (+29 -0)
➕ `backend/apps/filter/main.py` (+153 -0)
➕ `backend/apps/filter/trieNode.py` (+25 -0)
➕ `backend/apps/filter/wordsSearch.py` (+181 -0)
📝 `backend/apps/openai/main.py` (+22 -25)
📝 `backend/config.py` (+35 -0)
📝 `backend/main.py` (+2 -1)
➕ `src/lib/apis/filter/index.ts` (+66 -0)
📝 `src/lib/components/admin/Settings.svelte` (+33 -0)
➕ `src/lib/components/admin/Settings/Filter.svelte` (+149 -0)
📝 `src/lib/constants.ts` (+1 -0)
📝 `src/lib/i18n/locales/ar-BH/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/bg-BG/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/bn-BD/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/ca-ES/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/ceb-PH/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/de-DE/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/dg-DG/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/en-GB/translation.json` (+8 -0)
📝 `src/lib/i18n/locales/en-US/translation.json` (+8 -0)
_...and 30 more files_
</details>
### 📄 Description
👀 Supports bad words filtering of message
- [x] **Target branch:** Please verify that the pull request targets the `dev` branch.
- [x] **Description:** Provide a concise description of the changes made in this pull request.
- [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources?
- [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [x] **Testing:** Have you written and run sufficient tests for validating the changes?
- [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
# Changelog Entry
### Description
- Sometimes, I want to filter messagess by keywords so that we can better use the service. So that, I white this.
#### 1. I create 5 environment variables
| Variable Name | Description | Function |
|------------------------------|-------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|
| ENABLE_MESSAGE_FILTER | Enables message filtering functionality | When set to `true`, the system filters all incoming and outgoing messages to ensure they do not contain any prohibited content. |
| CHAT_FILTER_WORDS_FILE | File containing the list of words to filter | The system reads the words to be filtered from the `badwords.txt` file and checks them against message content. |
| CHAT_FILTER_WORDS | List of specific words to be filtered, separated by commas | The system checks message content against these words and filters them if they are found. |
| ENABLE_REPLACE_FILTER_WORDS | Enables the replacement functionality | When set to `true`, the system replaces filtered words with a preset replacement word. When set to `false`, it does not replace and send a error. |
| REPLACE_FILTER_WORDS | Content to replace filtered words with | If `ENABLE_REPLACE_FILTER_WORDS` is set to `true`, filtered words will be replaced with the content specified by this variable. |
#### 2. and I add the page in admin panel so that we can manage it very easy
#### 3. we can manage its switches and add all the bad words we need to desable or replace
#### 4. Global message check for bad words replacement
#### 5. Efficient inspection efficiency
### Passed the self-test
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/3627
Author: @Yanyutin753
Created: 7/4/2024
Status: ❌ Closed
Base:
dev← Head:new_filter📝 Commits (7)
0a4901d👀 Supports bad words filtering of messageb62762f🤯 feat global message check for bad words replacementb363863🤖 fix format and test9753dd7🤖 fix formate8a159c🤖 fix format and testf431b21🤖 fix formatb6c1e76🤖 fix format📊 Changes
50 files changed (+1024 additions, -43 deletions)
View changed files
➕
backend/apps/filter/hashNode.py(+29 -0)➕
backend/apps/filter/main.py(+153 -0)➕
backend/apps/filter/trieNode.py(+25 -0)➕
backend/apps/filter/wordsSearch.py(+181 -0)📝
backend/apps/openai/main.py(+22 -25)📝
backend/config.py(+35 -0)📝
backend/main.py(+2 -1)➕
src/lib/apis/filter/index.ts(+66 -0)📝
src/lib/components/admin/Settings.svelte(+33 -0)➕
src/lib/components/admin/Settings/Filter.svelte(+149 -0)📝
src/lib/constants.ts(+1 -0)📝
src/lib/i18n/locales/ar-BH/translation.json(+8 -0)📝
src/lib/i18n/locales/bg-BG/translation.json(+8 -0)📝
src/lib/i18n/locales/bn-BD/translation.json(+8 -0)📝
src/lib/i18n/locales/ca-ES/translation.json(+8 -0)📝
src/lib/i18n/locales/ceb-PH/translation.json(+8 -0)📝
src/lib/i18n/locales/de-DE/translation.json(+8 -0)📝
src/lib/i18n/locales/dg-DG/translation.json(+8 -0)📝
src/lib/i18n/locales/en-GB/translation.json(+8 -0)📝
src/lib/i18n/locales/en-US/translation.json(+8 -0)...and 30 more files
📄 Description
👀 Supports bad words filtering of message
devbranch.Changelog Entry
Description
1. I create 5 environment variables
true, the system filters all incoming and outgoing messages to ensure they do not contain any prohibited content.badwords.txtfile and checks them against message content.true, the system replaces filtered words with a preset replacement word. When set tofalse, it does not replace and send a error.ENABLE_REPLACE_FILTER_WORDSis set totrue, filtered words will be replaced with the content specified by this variable.2. and I add the page in admin panel so that we can manage it very easy
3. we can manage its switches and add all the bad words we need to desable or replace
4. Global message check for bad words replacement
5. Efficient inspection efficiency
Passed the self-test
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.