Note: Meta search engines like SearxNG do not support filtering results by domain natively, so this filtering is applied post-search.
Added a filter to restrict search results to specified domains.
Updated the processing of search results to apply the whitelist filter before returning the final results.
Ensured that only results from allowed domains (if specified in the whitelist) are included in the output.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/3112
**Author:** [@que-nguyen](https://github.com/que-nguyen)
**Created:** 6/13/2024
**Status:** ✅ Merged
**Merged:** 6/17/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `searxng`
---
### 📝 Commits (10+)
- [`7b5f434`](https://github.com/open-webui/open-webui/commit/7b5f434a079b335be8452af7a05982a779973fcd) Implement domain whitelisting for web search results
- [`a02ba52`](https://github.com/open-webui/open-webui/commit/a02ba52de8fc030466830744f4452ce42aba7f3e) Merge branch 'dev' into searxng
- [`9e4dd4b`](https://github.com/open-webui/open-webui/commit/9e4dd4b86f77653b59dc1fa97fe3f72e8252b359) Merge pull request #3159 from open-webui/dev
- [`75e51ec`](https://github.com/open-webui/open-webui/commit/75e51ecf6d373e152c9c182ca7d339fc50052253) Merge branch 'open-webui:main' into searxng
- [`a3ac9ee`](https://github.com/open-webui/open-webui/commit/a3ac9ee774f0cb9dc511970f596c462d678e5fb7) Refactor main.py
- [`b3d136b`](https://github.com/open-webui/open-webui/commit/b3d136b3b33781a4607ebdce952c3ec7daad204c) Refactored config.py
- [`a02139b`](https://github.com/open-webui/open-webui/commit/a02139ba9df0513696c3cb89aecf037e19aee4d2) Set filter_list as optional param in brave.py
- [`7d2ad8c`](https://github.com/open-webui/open-webui/commit/7d2ad8c4bf44e2c5b180310a462e9fc90d2ad2ec) Set filter_list as optional param in duckduckgo.py
- [`d8beed1`](https://github.com/open-webui/open-webui/commit/d8beed13b4a19281d114f6740a718a418a59e8c3) Set filter_list as optional param in google_pse.py
- [`3cc0e3e`](https://github.com/open-webui/open-webui/commit/3cc0e3ecb6f7ff6b4cd9ae9bf244fb237122fc99) Refactor rag/main.py
### 📊 Changes
**10 files changed** (+68 additions, -24 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/apps/rag/main.py` (+9 -1)
📝 `backend/apps/rag/search/brave.py` (+6 -3)
📝 `backend/apps/rag/search/duckduckgo.py` (+5 -4)
📝 `backend/apps/rag/search/google_pse.py` (+5 -3)
📝 `backend/apps/rag/search/main.py` (+11 -1)
📝 `backend/apps/rag/search/searxng.py` (+5 -3)
📝 `backend/apps/rag/search/serper.py` (+5 -3)
📝 `backend/apps/rag/search/serply.py` (+5 -3)
📝 `backend/apps/rag/search/serpstack.py` (+5 -3)
📝 `backend/config.py` (+12 -0)
</details>
### 📄 Description
Note: Meta search engines like SearxNG do not support filtering results by domain natively, so this filtering is applied post-search.
Added a filter to restrict search results to specified domains.
Updated the processing of search results to apply the whitelist filter before returning the final results.
Ensured that only results from allowed domains (if specified in the whitelist) are included in the output.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/3112
Author: @que-nguyen
Created: 6/13/2024
Status: ✅ Merged
Merged: 6/17/2024
Merged by: @tjbck
Base:
dev← Head:searxng📝 Commits (10+)
7b5f434Implement domain whitelisting for web search resultsa02ba52Merge branch 'dev' into searxng9e4dd4bMerge pull request #3159 from open-webui/dev75e51ecMerge branch 'open-webui:main' into searxnga3ac9eeRefactor main.pyb3d136bRefactored config.pya02139bSet filter_list as optional param in brave.py7d2ad8cSet filter_list as optional param in duckduckgo.pyd8beed1Set filter_list as optional param in google_pse.py3cc0e3eRefactor rag/main.py📊 Changes
10 files changed (+68 additions, -24 deletions)
View changed files
📝
backend/apps/rag/main.py(+9 -1)📝
backend/apps/rag/search/brave.py(+6 -3)📝
backend/apps/rag/search/duckduckgo.py(+5 -4)📝
backend/apps/rag/search/google_pse.py(+5 -3)📝
backend/apps/rag/search/main.py(+11 -1)📝
backend/apps/rag/search/searxng.py(+5 -3)📝
backend/apps/rag/search/serper.py(+5 -3)📝
backend/apps/rag/search/serply.py(+5 -3)📝
backend/apps/rag/search/serpstack.py(+5 -3)📝
backend/config.py(+12 -0)📄 Description
Note: Meta search engines like SearxNG do not support filtering results by domain natively, so this filtering is applied post-search.
Added a filter to restrict search results to specified domains.
Updated the processing of search results to apply the whitelist filter before returning the final results.
Ensured that only results from allowed domains (if specified in the whitelist) are included in the output.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.