mirror of
https://github.com/open-webui/open-webui.git
synced 2026-08-30 17:25:30 -05:00
feat: Add support for ZIM files #1461
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#1461
Reference in New Issue
Block a user
Originally created by @suncloudsmoon on GitHub (Jul 7, 2024).
Is your feature request related to a problem? Please describe.
The issue with a lot of LLMs is that they hallucinate if you ask them about obscure topics.
Describe the solution you'd like
There has been plenty of research that highlights the benefits of using Wikipedia in RAG to reduce hallucinations and increase the truthfulness of LLMs. Since Wikipedia is distributed as a single ZIM file via the ZIM format, I propose that Open WebUI should implement functionality to use ZIM files for RAG. One possible implementation route is to use the python binding of libzim to search for titles or do full text search (supported in some ZIM files) and use that information for RAG.
Describe alternatives you've considered
An alternative is to use the web search capability in Open WebUI for RAG. The drawback of web search is that it doesn't work offline and it doesn't always have the best information on a given topic (potential for misinformation). Another alternative solution is to create a pipeline addon that implements the above functionality.
Additional context
N/A