Manually concatenate all the documents into one single file to force it into a single collection.
Note
It is a design overlook more than a bug. The design of RAG doesn't consider scaling issue. For a complete set of documents, there isn't a need to have separate collections by each documents.
Originally created by @http403 on GitHub (Jun 15, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3200
# Bug Report
## Description
**Bug Summary:**
Extremely slow RAG when having 100+ documents.
**Steps to Reproduce:**
1. Upload 100 documents. (I used Docker docs in MD format)
2. Tag all with a common tag. (`docker-docs` or something make sense)
3. Ask question about docker with (`#docker-docs`)
**Expected Behavior:**
Finish RAG search in seconds
**Actual Behavior:**
Waiting 30 minutes+ for it to iterate every single collection
## Environment
- **Open WebUI Version:** 0.3.2
- **Ollama (if applicable):** 0.1.39
- **Operating System:** Linux Mint w/ Docker
- **Browser (if applicable):** Firefox 126
## Reproduction Details
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [ ] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [ ] I have included the Docker container logs.
## Logs and Screenshots
**Browser Console Logs:**
N/A
**Docker Container Logs:**
N/A
**Screenshots (if applicable):**
N/A
## Installation Method
Docker Compose
## Additional Information
[Discord Discussion](https://discord.com/channels/1170866489302188073/1251127195598983268)
## Alternative Solution
Manually concatenate all the documents into one single file to force it into a single collection.
## Note
It is a design overlook more than a bug. The design of RAG doesn't consider scaling issue. For a complete set of documents, there isn't a need to have separate collections by each documents.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @http403 on GitHub (Jun 15, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3200
Bug Report
Description
Bug Summary:
Extremely slow RAG when having 100+ documents.
Steps to Reproduce:
docker-docsor something make sense)#docker-docs)Expected Behavior:
Finish RAG search in seconds
Actual Behavior:
Waiting 30 minutes+ for it to iterate every single collection
Environment
Open WebUI Version: 0.3.2
Ollama (if applicable): 0.1.39
Operating System: Linux Mint w/ Docker
Browser (if applicable): Firefox 126
Reproduction Details
Confirmation:
Logs and Screenshots
Browser Console Logs:
N/A
Docker Container Logs:
N/A
Screenshots (if applicable):
N/A
Installation Method
Docker Compose
Additional Information
Discord Discussion
Alternative Solution
Manually concatenate all the documents into one single file to force it into a single collection.
Note
It is a design overlook more than a bug. The design of RAG doesn't consider scaling issue. For a complete set of documents, there isn't a need to have separate collections by each documents.
@tjbck commented on GitHub (Jun 16, 2024):
Duplicate #2044