[X ] I have read and followed all the instructions provided in the README.md.
[X ] I am on the latest version of both Open WebUI and Ollama.
[ ] I have included the browser console logs.
[ ] I have included the Docker container logs.
[X ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
I am running two instances of Open WebUI:
• Port 2000: Open WebUI v0.4.2
• Port 3000: Open WebUI v0.3.35
Both are fresh installations. I created the same knowledge object and added an identical markdown file to it on both instances. Subsequently, I created a custom model based on Llama:3.2 using the knowledge object in both instances, setting the TEMPERATURE value to 0 (accuracy over creativity).
Expectation: Both Open WebUI v0.4.2 and Open WebUI v0.3.35 should produce similar results since they are running the same custom RAG model.
Actual Behavior:
For the same questions, my custom model in Open WebUI v0.4.2 consistently provides worse responses compared to Open WebUI v0.3.35. I tested with different files, including various file formats, and multiple questions. The answers from v0.4.2 are consistently inferior to those from v0.3.35. Strangely, when I click on citations, they are mostly the same between the two versions. However, v0.4.2 tends not to follow the citations as closely.
Even tried changing the TEMPERATURE values. No luck.
What changes might account for this discrepancy?
Reproduction Details
Setup Instances:
Deploy two instances of Open WebUI on different ports:
Port 2000: Install Open WebUI v0.4.2
Port 3000: Install Open WebUI v0.3.35
Ensure both installations are fresh and have no prior configuration.
Create Knowledge Object:
On both instances, create the same knowledge object.
Add an identical markdown file to the knowledge object in each instance.
Configure Custom Model:
In both WebUI instances, create a custom model based on Llama:3.2 using the knowledge object.
Set the TEMPERATURE value to 0 for both models to prioritize accuracy over creativity.
Run Tests:
Ask identical questions to both models in each WebUI instance.
Observe and document the responses from each version.
Originally created by @firoagni on GitHub (Nov 21, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7143
# Bug Report
## Installation Method
Using Docker to deploy Open WebUI
## Environment
- Open WebUI Version: v0.4.2
- Ollama: 0.4.2
- Operating System: macOS 15.0.1
**Confirmation:**
- [X ] I have read and followed all the instructions provided in the README.md.
- [X ] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [ ] I have included the Docker container logs.
- [X ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
I am running two instances of Open WebUI:
• Port 2000: Open WebUI v0.4.2
• Port 3000: Open WebUI v0.3.35
Both are fresh installations. I created the same knowledge object and added an identical markdown file to it on both instances. Subsequently, I created a custom model based on `Llama:3.2` using the knowledge object in both instances, setting the TEMPERATURE value to 0 (accuracy over creativity).
Expectation: Both Open WebUI v0.4.2 and Open WebUI v0.3.35 should produce similar results since they are running the same custom RAG model.
## Actual Behavior:
For the same questions, my custom model in Open WebUI v0.4.2 consistently provides worse responses compared to Open WebUI v0.3.35. I tested with different files, including various file formats, and multiple questions. The answers from v0.4.2 are consistently inferior to those from v0.3.35. Strangely, when I click on citations, they are mostly the same between the two versions. However, v0.4.2 tends not to follow the citations as closely.
Even tried changing the TEMPERATURE values. No luck.
What changes might account for this discrepancy?
## Reproduction Details
1. Setup Instances:
- Deploy two instances of Open WebUI on different ports:
- Port 2000: Install Open WebUI v0.4.2
- Port 3000: Install Open WebUI v0.3.35
Ensure both installations are fresh and have no prior configuration.
2. Create Knowledge Object:
- On both instances, create the same knowledge object.
- Add an identical markdown file to the knowledge object in each instance.
3. Configure Custom Model:
- In both WebUI instances, create a custom model based on Llama:3.2 using the knowledge object.
- Set the TEMPERATURE value to 0 for both models to prioritize accuracy over creativity.
4. Run Tests:
- Ask identical questions to both models in each WebUI instance.
- Observe and document the responses from each version.
Dumb question - From where I can disable RAG query generation? Couldn't able to find the checkbox
<!-- gh-comment-id:2491909919 -->
@firoagni commented on GitHub (Nov 21, 2024):
@tjbck
> changelogs
Dumb question - From where I can disable RAG query generation? Couldn't able to find the checkbox
From where I can disable RAG query generation? Couldn't able to find the checkbox
In Interface tab.
Also, you can change Query Generation Prompt. This prompt is used both for internet search and RAG search. I found that better results will be with prompt like that:
`
Task:
Based on the user's message and interaction history, generate search query. Refrain from verbose responses or any additional commentary. Prefer suggesting a search if uncertain to provide comprehensive or updated information. Then output both search query and user message. Today's date is: {{CURRENT_DATE}}. Search query must be in a user's message language.
Guidelines:
Respond exclusively with a JSON object.
Be concise, focusing strictly on composing search queries with no additional commentary or text.
<!-- gh-comment-id:2493855997 -->
@morgan55555 commented on GitHub (Nov 22, 2024):
> From where I can disable RAG query generation? Couldn't able to find the checkbox
In Interface tab.
Also, you can change Query Generation Prompt. This prompt is used both for internet search and RAG search. I found that better results will be with prompt like that:
`
### Task:
Based on the user's message and interaction history, generate search query. Refrain from verbose responses or any additional commentary. Prefer suggesting a search if uncertain to provide comprehensive or updated information. Then output both search query and user message. Today's date is: {{CURRENT_DATE}}. Search query must be in a user's message language.
### Guidelines:
- Respond exclusively with a JSON object.
- Be concise, focusing strictly on composing search queries with no additional commentary or text.
### Output:
JSON format: {
"queries": ["Search query", "User message"]
}
### User message:
{{prompt:end:4000}}
### Chat History:
{{MESSAGES:END:6}}
`
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @firoagni on GitHub (Nov 21, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7143
Bug Report
Installation Method
Using Docker to deploy Open WebUI
Environment
Open WebUI Version: v0.4.2
Ollama: 0.4.2
Operating System: macOS 15.0.1
Confirmation:
Expected Behavior:
I am running two instances of Open WebUI:
• Port 2000: Open WebUI v0.4.2
• Port 3000: Open WebUI v0.3.35
Both are fresh installations. I created the same knowledge object and added an identical markdown file to it on both instances. Subsequently, I created a custom model based on
Llama:3.2using the knowledge object in both instances, setting the TEMPERATURE value to 0 (accuracy over creativity).Expectation: Both Open WebUI v0.4.2 and Open WebUI v0.3.35 should produce similar results since they are running the same custom RAG model.
Actual Behavior:
For the same questions, my custom model in Open WebUI v0.4.2 consistently provides worse responses compared to Open WebUI v0.3.35. I tested with different files, including various file formats, and multiple questions. The answers from v0.4.2 are consistently inferior to those from v0.3.35. Strangely, when I click on citations, they are mostly the same between the two versions. However, v0.4.2 tends not to follow the citations as closely.
Even tried changing the TEMPERATURE values. No luck.
What changes might account for this discrepancy?
Reproduction Details
Ensure both installations are fresh and have no prior configuration.
@tjbck commented on GitHub (Nov 21, 2024):
You can disable RAG query generation. Please check the changelogs.
@firoagni commented on GitHub (Nov 21, 2024):
@tjbck
Dumb question - From where I can disable RAG query generation? Couldn't able to find the checkbox
@morgan55555 commented on GitHub (Nov 22, 2024):
In Interface tab.
Also, you can change Query Generation Prompt. This prompt is used both for internet search and RAG search. I found that better results will be with prompt like that:
`
Task:
Based on the user's message and interaction history, generate search query. Refrain from verbose responses or any additional commentary. Prefer suggesting a search if uncertain to provide comprehensive or updated information. Then output both search query and user message. Today's date is: {{CURRENT_DATE}}. Search query must be in a user's message language.
Guidelines:
Output:
JSON format: {
"queries": ["Search query", "User message"]
}
User message:
{{prompt:end:4000}}
Chat History:
{{MESSAGES:END:6}}
`