Originally created by @Seth-Rothschild on GitHub (Jul 13, 2024).
Bug Report
Description
Bug Summary:
Currently system prompts and advanced settings can be set in 3 places
In the general tab of settings
In the model itself
In the new "chat controls" drawer
While the prompt/parameters in models overrides general settings, the interactions between chat controls and models are what I would call a bug, where some settings come from each in an unpredictable way for the user.
Steps to Reproduce:
To test and explore this, I would set a system prompt "prompt 1" in the general tab, "prompt 2" attached to a model and "prompt 3" in the chat controls after starting a new chat. I've also been using Seed 1/2/3 in the 3 spaces and various other parameters e.g. max-tokens to see what does and does not get overridden.
If you start a new chat with the model that has "prompt 2", and then go set the chat controls you'll get {'role': 'system', 'content': 'system prompt 3system prompt 2\nsystem prompt 3'} which certainly isn't right, though I would probably argue that reasoning about the precedence at all could be confusing for users.
The advanced parameters are also taken wholesale from the model, overriding a non-default setting put into chat controls.
Expected Behavior:
I think the least-work approach would be to only take the advanced parameters and system prompt from one place. My guess is that the parameters from models always have to take precedence because the advanced parameters are sometimes what makes the model work, so some small amount of text would be warranted to tell users of Chat Controls that their choices will be overridden by settings of particular models.
That being said I also think users expect the most recently done thing to be the highest priority, which means if they go into a chat and change the chat controls they absolutely will expect those to be taken into account in the chat. I think the approach where chat controls take precedence over models over general settings might be more intuitive for users.
In both cases, I only expect settings to be overridden when they've been changed off the default, which is currently not the case if you've changed at least one parameter.
Actual Behavior:
The system prompt from chat settings appears twice and the system prompt from the model appears once all in the same message
Advanced parameters from the model override all choices made in chat controls, even for items set as default in the model.
Environment
Open WebUI Version: [v0.3.8]
Ollama (if applicable): N/A
Operating System: [Ubuntu Linux]
Browser (if applicable): N/A
Reproduction Details
Confirmation:
I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
Logs and Screenshots
Installation Method
npm/conda
Additional Information
Happy to add additional information as needed.
Note
If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
Originally created by @Seth-Rothschild on GitHub (Jul 13, 2024).
# Bug Report
## Description
**Bug Summary:**
Currently system prompts and advanced settings can be set in 3 places
1. In the general tab of settings
2. In the model itself
3. In the new "chat controls" drawer
While the prompt/parameters in models overrides general settings, the interactions between chat controls and models are what I would call a bug, where some settings come from each in an unpredictable way for the user.
**Steps to Reproduce:**
To test and explore this, I would set a system prompt "prompt 1" in the general tab, "prompt 2" attached to a model and "prompt 3" in the chat controls after starting a new chat. I've also been using Seed 1/2/3 in the 3 spaces and various other parameters e.g. max-tokens to see what does and does not get overridden.
If you start a new chat with the model that has "prompt 2", and then go set the chat controls you'll get
```{'role': 'system', 'content': 'system prompt 3system prompt 2\nsystem prompt 3'}``` which certainly isn't right, though I would probably argue that reasoning about the precedence at all could be confusing for users.
The advanced parameters are also taken wholesale from the model, overriding a non-default setting put into chat controls.
**Expected Behavior:**
I think the least-work approach would be to only take the advanced parameters and system prompt from one place. My guess is that the parameters from models always have to take precedence because the advanced parameters are sometimes what makes the model work, so some small amount of text would be warranted to tell users of Chat Controls that their choices will be overridden by settings of particular models.
That being said I _also_ think users expect the most recently done thing to be the highest priority, which means if they go into a chat and change the chat controls they absolutely will expect those to be taken into account in the chat. I think the approach where chat controls take precedence over models over general settings might be more intuitive for users.
In both cases, I only expect settings to be overridden when they've been changed off the default, which is currently not the case if you've changed at least one parameter.
**Actual Behavior:**
1. The system prompt from chat settings appears twice and the system prompt from the model appears once all in the same message
2. Advanced parameters from the model override _all_ choices made in chat controls, even for items set as default in the model.
## Environment
- **Open WebUI Version:** [v0.3.8]
- **Ollama (if applicable):** N/A
- **Operating System:** [Ubuntu Linux]
- **Browser (if applicable):** N/A
## Reproduction Details
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
## Logs and Screenshots
## Installation Method
npm/conda
## Additional Information
Happy to add additional information as needed.
## Note
If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @Seth-Rothschild on GitHub (Jul 13, 2024).
Bug Report
Description
Bug Summary:
Currently system prompts and advanced settings can be set in 3 places
While the prompt/parameters in models overrides general settings, the interactions between chat controls and models are what I would call a bug, where some settings come from each in an unpredictable way for the user.
Steps to Reproduce:
To test and explore this, I would set a system prompt "prompt 1" in the general tab, "prompt 2" attached to a model and "prompt 3" in the chat controls after starting a new chat. I've also been using Seed 1/2/3 in the 3 spaces and various other parameters e.g. max-tokens to see what does and does not get overridden.
If you start a new chat with the model that has "prompt 2", and then go set the chat controls you'll get
{'role': 'system', 'content': 'system prompt 3system prompt 2\nsystem prompt 3'}which certainly isn't right, though I would probably argue that reasoning about the precedence at all could be confusing for users.The advanced parameters are also taken wholesale from the model, overriding a non-default setting put into chat controls.
Expected Behavior:
I think the least-work approach would be to only take the advanced parameters and system prompt from one place. My guess is that the parameters from models always have to take precedence because the advanced parameters are sometimes what makes the model work, so some small amount of text would be warranted to tell users of Chat Controls that their choices will be overridden by settings of particular models.
That being said I also think users expect the most recently done thing to be the highest priority, which means if they go into a chat and change the chat controls they absolutely will expect those to be taken into account in the chat. I think the approach where chat controls take precedence over models over general settings might be more intuitive for users.
In both cases, I only expect settings to be overridden when they've been changed off the default, which is currently not the case if you've changed at least one parameter.
Actual Behavior:
Environment
Open WebUI Version: [v0.3.8]
Ollama (if applicable): N/A
Operating System: [Ubuntu Linux]
Browser (if applicable): N/A
Reproduction Details
Confirmation:
Logs and Screenshots
Installation Method
npm/conda
Additional Information
Happy to add additional information as needed.
Note
If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!