Priority of system prompts and advanced parameters is hard to understand. #1511

Closed
opened 2025-11-11 14:46:39 -06:00 by GiteaMirror · 0 comments
Owner

Originally created by @Seth-Rothschild on GitHub (Jul 13, 2024).

Bug Report

Description

Bug Summary:
Currently system prompts and advanced settings can be set in 3 places

  1. In the general tab of settings
  2. In the model itself
  3. In the new "chat controls" drawer

While the prompt/parameters in models overrides general settings, the interactions between chat controls and models are what I would call a bug, where some settings come from each in an unpredictable way for the user.

Steps to Reproduce:
To test and explore this, I would set a system prompt "prompt 1" in the general tab, "prompt 2" attached to a model and "prompt 3" in the chat controls after starting a new chat. I've also been using Seed 1/2/3 in the 3 spaces and various other parameters e.g. max-tokens to see what does and does not get overridden.

If you start a new chat with the model that has "prompt 2", and then go set the chat controls you'll get
{'role': 'system', 'content': 'system prompt 3system prompt 2\nsystem prompt 3'} which certainly isn't right, though I would probably argue that reasoning about the precedence at all could be confusing for users.

The advanced parameters are also taken wholesale from the model, overriding a non-default setting put into chat controls.

Expected Behavior:
I think the least-work approach would be to only take the advanced parameters and system prompt from one place. My guess is that the parameters from models always have to take precedence because the advanced parameters are sometimes what makes the model work, so some small amount of text would be warranted to tell users of Chat Controls that their choices will be overridden by settings of particular models.

That being said I also think users expect the most recently done thing to be the highest priority, which means if they go into a chat and change the chat controls they absolutely will expect those to be taken into account in the chat. I think the approach where chat controls take precedence over models over general settings might be more intuitive for users.

In both cases, I only expect settings to be overridden when they've been changed off the default, which is currently not the case if you've changed at least one parameter.

Actual Behavior:

  1. The system prompt from chat settings appears twice and the system prompt from the model appears once all in the same message
  2. Advanced parameters from the model override all choices made in chat controls, even for items set as default in the model.

Environment

  • Open WebUI Version: [v0.3.8]

  • Ollama (if applicable): N/A

  • Operating System: [Ubuntu Linux]

  • Browser (if applicable): N/A

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.

Logs and Screenshots

Installation Method

npm/conda

Additional Information

Happy to add additional information as needed.

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

Originally created by @Seth-Rothschild on GitHub (Jul 13, 2024). # Bug Report ## Description **Bug Summary:** Currently system prompts and advanced settings can be set in 3 places 1. In the general tab of settings 2. In the model itself 3. In the new "chat controls" drawer While the prompt/parameters in models overrides general settings, the interactions between chat controls and models are what I would call a bug, where some settings come from each in an unpredictable way for the user. **Steps to Reproduce:** To test and explore this, I would set a system prompt "prompt 1" in the general tab, "prompt 2" attached to a model and "prompt 3" in the chat controls after starting a new chat. I've also been using Seed 1/2/3 in the 3 spaces and various other parameters e.g. max-tokens to see what does and does not get overridden. If you start a new chat with the model that has "prompt 2", and then go set the chat controls you'll get ```{'role': 'system', 'content': 'system prompt 3system prompt 2\nsystem prompt 3'}``` which certainly isn't right, though I would probably argue that reasoning about the precedence at all could be confusing for users. The advanced parameters are also taken wholesale from the model, overriding a non-default setting put into chat controls. **Expected Behavior:** I think the least-work approach would be to only take the advanced parameters and system prompt from one place. My guess is that the parameters from models always have to take precedence because the advanced parameters are sometimes what makes the model work, so some small amount of text would be warranted to tell users of Chat Controls that their choices will be overridden by settings of particular models. That being said I _also_ think users expect the most recently done thing to be the highest priority, which means if they go into a chat and change the chat controls they absolutely will expect those to be taken into account in the chat. I think the approach where chat controls take precedence over models over general settings might be more intuitive for users. In both cases, I only expect settings to be overridden when they've been changed off the default, which is currently not the case if you've changed at least one parameter. **Actual Behavior:** 1. The system prompt from chat settings appears twice and the system prompt from the model appears once all in the same message 2. Advanced parameters from the model override _all_ choices made in chat controls, even for items set as default in the model. ## Environment - **Open WebUI Version:** [v0.3.8] - **Ollama (if applicable):** N/A - **Operating System:** [Ubuntu Linux] - **Browser (if applicable):** N/A ## Reproduction Details **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. ## Logs and Screenshots ## Installation Method npm/conda ## Additional Information Happy to add additional information as needed. ## Note If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#1511