Originally created by @Tureti on GitHub (Jul 23, 2024).
Bug Report
Description
Bug Summary:
System message is added at the start with only one newline instead of adding it before the last user message with two newlines.
A space before [/INST] is added that shouldn't be there.
Space after [/INST] and before assistant reply isn't added.
Steps to Reproduce:
Start llama-server with Mistral-Nemo-Instruct model and use it as an openai compatible endpoint in open-webui.
Actual Behavior:
"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 1234+321?"},{"role":"assistant","content":"The sum of 1234 and 321 is 1555."},{"role":"user","content":"Good job!"},{"role":"assistant","content":"Thank you! Is there anything else I can help you with?"},{"role":"user","content":"No."}]
Open-webui prompt:
"prompt": "[INST] You are a helpful assistant.\nWhat is 1234+321? [/INST]The sum of 1234 and 321 is 1555.[INST] Good job! [/INST]Thank you! Is there anything else I can help you with?[INST] No. [/INST]"
Expected prompt:
"prompt":"[INST] What is 1234+321?[/INST] The sum of 1234 and 321 is 1555.[INST] Good job![/INST] Thank you! Is there anything else I can help you with?[INST] You are a helpful assistant.\n\nNo.[/INST]"
Environment
Open WebUI Version: v0.3.10
Operating System: Ubuntu 22.04.4 LTS
Confirmation:
I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
Originally created by @Tureti on GitHub (Jul 23, 2024).
# Bug Report
## Description
**Bug Summary:**
System message is added at the start with only one newline instead of adding it before the last user message with two newlines.
A space before [/INST] is added that shouldn't be there.
Space after [/INST] and before assistant reply isn't added.
**Steps to Reproduce:**
Start llama-server with Mistral-Nemo-Instruct model and use it as an openai compatible endpoint in open-webui.
**Expected Behavior:**
Apply the correct chat template to the prompt as specified in [https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407/blob/main/tokenizer_config.json](url)
**Actual Behavior:**
"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 1234+321?"},{"role":"assistant","content":"The sum of 1234 and 321 is 1555."},{"role":"user","content":"Good job!"},{"role":"assistant","content":"Thank you! Is there anything else I can help you with?"},{"role":"user","content":"No."}]
Open-webui prompt:
"prompt": "[INST] You are a helpful assistant.\nWhat is 1234+321? [/INST]The sum of 1234 and 321 is 1555.</s>[INST] Good job! [/INST]Thank you! Is there anything else I can help you with?</s>[INST] No. [/INST]"
Expected prompt:
"prompt":"[INST] What is 1234+321?[/INST] The sum of 1234 and 321 is 1555.</s>[INST] Good job![/INST] Thank you! Is there anything else I can help you with?</s>[INST] You are a helpful assistant.\n\nNo.[/INST]"
## Environment
- **Open WebUI Version:** v0.3.10
- **Operating System:** Ubuntu 22.04.4 LTS
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
WebUI does not handle the chat templates of any of the inference backends, you should take this issue up with llama-server.
@justinh-rahb commented on GitHub (Jul 23, 2024):
WebUI does not handle the chat templates of any of the inference backends, you should take this issue up with llama-server.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @Tureti on GitHub (Jul 23, 2024).
Bug Report
Description
Bug Summary:
System message is added at the start with only one newline instead of adding it before the last user message with two newlines.
A space before [/INST] is added that shouldn't be there.
Space after [/INST] and before assistant reply isn't added.
Steps to Reproduce:
Start llama-server with Mistral-Nemo-Instruct model and use it as an openai compatible endpoint in open-webui.
Expected Behavior:
Apply the correct chat template to the prompt as specified in https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407/blob/main/tokenizer_config.json
Actual Behavior:
"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 1234+321?"},{"role":"assistant","content":"The sum of 1234 and 321 is 1555."},{"role":"user","content":"Good job!"},{"role":"assistant","content":"Thank you! Is there anything else I can help you with?"},{"role":"user","content":"No."}]
Open-webui prompt:
"prompt": "[INST] You are a helpful assistant.\nWhat is 1234+321? [/INST]The sum of 1234 and 321 is 1555.[INST] Good job! [/INST]Thank you! Is there anything else I can help you with?[INST] No. [/INST]"
Expected prompt:
"prompt":"[INST] What is 1234+321?[/INST] The sum of 1234 and 321 is 1555.[INST] Good job![/INST] Thank you! Is there anything else I can help you with?[INST] You are a helpful assistant.\n\nNo.[/INST]"
Environment
Open WebUI Version: v0.3.10
Operating System: Ubuntu 22.04.4 LTS
Confirmation:
@justinh-rahb commented on GitHub (Jul 23, 2024):
WebUI does not handle the chat templates of any of the inference backends, you should take this issue up with llama-server.