enhancement: better title prompt #701

New Issue

GiteaMirror · 2025-11-11T14:29:22-06:00

GiteaMirror commented

2025-11-11 14:29:22 -06:00

Originally created by @coder543 on GitHub (Apr 22, 2024).

Bug Report

Description

The default title prompt doesn't seem to be respected by virtually any model I've tested it with. This is the current default title prompt on my installation of Open WebUI:

Create a concise, 3-5 word phrase as a header for the following query, strictly adhering to the 3-5 word limit and avoiding the use of the word 'title': {{prompt}}

Here is an example of the kind of title that often gets generated:

Sure, here's the concise, 3-5 word phrase for the query header:Image Object Detection

I've had models as big as Mixtral 8x7B frequently struggle to return just the title with the default title prompt.

I would like to propose a better title prompt:

{{prompt}}

The preceding text is a conversation thread that needs a concise but descriptive 3 to 5 word title so that readers will be able to easily find it again. Respond only with the 3 to 5 word title text. Do not add any quotation marks.

Even the tiny Gemma-2B-Instruct is able to follow this prompt with a fairly high degree of consistency, to say nothing of the larger models, which struggle even less.

I find that LLMs follow instructions best when the instructions are placed at the end of the input, especially when the input may contain other instructions that are directed at an LLM. They may see an instruction at the end of the {{prompt}} that tells it to ignore previous instructions, and then it will completely "forget" that it is trying to generate a title.

Anyways, the title prompt I'm proposing above is just a simple improvement, in my opinion. To really make title generation work as well as possible, I am tempted to say that a grammar would be helpful here... but unfortunately, ollama still does not support custom grammars AFAICT. Requiring the model to respond with a response matching the format {"title":"<title with up to 4 spaces goes here>"} seems like it would help even the weakest of models to generate a title that meets the requirements.

Originally created by @coder543 on GitHub (Apr 22, 2024). # Bug Report ## Description The default title prompt doesn't seem to be respected by virtually any model I've tested it with. This is the current default title prompt on my installation of Open WebUI: > Create a concise, 3-5 word phrase as a header for the following query, strictly adhering to the 3-5 word limit and avoiding the use of the word 'title': {{prompt}} Here is an example of the kind of title that often gets generated: > Sure, here's the concise, 3-5 word phrase for the query header:**Image Object Detection** I've had models as big as Mixtral 8x7B frequently struggle to return just the title with the default title prompt. I would like to propose a better title prompt: > {{prompt}} > > The preceding text is a conversation thread that needs a concise but descriptive 3 to 5 word title so that readers will be able to easily find it again. Respond only with the 3 to 5 word title text. Do not add any quotation marks. Even the tiny Gemma-2B-Instruct is able to follow this prompt with a fairly high degree of consistency, to say nothing of the larger models, which struggle even less. I find that LLMs follow instructions best when the instructions are placed at the end of the input, _especially_ when the input may contain other instructions that are directed at an LLM. They may see an instruction at the end of the {{prompt}} that tells it to ignore previous instructions, and then it will completely "forget" that it is trying to generate a title. Anyways, the title prompt I'm proposing above is just a simple improvement, in my opinion. To really make title generation work as well as possible, I am tempted to say that a grammar would be helpful here... but unfortunately, `ollama` still does not support custom grammars AFAICT. Requiring the model to respond with a response matching the format `{"title":"<title with up to 4 spaces goes here>"}` seems like it would help even the weakest of models to generate a title that meets the requirements.

GiteaMirror closed this issue

2025-11-11 14:29:22 -06:00

GiteaMirror commented

2025-11-11 14:29:23 -06:00

@justinh-rahb commented on GitHub (Apr 22, 2024):

If this can be demonstrated to be somewhat consistent across a variety of popular models it would have my full support as a new default prompt.

@justinh-rahb commented on GitHub (Apr 22, 2024): If this can be demonstrated to be somewhat consistent across a variety of popular models it would have my full support as a new default prompt.

GiteaMirror commented

2025-11-11 14:29:23 -06:00

@coder543 commented on GitHub (Apr 22, 2024):

I would recommend trying it out for awhile and seeing what happens anecdotally, but maybe I can find the motivation to write up a script to generate titles with a bunch of models and show the results in a more objective manner. I hesitated to even submit the issue because I recognize that a lot of prompt engineering is often done subjectively. I'm sure even better prompts are possible.

In the absence of grammar support, I would also like to propose that the title generator should tell ollama to set the max response tokens to 30 tokens or something... I've had a few rambling LLMs generate hundreds of tokens for the "title", and that many tokens will never fit where a title goes anyways.

@coder543 commented on GitHub (Apr 22, 2024): I would recommend trying it out for awhile and seeing what happens anecdotally, but maybe I can find the motivation to write up a script to generate titles with a bunch of models and show the results in a more objective manner. I hesitated to even submit the issue because I recognize that a lot of prompt engineering is often done subjectively. I'm sure even better prompts are possible. In the absence of grammar support, I would also like to propose that the title generator should tell ollama to set the max response tokens to 30 tokens or something... I've had a few rambling LLMs generate hundreds of tokens for the "title", and that many tokens will never fit where a title goes anyways.

GiteaMirror referenced this issue

2025-11-11 17:20:35 -06:00

[PR #701] [MERGED] fix: admin issue #7220

GiteaMirror referenced this issue

2026-04-20 02:57:25 -05:00

[PR #701] [MERGED] fix: admin issue #20424

GiteaMirror referenced this issue

2026-04-25 10:06:27 -05:00

[PR #701] [MERGED] fix: admin issue #36054

GiteaMirror referenced this issue

2026-04-29 17:34:29 -05:00

[PR #701] [MERGED] fix: admin issue #43472