feat: Customizable API Model for Title Generation #4907

New Issue

2025-11-11T16:06:08-06:00

GiteaMirror commented

2025-11-11 16:06:08 -06:00

Originally created by @Amamiya23 on GitHub (Apr 19, 2025).

Check Existing Issues

I have searched the existing issues and discussions.

Problem Description

I would like to propose the ability to use a customizable API model specifically for generating titles. The idea is to allow users to set up a lightweight model (e.g., a 3B parameter model) dedicated solely to generating conversation titles.

Currently, it seems that the title generation process relies on the same model being used for the main conversation or reasoning tasks. This can lead to inefficiencies, especially when using more complex models that involve chain-of-thought reasoning. For example, as shown in the red box in the attached image, generating a title with such models takes significantly longer than necessary.

Desired Solution you'd like

Customizable Model Selection: Allow users to specify a smaller, lightweight model for title generation tasks. This would reduce the computational overhead and speed up the process.
API Integration: Provide an API endpoint where users can define which model they want to use for title generation, separate from the main conversational or reasoning model.

Alternatives Considered

No response

Additional Context

No response

Originally created by @Amamiya23 on GitHub (Apr 19, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Problem Description I would like to propose the ability to use a customizable API model specifically for generating titles. The idea is to allow users to set up a lightweight model (e.g., a 3B parameter model) dedicated solely to generating conversation titles. Currently, it seems that the title generation process relies on the same model being used for the main conversation or reasoning tasks. This can lead to inefficiencies, especially when using more complex models that involve chain-of-thought reasoning. For example, as shown in the red box in the attached image, generating a title with such models takes significantly longer than necessary. ![Image](https://github.com/user-attachments/assets/79d41e07-b493-4a26-bb4d-99f1cb09c3a1) ### Desired Solution you'd like Customizable Model Selection: Allow users to specify a smaller, lightweight model for title generation tasks. This would reduce the computational overhead and speed up the process. API Integration: Provide an API endpoint where users can define which model they want to use for title generation, separate from the main conversational or reasoning model. ### Alternatives Considered _No response_ ### Additional Context _No response_

GiteaMirror closed this issue

2025-11-11 16:06:08 -06:00

GiteaMirror commented

2025-11-11 16:06:09 -06:00