[GH-ISSUE #20244] LLM as Agent: Delayed Image Trigger for Prompt Refinement #106132

New Issue

GiteaMirror · 2026-05-18T04:20:13-05:00

GiteaMirror commented

2026-05-18 04:20:13 -05:00

Originally created by @Bennowan on GitHub (Dec 29, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/20244

Check Existing Issues

I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Verify Feature Scope

I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.

Problem Description

Currently, Open WebUI triggers the image generation (ComfyUI/Automatic1111) immediately when the user sends a message. This makes it impossible for the LLM to act as a "Prompt Architect." If a user provides a simple or non-English request, the Image Engine receives a poor prompt, leading to suboptimal results. There is no "Chain of Thought" where the LLM can plan or refine the prompt before the GPU starts rendering.

Desired Solution you'd like

Implement a "Delayed Trigger" system:

LLM Refinement: The LLM first processes the user input, translates it into a detailed English prompt, and potentially selects a specific model/checkpoint.

Post-Response Trigger: Open WebUI should wait for the LLM to finish its response. Only then should it extract the final prompt (e.g., from brackets [...]) and send it to the Image Engine.

Dynamic Parameters: Allow the LLM to specify parameters like --model, --neg, or --cfg within its text output to dynamically control the image generation process.

Alternatives Considered

No response

Additional Context

No response

Originally created by @Bennowan on GitHub (Dec 29, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/20244 ### Check Existing Issues - [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request. ### Verify Feature Scope - [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions. ### Problem Description Currently, Open WebUI triggers the image generation (ComfyUI/Automatic1111) immediately when the user sends a message. This makes it impossible for the LLM to act as a "Prompt Architect." If a user provides a simple or non-English request, the Image Engine receives a poor prompt, leading to suboptimal results. There is no "Chain of Thought" where the LLM can plan or refine the prompt before the GPU starts rendering. ### Desired Solution you'd like Implement a "Delayed Trigger" system: LLM Refinement: The LLM first processes the user input, translates it into a detailed English prompt, and potentially selects a specific model/checkpoint. Post-Response Trigger: Open WebUI should wait for the LLM to finish its response. Only then should it extract the final prompt (e.g., from brackets [...]) and send it to the Image Engine. Dynamic Parameters: Allow the LLM to specify parameters like --model, --neg, or --cfg within its text output to dynamically control the image generation process. ### Alternatives Considered _No response_ ### Additional Context _No response_

GiteaMirror closed this issue

2026-05-18 04:20:13 -05:00

GiteaMirror commented

2026-05-18 04:20:14 -05:00

@owui-terminator[bot] commented on GitHub (Dec 29, 2025):

🔍 Similar Issues Found

I found some existing issues that might be related to this one. Please check if any of these are duplicates or contain helpful solutions:

#20099 feat: Need to Bypass Non-Multimodal Qwen LLM for ComfyUI Image Generation/Editing in open-webui
by freesunshine • Dec 22, 2025
#10533 Ability to edit an llm response AND have ui continue generating assistant response from that point
by thistleknot • Feb 21, 2025

💡 Tips:

If this is a duplicate, please consider closing this issue and adding any additional details to the existing one
If you found a solution in any of these issues, please share it here to help others

This comment was generated automatically by a bot. Please react with a 👍 if this comment was helpful, or a 👎 if it was not.

@owui-terminator[bot] commented on GitHub (Dec 29, 2025): 🔍 **Similar Issues Found** I found some existing issues that might be related to this one. Please check if any of these are duplicates or contain helpful solutions: 1. [#20099](https://github.com/open-webui/open-webui/issues/20099) **feat: Need to Bypass Non-Multimodal Qwen LLM for ComfyUI Image Generation/Editing in open-webui** *by freesunshine • Dec 22, 2025* 2. [#10533](https://github.com/open-webui/open-webui/issues/10533) **Ability to edit an llm response AND have ui continue generating assistant response from that point** *by thistleknot • Feb 21, 2025* --- 💡 **Tips:** - If this is a duplicate, please consider closing this issue and adding any additional details to the existing one - If you found a solution in any of these issues, please share it here to help others *This comment was generated automatically by a bot.* Please react with a 👍 if this comment was helpful, or a 👎 if it was not.

GiteaMirror commented

2026-05-18 04:20:14 -05:00

@Classic298 commented on GitHub (Dec 29, 2025):

You can already have the ai write the prompt and then press the little image button underneath the message which does exactly what you described.

@Classic298 commented on GitHub (Dec 29, 2025): You can already have the ai write the prompt and then press the little image button underneath the message which does exactly what you described.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#106132