mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-17 08:21:12 -05:00
[GH-ISSUE #3237] Enh: call outlet hook from the backend #28712
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @frederikschubert on GitHub (Jun 17, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3237
Is your feature request related to a problem? Please describe.
We are using Open WebUI as a general solution to manage access to LLMs and RAG applications in our company. Besides the Open WebUI web application, we are using the continue plugin with the following configuration:
Additionally, we are using pipelines such as the langfuse filter to track the usage. When called via the continue plugin, only the
inletis called. This leaves the response text empty, as it is set in theoutletfilter.Describe the solution you'd like
The
inletas well as theoutletfilter of the registered pipelines should be called when using the Open WebUI as an API proxy.Describe alternatives you've considered
The langfuse filter could be integrated in a general litellm setup, but I think that using inlets and outlets of pipelines in the same way for all chat interactions is a good feature in general.
@Ronan035 commented on GitHub (Sep 30, 2024):
Same issue with version 0.3.30 and current WebUI Pipelines docker image.
I use Continue and Open WebUI as a proxy to an external Ollama instance. The outlets are 'visible' via Open WebUI chat but not for calls made by Continue via /ollama/v1 endpoint. Only inlets are taken into account.
(Open WebUI is a fantastic tool! Huge huge thanks for your work!)
@Ronan035 commented on GitHub (Oct 17, 2024):
ok, it's not an issue (https://github.com/open-webui/open-webui/discussions/4460). But it would be very convenient to have a chat API call that takes into account outlet filters. The use of tools often does not allow to implement the Open WebUI client workflow.
@ADD-Carlos-Zamora commented on GitHub (Oct 17, 2024):
As @Ronan035 said, thanks for your amazing job, Open WebUI team!
At my company, we are facing the same problem. We tried to create a Pipe to intercept the call and process the response produced by Ollama, but we have not been able to make it executed :-/
How is this issue going? Is it being considered?
@sir3mat commented on GitHub (Nov 5, 2024):
Hi, i have seen a strange behaviour using rag and pipelines
The workflow is as follows:
Documents are loaded and fed into OpenWebUI.
OpenWebUI encodes the documents, splits them into chunks, and stores them.
The /inlet endpoint on the pipeline is called.
OpenWebUI then performs the RAG operation.
The pipeline’s /pipe endpoint is triggered, displaying the response on OpenWebUI.
Finally, the client calls the /outlet endpoint
Question on Implementation:
Why do the /inlet, /pipe, and /outlet endpoints each have different request bodies?
Why the process is openwebuiClient->inlet->openwebuiClient->pipe->openwebuiClient->outlet->openwebuiClient?
If I develop RAG on my custom pipeline in pipe logic the RAG of openwebui is still executed.
@rkconsulting commented on GitHub (Dec 8, 2024):
stumbling across this as well.. we def need to be able to get at the outlet from backend for various retrieval purposes. thanks oi team!
@Seniorsimo commented on GitHub (Dec 20, 2024):
I'm having the same problem using the continue plugin with a simple filter to count the token usage as metric. When using the model from the UI, booth the inlet filter and the outled are called, but when using the same model from the chat in the continue plugin it results in a call to the endpoint
/api/chat/completionsand the outlet filter is not called.@DmitriyAlergant commented on GitHub (Mar 10, 2025):
Hi @tjbck, your recent improvements to Filters including .stream() method works for both UI and API consumption, so this already moved things into the right direction.
Would you support a PR that moves outlet() filter calls from /chat/completed and into /chat/completions? Specifically into the process_chat_response() funciton. It will be somewhat tricky to do correctly for all non-streaming and streaming request avenues - types including tool calls, code interpreter, tasks, etc. That function is somewhat complicated. But should be possible.
Would you accept such a PR (if it passes the tests), or is it something that you are looking to do yourself any time soon?
@arunbugkiller commented on GitHub (Mar 12, 2025):
@DmitriyAlergant, @Seniorsimo @frederikschubert @rkconsulting @sir3mat
I am facing an issue where the filter is getting executed at the backend, but sometimes the results are shown on the front-end and sometimes they are not. Any idea on how this can be resolved.
@yz342 commented on GitHub (Apr 25, 2025):
A workaround would be to inject the connection to Open WebUI through pipeline and add an observability tool like openlit there
@DmitriyAlergant commented on GitHub (Apr 27, 2025):
I actually made an attempt at starting this as a PR (moving "outlet" call from /chat/completed to /chat/completion). Unfortunately the stream processing code pathway is very complicated (also with embedded tool call calls, etc), it would be a nightmare to test, so I gave up.
What can be done though (relatively easily) is to introduce a new filter method "batch_outlet()" which would be
With a combination of stream() and batch_outlet() filters, one will be able to achieve reliable output filtering for all requests including batch and streaming requests, from the app and the API. While existing community filters may continue relying on the legacy outlet() method until it is sunset.
@tjbck would you agree to this approach? It's not motivating to work on a PR if we don't know upfront whether you like the design in general.
@DmitriyAlergant commented on GitHub (Apr 27, 2025):
@yz342 in https://github.com/open-webui/open-webui/discussions/8722#discussioncomment-12193783 people are saying the the behavior is the same for pipelines as well... outlet() not being called for API usage, being an "Open WebUI exclusive feature"
@MotherEarth-AI commented on GitHub (Jun 12, 2025):
Same problem here ;(
@bennfocus commented on GitHub (Jun 20, 2025):
Same problem
@Jirubizu commented on GitHub (Aug 4, 2025):
Has there been any more light in regards to this. Really would like a proper deployment with langfuse, but not being able to monitor which user sends api requests is a bit worrying
@joni-graham commented on GitHub (Jan 23, 2026):
The lack of Langfuse trace data for api/chat/completions endpoint is a big issue for us as we are using OWUI API as a LLM-provider for Claude Code, continue.dev, custom scripts, etc. Our remedy was to disable the Langfuse pipeline and instead integrate our LiteLLM proxy (to AWS Bedrock) with Langfuse to catch both UI and API generated chat completions. It tested well in that both type of traces were visible in Langfuse, but we lost the user attribution on UI-generated chat completions. I reviewed the code and think that this conditional logic in openai.py might be why:
# Add user info to the payload if the model is a pipeline if "pipeline" in model and model.get("pipeline"): payload["user"] = { "name": user.name, "id": user.id, "email": user.email, "role": user.role, }Need some feedback as to whether we should expect OWUI to be updated for this issue, or need to continue to pursue some type of workaround.
@yz342 commented on GitHub (Jan 23, 2026):
We have a workaround to use litellm with langfuse callback. Litellm can integrate all connections.
A lot more work but litellm handles the connections better than OWUI, if you have multiple connections
@joni-graham commented on GitHub (Jan 23, 2026):
Are you getting all the user attributions in Langfuse for both UI and API chat completions? If so, do you mind sharing the LiteLLM config settings for Langfuse callback?
@yz342 commented on GitHub (Jan 23, 2026):
sorry can't share but you can start here https://langfuse.com/integrations/gateways/litellm
@Classic298 commented on GitHub (Apr 14, 2026):
OUTLET now being called - in dev.
@ulo commented on GitHub (Apr 21, 2026):
I tried using
ghcr.io/open-webui/open-webui:devwith a very simple filter function, but theoutletfunction is not called for an API call (I tried/api/chat/completionsand/api/v1/messages). Would love to see that functionality working!!This is also contradicting the documentation which apparently was already updated before this feature made it into the release: "outlet() now runs inline during /api/chat/completions for both WebUI and direct API requests." (https://docs.openwebui.com/reference/api-endpoints#-filter-and-function-behavior-with-api-requests)
@Classic298 commented on GitHub (Apr 21, 2026):
@ulo hi it should do now for non-streaming, unfortunately for streaming its not there yet :( and yeah the docs i updated them but forgot to pull back on the fact its not for streaming yet. need to update it.
@ulo commented on GitHub (Apr 21, 2026):
Thanks @Classic298 !
But for me neither streaming nor non-streaming works (tested on both open-webui:dev and open-webui:main).
I am not familiar with such large codebases, but I tried to debug a bit by editing the
middleware.pywithin the container: for me theoutlet_filter_handlerfunction is never called because thectx['event_emitter']within thenon_streaming_chat_response_handlerand also thestreaming_chat_response_handleris alwaysNonefor API calls, so the main part of those functions is skipped:0a8a620fb6/backend/open_webui/utils/middleware.py (L3307)0a8a620fb6/backend/open_webui/utils/middleware.py (L3467)@Classic298 commented on GitHub (Apr 21, 2026):
Thanks I'll look into it again. I already updated the docs a bit 1hr ago and I'll do it again if i have to
@Classic298 commented on GitHub (Apr 21, 2026):
@ulo thanks — you were right, and my earlier "non-streaming works" claim was wrong. I had someone audit the relevant code paths in
backend/open_webui/utils/middleware.pyon bothmainanddevand here's the ground truth:On
main(tagged releases):outlet()is not called by/api/chat/completionsat all — inlineoutlet_filter_handlerdoesn't exist there. The only path is stillPOST /api/chat/completed.On
dev:outlet_filter_handleris wired up, but both call sites (non-streaming atmiddleware.py:3420, streaming atmiddleware.py:4971) are gated byif event_emitter:. You're right that the check gates the block — butevent_emitterisn't unconditionallyNonefor API callers.get_event_emitter_and_caller(middleware.py:2836-2851) populates it whenever bothmetadata.chat_idandmetadata.message_idare truthy. Those come fromform_data["chat_id"]andform_data["id"]inmain.py:1647/main.py:2005. An API caller that supplies both gets a realevent_emitterand does reach outlet. Without either field — which is what a standard OpenAI-shaped request looks like —event_emitterstaysNoneand outlet is skipped. That matches what you saw.Two more gotchas the audit surfaced, which probably explain why even "correct" requests still looked broken:
outlet_filter_handlerupdates the persisted chat row and emits achat:outletWS event; the JSON returned to the HTTP client is built fromresponse_databefore outlet runs and never re-read. So even when outlet fires, a pure API consumer reading the HTTP response won't see the filtered content.chat_id+id, the handler consumes the upstream stream itself and routes deltas to the user's WS room — the HTTP streaming response to an API client is effectively empty. It's designed for a WebUI listening on the socket.I've rewritten the docs accordingly — release-specific matrix, dropped the
local:<uuid>"synthesize IDs in inlet()" workaround (ownership checks and the un-rewritten HTTP body mean it wasn't the escape hatch it looked like), and pointed API integrations at/api/chat/completedas the supported surface today.Sorry for the noise in the earlier revision.