[GH-ISSUE #3237] Enh: call outlet hook from the backend #116494

Closed
opened 2026-05-20 18:13:14 -05:00 by GiteaMirror · 24 comments
Owner

Originally created by @frederikschubert on GitHub (Jun 17, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3237

Is your feature request related to a problem? Please describe.
We are using Open WebUI as a general solution to manage access to LLMs and RAG applications in our company. Besides the Open WebUI web application, we are using the continue plugin with the following configuration:

"models": [
    {
      "model": "gpt-4o",
      "title": "GPT-4o",
      "apiKey": "sk-...,
      "completionOptions": {},
      "apiBase": "https://openwebui/openai",
      "provider": "openai",
      "requestOptions": {
        "headers": {
            "Content-Type": "application/json"
        }
      }
    },
...

Additionally, we are using pipelines such as the langfuse filter to track the usage. When called via the continue plugin, only the inlet is called. This leaves the response text empty, as it is set in the outlet filter.

Describe the solution you'd like
The inlet as well as the outlet filter of the registered pipelines should be called when using the Open WebUI as an API proxy.

Describe alternatives you've considered
The langfuse filter could be integrated in a general litellm setup, but I think that using inlets and outlets of pipelines in the same way for all chat interactions is a good feature in general.

Originally created by @frederikschubert on GitHub (Jun 17, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/3237 **Is your feature request related to a problem? Please describe.** We are using Open WebUI as a general solution to manage access to LLMs and RAG applications in our company. Besides the Open WebUI web application, we are using the [continue](https://docs.openwebui.com/tutorial/continue-dev/) plugin with the following configuration: ```json "models": [ { "model": "gpt-4o", "title": "GPT-4o", "apiKey": "sk-..., "completionOptions": {}, "apiBase": "https://openwebui/openai", "provider": "openai", "requestOptions": { "headers": { "Content-Type": "application/json" } } }, ... ``` Additionally, we are using pipelines such as the [langfuse filter ](https://github.com/open-webui/pipelines/blob/main/examples/filters/langfuse_filter_pipeline.py) to track the usage. When called via the continue plugin, only the `inlet` is called. This leaves the response text empty, as it is set in the `outlet` filter. **Describe the solution you'd like** The `inlet` as well as the `outlet` [filter](https://github.com/open-webui/open-webui/blob/main/backend/main.py#L1033) of the registered pipelines should be called when using the Open WebUI as an API proxy. **Describe alternatives you've considered** The langfuse filter could be integrated in a general litellm setup, but I think that using inlets and outlets of pipelines in the same way for all chat interactions is a good feature in general.
Author
Owner

@Ronan035 commented on GitHub (Sep 30, 2024):

Same issue with version 0.3.30 and current WebUI Pipelines docker image.
I use Continue and Open WebUI as a proxy to an external Ollama instance. The outlets are 'visible' via Open WebUI chat but not for calls made by Continue via /ollama/v1 endpoint. Only inlets are taken into account.
(Open WebUI is a fantastic tool! Huge huge thanks for your work!)

<!-- gh-comment-id:2383004864 --> @Ronan035 commented on GitHub (Sep 30, 2024): Same issue with version 0.3.30 and current WebUI Pipelines docker image. I use Continue and Open WebUI as a proxy to an external Ollama instance. The outlets are 'visible' via Open WebUI chat but not for calls made by Continue via <Open WebUI URL>/ollama/v1 endpoint. Only inlets are taken into account. (Open WebUI is a fantastic tool! Huge huge thanks for your work!)
Author
Owner

@Ronan035 commented on GitHub (Oct 17, 2024):

ok, it's not an issue (https://github.com/open-webui/open-webui/discussions/4460). But it would be very convenient to have a chat API call that takes into account outlet filters. The use of tools often does not allow to implement the Open WebUI client workflow.

<!-- gh-comment-id:2419867577 --> @Ronan035 commented on GitHub (Oct 17, 2024): ok, it's not an issue (https://github.com/open-webui/open-webui/discussions/4460). But it would be very convenient to have a chat API call that takes into account outlet filters. The use of tools often does not allow to implement the Open WebUI client workflow.
Author
Owner

@ADD-Carlos-Zamora commented on GitHub (Oct 17, 2024):

As @Ronan035 said, thanks for your amazing job, Open WebUI team!

At my company, we are facing the same problem. We tried to create a Pipe to intercept the call and process the response produced by Ollama, but we have not been able to make it executed :-/

How is this issue going? Is it being considered?

<!-- gh-comment-id:2419925728 --> @ADD-Carlos-Zamora commented on GitHub (Oct 17, 2024): As @Ronan035 said, thanks for your amazing job, Open WebUI team! At my company, we are facing the same problem. We tried to create a Pipe to intercept the call and process the response produced by Ollama, but we have not been able to make it executed :-/ How is this issue going? Is it being considered?
Author
Owner

@sir3mat commented on GitHub (Nov 5, 2024):

Hi, i have seen a strange behaviour using rag and pipelines

The workflow is as follows:

Documents are loaded and fed into OpenWebUI.
OpenWebUI encodes the documents, splits them into chunks, and stores them.
The /inlet endpoint on the pipeline is called.
OpenWebUI then performs the RAG operation.
The pipeline’s /pipe endpoint is triggered, displaying the response on OpenWebUI.
Finally, the client calls the /outlet endpoint

Question on Implementation:

Why do the /inlet, /pipe, and /outlet endpoints each have different request bodies?
Why the process is openwebuiClient->inlet->openwebuiClient->pipe->openwebuiClient->outlet->openwebuiClient?

If I develop RAG on my custom pipeline in pipe logic the RAG of openwebui is still executed.

<!-- gh-comment-id:2457750724 --> @sir3mat commented on GitHub (Nov 5, 2024): Hi, i have seen a strange behaviour using rag and pipelines The workflow is as follows: Documents are loaded and fed into OpenWebUI. OpenWebUI encodes the documents, splits them into chunks, and stores them. The /inlet endpoint on the pipeline is called. OpenWebUI then performs the RAG operation. The pipeline’s /pipe endpoint is triggered, displaying the response on OpenWebUI. Finally, the client calls the /outlet endpoint Question on Implementation: Why do the /inlet, /pipe, and /outlet endpoints each have different request bodies? Why the process is openwebuiClient->inlet->openwebuiClient->pipe->openwebuiClient->outlet->openwebuiClient? If I develop RAG on my custom pipeline in pipe logic the RAG of openwebui is still executed.
Author
Owner

@rkconsulting commented on GitHub (Dec 8, 2024):

stumbling across this as well.. we def need to be able to get at the outlet from backend for various retrieval purposes. thanks oi team!

<!-- gh-comment-id:2526227982 --> @rkconsulting commented on GitHub (Dec 8, 2024): stumbling across this as well.. we def need to be able to get at the outlet from backend for various retrieval purposes. thanks oi team!
Author
Owner

@Seniorsimo commented on GitHub (Dec 20, 2024):

I'm having the same problem using the continue plugin with a simple filter to count the token usage as metric. When using the model from the UI, booth the inlet filter and the outled are called, but when using the same model from the chat in the continue plugin it results in a call to the endpoint /api/chat/completions and the outlet filter is not called.

<!-- gh-comment-id:2557844161 --> @Seniorsimo commented on GitHub (Dec 20, 2024): I'm having the same problem using the continue plugin with a simple filter to count the token usage as metric. When using the model from the UI, booth the inlet filter and the outled are called, but when using the same model from the chat in the continue plugin it results in a call to the endpoint `/api/chat/completions` and the outlet filter is not called.
Author
Owner

@DmitriyAlergant commented on GitHub (Mar 10, 2025):

Hi @tjbck, your recent improvements to Filters including .stream() method works for both UI and API consumption, so this already moved things into the right direction.

Would you support a PR that moves outlet() filter calls from /chat/completed and into /chat/completions? Specifically into the process_chat_response() funciton. It will be somewhat tricky to do correctly for all non-streaming and streaming request avenues - types including tool calls, code interpreter, tasks, etc. That function is somewhat complicated. But should be possible.

Would you accept such a PR (if it passes the tests), or is it something that you are looking to do yourself any time soon?

<!-- gh-comment-id:2711486915 --> @DmitriyAlergant commented on GitHub (Mar 10, 2025): Hi @tjbck, your recent improvements to Filters including .stream() method works for both UI and API consumption, so this already moved things into the right direction. Would you support a PR that moves outlet() filter calls from /chat/completed and into /chat/completions? Specifically into the process_chat_response() funciton. It will be somewhat tricky to do correctly for all non-streaming and streaming request avenues - types including tool calls, code interpreter, tasks, etc. That function is somewhat complicated. But should be possible. Would you accept such a PR (if it passes the tests), or is it something that you are looking to do yourself any time soon?
Author
Owner

@arunbugkiller commented on GitHub (Mar 12, 2025):

@DmitriyAlergant, @Seniorsimo @frederikschubert @rkconsulting @sir3mat
I am facing an issue where the filter is getting executed at the backend, but sometimes the results are shown on the front-end and sometimes they are not. Any idea on how this can be resolved.

<!-- gh-comment-id:2716606992 --> @arunbugkiller commented on GitHub (Mar 12, 2025): @DmitriyAlergant, @Seniorsimo @frederikschubert @rkconsulting @sir3mat I am facing an issue where the filter is getting executed at the backend, but sometimes the results are shown on the front-end and sometimes they are not. Any idea on how this can be resolved.
Author
Owner

@yz342 commented on GitHub (Apr 25, 2025):

A workaround would be to inject the connection to Open WebUI through pipeline and add an observability tool like openlit there

<!-- gh-comment-id:2831364856 --> @yz342 commented on GitHub (Apr 25, 2025): A workaround would be to inject the connection to Open WebUI through [pipeline](https://github.com/open-webui/pipelines/tree/main/examples/pipelines/providers) and add an observability tool like [openlit](https://openlit.io/blogs/openlit-openwebui) there
Author
Owner

@DmitriyAlergant commented on GitHub (Apr 27, 2025):

I actually made an attempt at starting this as a PR (moving "outlet" call from /chat/completed to /chat/completion). Unfortunately the stream processing code pathway is very complicated (also with embedded tool call calls, etc), it would be a nightmare to test, so I gave up.

What can be done though (relatively easily) is to introduce a new filter method "batch_outlet()" which would be

  1. Only applicable to non-streaming requests (much easier to code and test)
  2. Invoked reliably from /chat/completions regardless of who is calling it whether the app itself or an API user
  3. Won't break compatibility with all existing filters that may have relied on the legacy behavior (calling of "outlet" from the app only, both for streaming and batch requests, etc).

With a combination of stream() and batch_outlet() filters, one will be able to achieve reliable output filtering for all requests including batch and streaming requests, from the app and the API. While existing community filters may continue relying on the legacy outlet() method until it is sunset.

@tjbck would you agree to this approach? It's not motivating to work on a PR if we don't know upfront whether you like the design in general.

<!-- gh-comment-id:2832903680 --> @DmitriyAlergant commented on GitHub (Apr 27, 2025): I actually made an attempt at starting this as a PR (moving "outlet" call from /chat/completed to /chat/completion). Unfortunately the stream processing code pathway is very complicated (also with embedded tool call calls, etc), it would be a nightmare to test, so I gave up. What can be done though (relatively easily) is to introduce a new filter method "**batch_outlet**()" which would be 1) Only applicable to non-streaming requests (much easier to code and test) 2) Invoked reliably from /chat/completions regardless of who is calling it whether the app itself or an API user 3) Won't break compatibility with all existing filters that may have relied on the legacy behavior (calling of "outlet" from the app only, both for streaming and batch requests, etc). With a combination of **stream()** and **batch_outlet()** filters, one will be able to achieve reliable output filtering for all requests including batch and streaming requests, from the app and the API. While existing community filters may continue relying on the legacy **outlet()** method until it is sunset. @tjbck would you agree to this approach? It's not motivating to work on a PR if we don't know upfront whether you like the design in general.
Author
Owner

@DmitriyAlergant commented on GitHub (Apr 27, 2025):

@yz342 in https://github.com/open-webui/open-webui/discussions/8722#discussioncomment-12193783 people are saying the the behavior is the same for pipelines as well... outlet() not being called for API usage, being an "Open WebUI exclusive feature"

<!-- gh-comment-id:2832905402 --> @DmitriyAlergant commented on GitHub (Apr 27, 2025): @yz342 in https://github.com/open-webui/open-webui/discussions/8722#discussioncomment-12193783 people are saying the the behavior is the same for pipelines as well... outlet() not being called for API usage, being an "Open WebUI exclusive feature"
Author
Owner

@MotherEarth-AI commented on GitHub (Jun 12, 2025):

Same problem here ;(

<!-- gh-comment-id:2966077690 --> @MotherEarth-AI commented on GitHub (Jun 12, 2025): Same problem here ;(
Author
Owner

@bennfocus commented on GitHub (Jun 20, 2025):

Same problem

<!-- gh-comment-id:2992377857 --> @bennfocus commented on GitHub (Jun 20, 2025): Same problem
Author
Owner

@Jirubizu commented on GitHub (Aug 4, 2025):

Has there been any more light in regards to this. Really would like a proper deployment with langfuse, but not being able to monitor which user sends api requests is a bit worrying

<!-- gh-comment-id:3152467907 --> @Jirubizu commented on GitHub (Aug 4, 2025): Has there been any more light in regards to this. Really would like a proper deployment with langfuse, but not being able to monitor which user sends api requests is a bit worrying
Author
Owner

@joni-graham commented on GitHub (Jan 23, 2026):

The lack of Langfuse trace data for api/chat/completions endpoint is a big issue for us as we are using OWUI API as a LLM-provider for Claude Code, continue.dev, custom scripts, etc. Our remedy was to disable the Langfuse pipeline and instead integrate our LiteLLM proxy (to AWS Bedrock) with Langfuse to catch both UI and API generated chat completions. It tested well in that both type of traces were visible in Langfuse, but we lost the user attribution on UI-generated chat completions. I reviewed the code and think that this conditional logic in openai.py might be why:

# Add user info to the payload if the model is a pipeline if "pipeline" in model and model.get("pipeline"): payload["user"] = { "name": user.name, "id": user.id, "email": user.email, "role": user.role, }

Need some feedback as to whether we should expect OWUI to be updated for this issue, or need to continue to pursue some type of workaround.

<!-- gh-comment-id:3791230175 --> @joni-graham commented on GitHub (Jan 23, 2026): The lack of Langfuse trace data for api/chat/completions endpoint is a big issue for us as we are using OWUI API as a LLM-provider for Claude Code, continue.dev, custom scripts, etc. Our remedy was to disable the Langfuse pipeline and instead integrate our LiteLLM proxy (to AWS Bedrock) with Langfuse to catch both UI and API generated chat completions. It tested well in that both type of traces were visible in Langfuse, but we lost the user attribution on UI-generated chat completions. I reviewed the code and think that this conditional logic in openai.py might be why: `# Add user info to the payload if the model is a pipeline if "pipeline" in model and model.get("pipeline"): payload["user"] = { "name": user.name, "id": user.id, "email": user.email, "role": user.role, }` Need some feedback as to whether we should expect OWUI to be updated for this issue, or need to continue to pursue some type of workaround.
Author
Owner

@yz342 commented on GitHub (Jan 23, 2026):

We have a workaround to use litellm with langfuse callback. Litellm can integrate all connections.
A lot more work but litellm handles the connections better than OWUI, if you have multiple connections

<!-- gh-comment-id:3791244255 --> @yz342 commented on GitHub (Jan 23, 2026): We have a workaround to use litellm with langfuse callback. Litellm can integrate all connections. A lot more work but litellm handles the connections better than OWUI, if you have multiple connections
Author
Owner

@joni-graham commented on GitHub (Jan 23, 2026):

We have a workaround to use litellm with langfuse callback. Litellm can integrate all connections. A lot more work but litellm handles the connections better than OWUI, if you have multiple connections

Are you getting all the user attributions in Langfuse for both UI and API chat completions? If so, do you mind sharing the LiteLLM config settings for Langfuse callback?

<!-- gh-comment-id:3791326589 --> @joni-graham commented on GitHub (Jan 23, 2026): > We have a workaround to use litellm with langfuse callback. Litellm can integrate all connections. A lot more work but litellm handles the connections better than OWUI, if you have multiple connections Are you getting all the user attributions in Langfuse for both UI and API chat completions? If so, do you mind sharing the LiteLLM config settings for Langfuse callback?
Author
Owner

@yz342 commented on GitHub (Jan 23, 2026):

sorry can't share but you can start here https://langfuse.com/integrations/gateways/litellm

<!-- gh-comment-id:3791428079 --> @yz342 commented on GitHub (Jan 23, 2026): sorry can't share but you can start here https://langfuse.com/integrations/gateways/litellm
Author
Owner

@Classic298 commented on GitHub (Apr 14, 2026):

OUTLET now being called - in dev.

<!-- gh-comment-id:4241963874 --> @Classic298 commented on GitHub (Apr 14, 2026): OUTLET now being called - in dev.
Author
Owner

@ulo commented on GitHub (Apr 21, 2026):

I tried using ghcr.io/open-webui/open-webui:dev with a very simple filter function, but the outlet function is not called for an API call (I tried /api/chat/completions and /api/v1/messages). Would love to see that functionality working!!

This is also contradicting the documentation which apparently was already updated before this feature made it into the release: "outlet() now runs inline during /api/chat/completions for both WebUI and direct API requests." (https://docs.openwebui.com/reference/api-endpoints#-filter-and-function-behavior-with-api-requests)

<!-- gh-comment-id:4289291350 --> @ulo commented on GitHub (Apr 21, 2026): I tried using `ghcr.io/open-webui/open-webui:dev` with a very simple filter function, but the `outlet` function is not called for an API call (I tried `/api/chat/completions` and `/api/v1/messages`). Would love to see that functionality working!! This is also contradicting the documentation which apparently was already updated before this feature made it into the release: _"outlet() now runs inline during /api/chat/completions for both WebUI and direct API requests."_ (https://docs.openwebui.com/reference/api-endpoints#-filter-and-function-behavior-with-api-requests)
Author
Owner

@Classic298 commented on GitHub (Apr 21, 2026):

@ulo hi it should do now for non-streaming, unfortunately for streaming its not there yet :( and yeah the docs i updated them but forgot to pull back on the fact its not for streaming yet. need to update it.

<!-- gh-comment-id:4289332796 --> @Classic298 commented on GitHub (Apr 21, 2026): @ulo hi it should do now for non-streaming, unfortunately for streaming its not there yet :( and yeah the docs i updated them but forgot to pull back on the fact its not for streaming yet. need to update it.
Author
Owner

@ulo commented on GitHub (Apr 21, 2026):

Thanks @Classic298 !
But for me neither streaming nor non-streaming works (tested on both open-webui:dev and open-webui:main).
I am not familiar with such large codebases, but I tried to debug a bit by editing the middleware.py within the container: for me the outlet_filter_handler function is never called because the ctx['event_emitter'] within the non_streaming_chat_response_handler and also the streaming_chat_response_handler is always None for API calls, so the main part of those functions is skipped:
0a8a620fb6/backend/open_webui/utils/middleware.py (L3307)
0a8a620fb6/backend/open_webui/utils/middleware.py (L3467)

<!-- gh-comment-id:4290729568 --> @ulo commented on GitHub (Apr 21, 2026): Thanks @Classic298 ! But for me neither streaming nor non-streaming works (tested on both open-webui:dev and open-webui:main). I am not familiar with such large codebases, but I tried to debug a bit by editing the `middleware.py` within the container: for me the `outlet_filter_handler` function is never called because the `ctx['event_emitter']` within the `non_streaming_chat_response_handler` and also the `streaming_chat_response_handler` is always `None` for API calls, so the main part of those functions is skipped: https://github.com/open-webui/open-webui/blob/0a8a620fb6fd4c914494f56ac06475bd5f95a985/backend/open_webui/utils/middleware.py#L3307 https://github.com/open-webui/open-webui/blob/0a8a620fb6fd4c914494f56ac06475bd5f95a985/backend/open_webui/utils/middleware.py#L3467
Author
Owner

@Classic298 commented on GitHub (Apr 21, 2026):

Thanks I'll look into it again. I already updated the docs a bit 1hr ago and I'll do it again if i have to

<!-- gh-comment-id:4290762158 --> @Classic298 commented on GitHub (Apr 21, 2026): Thanks I'll look into it again. I already updated the docs a bit 1hr ago and I'll do it again if i have to
Author
Owner

@Classic298 commented on GitHub (Apr 21, 2026):

@ulo thanks — you were right, and my earlier "non-streaming works" claim was wrong. I had someone audit the relevant code paths in backend/open_webui/utils/middleware.py on both main and dev and here's the ground truth:

On main (tagged releases): outlet() is not called by /api/chat/completions at all — inline outlet_filter_handler doesn't exist there. The only path is still POST /api/chat/completed.

On dev: outlet_filter_handler is wired up, but both call sites (non-streaming at middleware.py:3420, streaming at middleware.py:4971) are gated by if event_emitter:. You're right that the check gates the block — but event_emitter isn't unconditionally None for API callers. get_event_emitter_and_caller (middleware.py:2836-2851) populates it whenever both metadata.chat_id and metadata.message_id are truthy. Those come from form_data["chat_id"] and form_data["id"] in main.py:1647 / main.py:2005. An API caller that supplies both gets a real event_emitter and does reach outlet. Without either field — which is what a standard OpenAI-shaped request looks like — event_emitter stays None and outlet is skipped. That matches what you saw.

Two more gotchas the audit surfaced, which probably explain why even "correct" requests still looked broken:

  1. On the non-streaming path, outlet does not rewrite the HTTP response body. outlet_filter_handler updates the persisted chat row and emits a chat:outlet WS event; the JSON returned to the HTTP client is built from response_data before outlet runs and never re-read. So even when outlet fires, a pure API consumer reading the HTTP response won't see the filtered content.
  2. On the streaming path, if you do send chat_id + id, the handler consumes the upstream stream itself and routes deltas to the user's WS room — the HTTP streaming response to an API client is effectively empty. It's designed for a WebUI listening on the socket.

I've rewritten the docs accordingly — release-specific matrix, dropped the local:<uuid> "synthesize IDs in inlet()" workaround (ownership checks and the un-rewritten HTTP body mean it wasn't the escape hatch it looked like), and pointed API integrations at /api/chat/completed as the supported surface today.

Sorry for the noise in the earlier revision.

<!-- gh-comment-id:4291169936 --> @Classic298 commented on GitHub (Apr 21, 2026): @ulo thanks — you were right, and my earlier "non-streaming works" claim was wrong. I had someone audit the relevant code paths in `backend/open_webui/utils/middleware.py` on both `main` and `dev` and here's the ground truth: **On `main` (tagged releases):** `outlet()` is **not** called by `/api/chat/completions` at all — inline `outlet_filter_handler` doesn't exist there. The only path is still `POST /api/chat/completed`. **On `dev`:** `outlet_filter_handler` is wired up, but both call sites (non-streaming at `middleware.py:3420`, streaming at `middleware.py:4971`) are gated by `if event_emitter:`. You're right that the check gates the block — but `event_emitter` isn't unconditionally `None` for API callers. `get_event_emitter_and_caller` (`middleware.py:2836-2851`) populates it whenever **both** `metadata.chat_id` and `metadata.message_id` are truthy. Those come from `form_data["chat_id"]` and `form_data["id"]` in `main.py:1647` / `main.py:2005`. An API caller that supplies both gets a real `event_emitter` and does reach outlet. Without either field — which is what a standard OpenAI-shaped request looks like — `event_emitter` stays `None` and outlet is skipped. That matches what you saw. Two more gotchas the audit surfaced, which probably explain why even "correct" requests still looked broken: 1. On the non-streaming path, outlet **does not rewrite the HTTP response body**. `outlet_filter_handler` updates the persisted chat row and emits a `chat:outlet` WS event; the JSON returned to the HTTP client is built from `response_data` before outlet runs and never re-read. So even when outlet fires, a pure API consumer reading the HTTP response won't see the filtered content. 2. On the streaming path, if you do send `chat_id` + `id`, the handler consumes the upstream stream itself and routes deltas to the user's WS room — the HTTP streaming response to an API client is effectively empty. It's designed for a WebUI listening on the socket. I've rewritten the docs accordingly — release-specific matrix, dropped the `local:<uuid>` "synthesize IDs in inlet()" workaround (ownership checks and the un-rewritten HTTP body mean it wasn't the escape hatch it looked like), and pointed API integrations at `/api/chat/completed` as the supported surface today. Sorry for the noise in the earlier revision.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#116494