mirror of
https://github.com/open-webui/open-webui.git
synced 2026-08-30 17:25:30 -05:00
Filters are not active when using openwebui api /api/chat/completions #3331
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#3331
Reference in New Issue
Block a user
Originally created by @Classic298 on GitHub (Jan 21, 2025).
Bug Report
Installation Method
PIP on Debian in a venv
Environment
Open WebUI Version: 0.5.4
Operating System: Debian 12
Confirmation:
Expected Behavior:
When sending an API request to /api/chat/completions, any filters are applied to the model as set in the admin panel.
Actual Behavior:
Sending a request to the API seemingly bypasses all filters created/uploaded/activated for that model in the admin panel
Description
Bug Summary:
Applied filters to a model in the admin panel are bypassed when sending requests on the API.
Reproduction Details
Steps to Reproduce:
Create a simple filter (ideally one with a measurable impact), apply the filter to the model from the admin panel and then send a request via the API to that model.
{ "model": modelname, "messages": [ { "role": "user", "content": message } ] }Additional Information
Specifically annoying for applying rate limits via a rate limit filter or for tracking input tokens and output tokens. I have a filter for rate limits and a second filter for tracking my input tokens and my output tokens so I can compare it to the invoice by the AI cloud service. But these filters are not applied when going via the openwebui API.
I did search the prior issues but couldn't find anyone report this yet
@spammenotinoz commented on GitHub (Jan 21, 2025):
For me they are now working.
When i tested this, I was on v0.5.2, but turns out that had a bug with 'event_emitter' and 'outlet hooks' where none of my function filters were working.
Well I have some filters applied again custom models, but only raw models seem to be available via API
@Classic298 commented on GitHub (Jan 21, 2025):
For me they are not working on the API, still. What did you do to make them work?
Even when setting the filter to global, it does not work.
@Classic298 commented on GitHub (Jan 21, 2025):
I found out that the /api/chat/completions endpoint queries the "chat_completion_handler()" function.
The chat_completion_handler is an imported function that was renamed.
It's actual name (before getting renamed in the import stattement) is generate_chat_completion and was imported from the chat.py.
Seemingly, ollama.py and openai.py are not directly imported anywhere, except into the chat.py.
The generate_chat_completion() in the chat.py is always trying to call the process_pipeline_inlet_filter() function, to process the inlet filters.
This makes it even the more confusing, why requests to the API are not being processed by filters, but requests in the normal WebUI do go through all the filters. I thought I will find the answer in the code, but now I am even the more confused.
Perhaps someone can enlighten me how API calls are handled (differently) because I fail to see any difference to how API calls are handled versus 'normal' chat-requests. I could then try to fix the issue with a PR or even implement a toggle (or environment variable). But I really fail to see how and where they are handled differently.
openai.py and ollama.py are only imported into chat.py and chat.py always calls the filter inlets in the generate_chat_completion() function.