Is your feature request related to a problem? Please describe.
We use open-webui as the frontend of multiple OpenAI API capable backends (e.g. vllm/fastchat) without ollama thanks to the high extensibility of this project. However, we encountered a problem that some of out-of-scope extra parameters provided by inference backends such as vllm cannot be passed due to the lack of support.
For example, OpenAI Python SDK provides an extra_body paramter for users to additional JSON properties to the request.
Describe the solution you'd like
Provide extra_body as a configurable option at frontend, so that it can be passed to the proxied API.
Describe alternatives you've considered
N/A
Additional context
N/A
Originally created by @rudeigerc on GitHub (Apr 10, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1484
**Is your feature request related to a problem? Please describe.**
We use `open-webui` as the frontend of multiple OpenAI API capable backends (e.g. vllm/fastchat) without ollama thanks to the high extensibility of this project. However, we encountered a problem that some of out-of-scope extra parameters provided by inference backends such as [vllm](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#extra-parameters-for-chat-api) cannot be passed due to the lack of support.
For example, OpenAI Python SDK provides an `extra_body` paramter for users to additional JSON properties to the request.
**Describe the solution you'd like**
Provide `extra_body` as a configurable option at frontend, so that it can be passed to the proxied API.
**Describe alternatives you've considered**
N/A
**Additional context**
N/A
With slight modification to this Function, you should be able to achieve what you've outlined here.
<!-- gh-comment-id:2365308148 -->
@tjbck commented on GitHub (Sep 21, 2024):
This is currently supported using Functions, Here's an example of OpenAI Pipe Function: https://openwebui.com/f/hub/openai_manifold
With slight modification to this Function, you should be able to achieve what you've outlined here.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @rudeigerc on GitHub (Apr 10, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1484
Is your feature request related to a problem? Please describe.
We use
open-webuias the frontend of multiple OpenAI API capable backends (e.g. vllm/fastchat) without ollama thanks to the high extensibility of this project. However, we encountered a problem that some of out-of-scope extra parameters provided by inference backends such as vllm cannot be passed due to the lack of support.For example, OpenAI Python SDK provides an
extra_bodyparamter for users to additional JSON properties to the request.Describe the solution you'd like
Provide
extra_bodyas a configurable option at frontend, so that it can be passed to the proxied API.Describe alternatives you've considered
N/A
Additional context
N/A
@esceptico commented on GitHub (Sep 11, 2024):
Second that!
Are there any updates on this?
@tjbck commented on GitHub (Sep 21, 2024):
This is currently supported using Functions, Here's an example of OpenAI Pipe Function: https://openwebui.com/f/hub/openai_manifold
With slight modification to this Function, you should be able to achieve what you've outlined here.