Ollama's OpenAI Compatible APIs do not have '/api' prefix.
However, In current implentation of open-webui, OLLAMA_API_BASE_URL should have '/api' suffix.
If users want to communicate with Ollama's API through open-webui,
then they can't use Ollama's OpenAI Compatible APIs and use open-webui in browser at the same time.
Describe the solution you'd like
Are there any ways to support both Ollama's own APIs and Ollama's OpenAI Compatible APIs ?
Maybe adding a new route /ollama/api/v1/{path} and manually handling the OLLAMA_API_BASE_URL in the handler can solve this problem?
Originally created by @wlh320 on GitHub (Mar 2, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/999
**Is your feature request related to a problem? Please describe.**
According to https://ollama.com/blog/openai-compatibility and the code
https://github.com/ollama/ollama/blob/21347e1ed67a4ef36abc11bf314e90eaba9a0dc4/server/routes.go#L961-L962
Ollama's OpenAI Compatible APIs do not have '/api' prefix.
However, In current implentation of open-webui, `OLLAMA_API_BASE_URL` should have '/api' suffix.
If users want to communicate with Ollama's API through open-webui,
then they can't use Ollama's OpenAI Compatible APIs and use open-webui in browser at the same time.
**Describe the solution you'd like**
Are there any ways to support both Ollama's own APIs and Ollama's OpenAI Compatible APIs ?
Maybe adding a new route `/ollama/api/v1/{path}` and manually handling the `OLLAMA_API_BASE_URL` in the handler can solve this problem?
Why? The OpenAI compatible endpoint does not return the same information the native one does, so it's a degraded experience if used in Open WebUI. Besides, you can already add it as an "OpenAI" connection or LiteLLM "model". This request is redundant.
<!-- gh-comment-id:1974849687 -->
@justinh-rahb commented on GitHub (Mar 2, 2024):
Why? The OpenAI compatible endpoint does not return the same information the native one does, so it's a degraded experience if used in Open WebUI. Besides, you can already add it as an "OpenAI" connection or LiteLLM "model". This request is redundant.
Open WebUI provides security benefits for Ollama APIs, so some users would like to use other clients such as a desktop client which are compatible with OpenAI's API, and fill in open-webui's url to use Ollama, and some other users would like to use Open WebUI directly from the browser. In this condition, these two requirements cannot be met at the same time.
<!-- gh-comment-id:1974854106 -->
@wlh320 commented on GitHub (Mar 2, 2024):
Open WebUI provides security benefits for Ollama APIs, so some users would like to use other clients such as a desktop client which are compatible with OpenAI's API, and fill in open-webui's url to use Ollama, and some other users would like to use Open WebUI directly from the browser. In this condition, these two requirements cannot be met at the same time.
Maybe the first way is a wrong way to use Open WebUI because it only uses its exposed APIs.
<!-- gh-comment-id:1974856166 -->
@wlh320 commented on GitHub (Mar 2, 2024):
Maybe the first way is a wrong way to use Open WebUI because it only uses its exposed APIs.
Maybe the first way is a wrong way to use Open WebUI because it only uses its exposed APIs.
It certainly doesn't sound like an intended use-case. If you want authentication for Ollama API there are other ways to do it, you could front it with a reverse-proxy that adds an authentication mechanism. I don't think there's anything that can be done to service this need from WebUI's point of view though.
<!-- gh-comment-id:1974856793 -->
@justinh-rahb commented on GitHub (Mar 2, 2024):
> Maybe the first way is a wrong way to use Open WebUI because it only uses its exposed APIs.
It certainly doesn't sound like an _intended_ use-case. If you want authentication for Ollama API there are other ways to do it, you could front it with a reverse-proxy that adds an authentication mechanism. I don't think there's anything that can be done to service this need from WebUI's point of view though.
Thank you! I thought it is intended. I am sorry for misunderstanding this.
<!-- gh-comment-id:1974858622 -->
@wlh320 commented on GitHub (Mar 2, 2024):
Thank you! I thought it is intended. I am sorry for misunderstanding this.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @wlh320 on GitHub (Mar 2, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/999
Is your feature request related to a problem? Please describe.
According to https://ollama.com/blog/openai-compatibility and the code
https://github.com/ollama/ollama/blob/21347e1ed67a4ef36abc11bf314e90eaba9a0dc4/server/routes.go#L961-L962
Ollama's OpenAI Compatible APIs do not have '/api' prefix.
However, In current implentation of open-webui,
OLLAMA_API_BASE_URLshould have '/api' suffix.If users want to communicate with Ollama's API through open-webui,
then they can't use Ollama's OpenAI Compatible APIs and use open-webui in browser at the same time.
Describe the solution you'd like
Are there any ways to support both Ollama's own APIs and Ollama's OpenAI Compatible APIs ?
Maybe adding a new route
/ollama/api/v1/{path}and manually handling theOLLAMA_API_BASE_URLin the handler can solve this problem?@justinh-rahb commented on GitHub (Mar 2, 2024):
Why? The OpenAI compatible endpoint does not return the same information the native one does, so it's a degraded experience if used in Open WebUI. Besides, you can already add it as an "OpenAI" connection or LiteLLM "model". This request is redundant.
@wlh320 commented on GitHub (Mar 2, 2024):
Open WebUI provides security benefits for Ollama APIs, so some users would like to use other clients such as a desktop client which are compatible with OpenAI's API, and fill in open-webui's url to use Ollama, and some other users would like to use Open WebUI directly from the browser. In this condition, these two requirements cannot be met at the same time.
@wlh320 commented on GitHub (Mar 2, 2024):
Maybe the first way is a wrong way to use Open WebUI because it only uses its exposed APIs.
@justinh-rahb commented on GitHub (Mar 2, 2024):
It certainly doesn't sound like an intended use-case. If you want authentication for Ollama API there are other ways to do it, you could front it with a reverse-proxy that adds an authentication mechanism. I don't think there's anything that can be done to service this need from WebUI's point of view though.
@wlh320 commented on GitHub (Mar 2, 2024):
Thank you! I thought it is intended. I am sorry for misunderstanding this.