Bug Summary:
I'm trying to connect to the Ollama API via WebUI backend (as it is shown in README). WebUI works just fine, direct connection to Ollama API by port doesn't have any problems too, but /ollama/api/... endpoints don't seem to work properly.
Steps to Reproduce:
Make request using /ollama API endpoint:
~$ curl -s -X POST -H "Authorization: Bearer <token>"\
https://my_domain.example/ollama/api/generate \
-d '{"model":"openchat","prompt":"Why is the sky blue?"}'| jq
Expected Behavior:
Same as direct request to 11434 port:
~$ curl http://my_domain.example:11434/api/generate \
-d '{"model":"openchat","prompt":"Why is the sky blue?"}'| jq
Originally created by @georg3k on GitHub (Mar 31, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1366
# Bug Report
## Description
**Bug Summary:**
I'm trying to connect to the Ollama API via WebUI backend (as it is shown in README). WebUI works just fine, direct connection to Ollama API by port doesn't have any problems too, but /ollama/api/... endpoints don't seem to work properly.
**Steps to Reproduce:**
Make request using /ollama API endpoint:
```bash
~$ curl -s -X POST -H "Authorization: Bearer <token>" \
https://my_domain.example/ollama/api/generate \
-d '{"model":"openchat","prompt":"Why is the sky blue?"}' | jq
```
**Expected Behavior:**
Same as direct request to 11434 port:
```bash
~$ curl http://my_domain.example:11434/api/generate \
-d '{"model":"openchat","prompt":"Why is the sky blue?"}' | jq
```
```json
{
"model": "openchat",
"created_at": "2024-03-31T14:33:39.886274266Z",
"response": " The",
"done": false
}
{
"model": "openchat",
"created_at": "2024-03-31T14:33:40.007860549Z",
"response": " sky",
"done": false
}
{
"model": "openchat",
"created_at": "2024-03-31T14:33:40.127979905Z",
"response": " appears",
"done": false
}
{
"model": "openchat",
"created_at": "2024-03-31T14:33:40.248329815Z",
"response": " blue",
"done": false
}
{
"model": "openchat",
"created_at": "2024-03-31T14:33:40.368849403Z",
"response": " to",
"done": false
}
...
```
**Actual Behavior:**
Actual response:
```json
{
"detail": "error_detail"
}
```
## Environment
- Ubuntu 22.04 LTS
- Docker v20.10.24
- Ollama v0.1.30
- Open WebUI v0.1.116
## Reproduction Details
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
## Logs and Screenshots
**Docker Container Logs:**
```
open-webui | DEBUG:peewee:('SELECT "t1"."id", "t1"."name", "t1"."email", "t1"."role", "t1"."profile_image_url", "t1"."timestamp" FROM "user" AS "t1" WHERE ("t1"."id" = ?) LIMIT ? OFFSET ?', ['672e8a9a-12334-1234-1234-c5f3b21ada68', 1, 0])
open-webui | INFO: 172.22.0.1:42428 - "POST /ollama/api/generate HTTP/1.1" 400 Bad Request
```
## Installation Method
Deployed with docker-compose:
```yaml
version: '3.8'
services:
ollama:
volumes:
- ollama:/root/.ollama
container_name: ollama
pull_policy: always
tty: true
restart: unless-stopped
image: ollama/ollama:latest
open-webui:
image: ghcr.io/open-webui/open-webui:latest
container_name: open-webui
pull_policy: always
volumes:
- open-webui:/app/backend/data
depends_on:
- ollama
ports:
- "8080:8080"
environment:
- 'OLLAMA_BASE_URL=http://ollama:11434'
- 'WEBUI_SECRET_KEY='
- 'GLOBAL_LOG_LEVEL=DEBUG'
extra_hosts:
- host.docker.internal:host-gateway
restart: unless-stopped
volumes:
ollama: {}
open-webui: {}
```
Most likely the issue is occurring because Ollama couldn't find the model you specified in the body data, Here's the correct format for the payload:
{
"model": "mistral:latest",
"prompt": "Why is the sky blue?"
}
Updated the error message in our dev branch, let me know if the issue persists!
<!-- gh-comment-id:2028900071 -->
@tjbck commented on GitHub (Mar 31, 2024):
Most likely the issue is occurring because Ollama couldn't find the model you specified in the body data, Here's the correct format for the payload:
```
{
"model": "mistral:latest",
"prompt": "Why is the sky blue?"
}
```
Updated the error message in our dev branch, let me know if the issue persists!
@tjbck, direct requests to the ollama server work without model tag, so I thought that WebUI passes it as is and it should work the same way. With version specified it works now, thanks!
<!-- gh-comment-id:2028914761 -->
@georg3k commented on GitHub (Mar 31, 2024):
@tjbck, direct requests to the ollama server work without model tag, so I thought that WebUI passes it as is and it should work the same way. With version specified it works now, thanks!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @georg3k on GitHub (Mar 31, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1366
Bug Report
Description
Bug Summary:
I'm trying to connect to the Ollama API via WebUI backend (as it is shown in README). WebUI works just fine, direct connection to Ollama API by port doesn't have any problems too, but /ollama/api/... endpoints don't seem to work properly.
Steps to Reproduce:
Make request using /ollama API endpoint:
Expected Behavior:
Same as direct request to 11434 port:
Actual Behavior:
Actual response:
Environment
Reproduction Details
Confirmation:
Logs and Screenshots
Docker Container Logs:
Installation Method
Deployed with docker-compose:
@tjbck commented on GitHub (Mar 31, 2024):
Most likely the issue is occurring because Ollama couldn't find the model you specified in the body data, Here's the correct format for the payload:
Updated the error message in our dev branch, let me know if the issue persists!
@georg3k commented on GitHub (Mar 31, 2024):
@tjbck, direct requests to the ollama server work without model tag, so I thought that WebUI passes it as is and it should work the same way. With version specified it works now, thanks!