github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-11-14 05:02:01 -06:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #5904] No suitable llama servers found #68772

New Issue
Closed
opened 2026-05-13 01:09:45 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-13 01:09:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @AFwcxx on GitHub (Oct 4, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/5904

Bug Report

Installation Method

Docker

Environment

  • Open WebUI Version: latest from docker

  • Ollama (if applicable): latest from docker

  • Operating System: Ubuntu 22.04

  • Browser (if applicable): Chrome

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Description

Bug Summary:
Can't use any model. Tried llama3.2, llama3.1 and gemma2

Reproduction Details

Steps to Reproduce:

# Install nvidia toolkit
$ sudo apt-get install -y nvidia-container-toolkit
$ sudo nvidia-ctk runtime configure --runtime=docker
$ sudo systemctl restart docker

# Install and run
$ docker run -d --network=host -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Logs and Screenshots

Browser Console Logs:

Ollama: 500, message='Internal Server Error', url='http://localhost:11434/api/chat'

Docker Container Logs:

INFO  [open_webui.apps.ollama.main] url: http://localhost:11434
time=2024-10-04T04:07:55.762Z level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe gpu=GPU-9a8b1555-a8d6-f26d-69bf-b5dcbfb47f2e parallel=4 available=11889344512 required="6.2 GiB"
time=2024-10-04T04:07:55.762Z level=INFO source=server.go:103 msg="system memory" total="30.5 GiB" free="23.0 GiB" free_swap="20.0 GiB"
time=2024-10-04T04:07:55.762Z level=INFO source=memory.go:326 msg="offload to cuda" layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[11.1 GiB]" memory.gpu_overhead="0 B" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB"
time=2024-10-04T04:08:00.779Z level=ERROR source=common.go:214 msg="failed to extract files" error="copy payload linux/amd64/cuda_v12/libggml.so: flate: corrupt input before offset 417983866"
time=2024-10-04T04:08:01.061Z level=INFO source=sched.go:428 msg="NewLlamaServer failed" model=/root/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe error="no suitable llama servers found"
Originally created by @AFwcxx on GitHub (Oct 4, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/5904 # Bug Report ## Installation Method Docker ## Environment - **Open WebUI Version:** latest from docker - **Ollama (if applicable):** latest from docker - **Operating System:** Ubuntu 22.04 - **Browser (if applicable):** Chrome **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Description **Bug Summary:** Can't use any model. Tried llama3.2, llama3.1 and gemma2 ## Reproduction Details **Steps to Reproduce:** ```bash # Install nvidia toolkit $ sudo apt-get install -y nvidia-container-toolkit $ sudo nvidia-ctk runtime configure --runtime=docker $ sudo systemctl restart docker # Install and run $ docker run -d --network=host -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama ``` ## Logs and Screenshots **Browser Console Logs:** ``` Ollama: 500, message='Internal Server Error', url='http://localhost:11434/api/chat' ``` **Docker Container Logs:** ```bash INFO [open_webui.apps.ollama.main] url: http://localhost:11434 time=2024-10-04T04:07:55.762Z level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe gpu=GPU-9a8b1555-a8d6-f26d-69bf-b5dcbfb47f2e parallel=4 available=11889344512 required="6.2 GiB" time=2024-10-04T04:07:55.762Z level=INFO source=server.go:103 msg="system memory" total="30.5 GiB" free="23.0 GiB" free_swap="20.0 GiB" time=2024-10-04T04:07:55.762Z level=INFO source=memory.go:326 msg="offload to cuda" layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[11.1 GiB]" memory.gpu_overhead="0 B" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB" time=2024-10-04T04:08:00.779Z level=ERROR source=common.go:214 msg="failed to extract files" error="copy payload linux/amd64/cuda_v12/libggml.so: flate: corrupt input before offset 417983866" time=2024-10-04T04:08:01.061Z level=INFO source=sched.go:428 msg="NewLlamaServer failed" model=/root/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe error="no suitable llama servers found" ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#68772