github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #11041] llama3.2-vision enters infinite loop on Kubernetes with NVIDIA A10 GPU #7283

New Issue
Closed
opened 2026-04-12 19:20:00 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-12 19:20:00 -05:00
Owner
Copy Link

Originally created by @LaeraFelipe on GitHub (Jun 10, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11041

What is the issue?

I'm running Ollama with the llama3.2-vision:latest model on a Kubernetes cluster using an NVIDIA A10 GPU. Alongside this model, I'm also running others such as gemma3:4b, llama3.2:3b, etc.—all of which work normally. However, when I run prompts with llama3.2-vision, it initially appears to work fine but eventually enters an infinite loop, as shown in the image below:

Image

Image

In the following image, you can see that Ollama is the only process using the GPU, and during the loop, it fully utilizes the GPU's processing power. Note that this container has no CPU, memory, or GPU limits defined.

Image

The processing only stops when the prompt is manually canceled.

After this initial loop scenario, every subsequent attempt to send an image and ask a question results in the same behavior:

Image

After approximately 5 minutes, the model is unloaded from memory. If I then submit a new prompt, the loop scenario occurs again.

Here are the details of the GPU driver I'm using:

Image

I also have another server with the same configuration, running Ollama version 0.6.7 via Docker Compose, using a llama3.2-vision:latest model downloaded around six months ago—and that setup works perfectly.

Here is the nvidia-smi output from that machine:

Image

Relevant log output

[GIN] 2025/06/10 - 22:54:54 | 200 |   7.45266341s |     10.0.10.123 | POST     "/api/chat"
time=2025-06-10T22:54:54.256Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:54:54.256Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:54:54.257Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
[GIN] 2025/06/10 - 22:55:09 | 200 |     474.068µs |     10.0.10.194 | GET      "/api/tags"
time=2025-06-10T22:55:09.589Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:55:09.590Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:55:09.590Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:55:09.612Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=3769 format=""
time=2025-06-10T22:55:09.825Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=711 prompt=737 used=711 remaining=26
[GIN] 2025/06/10 - 22:55:16 | 200 |  7.456949066s |     10.0.10.194 | POST     "/api/chat"
time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
time=2025-06-10T22:56:12.674Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:56:12.675Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:56:12.675Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:56:12.710Z level=DEBUG source=server.go:729 msg="completion request" images=6 prompt=6397 format=""
time=2025-06-10T22:56:13.052Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1265 prompt=1286 used=1044 remaining=242
[GIN] 2025/06/10 - 22:56:22 | 200 | 10.528935852s |     10.0.10.123 | POST     "/api/chat"
time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
time=2025-06-10T22:56:34.461Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:56:34.462Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:56:34.462Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:56:34.513Z level=DEBUG source=server.go:729 msg="completion request" images=7 prompt=7727 format=""
time=2025-06-10T22:56:34.988Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1599 prompt=1619 used=1414 remaining=205
[GIN] 2025/06/10 - 22:56:39 | 200 |     275.564µs |     10.0.10.194 | GET      "/api/tags"
time=2025-06-10T22:59:32.871Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:59:32.872Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:59:32.872Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:59:32.872Z level=DEBUG source=server.go:729 msg="completion request" images=1 prompt=152 format=""
time=2025-06-10T22:59:32.957Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=2427 prompt=21 used=4 remaining=17
[GIN] 2025/06/10 - 22:59:35 | 200 |  2.686124122s |     10.0.10.232 | POST     "/api/chat"
time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
time=2025-06-10T22:59:35.581Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:59:35.582Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:59:35.582Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:59:35.582Z level=DEBUG source=server.go:729 msg="completion request" images=0 prompt=1470 format=""
time=2025-06-10T22:59:35.583Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=37 prompt=318 used=4 remaining=314

OS

Docker

GPU

Nvidia

CPU

Intel

Ollama version

0.9.0

Originally created by @LaeraFelipe on GitHub (Jun 10, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11041 ### What is the issue? I'm running Ollama with the llama3.2-vision:latest model on a Kubernetes cluster using an NVIDIA A10 GPU. Alongside this model, I'm also running others such as gemma3:4b, llama3.2:3b, etc.—all of which work normally. However, when I run prompts with llama3.2-vision, it initially appears to work fine but eventually enters an infinite loop, as shown in the image below: ![Image](https://github.com/user-attachments/assets/13964dd4-cd5c-4b28-a485-8d791932a4b8) ![Image](https://github.com/user-attachments/assets/05795e7b-6c0a-423d-b944-5482f4724a5f) In the following image, you can see that Ollama is the only process using the GPU, and during the loop, it fully utilizes the GPU's processing power. Note that this container has no CPU, memory, or GPU limits defined. ![Image](https://github.com/user-attachments/assets/cda21240-61fb-4bd3-a023-6cedb320eab2) The processing only stops when the prompt is manually canceled. After this initial loop scenario, every subsequent attempt to send an image and ask a question results in the same behavior: ![Image](https://github.com/user-attachments/assets/4da0e37f-dfdc-4ab1-a2e8-60a3e425e5e1) After approximately 5 minutes, the model is unloaded from memory. If I then submit a new prompt, the loop scenario occurs again. Here are the details of the GPU driver I'm using: ![Image](https://github.com/user-attachments/assets/7eb7c0cd-6746-4b71-8da5-10e203892144) I also have another server with the same configuration, running Ollama version 0.6.7 via Docker Compose, using a llama3.2-vision:latest model downloaded around six months ago—and that setup works perfectly. Here is the nvidia-smi output from that machine: ![Image](https://github.com/user-attachments/assets/5d88b1f2-fae9-4a07-b9b6-6586ec8c4ca6) ### Relevant log output ```shell [GIN] 2025/06/10 - 22:54:54 | 200 | 7.45266341s | 10.0.10.123 | POST "/api/chat" time=2025-06-10T22:54:54.256Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:54:54.256Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:54:54.257Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 [GIN] 2025/06/10 - 22:55:09 | 200 | 474.068µs | 10.0.10.194 | GET "/api/tags" time=2025-06-10T22:55:09.589Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:55:09.590Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:55:09.590Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:55:09.612Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=3769 format="" time=2025-06-10T22:55:09.825Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=711 prompt=737 used=711 remaining=26 [GIN] 2025/06/10 - 22:55:16 | 200 | 7.456949066s | 10.0.10.194 | POST "/api/chat" time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:55:16.884Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 time=2025-06-10T22:56:12.674Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:56:12.675Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:56:12.675Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:56:12.710Z level=DEBUG source=server.go:729 msg="completion request" images=6 prompt=6397 format="" time=2025-06-10T22:56:13.052Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1265 prompt=1286 used=1044 remaining=242 [GIN] 2025/06/10 - 22:56:22 | 200 | 10.528935852s | 10.0.10.123 | POST "/api/chat" time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:56:22.992Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 time=2025-06-10T22:56:34.461Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:56:34.462Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:56:34.462Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:56:34.513Z level=DEBUG source=server.go:729 msg="completion request" images=7 prompt=7727 format="" time=2025-06-10T22:56:34.988Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1599 prompt=1619 used=1414 remaining=205 [GIN] 2025/06/10 - 22:56:39 | 200 | 275.564µs | 10.0.10.194 | GET "/api/tags" time=2025-06-10T22:59:32.871Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:59:32.872Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:59:32.872Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:59:32.872Z level=DEBUG source=server.go:729 msg="completion request" images=1 prompt=152 format="" time=2025-06-10T22:59:32.957Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=2427 prompt=21 used=4 remaining=17 [GIN] 2025/06/10 - 22:59:35 | 200 | 2.686124122s | 10.0.10.232 | POST "/api/chat" time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:59:35.477Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 time=2025-06-10T22:59:35.581Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:59:35.582Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:59:35.582Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:59:35.582Z level=DEBUG source=server.go:729 msg="completion request" images=0 prompt=1470 format="" time=2025-06-10T22:59:35.583Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=37 prompt=318 used=4 remaining=314 ``` ### OS Docker ### GPU Nvidia ### CPU Intel ### Ollama version 0.9.0
GiteaMirror added the bug label 2026-04-12 19:20:00 -05:00
GiteaMirror commented 2026-04-12 19:20:01 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 10, 2025):

Do you have logs from around the time the request was manually cancelled?

<!-- gh-comment-id:2960801251 --> @rick-github commented on GitHub (Jun 10, 2025): Do you have logs from around the time the request was manually cancelled?
GiteaMirror commented 2026-04-12 19:20:03 -05:00
Author
Owner
Copy Link

@LaeraFelipe commented on GitHub (Jun 10, 2025):

Do you have logs from around the time the request was manually cancelled?

Yes! These are the latest logs:

time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046
time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
[GIN] 2025/06/10 - 22:57:45 | 200 |         1m11s |     10.0.10.123 | POST     "/api/chat"
time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
[GIN] 2025/06/10 - 22:58:09 | 200 |     517.229µs |     10.0.10.194 | GET      "/api/tags"
[GIN] 2025/06/10 - 22:58:39 | 200 |     948.107µs |     10.0.10.194 | GET      "/api/tags"
[GIN] 2025/06/10 - 22:59:09 | 200 |     840.522µs |     10.0.10.194 | GET      "/api/tags"
[GIN] 2025/06/10 - 22:59:10 | 200 |      41.432µs |      10.0.10.46 | GET      "/api/version"
[GIN] 2025/06/10 - 22:59:12 | 200 |      22.218µs |     10.0.10.123 | GET      "/api/version"
[GIN] 2025/06/10 - 22:59:18 | 200 |      22.566µs |      10.0.10.46 | GET      "/api/version"
time=2025-06-10T22:59:32.871Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:59:32.872Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:59:32.872Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:59:32.872Z level=DEBUG source=server.go:729 msg="completion request" images=1 prompt=152 format=""
time=2025-06-10T22:59:32.957Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=2427 prompt=21 used=4 remaining=17
time=2025-06-10T22:59:35.583Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=37 prompt=318 used=4 remaining=314
[GIN] 2025/06/10 - 22:59:35 | 200 |  315.450811ms |     10.0.10.194 | POST     "/api/chat"
time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
time=2025-06-10T22:59:35.969Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
time=2025-06-10T22:59:35.970Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
time=2025-06-10T22:59:35.970Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
time=2025-06-10T22:59:35.970Z level=DEBUG source=server.go:729 msg="completion request" images=0 prompt=930 format=""
time=2025-06-10T22:59:35.971Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=330 prompt=203 used=8 remaining=195
[GIN] 2025/06/10 - 22:59:36 | 200 |  263.572345ms |     10.0.10.123 | POST     "/api/chat"
time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
[GIN] 2025/06/10 - 22:59:39 | 200 |     297.219µs |     10.0.10.194 | GET      "/api/tags"
<!-- gh-comment-id:2960809780 --> @LaeraFelipe commented on GitHub (Jun 10, 2025): > Do you have logs from around the time the request was manually cancelled? Yes! These are the latest logs: ``` time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046 time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 [GIN] 2025/06/10 - 22:57:45 | 200 | 1m11s | 10.0.10.123 | POST "/api/chat" time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:57:45.896Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 [GIN] 2025/06/10 - 22:58:09 | 200 | 517.229µs | 10.0.10.194 | GET "/api/tags" [GIN] 2025/06/10 - 22:58:39 | 200 | 948.107µs | 10.0.10.194 | GET "/api/tags" [GIN] 2025/06/10 - 22:59:09 | 200 | 840.522µs | 10.0.10.194 | GET "/api/tags" [GIN] 2025/06/10 - 22:59:10 | 200 | 41.432µs | 10.0.10.46 | GET "/api/version" [GIN] 2025/06/10 - 22:59:12 | 200 | 22.218µs | 10.0.10.123 | GET "/api/version" [GIN] 2025/06/10 - 22:59:18 | 200 | 22.566µs | 10.0.10.46 | GET "/api/version" time=2025-06-10T22:59:32.871Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:59:32.872Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:59:32.872Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:59:32.872Z level=DEBUG source=server.go:729 msg="completion request" images=1 prompt=152 format="" time=2025-06-10T22:59:32.957Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=2427 prompt=21 used=4 remaining=17 time=2025-06-10T22:59:35.583Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=37 prompt=318 used=4 remaining=314 [GIN] 2025/06/10 - 22:59:35 | 200 | 315.450811ms | 10.0.10.194 | POST "/api/chat" time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:59:35.875Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 time=2025-06-10T22:59:35.969Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 time=2025-06-10T22:59:35.970Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" time=2025-06-10T22:59:35.970Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed time=2025-06-10T22:59:35.970Z level=DEBUG source=server.go:729 msg="completion request" images=0 prompt=930 format="" time=2025-06-10T22:59:35.971Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=330 prompt=203 used=8 remaining=195 [GIN] 2025/06/10 - 22:59:36 | 200 | 263.572345ms | 10.0.10.123 | POST "/api/chat" time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s time=2025-06-10T22:59:36.212Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=1051612 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 [GIN] 2025/06/10 - 22:59:39 | 200 | 297.219µs | 10.0.10.194 | GET "/api/tags" ```
GiteaMirror commented 2026-04-12 19:20:06 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 10, 2025):

time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046

Your context window is too small. The runner fills up the available context space and then has to shift the context buffer to make room for new tokens. This results in the head of the buffer being lost which can cause the model to lose coherence, after which it can get in a state of emitting random tokens until interrupted or the internal repeated token limit triggers and stops the generation.

In the ideal world a buffer shift shouldn't cause this loss of coherence, so this bears looking into. A similar problem was recently discussed in the discord. In the meantime, increasing the size of the context buffer or setting num_predict to limit the number of tokens generated will mitigate the issue.

<!-- gh-comment-id:2960823727 --> @rick-github commented on GitHub (Jun 10, 2025): ``` time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046 ``` Your context window is too small. The runner fills up the available context space and then has to shift the context buffer to make room for new tokens. This results in the head of the buffer being lost which can cause the model to lose coherence, after which it can get in a state of emitting random tokens until interrupted or the internal repeated token limit triggers and stops the generation. In the ideal world a buffer shift shouldn't cause this loss of coherence, so this bears looking into. A similar problem was recently discussed in the [discord](https://discord.com/channels/1128867683291627614/1381942795144527993). In the meantime, increasing the size of the context buffer or setting `num_predict` to limit the number of tokens generated will mitigate the issue.
GiteaMirror commented 2026-04-12 19:20:08 -05:00
Author
Owner
Copy Link

@LaeraFelipe commented on GitHub (Jun 11, 2025):

time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046

Your context window is too small. The runner fills up the available context space and then has to shift the context buffer to make room for new tokens. This results in the head of the buffer being lost which can cause the model to lose coherence, after which it can get in a state of emitting random tokens until interrupted or the internal repeated token limit triggers and stops the generation.

In the ideal world a buffer shift shouldn't cause this loss of coherence, so this bears looking into. A similar problem was recently discussed in the discord. In the meantime, increasing the size of the context buffer or setting num_predict to limit the number of tokens generated will mitigate the issue.

Thanks @rick-github!
Is there a way to set the num_predict variable through an environment variable, besides setting it via the API?
From what I understand, the default value is -1, which means infinite — is that correct?

And for the context size, the corresponding environment variable would be OLLAMA_CONTEXT_LENGTH, right?

<!-- gh-comment-id:2960840630 --> @LaeraFelipe commented on GitHub (Jun 11, 2025): > ``` > time=2025-06-10T22:57:39.869Z level=DEBUG source=cache.go:272 msg="context limit hit - shifting" id=0 limit=4096 input=4096 keep=4 discard=2046 > ``` > > Your context window is too small. The runner fills up the available context space and then has to shift the context buffer to make room for new tokens. This results in the head of the buffer being lost which can cause the model to lose coherence, after which it can get in a state of emitting random tokens until interrupted or the internal repeated token limit triggers and stops the generation. > > In the ideal world a buffer shift shouldn't cause this loss of coherence, so this bears looking into. A similar problem was recently discussed in the [discord](https://discord.com/channels/1128867683291627614/1381942795144527993). In the meantime, increasing the size of the context buffer or setting `num_predict` to limit the number of tokens generated will mitigate the issue. Thanks @rick-github! Is there a way to set the num_predict variable through an environment variable, besides setting it via the API? From what I understand, the default value is -1, which means infinite — is that correct? And for the context size, the corresponding environment variable would be OLLAMA_CONTEXT_LENGTH, right?
GiteaMirror commented 2026-04-12 19:20:09 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 11, 2025):

num_predict can be set in the API or in a Modelfile. Unfortunately there's no global environment variable that can be set. The default value of -1 is infinite. There is an open PR to allow setting it to -2 to limit it to the number of free token positions in the buffer.

Context size is settable via OLLAMA_CONTEXT_LENGTH or again via API or Modelfile. Since it's global, it will affect all loaded models.

<!-- gh-comment-id:2960849188 --> @rick-github commented on GitHub (Jun 11, 2025): `num_predict` can be set in the API or in a [Modelfile](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter). Unfortunately there's no global environment variable that can be set. The default value of -1 is infinite. There is an open [PR](https://github.com/ollama/ollama/pull/9547) to allow setting it to -2 to limit it to the number of free token positions in the buffer. Context size is settable via `OLLAMA_CONTEXT_LENGTH` or again via API or Modelfile. Since it's global, it will affect all loaded models.
GiteaMirror commented 2026-04-12 19:20:09 -05:00
Author
Owner
Copy Link

@LaeraFelipe commented on GitHub (Jun 11, 2025):

@rick-github ,I believe the last log you saw was already from the other chat I opened to show what it looked like afterward. I just ran a new simulation on the server, but I couldn’t find the log you mentioned.

I also managed to reproduce the scenario on my laptop using Docker Compose with an RTX 4060, and I didn’t see that log there either.

Here are the logs:

Image

The request I had to stop because it entered a loop is the last one, which ran for 5 minutes.

I also tried setting num_predict to -2 in the API requests, but it didn’t work.

<!-- gh-comment-id:2961028023 --> @LaeraFelipe commented on GitHub (Jun 11, 2025): @rick-github ,I believe the last log you saw was already from the other chat I opened to show what it looked like afterward. I just ran a new simulation on the server, but I couldn’t find the log you mentioned. I also managed to reproduce the scenario on my laptop using Docker Compose with an RTX 4060, and I didn’t see that log there either. Here are the logs: ![Image](https://github.com/user-attachments/assets/0b396b27-cc4f-4058-a01b-1628902ec1f1) The request I had to stop because it entered a loop is the last one, which ran for 5 minutes. I also tried setting num_predict to -2 in the API requests, but it didn’t work.
GiteaMirror commented 2026-04-12 19:20:10 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 11, 2025):

Set OLLAMA_DEBUG=1.

<!-- gh-comment-id:2961800263 --> @rick-github commented on GitHub (Jun 11, 2025): Set `OLLAMA_DEBUG=1`.
GiteaMirror commented 2026-04-12 19:20:10 -05:00
Author
Owner
Copy Link

@LaeraFelipe commented on GitHub (Jun 11, 2025):

@rick-github, here are the logs with OLLAMA_DEBUG=1 enabled, running via Docker Compose on an A10 GPU.

ollama  | time=2025-06-11T19:51:33.287Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
ollama  | time=2025-06-11T19:51:33.288Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
ollama  | time=2025-06-11T19:51:33.288Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
ollama  | time=2025-06-11T19:51:33.308Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=5646 format=""
ollama  | time=2025-06-11T19:51:33.722Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1177 prompt=1199 used=850 remaining=349
ollama  | [GIN] 2025/06/11 - 19:51:39 | 200 |  6.457159059s |  129.159.59.183 | POST     "/api/chat"
ollama  | time=2025-06-11T19:51:39.592Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
ollama  | time=2025-06-11T19:51:39.593Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
ollama  | time=2025-06-11T19:51:39.593Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
ollama  | time=2025-06-11T19:51:49.003Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
ollama  | time=2025-06-11T19:51:49.004Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
ollama  | time=2025-06-11T19:51:49.004Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
ollama  | time=2025-06-11T19:51:49.021Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=6505 format=""
ollama  | time=2025-06-11T19:51:49.448Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1342 prompt=1360 used=1342 remaining=18
ollama  | [GIN] 2025/06/11 - 19:51:52 | 200 |  3.674636933s |  129.159.59.183 | POST     "/api/chat"
ollama  | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
ollama  | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
ollama  | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0
ollama  | time=2025-06-11T19:52:02.492Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32
ollama  | time=2025-06-11T19:52:02.493Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests"
ollama  | time=2025-06-11T19:52:02.493Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed
ollama  | time=2025-06-11T19:52:02.519Z level=DEBUG source=server.go:729 msg="completion request" images=5 prompt=7257 format=""
ollama  | time=2025-06-11T19:52:03.054Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1483 prompt=1502 used=1483 remaining=19
ollama  | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096
ollama  | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s
ollama  | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0

Image

The logs above are from version 0.9.0 running the latest model.

Note: I cloned the model data from another server that was running Ollama version 0.6.7, along with a copy of the model downloaded six months ago, and deployed it to Kubernetes — it's now working correctly. It seems the issue is related to the latest versions of Ollama and the llama3.2-vision:latest model.

<!-- gh-comment-id:2964003834 --> @LaeraFelipe commented on GitHub (Jun 11, 2025): @rick-github, here are the logs with OLLAMA_DEBUG=1 enabled, running via Docker Compose on an A10 GPU. ``` ollama | time=2025-06-11T19:51:33.287Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 ollama | time=2025-06-11T19:51:33.288Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" ollama | time=2025-06-11T19:51:33.288Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed ollama | time=2025-06-11T19:51:33.308Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=5646 format="" ollama | time=2025-06-11T19:51:33.722Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1177 prompt=1199 used=850 remaining=349 ollama | [GIN] 2025/06/11 - 19:51:39 | 200 | 6.457159059s | 129.159.59.183 | POST "/api/chat" ollama | time=2025-06-11T19:51:39.592Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 ollama | time=2025-06-11T19:51:39.593Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s ollama | time=2025-06-11T19:51:39.593Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 ollama | time=2025-06-11T19:51:49.003Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 ollama | time=2025-06-11T19:51:49.004Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" ollama | time=2025-06-11T19:51:49.004Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed ollama | time=2025-06-11T19:51:49.021Z level=DEBUG source=server.go:729 msg="completion request" images=4 prompt=6505 format="" ollama | time=2025-06-11T19:51:49.448Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1342 prompt=1360 used=1342 remaining=18 ollama | [GIN] 2025/06/11 - 19:51:52 | 200 | 3.674636933s | 129.159.59.183 | POST "/api/chat" ollama | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 ollama | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s ollama | time=2025-06-11T19:51:52.464Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 ollama | time=2025-06-11T19:52:02.492Z level=DEBUG source=ggml.go:155 msg="key not found" key=general.alignment default=32 ollama | time=2025-06-11T19:52:02.493Z level=WARN source=sched.go:140 msg="mllama does not currently support parallel requests" ollama | time=2025-06-11T19:52:02.493Z level=DEBUG source=sched.go:615 msg="evaluating already loaded" model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed ollama | time=2025-06-11T19:52:02.519Z level=DEBUG source=server.go:729 msg="completion request" images=5 prompt=7257 format="" ollama | time=2025-06-11T19:52:03.054Z level=DEBUG source=cache.go:136 msg="loading cache slot" id=0 cache=1483 prompt=1502 used=1483 remaining=19 ollama | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:434 msg="context for request finished" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 ollama | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:343 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 duration=5m0s ollama | time=2025-06-11T19:52:32.239Z level=DEBUG source=sched.go:361 msg="after processing request finished event" runner.name=registry.ollama.ai/library/llama3.2-vision:latest runner.inference=cuda runner.devices=1 runner.size="11.2 GiB" runner.vram="11.2 GiB" runner.parallel=1 runner.pid=31 runner.model=/root/.ollama/models/blobs/sha256-9999d473417a8e179d993498195be5f42cab963acc75f4a6b15d981e8b68abed runner.num_ctx=4096 refCount=0 ``` ![Image](https://github.com/user-attachments/assets/651ed225-6905-4153-b41f-34a4535771dc) The logs above are from version 0.9.0 running the latest model. Note: I cloned the model data from another server that was running Ollama version 0.6.7, along with a copy of the model downloaded six months ago, and deployed it to Kubernetes — it's now working correctly. It seems the issue is related to the latest versions of Ollama and the llama3.2-vision:latest model.
GiteaMirror commented 2026-04-12 19:20:10 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 12, 2025):

This logs shows two apparently successful generations. It's more helpful if you post the full log rather than a handful of lines.

<!-- gh-comment-id:2966110622 --> @rick-github commented on GitHub (Jun 12, 2025): This logs shows two apparently successful generations. It's more helpful if you post the full log rather than a handful of lines.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7283
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?