github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #10801] Model qwen2.5vl:7b dosen't work with GPU #7093

New Issue
Closed
opened 2026-04-12 19:02:27 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 19:02:27 -05:00
Owner
Copy Link

Originally created by @tianfan007 on GitHub (May 21, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10801

What is the issue?

All other models are working normally, except qwen2.5vl:7b

For Example,run these command
ollama run llama3.1:lastest
and then ,run command
ollama ps
there will be show:

NAME ID SIZE PROCESSOR UNTIL
llama3.1:latest 91ab477bec9d 6.5 GB 47%/53% CPU/GPU 4 minutes from now

But if I use qwen2.5vl:7b
ollama run qwen2.5vl:7b ｀ and then ,run the command
ollama ps
`
there will be show :
NAME ID SIZE PROCESSOR UNTIL
qwen2.5vl:7b 5ced39dfa4ba 4.6 GB 100% CPU 4 minutes from now

It dosen't work with GPU.

Relevant log output

5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.567+08:00 level=INFO source=server.go:135 msg="system memory" total="14.8 GiB" free="9.7 GiB" free_swap="15.1 GiB"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB" projector.weights="1.2 GiB" projector.graph="1.6 GiB"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-a99b7f834d754b88f122d865f32758ba9f0994a83f8363df2c1e71c17605a025 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 36785"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=sched.go:472 msg="loaded runners" count=1
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.609+08:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.623+08:00 level=INFO source=runner.go:836 msg="starting ollama engine"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.624+08:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:36785"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.662+08:00 level=INFO source=ggml.go:73 msg="" architecture=qwen25vl file_type=Q4_K_M name="" description="" num_tensors=858 num_key_values=36
5月 21 22:19:26 Victus8 ollama[2984]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.673+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc)
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.678+08:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="5.6 GiB"
5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.860+08:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model"
5月 21 22:19:30 Victus8 ollama[2984]: time=2025-05-21T22:19:30.601+08:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="238.0 MiB"
5月 21 22:19:30 Victus8 ollama[2984]: time=2025-05-21T22:19:30.627+08:00 level=INFO source=server.go:630 msg="llama runner started in 4.02 seconds"
5月 21 22:19:30 Victus8 ollama[2984]: [GIN] 2025/05/21 - 22:19:30 | 200 |   4.34508053s |       127.0.0.1 | POST     "/api/generate"

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.7.0

Originally created by @tianfan007 on GitHub (May 21, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10801 ### What is the issue? All other models are working normally, except qwen2.5vl:7b For Example,run these command ` ollama run llama3.1:lastest ` and then ,run command ` ollama ps ` there will be show: NAME ID SIZE PROCESSOR UNTIL llama3.1:latest 91ab477bec9d 6.5 GB 47%/53% CPU/GPU 4 minutes from now But if I use qwen2.5vl:7b ` ollama run qwen2.5vl:7b ｀ and then ,run the command ` ollama ps ` there will be show : NAME ID SIZE PROCESSOR UNTIL qwen2.5vl:7b 5ced39dfa4ba 4.6 GB 100% CPU 4 minutes from now It dosen't work with GPU. ### Relevant log output ```shell 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.567+08:00 level=INFO source=server.go:135 msg="system memory" total="14.8 GiB" free="9.7 GiB" free_swap="15.1 GiB" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB" projector.weights="1.2 GiB" projector.graph="1.6 GiB" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-a99b7f834d754b88f122d865f32758ba9f0994a83f8363df2c1e71c17605a025 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 36785" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=sched.go:472 msg="loaded runners" count=1 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.608+08:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.609+08:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.623+08:00 level=INFO source=runner.go:836 msg="starting ollama engine" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.624+08:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:36785" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.662+08:00 level=INFO source=ggml.go:73 msg="" architecture=qwen25vl file_type=Q4_K_M name="" description="" num_tensors=858 num_key_values=36 5月 21 22:19:26 Victus8 ollama[2984]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.673+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc) 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.678+08:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="5.6 GiB" 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.860+08:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model" 5月 21 22:19:30 Victus8 ollama[2984]: time=2025-05-21T22:19:30.601+08:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="238.0 MiB" 5月 21 22:19:30 Victus8 ollama[2984]: time=2025-05-21T22:19:30.627+08:00 level=INFO source=server.go:630 msg="llama runner started in 4.02 seconds" 5月 21 22:19:30 Victus8 ollama[2984]: [GIN] 2025/05/21 - 22:19:30 | 200 | 4.34508053s | 127.0.0.1 | POST "/api/generate" ``` ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.7.0
GiteaMirror added the bug label 2026-04-12 19:02:27 -05:00
GiteaMirror commented 2026-04-12 19:02:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 21, 2025):

5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload
 library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]"
 memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB"
 memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB"
 memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB"
 projector.weights="1.2 GiB" projector.graph="1.6 GiB"

You have 3.3G available on the GPU, it is not enough for 0.7.0 to load any layers, see here for more details. 0.7.1 might work better.

<!-- gh-comment-id:2898228401 --> @rick-github commented on GitHub (May 21, 2025): ``` 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB" projector.weights="1.2 GiB" projector.graph="1.6 GiB" ``` You have 3.3G available on the GPU, it is not enough for 0.7.0 to load any layers, see [here](https://github.com/ollama/ollama/issues/10765#issuecomment-2890759340) for more details. 0.7.1 might work better.
GiteaMirror commented 2026-04-12 19:02:31 -05:00
Author
Owner
Copy Link

@tianfan007 commented on GitHub (May 21, 2025):

5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload
 library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]"
 memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB"
 memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB"
 memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB"
 projector.weights="1.2 GiB" projector.graph="1.6 GiB"

You have 3.3G available on the GPU, it is not enough for 0.7.0 to load any layers, see here for more details. 0.7.1 might work better.

But why can other models allocate CPUs and GPUs in a certain proportion? Is the qwen2.5vl model special for OLAMA?

<!-- gh-comment-id:2898484144 --> @tianfan007 commented on GitHub (May 21, 2025): > ``` > 5月 21 22:19:26 Victus8 ollama[2984]: time=2025-05-21T22:19:26.569+08:00 level=INFO source=server.go:168 msg=offload > library=cuda layers.requested=-1 layers.model=29 layers.offload=0 layers.split="" memory.available="[3.3 GiB]" > memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="0 B" memory.required.kv="224.0 MiB" > memory.required.allocations="[0 B]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" > memory.weights.nonrepeating="426.4 MiB" memory.graph.full="261.3 MiB" memory.graph.partial="261.3 MiB" > projector.weights="1.2 GiB" projector.graph="1.6 GiB" > ``` > > You have 3.3G available on the GPU, it is not enough for 0.7.0 to load any layers, see [here](https://github.com/ollama/ollama/issues/10765#issuecomment-2890759340) for more details. 0.7.1 might work better. But why can other models allocate CPUs and GPUs in a certain proportion? Is the qwen2.5vl model special for OLAMA?
GiteaMirror commented 2026-04-12 19:02:32 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 21, 2025):

Other models have different sizes. qwen2.5vl has memory requirements that don't fit in 3.3G. The linked issue points to a PR that may help the situation in 0.7.1.

<!-- gh-comment-id:2898495528 --> @rick-github commented on GitHub (May 21, 2025): Other models have different sizes. qwen2.5vl has memory requirements that don't fit in 3.3G. The linked issue points to a PR that may help the situation in 0.7.1.
GiteaMirror commented 2026-04-12 19:02:33 -05:00
Author
Owner
Copy Link

@tianfan007 commented on GitHub (May 30, 2025):

Ｉthink there is more than 3.3G memory

Image

<!-- gh-comment-id:2921141626 --> @tianfan007 commented on GitHub (May 30, 2025): Ｉthink there is more than 3.3G memory ![Image](https://github.com/user-attachments/assets/81ef15bd-d5fe-452d-b2f0-a6a1524a81b1)
GiteaMirror commented 2026-04-12 19:02:33 -05:00
Author
Owner
Copy Link

@ArchBase commented on GitHub (Feb 25, 2026):

Same issue. wanted to run a this model but its using only cpu. the problem is, when using 100% cpu it takes a lot time as the context we give in prompt increases. like adding an image increases the time it takes like 20x ig.

<!-- gh-comment-id:3960767602 --> @ArchBase commented on GitHub (Feb 25, 2026): Same issue. wanted to run a this model but its using only cpu. the problem is, when using 100% cpu it takes a lot time as the context we give in prompt increases. like adding an image increases the time it takes like 20x ig.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7093
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?