github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #10765] Ollama does not use nvidia gpu #32832

New Issue
Closed
opened 2026-04-22 14:41:19 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-22 14:41:19 -05:00
Owner
Copy Link

Originally created by @vladlen32230 on GitHub (May 18, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10765

What is the issue?

Ollama does not use gtx 1650 for gemma3:4b q4 model in docker and in host too.

Relevant log output

(base) vladlen32230@vladlen32230:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0


(base) vladlen32230@vladlen32230:~$ nvidia-smi
Sun May 18 16:42:35 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.144.03             Driver Version: 550.144.03     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1650        Off |   00000000:01:00.0  On |                  N/A |
| 26%   31C    P8              8W /   75W |     601MiB /   4096MiB |      6%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      2525      G   /usr/lib/xorg/Xorg                            192MiB |
|    0   N/A  N/A      2766      G   /usr/bin/gnome-shell                           34MiB |
|    0   N/A  N/A      3566      G   ...irefox/6159/usr/lib/firefox/firefox        156MiB |
|    0   N/A  N/A      4954      G   ...erProcess --variations-seed-version         81MiB |
|    0   N/A  N/A      6062      G   ...ures=SpareRendererForSitePerProcess         18MiB |
|    0   N/A  N/A      7087      G   ...832749ca37a9f3191bdb1cfdd91efa4d23d         65MiB |
|    0   N/A  N/A      7767      G   /usr/pgadmin4/bin/pgadmin4                     45MiB |
+-----------------------------------------------------------------------------------------+


(base) vladlen32230@vladlen32230:~$ ollama --version
ollama version is 0.7.0


(base) vladlen32230@vladlen32230:~$ ollama ps
NAME         ID              SIZE      PROCESSOR    UNTIL              
gemma3:4b    a2af6cc3eb7f    2.7 GB    100% CPU     4 minutes from now


(base) vladlen32230@vladlen32230:~$ journalctl -u ollama --no-pager --follow --pager-end 
May 19 08:08:41 vladlen32230 systemd[1]: Started ollama.service - Ollama Service.
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.457+05:00 level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:463 msg="total blobs: 5"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:470 msg="total unused blobs removed: 0"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.465+05:00 level=INFO source=routes.go:1258 msg="Listening on 127.0.0.1:11434 (version 0.7.0)"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.468+05:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.721+05:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c8d15b82-a97c-4d92-2c14-f92754c94792 library=cuda variant=v12 compute=7.5 driver=12.4 name="NVIDIA GeForce GTX 1650" total="3.8 GiB" available="3.7 GiB"
May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 |    2.063566ms |       127.0.0.1 | HEAD     "/"
May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 |    1.515224ms |       127.0.0.1 | GET      "/api/tags"
May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 |       27.17µs |       127.0.0.1 | HEAD     "/"
May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 |  232.463123ms |       127.0.0.1 | POST     "/api/show"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.623+05:00 level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="24.8 GiB" free_swap="4.0 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB" memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.706+05:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 43155"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=sched.go:472 msg="loaded runners" count=1
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:836 msg="starting ollama engine"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:43155"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.791+05:00 level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36
May 19 10:50:36 vladlen32230 ollama[1995]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.817+05:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc)
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.821+05:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="3.6 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.965+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model"
May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.626+05:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="73.0 MiB"
May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.759+05:00 level=INFO source=server.go:630 msg="llama runner started in 10.05 seconds"
May 19 10:50:46 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:46 | 200 | 10.407066165s |       127.0.0.1 | POST     "/api/generate"
May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 |       46.42µs |       127.0.0.1 | HEAD     "/"
May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 |     125.253µs |       127.0.0.1 | GET      "/api/ps"
May 19 10:55:51 vladlen32230 ollama[1995]: time=2025-05-19T10:55:51.876+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.114636874 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25
May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.126+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.365031151 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25
May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.377+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.615289448 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.7.0

Originally created by @vladlen32230 on GitHub (May 18, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10765 ### What is the issue? Ollama does not use gtx 1650 for gemma3:4b q4 model in docker and in host too. ### Relevant log output ```shell (base) vladlen32230@vladlen32230:~$ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Fri_Jan__6_16:45:21_PST_2023 Cuda compilation tools, release 12.0, V12.0.140 Build cuda_12.0.r12.0/compiler.32267302_0 (base) vladlen32230@vladlen32230:~$ nvidia-smi Sun May 18 16:42:35 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce GTX 1650 Off | 00000000:01:00.0 On | N/A | | 26% 31C P8 8W / 75W | 601MiB / 4096MiB | 6% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 2525 G /usr/lib/xorg/Xorg 192MiB | | 0 N/A N/A 2766 G /usr/bin/gnome-shell 34MiB | | 0 N/A N/A 3566 G ...irefox/6159/usr/lib/firefox/firefox 156MiB | | 0 N/A N/A 4954 G ...erProcess --variations-seed-version 81MiB | | 0 N/A N/A 6062 G ...ures=SpareRendererForSitePerProcess 18MiB | | 0 N/A N/A 7087 G ...832749ca37a9f3191bdb1cfdd91efa4d23d 65MiB | | 0 N/A N/A 7767 G /usr/pgadmin4/bin/pgadmin4 45MiB | +-----------------------------------------------------------------------------------------+ (base) vladlen32230@vladlen32230:~$ ollama --version ollama version is 0.7.0 (base) vladlen32230@vladlen32230:~$ ollama ps NAME ID SIZE PROCESSOR UNTIL gemma3:4b a2af6cc3eb7f 2.7 GB 100% CPU 4 minutes from now (base) vladlen32230@vladlen32230:~$ journalctl -u ollama --no-pager --follow --pager-end May 19 08:08:41 vladlen32230 systemd[1]: Started ollama.service - Ollama Service. May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.457+05:00 level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:463 msg="total blobs: 5" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:470 msg="total unused blobs removed: 0" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.465+05:00 level=INFO source=routes.go:1258 msg="Listening on 127.0.0.1:11434 (version 0.7.0)" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.468+05:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.721+05:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c8d15b82-a97c-4d92-2c14-f92754c94792 library=cuda variant=v12 compute=7.5 driver=12.4 name="NVIDIA GeForce GTX 1650" total="3.8 GiB" available="3.7 GiB" May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 | 2.063566ms | 127.0.0.1 | HEAD "/" May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 | 1.515224ms | 127.0.0.1 | GET "/api/tags" May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 | 27.17µs | 127.0.0.1 | HEAD "/" May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 | 232.463123ms | 127.0.0.1 | POST "/api/show" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.623+05:00 level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="24.8 GiB" free_swap="4.0 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB" memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.706+05:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 43155" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=sched.go:472 msg="loaded runners" count=1 May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:836 msg="starting ollama engine" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:43155" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.791+05:00 level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36 May 19 10:50:36 vladlen32230 ollama[1995]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.817+05:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc) May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.821+05:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="3.6 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.965+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model" May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.626+05:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="73.0 MiB" May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.759+05:00 level=INFO source=server.go:630 msg="llama runner started in 10.05 seconds" May 19 10:50:46 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:46 | 200 | 10.407066165s | 127.0.0.1 | POST "/api/generate" May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 | 46.42µs | 127.0.0.1 | HEAD "/" May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 | 125.253µs | 127.0.0.1 | GET "/api/ps" May 19 10:55:51 vladlen32230 ollama[1995]: time=2025-05-19T10:55:51.876+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.114636874 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.126+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.365031151 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.377+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.615289448 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.7.0
GiteaMirror added the performancememorybug labels 2026-04-22 14:41:19 -05:00
GiteaMirror commented 2026-04-22 14:41:22 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 18, 2025):

Server logs will aid in debugging.

<!-- gh-comment-id:2888949356 --> @rick-github commented on GitHub (May 18, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging.
GiteaMirror commented 2026-04-22 14:41:22 -05:00
Author
Owner
Copy Link

@zhangddjs commented on GitHub (May 18, 2025):

I also have this problem while running deepseek r1 7b model by ollama run deepseek-r1

<!-- gh-comment-id:2888983696 --> @zhangddjs commented on GitHub (May 18, 2025): I also have this problem while running deepseek r1 7b model by `ollama run deepseek-r1`
GiteaMirror commented 2026-04-22 14:41:23 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 18, 2025):

Server logs will aid in debugging.

<!-- gh-comment-id:2889009665 --> @rick-github commented on GitHub (May 18, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging.
GiteaMirror commented 2026-04-22 14:41:24 -05:00
Author
Owner
Copy Link

@abes200 commented on GitHub (May 18, 2025):

I am also experiencing a problem similar to this. Although it will still load into GPU memory, but will not utilize the GPU at all and take a long time to respond.

But I will not upload a server.log to a public forum while it still contains identifying information and personal details I don't want to share. Like the username for my PC.

<!-- gh-comment-id:2889226139 --> @abes200 commented on GitHub (May 18, 2025): I am also experiencing a problem similar to this. Although it will still load into GPU memory, but will not utilize the GPU at all and take a long time to respond. But I will not upload a server.log to a public forum while it still contains identifying information and personal details I don't want to share. Like the username for my PC.
GiteaMirror commented 2026-04-22 14:41:25 -05:00
Author
Owner
Copy Link

@UpscaleHistory commented on GitHub (May 19, 2025):

I also have this problem after the recent update. Sometimes ollama loads the models into vram, but runs on cpu instead of gpu.

<!-- gh-comment-id:2889431700 --> @UpscaleHistory commented on GitHub (May 19, 2025): I also have this problem after the recent update. Sometimes ollama loads the models into vram, but runs on cpu instead of gpu.
GiteaMirror commented 2026-04-22 14:41:26 -05:00
Author
Owner
Copy Link

@vladlen32230 commented on GitHub (May 19, 2025):

(base) vladlen32230@vladlen32230:~$ journalctl -u ollama --no-pager --follow --pager-end 
May 19 08:08:41 vladlen32230 systemd[1]: Started ollama.service - Ollama Service.
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.457+05:00 level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:463 msg="total blobs: 5"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:470 msg="total unused blobs removed: 0"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.465+05:00 level=INFO source=routes.go:1258 msg="Listening on 127.0.0.1:11434 (version 0.7.0)"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.468+05:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.721+05:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c8d15b82-a97c-4d92-2c14-f92754c94792 library=cuda variant=v12 compute=7.5 driver=12.4 name="NVIDIA GeForce GTX 1650" total="3.8 GiB" available="3.7 GiB"
May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 |    2.063566ms |       127.0.0.1 | HEAD     "/"
May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 |    1.515224ms |       127.0.0.1 | GET      "/api/tags"
May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 |       27.17µs |       127.0.0.1 | HEAD     "/"
May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 |  232.463123ms |       127.0.0.1 | POST     "/api/show"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.623+05:00 level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="24.8 GiB" free_swap="4.0 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB" memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.706+05:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 43155"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=sched.go:472 msg="loaded runners" count=1
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:836 msg="starting ollama engine"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:43155"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.791+05:00 level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36
May 19 10:50:36 vladlen32230 ollama[1995]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.817+05:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc)
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.821+05:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="3.6 GiB"
May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.965+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model"
May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.626+05:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="73.0 MiB"
May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.759+05:00 level=INFO source=server.go:630 msg="llama runner started in 10.05 seconds"
May 19 10:50:46 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:46 | 200 | 10.407066165s |       127.0.0.1 | POST     "/api/generate"
May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 |       46.42µs |       127.0.0.1 | HEAD     "/"
May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 |     125.253µs |       127.0.0.1 | GET      "/api/ps"
May 19 10:55:51 vladlen32230 ollama[1995]: time=2025-05-19T10:55:51.876+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.114636874 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25
May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.126+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.365031151 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25
May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.377+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.615289448 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25

Here is my server logs. It identifies GPU but does not use it.

<!-- gh-comment-id:2889715884 --> @vladlen32230 commented on GitHub (May 19, 2025): ```bash (base) vladlen32230@vladlen32230:~$ journalctl -u ollama --no-pager --follow --pager-end May 19 08:08:41 vladlen32230 systemd[1]: Started ollama.service - Ollama Service. May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.457+05:00 level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:463 msg="total blobs: 5" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.463+05:00 level=INFO source=images.go:470 msg="total unused blobs removed: 0" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.465+05:00 level=INFO source=routes.go:1258 msg="Listening on 127.0.0.1:11434 (version 0.7.0)" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.468+05:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs" May 19 08:08:41 vladlen32230 ollama[1995]: time=2025-05-19T08:08:41.721+05:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c8d15b82-a97c-4d92-2c14-f92754c94792 library=cuda variant=v12 compute=7.5 driver=12.4 name="NVIDIA GeForce GTX 1650" total="3.8 GiB" available="3.7 GiB" May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 | 2.063566ms | 127.0.0.1 | HEAD "/" May 19 10:50:31 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:31 | 200 | 1.515224ms | 127.0.0.1 | GET "/api/tags" May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 | 27.17µs | 127.0.0.1 | HEAD "/" May 19 10:50:36 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:36 | 200 | 232.463123ms | 127.0.0.1 | POST "/api/show" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.623+05:00 level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="24.8 GiB" free_swap="4.0 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB" memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.706+05:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 4096 --batch-size 512 --threads 6 --no-mmap --parallel 1 --port 43155" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=sched.go:472 msg="loaded runners" count=1 May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.707+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:836 msg="starting ollama engine" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.717+05:00 level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:43155" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.791+05:00 level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36 May 19 10:50:36 vladlen32230 ollama[1995]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.817+05:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc) May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.821+05:00 level=INFO source=ggml.go:299 msg="model weights" buffer=CPU size="3.6 GiB" May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.965+05:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server loading model" May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.626+05:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="73.0 MiB" May 19 10:50:46 vladlen32230 ollama[1995]: time=2025-05-19T10:50:46.759+05:00 level=INFO source=server.go:630 msg="llama runner started in 10.05 seconds" May 19 10:50:46 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:46 | 200 | 10.407066165s | 127.0.0.1 | POST "/api/generate" May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 | 46.42µs | 127.0.0.1 | HEAD "/" May 19 10:50:51 vladlen32230 ollama[1995]: [GIN] 2025/05/19 - 10:50:51 | 200 | 125.253µs | 127.0.0.1 | GET "/api/ps" May 19 10:55:51 vladlen32230 ollama[1995]: time=2025-05-19T10:55:51.876+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.114636874 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.126+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.365031151 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 May 19 10:55:52 vladlen32230 ollama[1995]: time=2025-05-19T10:55:52.377+05:00 level=WARN source=sched.go:676 msg="gpu VRAM usage didn't recover within timeout" seconds=5.615289448 runner.size="2.6 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=107200 runner.model=/usr/share/ollama/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 ``` Here is my server logs. It identifies GPU but does not use it.
GiteaMirror commented 2026-04-22 14:41:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 19, 2025):

May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168
 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split=""
 memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB"
 memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]"
 memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB"
 memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB"
 projector.graph="1.0 GiB"

There's a minimum amount of VRAM required for ollama to load layers into the GPU. The maths is currently :

FreeVRAM > projector.weights +
           projector.graph +
           max(memory.graph.partial, memory.graph.full) +
           VRAMbuffer +
           (2 * layerSize)

VRAMbuffer is a constant determined experimentally.

Plugging in the values from the logs, we get 1024 + 795.9 + 1024 + 457 + 1062 = 4362.9 = 4.26GiB, which exceeds the available free VRAM (3.2GiB) on the device.

There has been recent changes (eg) to the estimation logic to try to improve the results. The results have been poor for some users, so I expect further tuning. The estimation logic has been a pain point for some time and the ollama team is aware of it (#10726). In the short term you can try experimenting with flash attention, K/V quantization and reducing the size of the context window to reduce the VRAM footprint of the model.

Ultimately, I think the estimation logic needs to be re-worked. For example, the logic now uses the size of the largest layer (layerSize above) to estimate VRAM usage. In the case of gemma3:4b, this layer is token_embd, which is 10x the size of the average layer. Not only that, but that layer usually runs in CPU anyway, so should not factor in VRAM memory usage calculations.

<!-- gh-comment-id:2890759340 --> @rick-github commented on GitHub (May 19, 2025): ``` May 19 10:50:36 vladlen32230 ollama[1995]: time=2025-05-19T10:50:36.626+05:00 level=INFO source=server.go:168 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[3.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.6 GiB" memory.required.partial="0 B" memory.required.kv="254.0 MiB" memory.required.allocations="[0 B]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" ``` There's a minimum amount of VRAM required for ollama to load layers into the GPU. The maths is currently : ``` FreeVRAM > projector.weights + projector.graph + max(memory.graph.partial, memory.graph.full) + VRAMbuffer + (2 * layerSize) ``` VRAMbuffer is a constant determined [experimentally](https://github.com/ollama/ollama/commit/30a7d7096c1ca4e90e859efce23042a27487fa8c). Plugging in the values from the logs, we get 1024 + 795.9 + 1024 + 457 + 1062 = 4362.9 = 4.26GiB, which exceeds the available free VRAM (3.2GiB) on the device. There has been recent changes ([eg](https://github.com/ollama/ollama/commit/0478d440f0ba62202bc4b98043ae4a7d0b85e4ba)) to the estimation logic to try to improve the results. The results have been poor for some users, so I expect further tuning. The estimation logic has been a pain point for some time and the ollama team is aware of it (#10726). In the short term you can try experimenting with [flash attention](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-enable-flash-attention), [K/V quantization](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-set-the-quantization-type-for-the-kv-cache) and reducing the size of the [context window](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size) to reduce the VRAM footprint of the model. Ultimately, I think the estimation logic needs to be re-worked. For example, the logic now uses the size of the largest layer (layerSize above) to estimate VRAM usage. In the case of gemma3:4b, this layer is `token_embd`, which is 10x the size of the average layer. Not only that, but that layer usually runs in CPU anyway, so should not factor in VRAM memory usage calculations.
GiteaMirror commented 2026-04-22 14:41:29 -05:00
Author
Owner
Copy Link

@vladlen32230 commented on GitHub (May 19, 2025):

Anyways Ollama did CPU offloading if VRAM wasn't enough. Now there is no such feature?

<!-- gh-comment-id:2890879525 --> @vladlen32230 commented on GitHub (May 19, 2025): Anyways Ollama did CPU offloading if VRAM wasn't enough. Now there is no such feature?
GiteaMirror commented 2026-04-22 14:41:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 19, 2025):

There's a minimum amount of VRAM required for ollama to load layers into the GPU. The current version of ollama has increased the minimum. Your device is now too small.

<!-- gh-comment-id:2890896994 --> @rick-github commented on GitHub (May 19, 2025): There's a minimum amount of VRAM required for ollama to load layers into the GPU. The current version of ollama has increased the minimum. Your device is now too small.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug memory performance
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#32832
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?