github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3489] Ollama goes into uninterruptible sleep mode and cannot be shutdown #2149

New Issue
Closed
opened 2026-04-12 12:22:45 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-12 12:22:45 -05:00
Owner
Copy Link

Originally created by @sumitkm on GitHub (Apr 4, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3489

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

This is probably same as #2029 but I'll try and provide more details.

As mentioned in the above issue Ollama works fine from a clean boot. If I put computer to sleep and then wake it up, Ollama gets into a state where it can't be stopped via systemctl or killed via kill <pid>. Here is a screenshot from htop after I tried to stop it (after waking computer up from sleep).

image

Output of systemctl status ollama.service

sumitkm@PIOT-WORK-02:~$ sudo systemctl status ollama.service
× ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
     Active: failed (Result: timeout) since Thu 2024-04-04 13:34:55 BST; 4min 30s ago
   Duration: 30min 53.337s
   Main PID: 1675
      Tasks: 1 (limit: 114811)
     Memory: 339.5M
        CPU: 4.780s
     CGroup: /system.slice/ollama.service
             └─1675 "[ollama]"

Apr 04 13:30:24 PIOT-WORK-02 systemd[1]: ollama.service: State 'stop-sigterm' timed out. Killing.
Apr 04 13:30:24 PIOT-WORK-02 systemd[1]: ollama.service: Killing process 1675 (ollama) with signal SIGKILL.
Apr 04 13:31:54 PIOT-WORK-02 systemd[1]: ollama.service: Processes still around after SIGKILL. Ignoring.
Apr 04 13:33:24 PIOT-WORK-02 systemd[1]: ollama.service: State 'final-sigterm' timed out. Killing.
Apr 04 13:33:24 PIOT-WORK-02 systemd[1]: ollama.service: Killing process 1675 (ollama) with signal SIGKILL.
Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Processes still around after final SIGKILL. Entering failed mode.
Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Failed with result 'timeout'.
Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Unit process 1675 (ollama) remains running after unit stopped.
Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: Stopped ollama.service - Ollama Service.
Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Consumed 4.780s CPU time.

Even if I don't try to kill it, ollama prevents system from going to sleep completely. Reboot attempts to kill it thrice and then forces reboot anyway.

Workaround

If I do a sudo systemctl stop ollama.service before I put computer to sleep, everything works great.

journalctl -u ollama

logs-ollama.txt

Hardware

CPU: AMD Ryzen 7950x (AMD Eco mode 105W via bios settings)
RAM: 96 GB DDR 5
GPU: NVIDIA RTX 4090 (MSI Gaming X Slim card)

hostnamectl

Static hostname: PIOT-WORK-02
Icon name: computer-desktop
Chassis: desktop 🖥️
Machine ID: f3d60fb0a8c84b7aa9eb8e**********
Boot ID: 0b1a6283d6d04921a0f685**********
Operating System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.0-18-amd64
Architecture: x86-64
Hardware Vendor: ASUS
Hardware Model: ROG CROSSHAIR X670E HERO
Firmware Version: 1905

What did you expect to see?

Ollama should work correctly when the system wakes up from sleep, that includes going into sleep mode the second time and subsequent ocassions.

Steps to reproduce

Steps mentioned in description are reproducible reliably on my computer.

Are there any recent changes that introduced the issue?

This is a brand new system and apart from ollama and phronix test suite, there is barely anything else on the system.

OS

Linux

Architecture

amd64

Platform

No response

Ollama version

0.1.30

GPU

Nvidia

GPU info

nvidia-smi
Thu Apr  4 14:27:29 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:01:00.0  On |                  Off |
|  0%   45C    P8             17W /  450W |    1220MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      2042      G   /usr/lib/xorg/Xorg                            383MiB |
|    0   N/A  N/A      2224      G   /usr/bin/gnome-shell                          123MiB |
|    0   N/A  N/A      4123      G   ...sumitkm/AppImages/Telegram/Telegram          7MiB |
|    0   N/A  N/A      4381      G   /usr/lib/firefox/firefox-bin                    0MiB |
|    0   N/A  N/A      4790      G   ...sion,SpareRendererForSitePerProcess         79MiB |
|    0   N/A  N/A      5198      G   /usr/bin/nautilus                              24MiB |
|    0   N/A  N/A      5272      G   ...erProcess --variations-seed-version         23MiB |
|    0   N/A  N/A      5881      C   /usr/local/bin/ollama                           0MiB |
+-----------------------------------------------------------------------------------------+

CPU

AMD

Other software

No response

Originally created by @sumitkm on GitHub (Apr 4, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3489 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? This is probably same as #2029 but I'll try and provide more details. As mentioned in the above issue Ollama works fine from a clean boot. If I put computer to sleep and then wake it up, Ollama gets into a state where it can't be `stopped` via `systemctl` or killed via `kill <pid>`. Here is a screenshot from `htop` after I tried to `stop` it (after waking computer up from sleep). ![image](https://github.com/ollama/ollama/assets/578882/d8a06e37-6da2-4e96-a7e3-4ef773df1ea0) Output of `systemctl status ollama.service` ``` sumitkm@PIOT-WORK-02:~$ sudo systemctl status ollama.service × ollama.service - Ollama Service Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled) Active: failed (Result: timeout) since Thu 2024-04-04 13:34:55 BST; 4min 30s ago Duration: 30min 53.337s Main PID: 1675 Tasks: 1 (limit: 114811) Memory: 339.5M CPU: 4.780s CGroup: /system.slice/ollama.service └─1675 "[ollama]" Apr 04 13:30:24 PIOT-WORK-02 systemd[1]: ollama.service: State 'stop-sigterm' timed out. Killing. Apr 04 13:30:24 PIOT-WORK-02 systemd[1]: ollama.service: Killing process 1675 (ollama) with signal SIGKILL. Apr 04 13:31:54 PIOT-WORK-02 systemd[1]: ollama.service: Processes still around after SIGKILL. Ignoring. Apr 04 13:33:24 PIOT-WORK-02 systemd[1]: ollama.service: State 'final-sigterm' timed out. Killing. Apr 04 13:33:24 PIOT-WORK-02 systemd[1]: ollama.service: Killing process 1675 (ollama) with signal SIGKILL. Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Processes still around after final SIGKILL. Entering failed mode. Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Failed with result 'timeout'. Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Unit process 1675 (ollama) remains running after unit stopped. Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: Stopped ollama.service - Ollama Service. Apr 04 13:34:55 PIOT-WORK-02 systemd[1]: ollama.service: Consumed 4.780s CPU time. ``` Even if I don't try to kill it, ollama prevents system from going to sleep completely. Reboot attempts to kill it thrice and then forces reboot anyway. Workaround ========== If I do a `sudo systemctl stop ollama.service` before I put computer to sleep, everything works great. journalctl -u ollama ==================== [logs-ollama.txt](https://github.com/ollama/ollama/files/14871920/logs-ollama.txt) Hardware -------- CPU: AMD Ryzen 7950x (AMD Eco mode 105W via bios settings) RAM: 96 GB DDR 5 GPU: NVIDIA RTX 4090 (MSI Gaming X Slim card) hostnamectl ----------- Static hostname: PIOT-WORK-02 Icon name: computer-desktop Chassis: desktop 🖥️ Machine ID: f3d60fb0a8c84b7aa9eb8e********** Boot ID: 0b1a6283d6d04921a0f685********** Operating System: Debian GNU/Linux 12 (bookworm) Kernel: Linux 6.1.0-18-amd64 Architecture: x86-64 Hardware Vendor: ASUS Hardware Model: ROG CROSSHAIR X670E HERO Firmware Version: 1905 ### What did you expect to see? Ollama should work correctly when the system wakes up from sleep, that includes going into sleep mode the second time and subsequent ocassions. ### Steps to reproduce Steps mentioned in description are reproducible reliably on my computer. ### Are there any recent changes that introduced the issue? This is a brand new system and apart from `ollama` and `phronix` test suite, there is barely anything else on the system. ### OS Linux ### Architecture amd64 ### Platform _No response_ ### Ollama version 0.1.30 ### GPU Nvidia ### GPU info ``` nvidia-smi Thu Apr 4 14:27:29 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.14 Driver Version: 550.54.14 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4090 On | 00000000:01:00.0 On | Off | | 0% 45C P8 17W / 450W | 1220MiB / 24564MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 2042 G /usr/lib/xorg/Xorg 383MiB | | 0 N/A N/A 2224 G /usr/bin/gnome-shell 123MiB | | 0 N/A N/A 4123 G ...sumitkm/AppImages/Telegram/Telegram 7MiB | | 0 N/A N/A 4381 G /usr/lib/firefox/firefox-bin 0MiB | | 0 N/A N/A 4790 G ...sion,SpareRendererForSitePerProcess 79MiB | | 0 N/A N/A 5198 G /usr/bin/nautilus 24MiB | | 0 N/A N/A 5272 G ...erProcess --variations-seed-version 23MiB | | 0 N/A N/A 5881 C /usr/local/bin/ollama 0MiB | +-----------------------------------------------------------------------------------------+ ``` ### CPU AMD ### Other software _No response_
GiteaMirror added the bugnvidia labels 2026-04-12 12:22:45 -05:00
GiteaMirror commented 2026-04-12 12:22:46 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Apr 12, 2024):

In version 0.1.32 we're switching to a subprocess model which might mitigate this somewhat. As long as the system has been idle long enough for us to unload the subprocess, we will have fully released the GPU. If you run a prompt, and them quickly suspend, we'll still have the handle into the GPU though. (Default is 5 minute idle timeout to unload unused models, but adjustable)

<!-- gh-comment-id:2052695967 --> @dhiltgen commented on GitHub (Apr 12, 2024): In version 0.1.32 we're switching to a subprocess model which might mitigate this somewhat. As long as the system has been idle long enough for us to unload the subprocess, we will have fully released the GPU. If you run a prompt, and them quickly suspend, we'll still have the handle into the GPU though. (Default is 5 minute idle timeout to unload unused models, but adjustable)
GiteaMirror commented 2026-04-12 12:22:46 -05:00
Author
Owner
Copy Link

@sumitkm commented on GitHub (Apr 14, 2024):

Thanks @dhiltgen. That sounds like a reasonable solution. I can always set sleep timer to > ollama unload timer.

Is there an equivalent unload step today, or should I just continue to stop the service before suspend?

Much appreciate the team acknowledging and looking into the issue, which I am sure is very low on the priority pile :)

<!-- gh-comment-id:2054153560 --> @sumitkm commented on GitHub (Apr 14, 2024): Thanks @dhiltgen. That sounds like a reasonable solution. I can always set sleep timer to > ollama unload timer. Is there an equivalent `unload` step today, or should I just continue to `stop` the service before suspend? Much appreciate the team acknowledging and looking into the issue, which I am sure is very low on the priority pile :)
GiteaMirror commented 2026-04-12 12:22:47 -05:00
Author
Owner
Copy Link

@venondev commented on GitHub (Apr 15, 2024):

Hi,

I am also facing the same issue described above. When the system wakes up from suspend, it looks like the model is still in the GPU memory, but when I try to call it I receive the following error and ollama switches to CPU afterwards:
Error: Post "http://127.0.0.1:11434/api/chat": EOF

Stopping or restarting the service before suspend works for me too, but then the model is no longer in memory.

With the fix mentioned above, is it maybe possible to keep the model in gpu memory after a suspend, so that ollama does not have to load it again after waking up?

Thank you for your effort! :)

<!-- gh-comment-id:2056214056 --> @venondev commented on GitHub (Apr 15, 2024): Hi, I am also facing the same issue described above. When the system wakes up from suspend, it looks like the model is still in the GPU memory, but when I try to call it I receive the following error and ollama switches to CPU afterwards: `Error: Post "http://127.0.0.1:11434/api/chat": EOF` Stopping or restarting the service before suspend works for me too, but then the model is no longer in memory. With the fix mentioned above, is it maybe possible to keep the model in gpu memory after a suspend, so that ollama does not have to load it again after waking up? Thank you for your effort! :)
GiteaMirror commented 2026-04-12 12:22:47 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Apr 28, 2024):

We've refined the subprocess model further in 0.1.33 (available in pre-release now), so I'm curious if the system will be self-healing now. Please give the latest release a try and let us know if you see better behavior after a suspend, or if we still fall back to CPU after a resume.

<!-- gh-comment-id:2081601597 --> @dhiltgen commented on GitHub (Apr 28, 2024): We've refined the subprocess model further in [0.1.33](https://github.com/ollama/ollama/releases) (available in pre-release now), so I'm curious if the system will be self-healing now. Please give the latest release a try and let us know if you see better behavior after a suspend, or if we still fall back to CPU after a resume.
GiteaMirror commented 2026-04-12 12:22:47 -05:00
Author
Owner
Copy Link

@sumitkm commented on GitHub (Apr 29, 2024):

Sorry for the delay in response, but I just upgraded to 0.1.32 and the issue about going into uninterruptible sleep mode is gone. Thank you for that.

However, after recovering from sleep, it now refuses to use the GPU, it is only using CPU (I am going by the token generation rate, I don't know how to do it otherwise). A systemctl stop ollama.service followed by systemctl start ollama.service did not fix it.

 ● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-04-29 13:22:03 BST; 15s ago
   Main PID: 165802 (ollama)
      Tasks: 20 (limit: 114807)
     Memory: 490.9M
        CPU: 2.886s
     CGroup: /system.slice/ollama.service
             └─165802 /usr/local/bin/ollama serve

Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.621+01:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.621+01:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.622+01:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama2704931491/runners/cuda_v11/libcudart.so.11.0 /usr/local/c>
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.624+01:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama2704931491/runners/cuda_v11/libcudart.s>
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.628+01:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /usr/local/cuda/lib64/libcudart.so.12.4.99: cudart>
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.628+01:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.630+01:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so.550.54.14]"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.634+01:00 level=INFO source=gpu.go:137 msg="Nvidia GPU detected via nvidia-ml"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.634+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.643+01:00 level=INFO source=gpu.go:182 msg="[nvidia-ml] NVML CUDA Compute Capability detected: 8.9"
sumitkm@PIOT-WORK-02:~$ nvidia-smi
Mon Apr 29 13:42:30 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:01:00.0  On |                  Off |
|  0%   36C    P8             16W /  450W |    1099MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      2388      G   /usr/lib/xorg/Xorg                            594MiB |
|    0   N/A  N/A      2579      G   /usr/bin/gnome-shell                           67MiB |
|    0   N/A  N/A      4353      G   firefox                                         0MiB |
|    0   N/A  N/A     42749      G   gnome-control-center                           11MiB |
|    0   N/A  N/A    117112      G   /usr/bin/nautilus                              19MiB |
|    0   N/A  N/A    119452      G   ...sumitkm/AppImages/Telegram/Telegram          7MiB |
|    0   N/A  N/A    144681      G   ...sion,SpareRendererForSitePerProcess         69MiB |
|    0   N/A  N/A    144857      G   ...erProcess --variations-seed-version         32MiB |
|    0   N/A  N/A    144858      G   ...erProcess --variations-seed-version         17MiB |
+-----------------------------------------------------------------------------------------+

I will upgrade to 0.1.33 as soon as I've figured out how to install a pre-release version 🙈

<!-- gh-comment-id:2082658262 --> @sumitkm commented on GitHub (Apr 29, 2024): Sorry for the delay in response, but I just upgraded to 0.1.32 and the issue about going into uninterruptible sleep mode is gone. Thank you for that. However, after recovering from sleep, it now refuses to use the GPU, it is only using CPU (I am going by the token generation rate, I don't know how to do it otherwise). A `systemctl stop ollama.service` followed by `systemctl start ollama.service` did not fix it. ```bash ● ollama.service - Ollama Service Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled) Active: active (running) since Mon 2024-04-29 13:22:03 BST; 15s ago Main PID: 165802 (ollama) Tasks: 20 (limit: 114807) Memory: 490.9M CPU: 2.886s CGroup: /system.slice/ollama.service └─165802 /usr/local/bin/ollama serve Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.621+01:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.621+01:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.622+01:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama2704931491/runners/cuda_v11/libcudart.so.11.0 /usr/local/c> Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.624+01:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama2704931491/runners/cuda_v11/libcudart.s> Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.628+01:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /usr/local/cuda/lib64/libcudart.so.12.4.99: cudart> Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.628+01:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.630+01:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so.550.54.14]" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.634+01:00 level=INFO source=gpu.go:137 msg="Nvidia GPU detected via nvidia-ml" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.634+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" Apr 29 13:22:05 PIOT-WORK-02 ollama[165802]: time=2024-04-29T13:22:05.643+01:00 level=INFO source=gpu.go:182 msg="[nvidia-ml] NVML CUDA Compute Capability detected: 8.9" sumitkm@PIOT-WORK-02:~$ nvidia-smi Mon Apr 29 13:42:30 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.14 Driver Version: 550.54.14 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4090 On | 00000000:01:00.0 On | Off | | 0% 36C P8 16W / 450W | 1099MiB / 24564MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 2388 G /usr/lib/xorg/Xorg 594MiB | | 0 N/A N/A 2579 G /usr/bin/gnome-shell 67MiB | | 0 N/A N/A 4353 G firefox 0MiB | | 0 N/A N/A 42749 G gnome-control-center 11MiB | | 0 N/A N/A 117112 G /usr/bin/nautilus 19MiB | | 0 N/A N/A 119452 G ...sumitkm/AppImages/Telegram/Telegram 7MiB | | 0 N/A N/A 144681 G ...sion,SpareRendererForSitePerProcess 69MiB | | 0 N/A N/A 144857 G ...erProcess --variations-seed-version 32MiB | | 0 N/A N/A 144858 G ...erProcess --variations-seed-version 17MiB | +-----------------------------------------------------------------------------------------+ ``` I will upgrade to 0.1.33 as soon as I've figured out how to install a pre-release version 🙈
GiteaMirror commented 2026-04-12 12:22:47 -05:00
Author
Owner
Copy Link

@Aoriseth commented on GitHub (May 5, 2024):

In the meanwhile, if you want to switch back to using the GPU after resuming from a suspend, you can reload the nvidia_uvm:

First stop the service:
systemctl stop ollama

Reload nvidia_uvm:
sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm

Then restart the service:
systemclt start ollama

That should allow you to run a model again on the GPU.

<!-- gh-comment-id:2094665760 --> @Aoriseth commented on GitHub (May 5, 2024): In the meanwhile, if you want to switch back to using the GPU after resuming from a suspend, you can reload the nvidia_uvm: First stop the service: `systemctl stop ollama` Reload nvidia_uvm: `sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm` Then restart the service: `systemclt start ollama` That should allow you to run a model again on the GPU.
GiteaMirror commented 2026-04-12 12:22:48 -05:00
Author
Owner
Copy Link

@gyulii commented on GitHub (May 5, 2024):

Thanks @Aoriseth you're a genius, it's a quick and easy fix.

<!-- gh-comment-id:2094817934 --> @gyulii commented on GitHub (May 5, 2024): Thanks @Aoriseth you're a genius, it's a quick and easy fix.
GiteaMirror commented 2026-04-12 12:22:48 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 21, 2024):

Looks like we can resolve this one now. We've added these troubleshooting steps to our docs as well. https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#container-fails-to-run-on-nvidia-gpu

<!-- gh-comment-id:2123199709 --> @dhiltgen commented on GitHub (May 21, 2024): Looks like we can resolve this one now. We've added these troubleshooting steps to our docs as well. https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#container-fails-to-run-on-nvidia-gpu
GiteaMirror commented 2026-04-12 12:22:49 -05:00
Author
Owner
Copy Link

@Ks2wAu89jnEgCR0bGUAA commented on GitHub (Jul 19, 2025):

✅ Automatically Run a Script After Suspend on ubuntu/deb with systemd

You can create a systemd service that automatically runs after your Linux PC wakes from suspend. Here's how to do it, step by step.

✅ 1. Create the systemd service unit

We'll call it something like ollama-post-suspend.service.

Open a terminal and run:

sudo nano /etc/systemd/system/ollama-post-suspend.service

Paste this into the file:

[Unit]
Description=Reload nvidia_uvm and restart ollama after suspend
After=suspend.target
StopWhenUnneeded=yes

[Service]
Type=oneshot
ExecStart=/usr/local/bin/ollama-post-suspend.sh

[Install]
WantedBy=suspend.target

✅ 2. Create the script that will do the actual work

Create the script and make it executable:

sudo nano /usr/local/bin/ollama-post-suspend.sh

Paste this into the script:

#!/bin/bash

# Stop the service
systemctl stop ollama

# Reload the NVIDIA UVM kernel module
modprobe -r nvidia_uvm && modprobe nvidia_uvm

# Restart the service
systemctl start ollama

Make the script executable:

sudo chmod +x /usr/local/bin/ollama-post-suspend.sh

✅ 3. Enable the systemd service

Enable it so it runs after each resume from suspend:

sudo systemctl enable ollama-post-suspend.service

Note: Technically, systemd services can't trigger directly after resume, but suspend.target is reached on resume as well.

✅ How it works

  • suspend.target: A special systemd target that is reached when the system resumes from suspend.
  • The service runs once (Type=oneshot) when that target is reached.
  • It executes a script that:
    • Stops the ollama service
    • Reloads the nvidia_uvm kernel module
    • Restarts the ollama service

This setup ensures that after resuming from suspend, the GPU driver is reloaded and ollama can restart with a clean state.

🧪 Testing

You can simulate the behavior by running the script manually:

sudo /usr/local/bin/ollama-post-suspend.sh

Or test the full service manually:

sudo systemctl start ollama-post-suspend.service

Then suspend your computer, wake it up, and check if ollama restarted:

systemctl status ollama
<!-- gh-comment-id:3092566724 --> @Ks2wAu89jnEgCR0bGUAA commented on GitHub (Jul 19, 2025): # ✅ Automatically Run a Script After Suspend on ubuntu/deb with systemd You can create a `systemd` service that automatically runs after your Linux PC wakes from suspend. Here's how to do it, step by step. --- ## ✅ 1. Create the systemd service unit We'll call it something like `ollama-post-suspend.service`. Open a terminal and run: ```bash sudo nano /etc/systemd/system/ollama-post-suspend.service ``` Paste this into the file: ```ini [Unit] Description=Reload nvidia_uvm and restart ollama after suspend After=suspend.target StopWhenUnneeded=yes [Service] Type=oneshot ExecStart=/usr/local/bin/ollama-post-suspend.sh [Install] WantedBy=suspend.target ``` --- ## ✅ 2. Create the script that will do the actual work Create the script and make it executable: ```bash sudo nano /usr/local/bin/ollama-post-suspend.sh ``` Paste this into the script: ```bash #!/bin/bash # Stop the service systemctl stop ollama # Reload the NVIDIA UVM kernel module modprobe -r nvidia_uvm && modprobe nvidia_uvm # Restart the service systemctl start ollama ``` Make the script executable: ```bash sudo chmod +x /usr/local/bin/ollama-post-suspend.sh ``` --- ## ✅ 3. Enable the systemd service Enable it so it runs after each resume from suspend: ```bash sudo systemctl enable ollama-post-suspend.service ``` > **Note:** Technically, `systemd` services can't trigger directly *after resume*, but `suspend.target` is reached on resume as well. --- ## ✅ How it works - **suspend.target**: A special `systemd` target that is reached when the system resumes from suspend. - The service runs **once** (`Type=oneshot`) when that target is reached. - It executes a script that: - Stops the `ollama` service - Reloads the `nvidia_uvm` kernel module - Restarts the `ollama` service This setup ensures that after resuming from suspend, the GPU driver is reloaded and `ollama` can restart with a clean state. --- ## 🧪 Testing You can simulate the behavior by running the script manually: ```bash sudo /usr/local/bin/ollama-post-suspend.sh ``` Or test the full service manually: ```bash sudo systemctl start ollama-post-suspend.service ``` Then suspend your computer, wake it up, and check if `ollama` restarted: ```bash systemctl status ollama ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2149
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?