github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6541] llama runner process has terminated: exit status127 #66152

New Issue
Closed
opened 2026-05-04 00:16:53 -05:00 by GiteaMirror · 25 comments
GiteaMirror commented 2026-05-04 00:16:53 -05:00
Owner
Copy Link

Originally created by @sosojust1984 on GitHub (Aug 28, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6541

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

llama runner process has terminated: exit status127 ?

OS

Linux

GPU

Other

CPU

Other

Ollama version

0.3*

Originally created by @sosojust1984 on GitHub (Aug 28, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6541 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? llama runner process has terminated: exit status127 ? ### OS Linux ### GPU Other ### CPU Other ### Ollama version 0.3*
GiteaMirror added the bug label 2026-05-04 00:16:53 -05:00
GiteaMirror commented 2026-05-04 00:16:57 -05:00
Author
Owner
Copy Link

@pintofbeer commented on GitHub (Aug 28, 2024):

I also get this on Raspberry Pi 5, it seems to be looking for the GPU runner libllama.so when there isn't a GPU.

I checked older versions and appears to be a change in 0.3.7.
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.3 sh (works)
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.4 sh (works)
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.5 sh (works)
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.6 sh (works)
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.7 sh (breaks)

Log:
Aug 28 13:53:29 dev-server ollama[2771]: [GIN] 2024/08/28 - 13:53:29 | 200 | 106.193289ms | 127.0.0.1 | POST "/api/show"
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.216+01:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=33 layers.offload=0 layers.split="" memory.available="[7.1 GiB]" memory.required.full="5.8 GiB" memory.required.partial="0 B" memory.required.kv="1.0 GiB" memory.required.allocations="[5.8 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB"
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.216+01:00 level=INFO source=server.go:391 msg="starting llama server" cmd="/tmp/ollama2332254194/runners/cpu/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe --ctx-size 8192 --batch-size 512 --embedding --log-disable --no-mmap --parallel 4 --port 35227"
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=sched.go:450 msg="loaded runners" count=1
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
Aug 28 13:53:29 dev-server ollama[2771]: /tmp/ollama2332254194/runners/cpu/ollama_llama_server: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server error"
Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.467+01:00 level=ERROR source=sched.go:456 msg="error loading llama server" error="llama runner process has terminated: exit status 127"
Aug 28 13:53:29 dev-server ollama[2771]: [GIN] 2024/08/28 - 13:53:29 | 500 | 348.302988ms | 127.0.0.1 | POST "/api/chat"

<!-- gh-comment-id:2315248836 --> @pintofbeer commented on GitHub (Aug 28, 2024): I also get this on Raspberry Pi 5, it seems to be looking for the GPU runner libllama.so when there isn't a GPU. I checked older versions and appears to be a change in 0.3.7. curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.3 sh (works) curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.4 sh (works) curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.5 sh (works) curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.6 sh (works) curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.7 sh (breaks) Log: Aug 28 13:53:29 dev-server ollama[2771]: [GIN] 2024/08/28 - 13:53:29 | 200 | 106.193289ms | 127.0.0.1 | POST "/api/show" Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.216+01:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=33 layers.offload=0 layers.split="" memory.available="[7.1 GiB]" memory.required.full="5.8 GiB" memory.required.partial="0 B" memory.required.kv="1.0 GiB" memory.required.allocations="[5.8 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB" Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.216+01:00 level=INFO source=server.go:391 msg="starting llama server" cmd="/tmp/ollama2332254194/runners/cpu/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe --ctx-size 8192 --batch-size 512 --embedding --log-disable --no-mmap --parallel 4 --port 35227" Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding" Aug 28 13:53:29 dev-server ollama[2771]: /tmp/ollama2332254194/runners/cpu/ollama_llama_server: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.217+01:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server error" Aug 28 13:53:29 dev-server ollama[2771]: time=2024-08-28T13:53:29.467+01:00 level=ERROR source=sched.go:456 msg="error loading llama server" error="llama runner process has terminated: exit status 127" Aug 28 13:53:29 dev-server ollama[2771]: [GIN] 2024/08/28 - 13:53:29 | 500 | 348.302988ms | 127.0.0.1 | POST "/api/chat"
GiteaMirror commented 2026-05-04 00:16:58 -05:00
Author
Owner
Copy Link

@rikbon commented on GitHub (Aug 28, 2024):

same here, running Ollama in a docker container on ARM on Oracle Linux + Open WebUi on a Oracle Cloud VPS...

Running the same configuration (docker with ollama+open webui) and version on debian x64 works smoothly (no gpu, it's a low end NUC)...

<!-- gh-comment-id:2315253453 --> @rikbon commented on GitHub (Aug 28, 2024): same here, running Ollama in a docker container on ARM on Oracle Linux + Open WebUi on a Oracle Cloud VPS... Running the same configuration (docker with ollama+open webui) and version on debian x64 works smoothly (no gpu, it's a low end NUC)...
GiteaMirror commented 2026-05-04 00:16:59 -05:00
Author
Owner
Copy Link

@thelastblt commented on GitHub (Aug 28, 2024):

Ditto- running into this issue on Ollama in docker container on ARM Oracle Ubuntu on Oracle Cloud VPS.

Can post logs, but they're basically the same as pintofbeer's

Downgrade to 3.6 worked.

<!-- gh-comment-id:2315896020 --> @thelastblt commented on GitHub (Aug 28, 2024): Ditto- running into this issue on Ollama in docker container on ARM Oracle Ubuntu on Oracle Cloud VPS. Can post logs, but they're basically the same as pintofbeer's Downgrade to 3.6 worked.
GiteaMirror commented 2026-05-04 00:17:00 -05:00
Author
Owner
Copy Link

@benpatt-bp commented on GitHub (Aug 28, 2024):

Same here, Ollama in docker container on Raspberry Pi 5. Logs look the same as above. And yes, 0.3.6 works.

<!-- gh-comment-id:2315914550 --> @benpatt-bp commented on GitHub (Aug 28, 2024): Same here, Ollama in docker container on Raspberry Pi 5. Logs look the same as above. And yes, 0.3.6 works.
GiteaMirror commented 2026-05-04 00:17:01 -05:00
Author
Owner
Copy Link

@PaulaScholz commented on GitHub (Aug 29, 2024):

Just got this after installing Ollama with curl on a Raspberry Pi 5. Version 0.3.8

<!-- gh-comment-id:2316560841 --> @PaulaScholz commented on GitHub (Aug 29, 2024): Just got this after installing Ollama with curl on a Raspberry Pi 5. Version 0.3.8
GiteaMirror commented 2026-05-04 00:17:02 -05:00
Author
Owner
Copy Link

@barnaclebarnes commented on GitHub (Aug 29, 2024):

Got the same on latest on Apple Silicon. Rolling back to 0.3.6 works for me.

<!-- gh-comment-id:2316620749 --> @barnaclebarnes commented on GitHub (Aug 29, 2024): Got the same on `latest` on Apple Silicon. Rolling back to `0.3.6` works for me.
GiteaMirror commented 2026-05-04 00:17:05 -05:00
Author
Owner
Copy Link

@devskale commented on GitHub (Aug 29, 2024):

on Oracle Cloud Instance with Ampere CPU

OLLAMA_DEBUG=1 ./ollama run gemma2:2b
Error: llama runner process has terminated: exit status 127

architecture:
Linux ampere aarch64
Operating System: Ubuntu 24.04 LTS
Kernel: Linux 6.8.0-1011-oracle

<!-- gh-comment-id:2317568544 --> @devskale commented on GitHub (Aug 29, 2024): on Oracle Cloud Instance with Ampere CPU OLLAMA_DEBUG=1 ./ollama run gemma2:2b Error: llama runner process has terminated: exit status 127 architecture: Linux ampere aarch64 Operating System: Ubuntu 24.04 LTS Kernel: Linux 6.8.0-1011-oracle
GiteaMirror commented 2026-05-04 00:17:06 -05:00
Author
Owner
Copy Link

@jpfox commented on GitHub (Aug 29, 2024):

I use Archlinux ollama-cuda package and it seems that libcudart.so lib is no more embedded in this package ??
I've installed cuda package and it works now.

<!-- gh-comment-id:2317692522 --> @jpfox commented on GitHub (Aug 29, 2024): I use Archlinux `ollama-cuda` package and it seems that `libcudart.so` lib is no more embedded in this package ?? I've installed `cuda` package and it works now.
GiteaMirror commented 2026-05-04 00:17:06 -05:00
Author
Owner
Copy Link

@unclemusclez commented on GitHub (Aug 29, 2024):

one of these updates also killed my raspberry pi 3b+ how dare

<!-- gh-comment-id:2318211538 --> @unclemusclez commented on GitHub (Aug 29, 2024): one of these updates also killed my raspberry pi 3b+ how dare
GiteaMirror commented 2026-05-04 00:17:07 -05:00
Author
Owner
Copy Link

@sjohn2 commented on GitHub (Aug 29, 2024):

Thank you, @barnaclebarnes . Downgrading to 0.3.6 fixed it for me on my Pi 5.

<!-- gh-comment-id:2318372598 --> @sjohn2 commented on GitHub (Aug 29, 2024): Thank you, @barnaclebarnes . Downgrading to 0.3.6 fixed it for me on my Pi 5.
GiteaMirror commented 2026-05-04 00:17:08 -05:00
Author
Owner
Copy Link

@unclemusclez commented on GitHub (Aug 29, 2024):

Thank you, @barnaclebarnes . Downgrading to 0.3.6 fixed it for me on my Pi 5.

also worked for me

<!-- gh-comment-id:2318466415 --> @unclemusclez commented on GitHub (Aug 29, 2024): > Thank you, @barnaclebarnes . Downgrading to 0.3.6 fixed it for me on my Pi 5. also worked for me
GiteaMirror commented 2026-05-04 00:17:09 -05:00
Author
Owner
Copy Link

@SpudGunMan commented on GitHub (Aug 30, 2024):

mee too thanks

<!-- gh-comment-id:2320216404 --> @SpudGunMan commented on GitHub (Aug 30, 2024): mee too thanks
GiteaMirror commented 2026-05-04 00:17:10 -05:00
Author
Owner
Copy Link

@mt50su commented on GitHub (Aug 30, 2024):

same here, running ollama on a Raspberry Pi 5, and the log showed the same as pintofbeer's.
ver.0.3.6 works.

<!-- gh-comment-id:2320792020 --> @mt50su commented on GitHub (Aug 30, 2024): same here, running ollama on a Raspberry Pi 5, and the log showed the same as pintofbeer's. ver.0.3.6 works.
GiteaMirror commented 2026-05-04 00:17:11 -05:00
Author
Owner
Copy Link

@PaulaScholz commented on GitHub (Aug 31, 2024):

How do you install 3.6? I have a download from Github but don't know how to install it. Any instructions? All the instructions on the web involve downloading the default from Ollama, which is 3.8.

<!-- gh-comment-id:2322634696 --> @PaulaScholz commented on GitHub (Aug 31, 2024): How do you install 3.6? I have a download from Github but don't know how to install it. Any instructions? All the instructions on the web involve downloading the default from Ollama, which is 3.8.
GiteaMirror commented 2026-05-04 00:17:12 -05:00
Author
Owner
Copy Link

@MikeLP commented on GitHub (Aug 31, 2024):

https://github.com/ollama/ollama/issues/6568#issuecomment-2322023759

This issue affects ROCm build as well.

<!-- gh-comment-id:2322861651 --> @MikeLP commented on GitHub (Aug 31, 2024): https://github.com/ollama/ollama/issues/6568#issuecomment-2322023759 This issue affects ROCm build as well.
GiteaMirror commented 2026-05-04 00:17:12 -05:00
Author
Owner
Copy Link

@khuezy commented on GitHub (Aug 31, 2024):

How do you install 3.6? I have a download from Github but don't know how to install it. Any instructions? All the instructions on the web involve downloading the default from Ollama, which is 3.8.

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.6 sh

<!-- gh-comment-id:2322966481 --> @khuezy commented on GitHub (Aug 31, 2024): > How do you install 3.6? I have a download from Github but don't know how to install it. Any instructions? All the instructions on the web involve downloading the default from Ollama, which is 3.8. curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.3.6 sh
GiteaMirror commented 2026-05-04 00:17:13 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Aug 31, 2024):

Hi folks, so sorry about this. A new update is being prepared that will fix this (0.3.9). It should be available later today

<!-- gh-comment-id:2323008422 --> @jmorganca commented on GitHub (Aug 31, 2024): Hi folks, so sorry about this. A new update is being prepared that will fix this (0.3.9). It should be available later today
GiteaMirror commented 2026-05-04 00:17:15 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Aug 31, 2024):

Hi all, this should be fixed in Ollama 0.3.9: https://github.com/ollama/ollama/releases/tag/v0.3.9

Sorry again!

<!-- gh-comment-id:2323046067 --> @jmorganca commented on GitHub (Aug 31, 2024): Hi all, this should be fixed in Ollama 0.3.9: https://github.com/ollama/ollama/releases/tag/v0.3.9 Sorry again!
GiteaMirror commented 2026-05-04 00:17:16 -05:00
Author
Owner
Copy Link

@devskale commented on GitHub (Sep 1, 2024):

i can confirm that Ollama 0.3.9 is now again working ✅ on
OCI Ampere CPU
Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-1011-oracle aarch64)

thanks for the prompt fix

<!-- gh-comment-id:2323481227 --> @devskale commented on GitHub (Sep 1, 2024): i can confirm that Ollama 0.3.9 is now again working ✅ on OCI Ampere CPU Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-1011-oracle aarch64) thanks for the prompt fix
GiteaMirror commented 2026-05-04 00:17:17 -05:00
Author
Owner
Copy Link

@MikeLP commented on GitHub (Sep 2, 2024):

@jmorganca I still have the issue with ollama 0.3.9 (0.3.6 works well)
➜ ~ ollama run dbrx:132b-instruct-q8_0
Error: llama runner process has terminated: signal: segmentation fault (core dumped)

Should I create a new issue?

<!-- gh-comment-id:2323775498 --> @MikeLP commented on GitHub (Sep 2, 2024): @jmorganca I still have the issue with ollama 0.3.9 (0.3.6 works well) ➜ ~ ollama run dbrx:132b-instruct-q8_0 Error: llama runner process has terminated: signal: segmentation fault (core dumped) Should I create a new issue?
GiteaMirror commented 2026-05-04 00:17:18 -05:00
Author
Owner
Copy Link

@sjohn2 commented on GitHub (Sep 2, 2024):

0.3.9 works fine on RPI 5.0. Thank you, @jmorganca for the quick fix.

<!-- gh-comment-id:2323822692 --> @sjohn2 commented on GitHub (Sep 2, 2024): 0.3.9 works fine on RPI 5.0. Thank you, @jmorganca for the quick fix.
GiteaMirror commented 2026-05-04 00:17:22 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Sep 3, 2024):

@MikeLP your crash is most likely unrelated to the root cause of this issue, so I'd suggest opening a new issue. Please include server logs so we can see how far along it got before crashing on loading this model.

<!-- gh-comment-id:2327059853 --> @dhiltgen commented on GitHub (Sep 3, 2024): @MikeLP your crash is most likely unrelated to the root cause of this issue, so I'd suggest opening a new issue. Please include server logs so we can see how far along it got before crashing on loading this model.
GiteaMirror commented 2026-05-04 00:17:25 -05:00
Author
Owner
Copy Link

@sarrcom commented on GitHub (Oct 22, 2024):

Same here, running 0.3.14, I noticed this in the server logs:

time=2024-10-22T20:24:19.453+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory"
time=2024-10-22T20:24:19.707+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.0166048 model=C:\Users\xxxxxxx.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8...
time=2024-10-22T20:24:19.707+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory"
time=2024-10-22T20:24:19.945+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.2552733 model=C:\Users\xxxxxxx.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8...
time=2024-10-22T20:24:19.946+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory"
time=2024-10-22T20:24:20.200+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.5102787 model=C:\Users\xxxxxxx.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8...
[GIN] 2024/10/22 - 20:27:02 | 200 | 0s | 127.0.0.1 | GET "/api/version"

Looks like this is related to:
https://github.com/ollama/ollama/issues/6928

<!-- gh-comment-id:2429976496 --> @sarrcom commented on GitHub (Oct 22, 2024): Same here, running 0.3.14, I noticed this in the server logs: time=2024-10-22T20:24:19.453+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory" time=2024-10-22T20:24:19.707+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.0166048 model=C:\Users\xxxxxxx\.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8... time=2024-10-22T20:24:19.707+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory" time=2024-10-22T20:24:19.945+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.2552733 model=C:\Users\xxxxxxx\.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8... time=2024-10-22T20:24:19.946+02:00 level=WARN source=gpu.go:437 msg="error looking up nvidia GPU memory" time=2024-10-22T20:24:20.200+02:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.5102787 model=C:\Users\xxxxxxx\.ollama\models\blobs\sha256-dde5aa3fc5ffc17176b5e8... [GIN] 2024/10/22 - 20:27:02 | 200 | 0s | 127.0.0.1 | GET "/api/version" Looks like this is related to: [https://github.com/ollama/ollama/issues/6928](https://github.com/ollama/ollama/issues/6928)
GiteaMirror commented 2026-05-04 00:17:27 -05:00
Author
Owner
Copy Link

@buddha314 commented on GitHub (May 17, 2025):

Getting this error now on 0.7.0 with qwen3:0.6b on Nobara linux

<!-- gh-comment-id:2888523367 --> @buddha314 commented on GitHub (May 17, 2025): Getting this error now on 0.7.0 with qwen3:0.6b on Nobara linux
GiteaMirror commented 2026-05-04 00:17:29 -05:00
Author
Owner
Copy Link

@ErfanFathi commented on GitHub (May 21, 2025):

This command works for me:
systemctl restart ollama.service

<!-- gh-comment-id:2899196049 --> @ErfanFathi commented on GitHub (May 21, 2025): This command works for me: `systemctl restart ollama.service `
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#66152
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?