github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-13 17:34:40 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5350] Gemma 2 9B cannot run #3349

New Issue
Closed
opened 2026-04-12 13:57:33 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-12 13:57:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @Forevery1 on GitHub (Jun 28, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5350

What is the issue?

image

OS

Ubuntu 22.04.4 LTS

GPU

Nvidia 4060

CPU

Intel

Ollama version

0.1.47

Originally created by @Forevery1 on GitHub (Jun 28, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5350 ### What is the issue? <img width="732" alt="image" src="https://github.com/ollama/ollama/assets/19872771/e28dac56-9a8b-4310-84d3-97bf3b2594f4"> ### OS Ubuntu 22.04.4 LTS ### GPU Nvidia 4060 ### CPU Intel ### Ollama version 0.1.47
GiteaMirror added the bug label 2026-04-12 13:57:33 -05:00
GiteaMirror commented 2026-04-12 13:57:35 -05:00
Author
Owner
Copy Link
Copy Source

@tianfan007 commented on GitHub (Jun 28, 2024):

the same issue.
when I run command:
ollama run gemma2
there is an error occurd
Error: llama runner process has terminated: signal: aborted (core dumped)

any other model works well
截图 2024-06-28 13-44-56

CPU: AMD R5-6600
RAM:16GB
OS:ubuntu 22.04
GPU: nvidia 3050Ti 4GB
GPU Dirver: 535.183.01
CUDA Version: 12.2

<!-- gh-comment-id:2196188227 --> @tianfan007 commented on GitHub (Jun 28, 2024): the same issue. when I run command: `ollama run gemma2` there is an error occurd `Error: llama runner process has terminated: signal: aborted (core dumped) ` any other model works well ![截图 2024-06-28 13-44-56](https://github.com/ollama/ollama/assets/5567566/247fd174-3f2e-4383-8959-be753328dd89) CPU: AMD R5-6600 RAM:16GB OS:ubuntu 22.04 GPU: nvidia 3050Ti 4GB GPU Dirver: 535.183.01 CUDA Version: 12.2
GiteaMirror commented 2026-04-12 13:57:35 -05:00
Author
Owner
Copy Link
Copy Source

@mhjabreel commented on GitHub (Jun 28, 2024):

Same here
image

Windows
CPU: Intel(R) Core(TM) i9-14900HX

<!-- gh-comment-id:2196252162 --> @mhjabreel commented on GitHub (Jun 28, 2024): Same here <img width="871" alt="image" src="https://github.com/ollama/ollama/assets/9088025/194ec49c-8c19-4f8a-a67d-c20616b1e566"> Windows CPU: Intel(R) Core(TM) i9-14900HX
GiteaMirror commented 2026-04-12 13:57:37 -05:00
Author
Owner
Copy Link
Copy Source

@SP4595 commented on GitHub (Jun 28, 2024):

pic
Same bug for WSL2
Distributor ID: Ubuntu
Description: Ubuntu 22.04.4 LTS
Release: 22.04
Codename: jammy

<!-- gh-comment-id:2196407973 --> @SP4595 commented on GitHub (Jun 28, 2024): ![pic](https://github.com/ollama/ollama/assets/109747786/114b3804-9e67-4b21-a3b7-10a07bd6d026) Same bug for WSL2 Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammy
GiteaMirror commented 2026-04-12 13:57:37 -05:00
Author
Owner
Copy Link
Copy Source

@rick-github commented on GitHub (Jun 28, 2024):

May be a problem with ollama not properly calculating the amount of required VRAM. https://github.com/ollama/ollama/commit/1ed4f521c403025050c509394fb4ac3ca2466865 resolves (for me) the problem of OOM during model load. You can get the model to load without this patch by setting num_gpu lower (search logs for --n-gpu-layers to see what the default value is for your config).

<!-- gh-comment-id:2196929454 --> @rick-github commented on GitHub (Jun 28, 2024): May be a problem with ollama not properly calculating the amount of required VRAM. https://github.com/ollama/ollama/commit/1ed4f521c403025050c509394fb4ac3ca2466865 resolves (for me) the problem of OOM during model load. You can get the model to load without this patch by setting `num_gpu` lower (search logs for `--n-gpu-layers` to see what the default value is for your config).
GiteaMirror commented 2026-04-12 13:57:38 -05:00
Author
Owner
Copy Link
Copy Source

@chirag-urb commented on GitHub (Jun 28, 2024):

It is resolved for me.

sudo journalctl -u ollama | grep -- "--n-gpu-layers"

Logs

Jun 28 13:40:14 archlinux ollama[211707]: time=2024-06-28T13:40:14.215+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 33021"
Jun 28 13:40:36 archlinux ollama[211707]: time=2024-06-28T13:40:36.915+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 18 --parallel 1 --port 36953"
Jun 28 13:40:51 archlinux ollama[211707]: time=2024-06-28T13:40:51.293+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b6ee2328408ebc031359e9745973b09963df9269468d37e1ea7912862aadec72 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 8 --parallel 1 --port 33949"
Jun 28 13:44:57 archlinux ollama[1036]: time=2024-06-28T13:44:57.033+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 41111"
Jun 28 13:45:39 archlinux ollama[1036]: time=2024-06-28T13:45:39.994+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 41445"
Jun 28 13:53:46 archlinux ollama[1036]: time=2024-06-28T13:53:46.235+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b6ee2328408ebc031359e9745973b09963df9269468d37e1ea7912862aadec72 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 8 --parallel 1 --port 45083"
Jun 28 14:08:13 archlinux ollama[1036]: time=2024-06-28T14:08:13.101+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b26e6713dc749dda35872713fa19a568040f475cc71cb132cff332fe7e216462 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 31 --parallel 1 --port 41101"
Jun 28 14:11:10 archlinux ollama[1036]: time=2024-06-28T14:11:10.351+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 42557"
Jun 28 14:23:09 archlinux ollama[1036]: time=2024-06-28T14:23:09.476+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 43707"
Jun 28 14:26:33 archlinux ollama[1036]: time=2024-06-28T14:26:33.030+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 1024 --batch-size 512 --embedding --log-disable --n-gpu-layers 25 --parallel 1 --port 45287"
Jun 28 14:27:35 archlinux ollama[1036]: time=2024-06-28T14:27:35.763+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 18 --parallel 1 --port 43549"
Jun 28 15:24:47 archlinux ollama[1036]: time=2024-06-28T15:24:47.997+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 35471"

Defaults
8 gpu layers for gemma2:27b
22 gpu layers for gemma2:9b
What worked
6 gpu layers for gemma2:27b
19 gpu layers for gemma2:9b

gemma2l

FROM gemma2:27b
PARAMETER num_gpu 6

ollama create gemma2l -f pathtofile/gemma2l

gemma2s

FROM gemma2
PARAMETER num_gpu 19

ollama create gemma2s -f pathtofile/gemma2s
<!-- gh-comment-id:2197000669 --> @chirag-urb commented on GitHub (Jun 28, 2024): It is resolved for me. ``` sudo journalctl -u ollama | grep -- "--n-gpu-layers" ``` Logs ``` Jun 28 13:40:14 archlinux ollama[211707]: time=2024-06-28T13:40:14.215+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 33021" Jun 28 13:40:36 archlinux ollama[211707]: time=2024-06-28T13:40:36.915+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 18 --parallel 1 --port 36953" Jun 28 13:40:51 archlinux ollama[211707]: time=2024-06-28T13:40:51.293+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama4270095903/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b6ee2328408ebc031359e9745973b09963df9269468d37e1ea7912862aadec72 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 8 --parallel 1 --port 33949" Jun 28 13:44:57 archlinux ollama[1036]: time=2024-06-28T13:44:57.033+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 41111" Jun 28 13:45:39 archlinux ollama[1036]: time=2024-06-28T13:45:39.994+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 41445" Jun 28 13:53:46 archlinux ollama[1036]: time=2024-06-28T13:53:46.235+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b6ee2328408ebc031359e9745973b09963df9269468d37e1ea7912862aadec72 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 8 --parallel 1 --port 45083" Jun 28 14:08:13 archlinux ollama[1036]: time=2024-06-28T14:08:13.101+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-b26e6713dc749dda35872713fa19a568040f475cc71cb132cff332fe7e216462 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 31 --parallel 1 --port 41101" Jun 28 14:11:10 archlinux ollama[1036]: time=2024-06-28T14:11:10.351+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 42557" Jun 28 14:23:09 archlinux ollama[1036]: time=2024-06-28T14:23:09.476+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 43707" Jun 28 14:26:33 archlinux ollama[1036]: time=2024-06-28T14:26:33.030+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 1024 --batch-size 512 --embedding --log-disable --n-gpu-layers 25 --parallel 1 --port 45287" Jun 28 14:27:35 archlinux ollama[1036]: time=2024-06-28T14:27:35.763+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 18 --parallel 1 --port 43549" Jun 28 15:24:47 archlinux ollama[1036]: time=2024-06-28T15:24:47.997+02:00 level=INFO source=server.go:368 msg="starting llama server" cmd="/tmp/ollama2544715102/runners/cuda_v12/ollama_llama_server --model /var/lib/ollama/.ollama/models/blobs/sha256-e84ed7399c82fbf7dbd6cdef3f12d356c3cdb5512e5d8b2a9898080cbcdd72e5 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 22 --parallel 1 --port 35471" ``` Defaults 8 gpu layers for gemma2:27b 22 gpu layers for gemma2:9b What worked 6 gpu layers for gemma2:27b 19 gpu layers for gemma2:9b gemma2l ``` FROM gemma2:27b PARAMETER num_gpu 6 ``` ``` ollama create gemma2l -f pathtofile/gemma2l ``` gemma2s ``` FROM gemma2 PARAMETER num_gpu 19 ``` ``` ollama create gemma2s -f pathtofile/gemma2s ```
GiteaMirror commented 2026-04-12 13:57:38 -05:00
Author
Owner
Copy Link
Copy Source

@Forevery1 commented on GitHub (Jun 29, 2024):

The issue has been resolved after upgrading to v0.1.48. Thank you very much

<!-- gh-comment-id:2198211881 --> @Forevery1 commented on GitHub (Jun 29, 2024): The issue has been resolved after upgrading to v0.1.48. Thank you very much
GiteaMirror commented 2026-04-12 13:57:39 -05:00
Author
Owner
Copy Link
Copy Source

@llwwei168 commented on GitHub (Jul 3, 2024):

root@585edd6d76c1:/# ollama run gemma2:latest
Error: llama runner process has terminated: signal: aborted (core dumped)
root@585edd6d76c1:/# ollama --version
ollama version is 0.1.42

Ubuntu 22.04
nvidia 4090D 24G

<!-- gh-comment-id:2206308067 --> @llwwei168 commented on GitHub (Jul 3, 2024): root@585edd6d76c1:/# ollama run gemma2:latest Error: llama runner process has terminated: signal: aborted (core dumped) root@585edd6d76c1:/# ollama --version ollama version is 0.1.42 Ubuntu 22.04 nvidia 4090D 24G
GiteaMirror commented 2026-04-12 13:57:40 -05:00
Author
Owner
Copy Link
Copy Source

@llwwei168 commented on GitHub (Jul 3, 2024):

docker logs ollama
log infomation is follow:
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma2'
llama_load_model_from_file: exception loading model
terminate called after throwing an instance of 'std::runtime_error'
what(): error loading model architecture: unknown model architecture: 'gemma2'
time=2024-07-03T14:34:13.541Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error"
time=2024-07-03T14:34:13.791Z level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) "

<!-- gh-comment-id:2206327940 --> @llwwei168 commented on GitHub (Jul 3, 2024): docker logs ollama log infomation is follow: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma2' llama_load_model_from_file: exception loading model terminate called after throwing an instance of 'std::runtime_error' what(): error loading model architecture: unknown model architecture: 'gemma2' time=2024-07-03T14:34:13.541Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error" time=2024-07-03T14:34:13.791Z level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) "
GiteaMirror commented 2026-04-12 13:57:40 -05:00
Author
Owner
Copy Link
Copy Source

@chirag-urb commented on GitHub (Jul 3, 2024):

The issue has been resolved after upgrading to v0.1.48. Upgrade your docker image.

<!-- gh-comment-id:2206774175 --> @chirag-urb commented on GitHub (Jul 3, 2024): The issue has been resolved after upgrading to v0.1.48. Upgrade your docker image.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels bug
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3349