github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #11280] Memory error when running gemma-3-12b-it-qat-q4_0-gguf #7438

New Issue
Open
opened 2026-04-12 19:31:07 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 19:31:07 -05:00
Owner
Copy Link

Originally created by @villivateur on GitHub (Jul 3, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11280

What is the issue?

When running hf.co/google/gemma-3-12b-it-qat-q4_0-gguf , there is an error:

Error: llama runner process has terminated: error:invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff6dad15619]

goroutine 50 [running]:
github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...)
        C:/a/ollama/ollama/ml/nn/convolution.go:10
github.com/ollama/ollama/model/models/gemma3.(*VisionModel).Forward(0xc000cf5c40, {0x7ff6dbc0b930, 0xc000fc7f00

But other models, such as gemma3 from ollama's official site, runs well.

Here is my environment:

  • Windows 11 pro, build 26100.4349
  • RTX5070 (12GB VRAM) with CUDA Version: 12.9
  • 32 GB RAM
  • ollama 0.9.5
Originally created by @villivateur on GitHub (Jul 3, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11280 ### What is the issue? When running hf.co/google/gemma-3-12b-it-qat-q4_0-gguf , there is an error: ``` Error: llama runner process has terminated: error:invalid memory address or nil pointer dereference [signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff6dad15619] goroutine 50 [running]: github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...) C:/a/ollama/ollama/ml/nn/convolution.go:10 github.com/ollama/ollama/model/models/gemma3.(*VisionModel).Forward(0xc000cf5c40, {0x7ff6dbc0b930, 0xc000fc7f00 ``` But other models, such as gemma3 from ollama's official site, runs well. Here is my environment: - Windows 11 pro, build 26100.4349 - RTX5070 (12GB VRAM) with CUDA Version: 12.9 - 32 GB RAM - ollama 0.9.5
GiteaMirror added the bug label 2026-04-12 19:31:07 -05:00
GiteaMirror commented 2026-04-12 19:31:09 -05:00
Author
Owner
Copy Link

@martijk commented on GitHub (Jul 4, 2025):

I have the same issue and I just want to add that this was not always the case. It stopped working at a certain point, but sadly I don't remember exactly when.

<!-- gh-comment-id:3035416379 --> @martijk commented on GitHub (Jul 4, 2025): I have the same issue and I just want to add that this was not always the case. It stopped working at a certain point, but sadly I don't remember exactly when.
GiteaMirror commented 2026-04-12 19:31:09 -05:00
Author
Owner
Copy Link

@villivateur commented on GitHub (Jul 4, 2025):

@martijk I meet this error every time I start running this model.

<!-- gh-comment-id:3036570873 --> @villivateur commented on GitHub (Jul 4, 2025): @martijk I meet this error every time I start running this model.
GiteaMirror commented 2026-04-12 19:31:10 -05:00
Author
Owner
Copy Link

@akoblentz commented on GitHub (Jul 13, 2025):

I am also having this issue. I tried the 27b and 4b. I have plenty of ram - m4max 64gb and can run the same models through LMStudio no problem with MLX.

`ollama run hf.co/google/gemma-3-4b-it-qat-q4_0-gguf [07/13/25| 5:17PM]
pulling manifest
pulling 76aed0a8285b: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 3.2 GB
pulling e0a42594d802: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 358 B
pulling 8c0fb064b019: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 851 MB
pulling c5157d17cceb: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 44 B
pulling 26c93cff8d91: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 629 B
verifying sha256 digest
writing manifest
success
Error: llama runner process has terminated: error:invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1053767c0]

goroutine 11 [running]:
github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...)
/Users/runner/work/ollama/ollama/ml/nn/convolution.go`

<!-- gh-comment-id:3067307784 --> @akoblentz commented on GitHub (Jul 13, 2025): I am also having this issue. I tried the 27b and 4b. I have plenty of ram - m4max 64gb and can run the same models through LMStudio no problem with MLX. `ollama run hf.co/google/gemma-3-4b-it-qat-q4_0-gguf [07/13/25| 5:17PM] pulling manifest pulling 76aed0a8285b: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 3.2 GB pulling e0a42594d802: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 358 B pulling 8c0fb064b019: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 851 MB pulling c5157d17cceb: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 44 B pulling 26c93cff8d91: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████▏ 629 B verifying sha256 digest writing manifest success Error: llama runner process has terminated: error:invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1053767c0] goroutine 11 [running]: github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...) /Users/runner/work/ollama/ollama/ml/nn/convolution.go`
GiteaMirror commented 2026-04-12 19:31:11 -05:00
Author
Owner
Copy Link

@shanbady commented on GitHub (Jul 18, 2025):

having the same issue on my raspberry pi. the 1b-it version works fine. 4b-it has same issue

<!-- gh-comment-id:3086591172 --> @shanbady commented on GitHub (Jul 18, 2025): having the same issue on my raspberry pi. the 1b-it version works fine. 4b-it has same issue
GiteaMirror commented 2026-04-12 19:31:11 -05:00
Author
Owner
Copy Link

@NeroYuki commented on GitHub (Jul 20, 2025):

same problem running 27b it q4_0 gguf on 64GB of RAM and RTX 4060Ti 16GB

time=2025-07-20T20:25:38.558+07:00 level=INFO source=server.go:135 msg="system memory" total="63.9 GiB" free="35.6 GiB" free_swap="23.1 GiB"
time=2025-07-20T20:25:38.574+07:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=63 layers.offload=28 layers.split="" memory.available="[13.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="24.6 GiB" memory.required.partial="13.2 GiB" memory.required.kv="2.7 GiB" memory.required.allocations="[13.2 GiB]" memory.weights.total="16.0 GiB" memory.weights.repeating="13.4 GiB" memory.weights.nonrepeating="2.6 GiB" memory.graph.full="4.1 GiB" memory.graph.partial="4.3 GiB" projector.weights="818.0 MiB" projector.graph="0 B"
time=2025-07-20T20:25:38.574+07:00 level=INFO source=server.go:218 msg="enabling flash attention"
time=2025-07-20T20:25:38.602+07:00 level=INFO source=server.go:438 msg="starting llama server" cmd="C:\\Users\\ADMIN\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model C:\\Users\\ADMIN\\.ollama\\models\\blobs\\sha256-7b131f721b95bd06ef4066e18ed2febed15ad6620f61c6eaf37832837ca109fe --ctx-size 64000 --batch-size 512 --n-gpu-layers 28 --threads 6 --flash-attn --kv-cache-type q8_0 --no-mmap --parallel 1 --port 57624"
time=2025-07-20T20:25:38.604+07:00 level=INFO source=sched.go:483 msg="loaded runners" count=1
time=2025-07-20T20:25:38.604+07:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding"
time=2025-07-20T20:25:38.604+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error"
time=2025-07-20T20:25:38.640+07:00 level=INFO source=runner.go:925 msg="starting ollama engine"
time=2025-07-20T20:25:38.642+07:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:57624"
time=2025-07-20T20:25:38.666+07:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_0 name="" description="" num_tensors=808 num_key_values=40
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
load_backend: loaded CUDA backend from C:\Users\ADMIN\AppData\Local\Programs\Ollama\lib\ollama\ggml-cuda.dll
load_backend: loaded CPU backend from C:\Users\ADMIN\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll
time=2025-07-20T20:25:38.755+07:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:359 msg="offloading 27 repeating layers to GPU"
time=2025-07-20T20:25:38.855+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model"
time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:363 msg="offloading output layer to CPU"
time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:375 msg="offloaded 28/63 layers to GPU"
time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:377 msg="model weights" buffer=CPU size="12.6 GiB"
time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:377 msg="model weights" buffer=CUDA0 size="6.1 GiB"
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff668c459d9]

goroutine 11 [running]:
github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...)
	C:/a/ollama/ollama/ml/nn/convolution.go:10
github.com/ollama/ollama/model/models/gemma3.(*VisionModel).Forward(0xc000d7afc0, {0x7ff669b3bc70, 0xc000fdd980}, {0x7ff669b48dc0?, 0xc000ea6000?})
	C:/a/ollama/ollama/model/models/gemma3/model_vision.go:91 +0x99
github.com/ollama/ollama/model/models/gemma3.(*Model).EncodeMultimodal(0xc000090420, {0x7ff669b3bc70, 0xc000fdd980}, {0xc001ee4000, 0x400436, 0x700000})
	C:/a/ollama/ollama/model/models/gemma3/model.go:110 +0x1c5
github.com/ollama/ollama/runner/ollamarunner.(*Server).reserveWorstCaseGraph(0xc0001670e0)
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:761 +0x2b0
github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc0001670e0, {0xc00003a1c0?, 0x0?}, {0x6, 0x0, 0x1c, {0x0, 0x0, 0x0}, 0x1}, ...)
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:865 +0x270
github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc0001670e0, {0x7ff669b33510, 0xc000340190}, {0xc00003a1c0?, 0x0?}, {0x6, 0x0, 0x1c, {0x0, 0x0, ...}, ...}, ...)
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11
time=2025-07-20T20:25:39.156+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error"
time=2025-07-20T20:25:39.184+07:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2"
time=2025-07-20T20:25:39.407+07:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:invalid memory address or nil pointer dereference\n[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff668c459d9]\n\ngoroutine 11 [running]:"
[GIN] 2025/07/20 - 20:25:39 | 500 |    1.0498729s |     192.168.1.9 | POST     "/api/generate"
<!-- gh-comment-id:3094521609 --> @NeroYuki commented on GitHub (Jul 20, 2025): same problem running 27b it q4_0 gguf on 64GB of RAM and RTX 4060Ti 16GB ``` time=2025-07-20T20:25:38.558+07:00 level=INFO source=server.go:135 msg="system memory" total="63.9 GiB" free="35.6 GiB" free_swap="23.1 GiB" time=2025-07-20T20:25:38.574+07:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=63 layers.offload=28 layers.split="" memory.available="[13.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="24.6 GiB" memory.required.partial="13.2 GiB" memory.required.kv="2.7 GiB" memory.required.allocations="[13.2 GiB]" memory.weights.total="16.0 GiB" memory.weights.repeating="13.4 GiB" memory.weights.nonrepeating="2.6 GiB" memory.graph.full="4.1 GiB" memory.graph.partial="4.3 GiB" projector.weights="818.0 MiB" projector.graph="0 B" time=2025-07-20T20:25:38.574+07:00 level=INFO source=server.go:218 msg="enabling flash attention" time=2025-07-20T20:25:38.602+07:00 level=INFO source=server.go:438 msg="starting llama server" cmd="C:\\Users\\ADMIN\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model C:\\Users\\ADMIN\\.ollama\\models\\blobs\\sha256-7b131f721b95bd06ef4066e18ed2febed15ad6620f61c6eaf37832837ca109fe --ctx-size 64000 --batch-size 512 --n-gpu-layers 28 --threads 6 --flash-attn --kv-cache-type q8_0 --no-mmap --parallel 1 --port 57624" time=2025-07-20T20:25:38.604+07:00 level=INFO source=sched.go:483 msg="loaded runners" count=1 time=2025-07-20T20:25:38.604+07:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding" time=2025-07-20T20:25:38.604+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error" time=2025-07-20T20:25:38.640+07:00 level=INFO source=runner.go:925 msg="starting ollama engine" time=2025-07-20T20:25:38.642+07:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:57624" time=2025-07-20T20:25:38.666+07:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_0 name="" description="" num_tensors=808 num_key_values=40 ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes load_backend: loaded CUDA backend from C:\Users\ADMIN\AppData\Local\Programs\Ollama\lib\ollama\ggml-cuda.dll load_backend: loaded CPU backend from C:\Users\ADMIN\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll time=2025-07-20T20:25:38.755+07:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:359 msg="offloading 27 repeating layers to GPU" time=2025-07-20T20:25:38.855+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model" time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:363 msg="offloading output layer to CPU" time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:375 msg="offloaded 28/63 layers to GPU" time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:377 msg="model weights" buffer=CPU size="12.6 GiB" time=2025-07-20T20:25:38.855+07:00 level=INFO source=ggml.go:377 msg="model weights" buffer=CUDA0 size="6.1 GiB" panic: runtime error: invalid memory address or nil pointer dereference [signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff668c459d9] goroutine 11 [running]: github.com/ollama/ollama/ml/nn.(*Conv2D).Forward(...) C:/a/ollama/ollama/ml/nn/convolution.go:10 github.com/ollama/ollama/model/models/gemma3.(*VisionModel).Forward(0xc000d7afc0, {0x7ff669b3bc70, 0xc000fdd980}, {0x7ff669b48dc0?, 0xc000ea6000?}) C:/a/ollama/ollama/model/models/gemma3/model_vision.go:91 +0x99 github.com/ollama/ollama/model/models/gemma3.(*Model).EncodeMultimodal(0xc000090420, {0x7ff669b3bc70, 0xc000fdd980}, {0xc001ee4000, 0x400436, 0x700000}) C:/a/ollama/ollama/model/models/gemma3/model.go:110 +0x1c5 github.com/ollama/ollama/runner/ollamarunner.(*Server).reserveWorstCaseGraph(0xc0001670e0) C:/a/ollama/ollama/runner/ollamarunner/runner.go:761 +0x2b0 github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc0001670e0, {0xc00003a1c0?, 0x0?}, {0x6, 0x0, 0x1c, {0x0, 0x0, 0x0}, 0x1}, ...) C:/a/ollama/ollama/runner/ollamarunner/runner.go:865 +0x270 github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc0001670e0, {0x7ff669b33510, 0xc000340190}, {0xc00003a1c0?, 0x0?}, {0x6, 0x0, 0x1c, {0x0, 0x0, ...}, ...}, ...) C:/a/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11 time=2025-07-20T20:25:39.156+07:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error" time=2025-07-20T20:25:39.184+07:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2" time=2025-07-20T20:25:39.407+07:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:invalid memory address or nil pointer dereference\n[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff668c459d9]\n\ngoroutine 11 [running]:" [GIN] 2025/07/20 - 20:25:39 | 500 | 1.0498729s | 192.168.1.9 | POST "/api/generate" ```
GiteaMirror commented 2026-04-12 19:31:12 -05:00
Author
Owner
Copy Link

@ashleyabraham commented on GitHub (Aug 29, 2025):

Any update or fix for this issue?

<!-- gh-comment-id:3237943838 --> @ashleyabraham commented on GitHub (Aug 29, 2025): Any update or fix for this issue?
GiteaMirror commented 2026-04-12 19:31:14 -05:00
Author
Owner
Copy Link

@xcjs commented on GitHub (Sep 3, 2025):

Just verifying that I just encountered this issue in the Windows version of Ollama with gemma3:12b-it-qat.

The system in question has:

  • A GPU with 12 GB of RAM
  • 64 GB of DDR5 RAM

I'm able to run gemma3:27b with no issue.

<!-- gh-comment-id:3247486146 --> @xcjs commented on GitHub (Sep 3, 2025): Just verifying that I just encountered this issue in the Windows version of Ollama with `gemma3:12b-it-qat`. The system in question has: * A GPU with 12 GB of RAM * 64 GB of DDR5 RAM I'm able to run `gemma3:27b` with no issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7438
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?