github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9699] EOF with Gemma3:27b | POST predict: Post "http://127.0.0.1:35737/completion": EOF (status code: 500) #32093

New Issue
Closed
opened 2026-04-22 13:01:13 -05:00 by GiteaMirror · 31 comments
GiteaMirror commented 2026-04-22 13:01:13 -05:00
Owner
Copy Link

Originally created by @VistritPandey on GitHub (Mar 12, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9699

Originally assigned to: @jessegross on GitHub.

What is the issue?

Getting the following error when running Gemma3:27b inside a script. The same script works for Gemma3:4b, Llama3.2:11b, Llava, and other vision models.

Relevant log output

POST predict: Post "http://127.0.0.1:35737/completion": EOF (status code: 500)
The port changed consistently, despite hardcoding it to port 11434:

http://127.0.0.1:35737/completion
http://127.0.0.1:39649/completion
http://127.0.0.1:39123/completion
http://127.0.0.1:38255/completion
http://127.0.0.1:38299/completion
http://127.0.0.1:43129/completion
http://127.0.0.1:44591/completion
http://127.0.0.1:33493/completion
http://127.0.0.1:42997/completion
http://127.0.0.1:42431/completion
http://127.0.0.1:33473/completion
http://127.0.0.1:33663/completion
http://127.0.0.1:40771/completion

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.6.0

Originally created by @VistritPandey on GitHub (Mar 12, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9699 Originally assigned to: @jessegross on GitHub. ### What is the issue? Getting the following error when running `Gemma3:27b` inside a script. The same script works for `Gemma3:4b`, `Llama3.2:11b`, `Llava`, and other vision models. ### Relevant log output ```shell POST predict: Post "http://127.0.0.1:35737/completion": EOF (status code: 500) The port changed consistently, despite hardcoding it to port 11434: http://127.0.0.1:35737/completion http://127.0.0.1:39649/completion http://127.0.0.1:39123/completion http://127.0.0.1:38255/completion http://127.0.0.1:38299/completion http://127.0.0.1:43129/completion http://127.0.0.1:44591/completion http://127.0.0.1:33493/completion http://127.0.0.1:42997/completion http://127.0.0.1:42431/completion http://127.0.0.1:33473/completion http://127.0.0.1:33663/completion http://127.0.0.1:40771/completion ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.6.0
GiteaMirror added the bug label 2026-04-22 13:01:13 -05:00
GiteaMirror commented 2026-04-22 13:01:15 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 12, 2025):

The port number changes because a new runner is created whenever a model is started. The reason for the runner restarts is likely due to OOMs with gemma3, see here. Long story short, reduce the memory footprint as described here.

gemma3 is a new model and ollama has just started the 0.6 series so there may be some fine-tuning to be done to get the new model properly integrated with the new runner.

<!-- gh-comment-id:2718957948 --> @rick-github commented on GitHub (Mar 12, 2025): The port number changes because a new runner is created whenever a model is started. The reason for the runner restarts is likely due to OOMs with gemma3, see [here](https://github.com/ollama/ollama/issues/9685). Long story short, reduce the memory footprint as described [here](https://github.com/ollama/ollama/issues/8597#issuecomment-2614533288). gemma3 is a new model and ollama has just started the 0.6 series so there may be some fine-tuning to be done to get the new model properly integrated with the new runner.
GiteaMirror commented 2026-04-22 13:01:16 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Mar 12, 2025):

Same issue, didn't have issues with Gemma2 27B. I have plenty of memory, 16GBs of VRAM and 64GBs of RAM and even with 4096 context it fails with just a few tokens as input but just hi or something short works

<!-- gh-comment-id:2719316732 --> @nonetrix commented on GitHub (Mar 12, 2025): Same issue, didn't have issues with Gemma2 27B. I have plenty of memory, 16GBs of VRAM and 64GBs of RAM and even with 4096 context it fails with just a few tokens as input but just `hi` or something short works
GiteaMirror commented 2026-04-22 13:01:18 -05:00
Author
Owner
Copy Link

@VistritPandey commented on GitHub (Mar 12, 2025):

Thanks for the suggestion @rick-github. I tried the steps in the above comment, but unfortunately, it did not work. Both 12B and 27B seem to be having the same issue. But it works on Mac, so guess I will work on Mac in the meantime

<!-- gh-comment-id:2719324612 --> @VistritPandey commented on GitHub (Mar 12, 2025): Thanks for the suggestion @rick-github. I tried the steps in the above comment, but unfortunately, it did not work. Both 12B and 27B seem to be having the same issue. But it works on Mac, so guess I will work on Mac in the meantime
GiteaMirror commented 2026-04-22 13:01:19 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 12, 2025):

I have found that setting GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 reduces crashing quite a lot.

<!-- gh-comment-id:2719330180 --> @rick-github commented on GitHub (Mar 12, 2025): I have found that setting `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` reduces crashing quite a lot.
GiteaMirror commented 2026-04-22 13:01:19 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Mar 13, 2025):

For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly

<!-- gh-comment-id:2722485402 --> @nonetrix commented on GitHub (Mar 13, 2025): For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly
GiteaMirror commented 2026-04-22 13:01:20 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Mar 14, 2025):

Can you try out the 0.6.1 prerelease? This should be fixed.

<!-- gh-comment-id:2725895783 --> @pdevine commented on GitHub (Mar 14, 2025): Can you try out the [0.6.1](https://github.com/ollama/ollama/releases/tag/v0.6.1) prerelease? This _should_ be fixed.
GiteaMirror commented 2026-04-22 13:01:21 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 14, 2025):

@pdevine gemma3 uses a /lot/ of system RAM (RSS) compared to other models. 0.6.1 has reduced that by 10%, but it hasn't made much of a difference to VRAM or KV cache allocation.

Image

<!-- gh-comment-id:2725973592 --> @rick-github commented on GitHub (Mar 14, 2025): @pdevine gemma3 uses a /lot/ of system RAM (RSS) compared to other models. 0.6.1 has reduced that by 10%, but it hasn't made much of a difference to VRAM or KV cache allocation. ![Image](https://github.com/user-attachments/assets/0d921427-0cf9-4c1d-b74b-606ba021ac48)
GiteaMirror commented 2026-04-22 13:01:22 -05:00
Author
Owner
Copy Link

@mehditahmasebi commented on GitHub (Mar 15, 2025):

Not fixed yet
in ollama version 0.6.1
ollama run gemma3:27b-it-q8_0
Error: Post "http://127.0.0.1:11434/api/generate": EOF

I have overall 64gb RTX vram

<!-- gh-comment-id:2726421035 --> @mehditahmasebi commented on GitHub (Mar 15, 2025): Not fixed yet in ollama version 0.6.1 ollama run gemma3:27b-it-q8_0 Error: Post "http://127.0.0.1:11434/api/generate": EOF I have overall 64gb RTX vram
GiteaMirror commented 2026-04-22 13:01:24 -05:00
Author
Owner
Copy Link

@AlpSha commented on GitHub (Mar 15, 2025):

For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly

How did you disable ROCm exactly? I'm having the same problem. Any help is appreciated.

<!-- gh-comment-id:2726562748 --> @AlpSha commented on GitHub (Mar 15, 2025): > For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly How did you disable ROCm exactly? I'm having the same problem. Any help is appreciated.
GiteaMirror commented 2026-04-22 13:01:24 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Mar 15, 2025):

For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly

How did you disable ROCm exactly? I'm having the same problem. Any help is appreciated.

For me I just disable it in my NixOS configs but that doesn't apply to everyone, just install the normal package instead of CUDA/ROCm one I guess? On Windows I have no idea at all. That will make it use CPU instead of GPU so it will be a hell of a lot slower, but like me if you have a crap ton of RAM and not much VRAM you can just bypass the problem entirely

<!-- gh-comment-id:2726641910 --> @nonetrix commented on GitHub (Mar 15, 2025): > > For me disabling ROCm also does the trick, it's allocating too much VRAM and crashing seemingly > > How did you disable ROCm exactly? I'm having the same problem. Any help is appreciated. For me I just disable it in my NixOS configs but that doesn't apply to everyone, just install the normal package instead of CUDA/ROCm one I guess? On Windows I have no idea at all. That will make it use CPU instead of GPU so it will be a hell of a lot slower, but like me if you have a crap ton of RAM and not much VRAM you can just bypass the problem entirely
GiteaMirror commented 2026-04-22 13:01:26 -05:00
Author
Owner
Copy Link

@AlpSha commented on GitHub (Mar 16, 2025):

For me I just disable it in my NixOS configs but that doesn't apply to everyone, just install the normal package instead of CUDA/ROCm one I guess? On Windows I have no idea at all. That will make it use CPU instead of GPU so it will be a hell of a lot slower, but like me if you have a crap ton of RAM and not much VRAM you can just bypass the problem entirely

For me, it is a bit different case. I have 2 GPUs. Only one of them supports ROCm. I can use both if I run it on Vulkan instead of ROCm. It is clear and easy to setup which runtime to use on LM Studio. However on Ollama I'm not sure how this is managed. That's why I asked. Thanks for your response. I use arch btw.

<!-- gh-comment-id:2727277548 --> @AlpSha commented on GitHub (Mar 16, 2025): > For me I just disable it in my NixOS configs but that doesn't apply to everyone, just install the normal package instead of CUDA/ROCm one I guess? On Windows I have no idea at all. That will make it use CPU instead of GPU so it will be a hell of a lot slower, but like me if you have a crap ton of RAM and not much VRAM you can just bypass the problem entirely For me, it is a bit different case. I have 2 GPUs. Only one of them supports ROCm. I can use both if I run it on Vulkan instead of ROCm. It is clear and easy to setup which runtime to use on LM Studio. However on Ollama I'm not sure how this is managed. That's why I asked. Thanks for your response. I use arch btw.
GiteaMirror commented 2026-04-22 13:01:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 16, 2025):

In previous versions it was possible to set OLLAMA_LLM_LIBRARY to control the runtime. However the runner architecture in the 0.6.0 series has undergone a lot of surgery and this doesn't appear to work at the moment.

<!-- gh-comment-id:2727594723 --> @rick-github commented on GitHub (Mar 16, 2025): In previous versions it was possible to set [`OLLAMA_LLM_LIBRARY`](https://github.com/ollama/ollama/blob/2c8b4846437747bd23e7a176f83011e39ec2128b/envconfig/config.go#L247) to control the runtime. However the runner architecture in the 0.6.0 series has undergone a lot of surgery and this doesn't appear to work at the moment.
GiteaMirror commented 2026-04-22 13:01:28 -05:00
Author
Owner
Copy Link

@joyava commented on GitHub (Mar 17, 2025):

In short, it means your computer does not have enough RAM. If 27B fails, try quantize version. If 27B quantize version fails, try 12B model. If Gemma3:12B fails, try quantize version. If it fails, try Gemma3:4B. If it fail, try quantize version. If it fails, try Gemmae:1b model. I don't expect to have failure at this 1B model. Gemma3:12b-it-q4_K_M ran successful on 36GB RAM laptop without Nvidia GPU. If you guys have success with Gemma3, please tell us your physical RAM.

<!-- gh-comment-id:2727782590 --> @joyava commented on GitHub (Mar 17, 2025): In short, it means your computer does not have enough RAM. If 27B fails, try quantize version. If 27B quantize version fails, try 12B model. If Gemma3:12B fails, try quantize version. If it fails, try Gemma3:4B. If it fail, try quantize version. If it fails, try Gemmae:1b model. I don't expect to have failure at this 1B model. Gemma3:12b-it-q4_K_M ran successful on 36GB RAM laptop without Nvidia GPU. If you guys have success with Gemma3, please tell us your physical RAM.
GiteaMirror commented 2026-04-22 13:01:28 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Mar 17, 2025):

In short, it means your computer does not have enough RAM. If 27B fails, try quantize version. If 27B quantize version fails, try 12B model. If Gemma3:12B fails, try quantize version. If it fails, try Gemma3:4B. If it fail, try quantize version. If it fails, try Gemmae:1b model. I don't expect to have failure at this 1B model. Gemma3:12b-it-q4_K_M ran successful on 36GB RAM laptop without Nvidia GPU. If you guys have success with Gemma3, please tell us your physical RAM.

No, the issue is it's not falling back to system ram correctly for me and filling the GPU waaaaay too much. I can run even llama 3.3 70B, but I'm pretty sure Gemma 27B is using the Go backend hence the issue

<!-- gh-comment-id:2727828786 --> @nonetrix commented on GitHub (Mar 17, 2025): > In short, it means your computer does not have enough RAM. If 27B fails, try quantize version. If 27B quantize version fails, try 12B model. If Gemma3:12B fails, try quantize version. If it fails, try Gemma3:4B. If it fail, try quantize version. If it fails, try Gemmae:1b model. I don't expect to have failure at this 1B model. Gemma3:12b-it-q4_K_M ran successful on 36GB RAM laptop without Nvidia GPU. If you guys have success with Gemma3, please tell us your physical RAM. No, the issue is it's not falling back to system ram correctly for me and filling the GPU waaaaay too much. I can run even llama 3.3 70B, but I'm pretty sure Gemma 27B is using the Go backend hence the issue
GiteaMirror commented 2026-04-22 13:01:29 -05:00
Author
Owner
Copy Link

@notolac commented on GitHub (Mar 17, 2025):

I am using ollama 0.6.1.

Installed on ubuntu 24.04 with AMD 7800XT 16gb VRAM GPU, R9 7900X3D CPU, 64GB RAM latest official AMD driver and tested and running R1 32b and qwen2.5 of 32b without problems, instead with gemma3:27b I get the error:

Error: POST predict: Post “http://127.0.0.1:43657/completion”: EOF

But, On MacBook Pro with M2 Max, 32gb OS Sequoia 15.3.2 ollama 0.6.1
Yes, it works without problem.

<!-- gh-comment-id:2730612990 --> @notolac commented on GitHub (Mar 17, 2025): I am using ollama 0.6.1. Installed on ubuntu 24.04 with AMD 7800XT 16gb VRAM GPU, R9 7900X3D CPU, 64GB RAM latest official AMD driver and tested and running R1 32b and qwen2.5 of 32b without problems, instead with gemma3:27b I get the error: Error: POST predict: Post “http://127.0.0.1:43657/completion”: EOF But, On MacBook Pro with M2 Max, 32gb OS Sequoia 15.3.2 ollama 0.6.1 Yes, it works without problem.
GiteaMirror commented 2026-04-22 13:01:30 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Mar 17, 2025):

Sorry about this guys. There are some fixes for the memory issues coming out with 0.6.2 that we've been working through. This is just some teething pains with the new Ollama engine (vs. using llama.cpp). We're hoping to get 0.6.2 out today or tomorrow.

<!-- gh-comment-id:2730750032 --> @pdevine commented on GitHub (Mar 17, 2025): Sorry about this guys. There are some fixes for the memory issues coming out with `0.6.2` that we've been working through. This is just some teething pains with the new Ollama engine (vs. using llama.cpp). We're hoping to get `0.6.2` out today or tomorrow.
GiteaMirror commented 2026-04-22 13:01:31 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Mar 17, 2025):

Sorry about this guys. There are some fixes for the memory issues coming out with 0.6.2 that we've been working through. This is just some teething pains with the new Ollama engine (vs. using llama.cpp). We're hoping to get 0.6.2 out today or tomorrow.

No need to apologize :)

<!-- gh-comment-id:2730897684 --> @nonetrix commented on GitHub (Mar 17, 2025): > Sorry about this guys. There are some fixes for the memory issues coming out with `0.6.2` that we've been working through. This is just some teething pains with the new Ollama engine (vs. using llama.cpp). We're hoping to get `0.6.2` out today or tomorrow. No need to apologize :)
GiteaMirror commented 2026-04-22 13:01:32 -05:00
Author
Owner
Copy Link

@RomanMunar commented on GitHub (Mar 19, 2025):

The update resolved this issue on my end.

<!-- gh-comment-id:2735247780 --> @RomanMunar commented on GitHub (Mar 19, 2025): The update resolved this issue on my end.
GiteaMirror commented 2026-04-22 13:01:32 -05:00
Author
Owner
Copy Link

@Master-Pr0grammer commented on GitHub (Mar 19, 2025):

I am still having the issue after the update. Under certain images, I just get EOF, I have 12gb vram on my one gpu, or 16 in total if split, doesn't matter if I use gemma 3 12b, 4b, or even 4b full precision, I just get EOF under certain images, and I have more than enough VRAM for the 4b quantized.

But for every model, they all work perfectly for text only.

EDIT: It definitely is a VRAM issue, I forgot that I am running the models at the longest possible context size I can fit on a single GPU (12gb), this is what is causing the crash, if I set it to 2,048 it works. but only crashes with images.

I think the crash is related to trying to allocate extra room for tokenized/processed images in the context that arent accounted for in the num_ctx option

<!-- gh-comment-id:2735258071 --> @Master-Pr0grammer commented on GitHub (Mar 19, 2025): I am still having the issue after the update. Under certain images, I just get EOF, I have 12gb vram on my one gpu, or 16 in total if split, doesn't matter if I use gemma 3 12b, 4b, or even 4b full precision, I just get EOF under certain images, and I have more than enough VRAM for the 4b quantized. But for every model, they all work perfectly for text only. EDIT: It definitely is a VRAM issue, I forgot that I am running the models at the longest possible context size I can fit on a single GPU (12gb), this is what is causing the crash, if I set it to 2,048 it works. but only crashes with images. I think the crash is related to trying to allocate extra room for tokenized/processed images in the context that arent accounted for in the num_ctx option
GiteaMirror commented 2026-04-22 13:01:33 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Mar 19, 2025):

@Master-Pr0grammer Thanks for updating us on that. I think we can potentially offload the kvcache onto system memory to free up the GPU to hold the model which should allow you to have a much longer context. It wouldn't be as fast as putting the kvcache into VRAM, but still useful I think.

<!-- gh-comment-id:2735342935 --> @pdevine commented on GitHub (Mar 19, 2025): @Master-Pr0grammer Thanks for updating us on that. I think we can potentially offload the kvcache onto system memory to free up the GPU to hold the model which should allow you to have a much longer context. It wouldn't be as fast as putting the kvcache into VRAM, but still useful I think.
GiteaMirror commented 2026-04-22 13:01:33 -05:00
Author
Owner
Copy Link

@Suletta-Majo commented on GitHub (Mar 29, 2025):

I am trying out ollama from the termux repository on my smartphone.
I found a language model that shows almost the same error as this issue, and I am reporting it here.
The language models are hf.co/Felladrin/gguf-flan-t5-small:Q3_K_S and hf.co/Felladrin/gguf-flan-t5-small:Q3_K_M.
here huggingface.co
Both are lightweight and seem to meet the required storage, so I can get to the question prompt,
but when I enter something like hello and confirm, the following error appears:
Error: POST predict: Post "http://127.0.0.1:38003/completion": EOF
Other models that work smoothly in my environment include Qwen_0.5_python_codes_mbpp_GGUF:Q4_K_M. :)
The standard version of qwen0.5 is when using gradually becoming heavy and unusabled my phone case.
I'm sorry if this is just a basic problem that has nothing to do with the issue.

Snapdragon 695 (adreno619), I don't use vscode.
The remaining storage is 4GB, and the system storage is about 1.7GB.

<!-- gh-comment-id:2763272669 --> @Suletta-Majo commented on GitHub (Mar 29, 2025): I am trying out ollama from the termux repository on my smartphone. I found a language model that shows almost the same error as this issue, and I am reporting it here. The language models are `hf.co/Felladrin/gguf-flan-t5-small:Q3_K_S` and `hf.co/Felladrin/gguf-flan-t5-small:Q3_K_M`. [here huggingface.co](https://huggingface.co/Felladrin/gguf-flan-t5-small) Both are lightweight and seem to meet the required storage, so I can get to the question prompt, but when I enter something like hello and confirm, the following error appears: `Error: POST predict: Post "http://127.0.0.1:38003/completion": EOF` Other models that work smoothly in my environment include Qwen_0.5_python_codes_mbpp_GGUF:Q4_K_M. :) The standard version of qwen0.5 is when using gradually becoming heavy and unusabled my phone case. I'm sorry if this is just a basic problem that has nothing to do with the issue. _Snapdragon 695 (adreno619), I don't use vscode. The remaining storage is 4GB, and the system storage is about 1.7GB._
GiteaMirror commented 2026-04-22 13:01:34 -05:00
Author
Owner
Copy Link

@jessegross commented on GitHub (Apr 14, 2025):

I think there are several issues here, some of which have been fixed. I believe there is one remaining issue, which is the one described in #10041

<!-- gh-comment-id:2802965622 --> @jessegross commented on GitHub (Apr 14, 2025): I think there are several issues here, some of which have been fixed. I believe there is one remaining issue, which is the one described in #10041
GiteaMirror commented 2026-04-22 13:01:35 -05:00
Author
Owner
Copy Link

@kripper commented on GitHub (Apr 27, 2025):

Same issue with:

  • ollama version 0.6.6
  • gemma3:27b-it-qat
  • CPU only
  • over 270 GB free RAM
  • only fails with contexts larget then aprox. 10 lines of text, but works fine with 1 line of text.
  • HP DL 380p
  • 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
  • Oracle Linux 8

No problem with other models (ie deepseek-r1:8b)

<!-- gh-comment-id:2833587797 --> @kripper commented on GitHub (Apr 27, 2025): Same issue with: - ollama version 0.6.6 - `gemma3:27b-it-qat` - **CPU only** - over 270 GB free RAM - only fails with contexts larget then aprox. 10 lines of text, but works fine with 1 line of text. - HP DL 380p - 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz - Oracle Linux 8 No problem with other models (ie `deepseek-r1:8b`)
GiteaMirror commented 2026-04-22 13:01:35 -05:00
Author
Owner
Copy Link

@littlebull commented on GitHub (Apr 27, 2025):

Same for me

<!-- gh-comment-id:2833588597 --> @littlebull commented on GitHub (Apr 27, 2025): Same for me
GiteaMirror commented 2026-04-22 13:01:36 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Apr 27, 2025):

@kripper @littlebull can you give more details about your systems? What OS/CPU type?

<!-- gh-comment-id:2833592743 --> @pdevine commented on GitHub (Apr 27, 2025): @kripper @littlebull can you give more details about your systems? What OS/CPU type?
GiteaMirror commented 2026-04-22 13:01:37 -05:00
Author
Owner
Copy Link

@kripper commented on GitHub (Apr 27, 2025):

@kripper @littlebull can you give more details about your systems? What OS/CPU type?

  • HP DL 380p
  • 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
  • Oracle Linux 8
<!-- gh-comment-id:2833604832 --> @kripper commented on GitHub (Apr 27, 2025): > [@kripper](https://github.com/kripper) [@littlebull](https://github.com/littlebull) can you give more details about your systems? What OS/CPU type? - HP DL 380p - 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz - Oracle Linux 8
GiteaMirror commented 2026-04-22 13:01:38 -05:00
Author
Owner
Copy Link

@littlebull commented on GitHub (Apr 28, 2025):

Hi @pdevine hope that helps:

  • AMD Ryzen 7 8845HS w/ Radeon 780M Graphics
  • Win 11 Home
  • 96GB RAM

Tried with gemma3:12b and gemma3:12b-it-qat.

Others like granite-3.3:8b work.

The logs of the crash:

time=2025-04-28T07:34:45.240+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-04-28T07:34:45.296+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-04-28T07:34:45.354+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-04-28T07:34:45.358+02:00 level=INFO source=server.go:105 msg="system memory" total="93.8 GiB" free="14.7 GiB" free_swap="46.2 GiB"
time=2025-04-28T07:34:45.360+02:00 level=INFO source=server.go:138 msg=offload library=cpu layers.requested=-1 layers.model=49 layers.offload=0 layers.split="" memory.available="[14.8 GiB]" memory.gpu_overhead="0 B" memory.required.full="9.8 GiB" memory.required.partial="0 B" memory.required.kv="608.0 MiB" memory.required.allocations="[9.8 GiB]" memory.weights.total="6.8 GiB" memory.weights.repeating="6.0 GiB" memory.weights.nonrepeating="787.5 MiB" memory.graph.full="519.5 MiB" memory.graph.partial="1.3 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
time=2025-04-28T07:34:45.488+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-04-28T07:34:45.495+02:00 level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.add_eot_token default=false
time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.attention.layer_norm_rms_epsilon default=9.999999974752427e-07
time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.local.freq_base default=10000
time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.global.freq_base default=1e+06
time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.freq_scale default=1
time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.mm_tokens_per_image default=256
time=2025-04-28T07:34:45.514+02:00 level=INFO source=server.go:405 msg="starting llama server" cmd="C:\Users\julia\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --model C:\Users\julia\.ollama\models\blobs\sha256-e8ad13eff07a78d89926e9e8b882317d082ef5bf9768ad7b50fcdbbcd63748de --ctx-size 2048 --batch-size 512 --threads 8 --no-mmap --parallel 1 --port 65080"
time=2025-04-28T07:34:45.529+02:00 level=INFO source=sched.go:451 msg="loaded runners" count=1
time=2025-04-28T07:34:45.529+02:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding"
time=2025-04-28T07:34:45.530+02:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error"
time=2025-04-28T07:34:46.513+02:00 level=INFO source=runner.go:866 msg="starting ollama engine"
time=2025-04-28T07:34:46.536+02:00 level=INFO source=runner.go:929 msg="Server listening on 127.0.0.1:65080"
[GIN] 2025/04/28 - 07:34:46 | 200 | 0s | 192.168.178.39 | GET "/"
time=2025-04-28T07:34:46.607+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-04-28T07:34:46.610+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.name default=""
time=2025-04-28T07:34:46.610+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.description default=""
time=2025-04-28T07:34:46.610+02:00 level=INFO source=ggml.go:72 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1065 num_key_values=37
load_backend: loaded CPU backend from C:\Users\julia\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-icelake.dll
time=2025-04-28T07:34:46.632+02:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(clang)
time=2025-04-28T07:34:46.636+02:00 level=INFO source=ggml.go:298 msg="model weights" buffer=CPU size="8.3 GiB"
time=2025-04-28T07:34:46.784+02:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model"
time=2025-04-28T07:34:48.516+02:00 level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.add_eot_token default=false
time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.attention.layer_norm_rms_epsilon default=9.999999974752427e-07
time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.local.freq_base default=10000
time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.global.freq_base default=1e+06
time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.freq_scale default=1
time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.mm_tokens_per_image default=256
time=2025-04-28T07:34:48.749+02:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="95.0 MiB"
time=2025-04-28T07:34:48.791+02:00 level=INFO source=server.go:619 msg="llama runner started in 3.26 seconds"
ggml-alloc.c:819: GGML_ASSERT(talloc->buffer_id >= 0) failed
[GIN] 2025/04/28 - 07:34:49 | 500 | 4.3680197s | 192.168.178.201 | POST "/api/chat"
time=2025-04-28T07:34:50.089+02:00 level=ERROR source=server.go:449 msg="llama runner terminated" error="exit status 0xc0000409"

<!-- gh-comment-id:2834037879 --> @littlebull commented on GitHub (Apr 28, 2025): Hi @pdevine hope that helps: - AMD Ryzen 7 8845HS w/ Radeon 780M Graphics - Win 11 Home - 96GB RAM Tried with gemma3:12b and gemma3:12b-it-qat. Others like granite-3.3:8b work. The logs of the crash: time=2025-04-28T07:34:45.240+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-04-28T07:34:45.296+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-04-28T07:34:45.354+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-04-28T07:34:45.358+02:00 level=INFO source=server.go:105 msg="system memory" total="93.8 GiB" free="14.7 GiB" free_swap="46.2 GiB" time=2025-04-28T07:34:45.360+02:00 level=INFO source=server.go:138 msg=offload library=cpu layers.requested=-1 layers.model=49 layers.offload=0 layers.split="" memory.available="[14.8 GiB]" memory.gpu_overhead="0 B" memory.required.full="9.8 GiB" memory.required.partial="0 B" memory.required.kv="608.0 MiB" memory.required.allocations="[9.8 GiB]" memory.weights.total="6.8 GiB" memory.weights.repeating="6.0 GiB" memory.weights.nonrepeating="787.5 MiB" memory.graph.full="519.5 MiB" memory.graph.partial="1.3 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" time=2025-04-28T07:34:45.488+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-04-28T07:34:45.495+02:00 level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.add_eot_token default=false time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.attention.layer_norm_rms_epsilon default=9.999999974752427e-07 time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.local.freq_base default=10000 time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.global.freq_base default=1e+06 time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.freq_scale default=1 time=2025-04-28T07:34:45.504+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.mm_tokens_per_image default=256 time=2025-04-28T07:34:45.514+02:00 level=INFO source=server.go:405 msg="starting llama server" cmd="C:\\Users\\julia\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model C:\\Users\\julia\\.ollama\\models\\blobs\\sha256-e8ad13eff07a78d89926e9e8b882317d082ef5bf9768ad7b50fcdbbcd63748de --ctx-size 2048 --batch-size 512 --threads 8 --no-mmap --parallel 1 --port 65080" time=2025-04-28T07:34:45.529+02:00 level=INFO source=sched.go:451 msg="loaded runners" count=1 time=2025-04-28T07:34:45.529+02:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding" time=2025-04-28T07:34:45.530+02:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error" time=2025-04-28T07:34:46.513+02:00 level=INFO source=runner.go:866 msg="starting ollama engine" time=2025-04-28T07:34:46.536+02:00 level=INFO source=runner.go:929 msg="Server listening on 127.0.0.1:65080" [GIN] 2025/04/28 - 07:34:46 | 200 | 0s | 192.168.178.39 | GET "/" time=2025-04-28T07:34:46.607+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-04-28T07:34:46.610+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.name default="" time=2025-04-28T07:34:46.610+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.description default="" time=2025-04-28T07:34:46.610+02:00 level=INFO source=ggml.go:72 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1065 num_key_values=37 load_backend: loaded CPU backend from C:\Users\julia\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-icelake.dll time=2025-04-28T07:34:46.632+02:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(clang) time=2025-04-28T07:34:46.636+02:00 level=INFO source=ggml.go:298 msg="model weights" buffer=CPU size="8.3 GiB" time=2025-04-28T07:34:46.784+02:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model" time=2025-04-28T07:34:48.516+02:00 level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.add_eot_token default=false time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.attention.layer_norm_rms_epsilon default=9.999999974752427e-07 time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.local.freq_base default=10000 time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.global.freq_base default=1e+06 time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.rope.freq_scale default=1 time=2025-04-28T07:34:48.523+02:00 level=WARN source=ggml.go:152 msg="key not found" key=gemma3.mm_tokens_per_image default=256 time=2025-04-28T07:34:48.749+02:00 level=INFO source=ggml.go:556 msg="compute graph" backend=CPU buffer_type=CPU size="95.0 MiB" time=2025-04-28T07:34:48.791+02:00 level=INFO source=server.go:619 msg="llama runner started in 3.26 seconds" ggml-alloc.c:819: GGML_ASSERT(talloc->buffer_id >= 0) failed [GIN] 2025/04/28 - 07:34:49 | 500 | 4.3680197s | 192.168.178.201 | POST "/api/chat" time=2025-04-28T07:34:50.089+02:00 level=ERROR source=server.go:449 msg="llama runner terminated" error="exit status 0xc0000409"
GiteaMirror commented 2026-04-22 13:01:39 -05:00
Author
Owner
Copy Link

@DraculaVladimir commented on GitHub (Apr 28, 2025):

I am also experiencing the same issue with version 0.6.7-rc0. Running it in Docker.

When i pull ollama:latest it pulls version 5.7

<!-- gh-comment-id:2834430012 --> @DraculaVladimir commented on GitHub (Apr 28, 2025): I am also experiencing the same issue with version 0.6.7-rc0. Running it in Docker. When i pull `ollama:latest` it pulls version 5.7
GiteaMirror commented 2026-04-22 13:01:39 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Apr 28, 2025):

I believe this might be the same as #10410 and is related to us updating GGML in 0.6.6.

<!-- gh-comment-id:2835996087 --> @pdevine commented on GitHub (Apr 28, 2025): I believe this might be the same as #10410 and is related to us updating GGML in `0.6.6`.
GiteaMirror commented 2026-04-22 13:01:40 -05:00
Author
Owner
Copy Link

@jWeck-Software commented on GitHub (May 14, 2025):

Same, crashing over and over. -_-

Ubuntu 22.04 LTS
RTX 4090 24GB
System 32GB
Ollama 0.6.8

gemma3:27b

{"error":"POST predict: Post \"http://127.0.0.1:42367/completion\": EOF"}

Did not happen with older Version of Ollama < 0.6.8, but Ollama forced me to upgrade, to be able to download qwen3........................... which also did not work at all.................... mismatch download again!

<!-- gh-comment-id:2880080311 --> @jWeck-Software commented on GitHub (May 14, 2025): Same, crashing over and over. -_- Ubuntu 22.04 LTS RTX 4090 24GB System 32GB Ollama 0.6.8 gemma3:27b ``` {"error":"POST predict: Post \"http://127.0.0.1:42367/completion\": EOF"} ``` Did not happen with older Version of Ollama < 0.6.8, but Ollama forced me to upgrade, to be able to download qwen3........................... which also did not work at all.................... mismatch download again!
GiteaMirror commented 2026-04-22 13:01:41 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 14, 2025):

@jWeck-Software can you open a new issue for this? I don't think it's probably related to this older issue.

<!-- gh-comment-id:2881542989 --> @pdevine commented on GitHub (May 14, 2025): @jWeck-Software can you open a new issue for this? I don't think it's probably related to this older issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#32093
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?