github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8602] Deepseek-R1 671B - Segmentation Fault Bug #31324

New Issue
Closed
opened 2026-04-22 11:40:47 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-22 11:40:47 -05:00
Owner
Copy Link

Originally created by @Notbici on GitHub (Jan 27, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8602

What is the issue?

Hi,

I've been using the Deepseek-R1 671B model from Ollama on my 8x H100 machine and keep running into a segmentation fault, I've noticed that the frequency of the segfault happens the larger the context becomes.

I'm using the latest Ollama release.
Hardware Specs:

  • 8x H100 - 80GB SXM
  • Xeon Platinum 8468 (160c)
  • Micron 7450 ssd
  • 1548gb of ram
  • OS is ubuntu 22.04
  • CUDA: 12.6
  • NVIDIA driver: 560.35.05

Happy to test params or gather more data, I'm having a hard time working around this. The distilled models like the deepseek llama 70B work just fine.

server.err.log

Any advice is appreciated.

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.5.7

Originally created by @Notbici on GitHub (Jan 27, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8602 ### What is the issue? Hi, I've been using the Deepseek-R1 671B model from Ollama on my 8x H100 machine and keep running into a segmentation fault, I've noticed that the frequency of the segfault happens the larger the context becomes. I'm using the latest Ollama release. Hardware Specs: - 8x H100 - 80GB SXM - Xeon Platinum 8468 (160c) - Micron 7450 ssd - 1548gb of ram - OS is ubuntu 22.04 - CUDA: 12.6 - NVIDIA driver: 560.35.05 Happy to test params or gather more data, I'm having a hard time working around this. The distilled models like the deepseek llama 70B work just fine. [server.err.log](https://github.com/user-attachments/files/18554764/server.err.log) Any advice is appreciated. ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.7
GiteaMirror added the bug label 2026-04-22 11:40:47 -05:00
GiteaMirror commented 2026-04-22 11:40:50 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 27, 2025):

https://github.com/ollama/ollama/issues/5975

<!-- gh-comment-id:2615072167 --> @rick-github commented on GitHub (Jan 27, 2025): https://github.com/ollama/ollama/issues/5975
GiteaMirror commented 2026-04-22 11:40:53 -05:00
Author
Owner
Copy Link

@Notbici commented on GitHub (Jan 27, 2025):

#5975

Hey, thanks and the thread along with other adjacent threads spark a few headaches, could we get some idea on where the point of contention is with the solution, can we donate for developer time specific to this issue or is it a complicated issue with the implementation of llamacpp?

I see that there are some workarounds exist where cloning the model file and changing 2 params could ease the symptoms, can we deploy that workaround
to the ollama model provided with the box until llamacpp/ollama work on a concrete solution?

<!-- gh-comment-id:2615355610 --> @Notbici commented on GitHub (Jan 27, 2025): > [#5975](https://github.com/ollama/ollama/issues/5975) Hey, thanks and the thread along with other adjacent threads spark a few headaches, could we get some idea on where the point of contention is with the solution, can we donate for developer time specific to this issue or is it a complicated issue with the implementation of llamacpp? I see that there are some workarounds exist where cloning the model file and changing 2 params could ease the symptoms, can we deploy that workaround to the [ollama model](https://ollama.com/library/deepseek-r1:671b) provided with the box until llamacpp/ollama work on a concrete solution?
GiteaMirror commented 2026-04-22 11:40:54 -05:00
Author
Owner
Copy Link

@Notbici commented on GitHub (Jan 28, 2025):

For now this worked:

Modelfile

FROM deepseek-r1:671b
PARAMETER num_ctx 24576
PARAMETER num_predict 8192

ollama create deepseek-r1-fixed -f Modelfile

<!-- gh-comment-id:2618584828 --> @Notbici commented on GitHub (Jan 28, 2025): For now this worked: Modelfile ``` FROM deepseek-r1:671b PARAMETER num_ctx 24576 PARAMETER num_predict 8192 ``` > ollama create deepseek-r1-fixed -f Modelfile
GiteaMirror commented 2026-04-22 11:40:55 -05:00
Author
Owner
Copy Link

@daniel-adriano commented on GitHub (Feb 1, 2025):

Running into the same problem, with both deepseek-r1:671b and deepseek-r1:70b (machine has 16xV100-32GB)

For deepseek-r1:70b I seem to get away with modifying it to pretty large values:

num_ctx=131072
num_predict=32768

But for deepseek-r1:671b, anything similar to those VRAM amounts is prohibitive. So, I have tried:

PARAMETER num_ctx 4096
PARAMETER num_predict 2048

... but, predictably, I still run into the same problem pretty quickly, seemly due to the long responses that deepseek-r1 can frequently produce.

Is there a better fix? Like a setting preventing the context to grow beyond the allocated num_ctx and crashing the model? Anything else?

<!-- gh-comment-id:2628917325 --> @daniel-adriano commented on GitHub (Feb 1, 2025): Running into the same problem, with both deepseek-r1:671b and deepseek-r1:70b (machine has 16xV100-32GB) For deepseek-r1:70b I seem to get away with modifying it to pretty large values: `num_ctx=131072` `num_predict=32768` But for deepseek-r1:671b, anything similar to those VRAM amounts is prohibitive. So, I have tried: `PARAMETER num_ctx 4096` `PARAMETER num_predict 2048` ... but, predictably, I still run into the same problem pretty quickly, seemly due to the long responses that deepseek-r1 can frequently produce. Is there a better fix? Like a setting preventing the context to grow beyond the allocated num_ctx and crashing the model? Anything else?
GiteaMirror commented 2026-04-22 11:40:56 -05:00
Author
Owner
Copy Link

@ice6 commented on GitHub (Feb 27, 2025):

@daniel-adriano check here: https://github.com/ggml-org/llama.cpp/issues/8862

num_predict must be less than or equal to num_ctx / process count.
If not then when the context grows beyond what's been provisioned the program hard exits.
That feels like a bug to me. But apparently it's as designed.

<!-- gh-comment-id:2687126115 --> @ice6 commented on GitHub (Feb 27, 2025): @daniel-adriano check here: https://github.com/ggml-org/llama.cpp/issues/8862 > num_predict must be less than or equal to num_ctx / process count. > If not then when the context grows beyond what's been provisioned the program hard exits. > That feels like a bug to me. But apparently it's as designed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#31324
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?