github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #1236] deepseek-coder:6.7b cuda error #26389

New Issue
Closed
opened 2026-04-22 02:39:04 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-22 02:39:04 -05:00
Owner
Copy Link

Originally created by @SAXN-SYNX on GitHub (Nov 22, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/1236

-- ollama falls back to CPU for deepseek-coder:6.7b
-- Any other model runs just fine with cuda with nvida 1660 ti .

-- Meanwhile i can run openchat:latest and zephyr:lastest with cuda just fine.

CUDA error 2 at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/gguf/ggml-cuda.cu:7358: out of memory
current device: 0
2023/11/22 11:37:21 llama.go:430: 2 at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/gguf/ggml-cuda.cu:7358: out of memory
current device: 0

Originally created by @SAXN-SYNX on GitHub (Nov 22, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/1236 -- ollama falls back to CPU for deepseek-coder:6.7b -- Any other model runs just fine with cuda with nvida 1660 ti . -- Meanwhile i can run openchat:latest and zephyr:lastest with cuda just fine. CUDA error 2 at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/gguf/ggml-cuda.cu:7358: out of memory current device: 0 2023/11/22 11:37:21 llama.go:430: 2 at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/gguf/ggml-cuda.cu:7358: out of memory current device: 0
GiteaMirror commented 2026-04-22 02:39:06 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Nov 22, 2023):

Hi @SAXN-SYNX, it's not obvious but deepseek coder has a large context window so it will take more memory than other 7B models, it will require about 8GB. I see the 1660TI has 6GB, so that should be the issue.

<!-- gh-comment-id:1823070174 --> @BruceMacD commented on GitHub (Nov 22, 2023): Hi @SAXN-SYNX, it's not obvious but deepseek coder has a large context window so it will take more memory than other 7B models, it will require about 8GB. I see the 1660TI has 6GB, so that should be the issue.
GiteaMirror commented 2026-04-22 02:39:07 -05:00
Author
Owner
Copy Link

@madsamjp commented on GitHub (Nov 24, 2023):

I have the same issue. My GPU has 8GB of VRAM. Is it not possible to run on both GPU and CPU like other larger 13b/30b models?

<!-- gh-comment-id:1825791213 --> @madsamjp commented on GitHub (Nov 24, 2023): I have the same issue. My GPU has 8GB of VRAM. Is it not possible to run on both GPU and CPU like other larger 13b/30b models?
GiteaMirror commented 2026-04-22 02:39:07 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Nov 24, 2023):

@madsamjp it's pushing it on 8GB of VRAM so it may or may not run depending on the memory you have available at the time.

<!-- gh-comment-id:1826055431 --> @BruceMacD commented on GitHub (Nov 24, 2023): @madsamjp it's pushing it on 8GB of VRAM so it may or may not run depending on the memory you have available at the time.
GiteaMirror commented 2026-04-22 02:39:08 -05:00
Author
Owner
Copy Link

@Dav1nx1 commented on GitHub (Dec 1, 2023):

I have 12 GB of ram, and I have the same error with this model @SAXN-SYNX

<!-- gh-comment-id:1836413013 --> @Dav1nx1 commented on GitHub (Dec 1, 2023): I have 12 GB of ram, and I have the same error with this model @SAXN-SYNX
GiteaMirror commented 2026-04-22 02:39:10 -05:00
Author
Owner
Copy Link

@SAXN-SYNX commented on GitHub (Dec 2, 2023):

@BruceMacD ??

<!-- gh-comment-id:1837101754 --> @SAXN-SYNX commented on GitHub (Dec 2, 2023): @BruceMacD ??
GiteaMirror commented 2026-04-22 02:39:11 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Dec 4, 2023):

It still looks like not enough memory to me, for reference I'm running it locally and it uses 8.10GB. If you have more than that try looking at the available memory before running the model, it's very possible that not all memory is available.

My bad y'all just reproduced this, it is either a linux of GPU issue, I'll take a look at this when I have a free moment.

<!-- gh-comment-id:1839509740 --> @BruceMacD commented on GitHub (Dec 4, 2023): ~It still looks like not enough memory to me, for reference I'm running it locally and it uses 8.10GB. If you have more than that try looking at the available memory before running the model, it's very possible that not all memory is available.~ My bad y'all just reproduced this, it is either a linux of GPU issue, I'll take a look at this when I have a free moment.
GiteaMirror commented 2026-04-22 02:39:12 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Mar 12, 2024):

Hi @SAXN-SYNX this should be fixed in the recent versions of Ollama (current version is 0.1.28, and 0.1.29 is in pre-release).

Closing this for now. Sorry about this. Please let me know if anyone is still encountering problems. Thank you so much

<!-- gh-comment-id:1992501768 --> @mchiang0610 commented on GitHub (Mar 12, 2024): Hi @SAXN-SYNX this should be fixed in the recent versions of Ollama (current version is 0.1.28, and 0.1.29 is in pre-release). Closing this for now. Sorry about this. Please let me know if anyone is still encountering problems. Thank you so much
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#26389
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?