github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #12612] Unable to allocate VRAM or system RAM despite both reporting enough available #54886

New Issue
Open
opened 2026-04-29 07:48:28 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-29 07:48:28 -05:00
Owner
Copy Link

Originally created by @dwdali on GitHub (Oct 14, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12612

What is the issue?

I'm setting ollama to run deepseek:8b with my Nvidia RTX 2080ti, but seems GPU VRAM malloc always fail... but actually I don't run any other programs on VRAM almost. Could you please help?
here is my server.log , wish it will help.

Relevant log output

OS

Windows

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @dwdali on GitHub (Oct 14, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12612 ### What is the issue? I'm setting ollama to run deepseek:8b with my Nvidia RTX 2080ti, but seems GPU VRAM malloc always fail... but actually I don't run any other programs on VRAM almost. Could you please help? here is my [server.log](https://github.com/user-attachments/files/22908501/server.log) , wish it will help. ### Relevant log output ```shell ``` ### OS Windows ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-29 07:48:28 -05:00
GiteaMirror commented 2026-04-29 07:48:30 -05:00
Author
Owner
Copy Link

@dwdali commented on GitHub (Oct 14, 2025):

Another thing, my ollama version is 0.12.3.

<!-- gh-comment-id:3402590521 --> @dwdali commented on GitHub (Oct 14, 2025): Another thing, my ollama version is 0.12.3.
GiteaMirror commented 2026-04-29 07:48:34 -05:00
Author
Owner
Copy Link

@jessegross commented on GitHub (Oct 14, 2025):

There is something surprising about your environment. From Ollama's perspective, this is what happens:

  1. Free memory is reported as 18.3G VRAM (taking into account the 2G overhead you applied) and 8.2G system memory (plus 5.5G swap).
  2. We successfully allocate 4.5G of VRAM but fail to allocate the remaining 676M that we need.
  3. We can accommodate some misreporting of free VRAM but since the attempted allocation is already so much smaller than the reported free VRAM, we switch to system memory allocations.
  4. We attempt to allocate 4.9G on system RAM, which fails despite being much lower than the available memory.
  5. Ollama gives up and returns "memory layout cannot be allocated".

You might want to try fully removing the Ollama installation, including any libraries and reinstalling. Sometimes old DLLs can have odd results. Otherwise, it might be something strange going on with your system.

<!-- gh-comment-id:3403692804 --> @jessegross commented on GitHub (Oct 14, 2025): There is something surprising about your environment. From Ollama's perspective, this is what happens: 1. Free memory is reported as 18.3G VRAM (taking into account the 2G overhead you applied) and 8.2G system memory (plus 5.5G swap). 2. We successfully allocate 4.5G of VRAM but fail to allocate the remaining 676M that we need. 3. We can accommodate some misreporting of free VRAM but since the attempted allocation is already so much smaller than the reported free VRAM, we switch to system memory allocations. 4. We attempt to allocate 4.9G on system RAM, which fails despite being much lower than the available memory. 5. Ollama gives up and returns "memory layout cannot be allocated". You might want to try fully removing the Ollama installation, including any libraries and reinstalling. Sometimes old DLLs can have odd results. Otherwise, it might be something strange going on with your system.
GiteaMirror commented 2026-04-29 07:48:38 -05:00
Author
Owner
Copy Link

@dwdali commented on GitHub (Oct 15, 2025):

Hi Jessegross,

Thanks for your information. Actually, my PC has total 16GiB memory and I'm running a docker-desktop for dify, and they will use about 9.2GiB like this:
Image
If I don't run docker( compose down ), just run Ollama only, it could be OK for Ollama using system ram. I think it just can't use VRAM.
I really modified many environment variables, following others advices, but they all don't work... like this:
Image

Any other ideas?

<!-- gh-comment-id:3406039247 --> @dwdali commented on GitHub (Oct 15, 2025): Hi Jessegross, Thanks for your information. Actually, my PC has total 16GiB memory and I'm running a docker-desktop for dify, and they will use about 9.2GiB like this: <img width="198" height="59" alt="Image" src="https://github.com/user-attachments/assets/2e946665-c147-43cf-b086-1ed9234db94f" /> If I don't run docker( compose down ), just run Ollama only, it could be OK for Ollama using system ram. I think it just can't use VRAM. I really modified many environment variables, following others advices, but they all don't work... like this: <img width="304" height="198" alt="Image" src="https://github.com/user-attachments/assets/5ee16603-a1c1-4a5f-929b-b0073510adf1" /> Any other ideas?
GiteaMirror commented 2026-04-29 07:48:41 -05:00
Author
Owner
Copy Link

@dwdali commented on GitHub (Oct 15, 2025):

And I agree with you that my system may have something wrong, I see there is even smallest VRAM allocating can be fail. But what's the problem? I still don't know..

ggml_backend_cuda_buffer_type_alloc_buffer: allocating 8.00 MiB on device 0: cudaMalloc failed: out of memory

<!-- gh-comment-id:3406939416 --> @dwdali commented on GitHub (Oct 15, 2025): And I agree with you that my system may have something wrong, I see there is even smallest VRAM allocating can be fail. But what's the problem? I still don't know.. ====================== ggml_backend_cuda_buffer_type_alloc_buffer: allocating **8.00 MiB** on device 0: cudaMalloc failed: out of memory
GiteaMirror commented 2026-04-29 07:48:45 -05:00
Author
Owner
Copy Link

@dwdali commented on GitHub (Oct 16, 2025):

I find if I shutdown and exit docker-desktop, then type 'ollama run deepseek-r1:8b' in command line or run ollama app directly, it will run correctly. ollama can use GPU as well, usage of GPU memory is 5.7G/22G. But I think ollama malloc memory fail is not because docker-desktop use more. I've confirmed the VRAM usage is 0.4G/22G when docker-desktop is running. But it is really related.

Does anyone encounter this situation and help me? Many thanks.

<!-- gh-comment-id:3411103176 --> @dwdali commented on GitHub (Oct 16, 2025): I find if I shutdown and exit docker-desktop, then type 'ollama run deepseek-r1:8b' in command line or run ollama app directly, it will run correctly. ollama can use GPU as well, usage of GPU memory is 5.7G/22G. But I think ollama malloc memory fail is not because docker-desktop use more. I've confirmed the VRAM usage is 0.4G/22G when docker-desktop is running. But it is really related. Does anyone encounter this situation and help me? Many thanks.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#54886
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?