github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #8616] Ollama: torch.OutOfMemoryError: CUDA out of memory #5575

New Issue
Open
opened 2026-04-12 16:50:20 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-12 16:50:20 -05:00
Owner
Copy Link

Originally created by @kennethwork101 on GitHub (Jan 27, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8616

What is the issue?

Running some tests using pytest with the following 6 models. What I find is that if I run all tests with each model before go on to the next model, the tests mostly worked fine. 123/126 passed. But if I run each test against all 6 models sequentially and then go to the next test then I see hangs or out of memory error. Is this a known issue? I expect the order of running tests using Ollama should not matter.

ollama version is 0.5.7
pytest 8.3.4
| NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 |
| 0 NVIDIA GeForce RTX 4070 Ti Off | 00000000:01:00.0 On | N/A |
Linux kennethpc 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

qwen2:latest
qwen2.5:latest
mistral:latest
llama3-groq-tool-use:latest
llama3.2:latest
llama3.2:latest

Here is some examples of the errors. Sometimes I simply see hangs:
FAILED tests/_1_misc_test.py::test_t6_func[mistral:latest] - assert None is not None
FAILED tests/_1_misc_test.py::test_t6_func[llama3-groq-tool-use:latest] - assert None is not None
FAILED tests/_2_rag_test.py::test_t7_func[qwen2:latest-chroma] - assert 768 == 384
FAILED tests/_2_rag_test.py::test_t7_func[qwen2.5:latest-chroma] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.72 GiB of which 41.50 MiB is free. Process 263255 has 3.40 GiB memory in use. Process 263532 has 6.09 GiB memory in use. In...
FAILED tests/_2_rag_test.py::test_t7_func[qwen2.5:latest-huggingface] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.72 GiB of which 41.50 MiB is free. Process 263255 has 3.40 GiB memory in use. Process 263532 has 6.09 GiB memory in use. In...

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.5.7

Originally created by @kennethwork101 on GitHub (Jan 27, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8616 ### What is the issue? Running some tests using pytest with the following 6 models. What I find is that if I run all tests with each model before go on to the next model, the tests mostly worked fine. 123/126 passed. But if I run each test against all 6 models sequentially and then go to the next test then I see hangs or out of memory error. Is this a known issue? I expect the order of running tests using Ollama should not matter. ollama version is 0.5.7 pytest 8.3.4 | NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 | | 0 NVIDIA GeForce RTX 4070 Ti Off | 00000000:01:00.0 On | N/A | Linux kennethpc 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux qwen2:latest qwen2.5:latest mistral:latest llama3-groq-tool-use:latest llama3.2:latest llama3.2:latest Here is some examples of the errors. Sometimes I simply see hangs: FAILED tests/_1_misc_test.py::test_t6_func[mistral:latest] - assert None is not None FAILED tests/_1_misc_test.py::test_t6_func[llama3-groq-tool-use:latest] - assert None is not None FAILED tests/_2_rag_test.py::test_t7_func[qwen2:latest-chroma] - assert 768 == 384 FAILED tests/_2_rag_test.py::test_t7_func[qwen2.5:latest-chroma] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.72 GiB of which 41.50 MiB is free. Process 263255 has 3.40 GiB memory in use. Process 263532 has 6.09 GiB memory in use. In... FAILED tests/_2_rag_test.py::test_t7_func[qwen2.5:latest-huggingface] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.72 GiB of which 41.50 MiB is free. Process 263255 has 3.40 GiB memory in use. Process 263532 has 6.09 GiB memory in use. In... ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.7
GiteaMirror added the bug label 2026-04-12 16:50:21 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5575
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?