github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #11977] configure OLLAMA_NUM_PARALLEL per model #70011

New Issue
Closed
opened 2026-05-04 20:03:59 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-04 20:03:59 -05:00
Owner
Copy Link

Originally created by @Robinsane on GitHub (Aug 20, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11977

Hi Ollama community

I'd love for the ability to bring up multiple models in VRAM, with different OLLAMA_NUM_PARALLEL capabilities.
If I want 1 model to allow for e.g. 3 parallel requests, it sort of triples the reserved context. Also tripling context for models where I will not need parallel capabilities.

Previously I solved this by running different containers, each with their specific OLLAMA_NUM_PARALLEL.
However since the changes in V0.11.5, it seems every single Ollama instance reserves / uses 146MiB of VRAM on each GPU. This makes my previous solution also lose available VRAM.

Screenshot of VRAM usage with 2 models, each in their own Ollama container:
Image

Originally created by @Robinsane on GitHub (Aug 20, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11977 Hi Ollama community I'd love for the ability to bring up multiple models in VRAM, with different OLLAMA_NUM_PARALLEL capabilities. If I want 1 model to allow for e.g. 3 parallel requests, it sort of triples the reserved context. Also tripling context for models where I will not need parallel capabilities. Previously I solved this by running different containers, each with their specific OLLAMA_NUM_PARALLEL. However since the changes in V0.11.5, it seems every single Ollama instance reserves / uses 146MiB of VRAM on each GPU. This makes my previous solution also lose available VRAM. Screenshot of VRAM usage with 2 models, each in their own Ollama container: <img width="825" height="315" alt="Image" src="https://github.com/user-attachments/assets/41ab81f1-1896-4cb5-8f7b-ffd40db06365" />
GiteaMirror added the feature request label 2026-05-04 20:03:59 -05:00
GiteaMirror commented 2026-05-04 20:04:05 -05:00
Author
Owner
Copy Link

@Robinsane commented on GitHub (Aug 20, 2025):

relevant issue:
https://github.com/ollama/ollama/issues/4170

<!-- gh-comment-id:3204755019 --> @Robinsane commented on GitHub (Aug 20, 2025): relevant issue: https://github.com/ollama/ollama/issues/4170
GiteaMirror commented 2026-05-04 20:04:07 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 20, 2025):

#9546

<!-- gh-comment-id:3204825269 --> @rick-github commented on GitHub (Aug 20, 2025): #9546
GiteaMirror commented 2026-05-04 20:04:10 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Aug 20, 2025):

Definitely appreciate the issue (and thank you @rick-github for the PR). I'm going to close it as a dupe of #4170

cc @jessegross @mxyng

<!-- gh-comment-id:3207584694 --> @pdevine commented on GitHub (Aug 20, 2025): Definitely appreciate the issue (and thank you @rick-github for the PR). I'm going to close it as a dupe of #4170 cc @jessegross @mxyng
GiteaMirror commented 2026-05-04 20:04:11 -05:00
Author
Owner
Copy Link

@jessegross commented on GitHub (Aug 29, 2025):

@Robinsane We have reduced the size of these additional allocations in 0.11.8.

<!-- gh-comment-id:3237807637 --> @jessegross commented on GitHub (Aug 29, 2025): @Robinsane We have reduced the size of these additional allocations in 0.11.8.
GiteaMirror commented 2026-05-04 20:04:11 -05:00
Author
Owner
Copy Link

@Robinsane commented on GitHub (Aug 29, 2025):

@Robinsane We have reduced the size of these additional allocations in 0.11.8.

@jessegross
I greatly appreciate both your commit and comment, thank you!

<!-- gh-comment-id:3238000768 --> @Robinsane commented on GitHub (Aug 29, 2025): > [@Robinsane](https://github.com/Robinsane) We have reduced the size of these additional allocations in 0.11.8. @jessegross I greatly appreciate both your commit and comment, thank you!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#70011
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?