github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #7080] Support for NVLM #66552

New Issue
Open
opened 2026-05-04 07:23:47 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-05-04 07:23:47 -05:00
Owner
Copy Link

Originally created by @mitar on GitHub (Oct 2, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7080

NVLM is model from Nvidia: https://nvlm-project.github.io/

Originally created by @mitar on GitHub (Oct 2, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7080 NVLM is model from Nvidia: https://nvlm-project.github.io/
GiteaMirror added the model label 2026-05-04 07:23:47 -05:00
GiteaMirror commented 2026-05-04 07:24:02 -05:00
Author
Owner
Copy Link

@duaneking commented on GitHub (Oct 2, 2024):

Yeah looking at he numbers, its behind the new Lamav3 stuff a lil but as they release updates, support for this "open" decoder only model would ne nice. I just worry that its not as open as it seems. The hardware needed to load it is also an issue, at 72gb

<!-- gh-comment-id:2389970104 --> @duaneking commented on GitHub (Oct 2, 2024): Yeah looking at he numbers, its behind the new Lamav3 stuff a lil but as they release updates, support for this "open" decoder only model would ne nice. I just worry that its not as open as it seems. The hardware needed to load it is also an issue, at 72gb
GiteaMirror commented 2026-05-04 07:24:06 -05:00
Author
Owner
Copy Link

@TKaluza commented on GitHub (Oct 4, 2024):

So it would run a A100 /H100?

<!-- gh-comment-id:2393723860 --> @TKaluza commented on GitHub (Oct 4, 2024): So it would run a A100 /H100?
GiteaMirror commented 2026-05-04 07:24:09 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Oct 5, 2024):

I'm able to run Qwen 2.5 72B on 4 bits with 16GBs of VRAM and 64GBs of RAM adding up to 80GBs, but I think you are getting parameters and gigabytes confused. The model is presumably trained at FP16 and the accuracy can be lowered with quantization to FP8, FP4, and so on to fit in smaller amounts of memory sacrificing accuracy

Also, I'm running a AMD GPU so running a NVIDIA model is mildly funny. I just hope it's good maybe a step above Qwen 2.5

<!-- gh-comment-id:2394869433 --> @nonetrix commented on GitHub (Oct 5, 2024): I'm able to run Qwen 2.5 72B on 4 bits with 16GBs of VRAM and 64GBs of RAM adding up to 80GBs, but I think you are getting parameters and gigabytes confused. The model is presumably trained at FP16 and the accuracy can be lowered with quantization to FP8, FP4, and so on to fit in smaller amounts of memory sacrificing accuracy Also, I'm running a AMD GPU so running a NVIDIA model is mildly funny. I just hope it's good maybe a step above Qwen 2.5
GiteaMirror commented 2026-05-04 07:24:12 -05:00
Author
Owner
Copy Link

@angiopteris commented on GitHub (Oct 7, 2024):

Yes, would be a great addition ! Looking forward for chatting with it

<!-- gh-comment-id:2396956375 --> @angiopteris commented on GitHub (Oct 7, 2024): Yes, would be a great addition ! Looking forward for chatting with it
GiteaMirror commented 2026-05-04 07:24:22 -05:00
Author
Owner
Copy Link

@defaultsecurity commented on GitHub (Oct 9, 2024):

Looking forward to it!

<!-- gh-comment-id:2402793180 --> @defaultsecurity commented on GitHub (Oct 9, 2024): Looking forward to it!
GiteaMirror commented 2026-05-04 07:24:26 -05:00
Author
Owner
Copy Link

@mikeazurek commented on GitHub (Oct 9, 2024):

I don't think your request is even possible, because I've never seen any multi-modal AI supported by ollama, so I deduce that ollama is bound to only run purely language models, for some technical reason.

<!-- gh-comment-id:2403226638 --> @mikeazurek commented on GitHub (Oct 9, 2024): I don't think your request is even possible, because I've never seen any multi-modal AI supported by ollama, so I deduce that ollama is bound to only run purely language models, for some technical reason.
GiteaMirror commented 2026-05-04 07:24:28 -05:00
Author
Owner
Copy Link

@nonetrix commented on GitHub (Oct 9, 2024):

No, it does support them quite a few. This one has weird architecture I think, and Llama cpp seems quite slow to add them or more likely I think doesn't want to because it's out of scope (their project cool with me, not bashing them). I almost wish Ollama forked Llama cpp at this point, or made their own inference engine. Soft fork would probably be ideal, think Xenocara with XOrg where it's just a patch set

<!-- gh-comment-id:2403251133 --> @nonetrix commented on GitHub (Oct 9, 2024): No, it does support them quite a few. This one has weird architecture I think, and Llama cpp seems quite slow to add them or more likely I think doesn't want to because it's out of scope (their project cool with me, not bashing them). I almost wish Ollama forked Llama cpp at this point, or made their own inference engine. Soft fork would probably be ideal, think Xenocara with XOrg where it's just a patch set
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#66552
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?