[GH-ISSUE #1575] Add support for ViP-LLaVA? #864

Open
opened 2026-04-12 10:31:38 -05:00 by GiteaMirror · 1 comment
Owner

Originally created by @mu-cai on GitHub (Dec 17, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/1575

Hi OLLaMa team,

ViP-LLaVA is a region-level large multimodal model from LLaVA team that is capable of understanding visual prompts such as scribbles, bounding boxes, arrows, etc.

There are only several lines of changes to the original LLaVA code. Huggingface already integrate ViP-LLaVA into the official transformers library. https://huggingface.co/docs/transformers/main/model_doc/vipllava

Consider adding ViP-LLaVA here?

Thank you!
Mu Cai

Originally created by @mu-cai on GitHub (Dec 17, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/1575 Hi OLLaMa team, [ViP-LLaVA](https://vip-llava.github.io/) is a region-level large multimodal model from LLaVA team that is capable of understanding visual prompts such as scribbles, bounding boxes, arrows, etc. There are only several lines of changes to the original LLaVA code. Huggingface already integrate ViP-LLaVA into the official transformers library. [https://huggingface.co/docs/transformers/main/model_doc/vipllava](https://huggingface.co/docs/transformers/main/model_doc/vipllava) Consider adding ViP-LLaVA here? Thank you! Mu Cai
GiteaMirror added the model label 2026-04-12 10:31:38 -05:00
Author
Owner

@easp commented on GitHub (Dec 18, 2023):

Issue in Llama.cpp repo https://github.com/ggerganov/llama.cpp/issues/4515

<!-- gh-comment-id:1861258153 --> @easp commented on GitHub (Dec 18, 2023): Issue in Llama.cpp repo https://github.com/ggerganov/llama.cpp/issues/4515
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#864