github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-18 14:16:30 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3823] Can we add support for LLaVA-Llama-3-8B? #48877

New Issue
Closed
opened 2026-04-28 09:56:54 -05:00 by GiteaMirror · 13 comments
GiteaMirror commented 2026-04-28 09:56:54 -05:00
Owner
Copy Link

Originally created by @octavioccl on GitHub (Apr 22, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3823

Hi I just saw in redis that there is a llava model based on llama-3, can be added it to the library? Thanks

Source:https://www.reddit.com/r/LocalLLaMA/comments/1ca8uxo/llavallama38b_is_released/

Originally created by @octavioccl on GitHub (Apr 22, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3823 Hi I just saw in redis that there is a llava model based on llama-3, can be added it to the library? Thanks Source:https://www.reddit.com/r/LocalLLaMA/comments/1ca8uxo/llavallama38b_is_released/
GiteaMirror added the model label 2026-04-28 09:56:54 -05:00
GiteaMirror commented 2026-04-28 09:56:56 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Apr 22, 2024):

https://ollama.com/library/llama3

<!-- gh-comment-id:2070297392 --> @olumolu commented on GitHub (Apr 22, 2024): https://ollama.com/library/llama3
GiteaMirror commented 2026-04-28 09:56:56 -05:00
Author
Owner
Copy Link

@octavioccl commented on GitHub (Apr 22, 2024):

https://ollama.com/library/llama3 Before posting anything do your own research.

@userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text

<!-- gh-comment-id:2070315090 --> @octavioccl commented on GitHub (Apr 22, 2024): > https://ollama.com/library/llama3 Before posting anything do your own research. @userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text
GiteaMirror commented 2026-04-28 09:56:57 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Apr 22, 2024):

https://ollama.com/library/llama3 Before posting anything do your own research.

@userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text

Are you looking for this

<!-- gh-comment-id:2070723515 --> @olumolu commented on GitHub (Apr 22, 2024): > > https://ollama.com/library/llama3 Before posting anything do your own research. > > @userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text Are you looking for this
GiteaMirror commented 2026-04-28 09:56:57 -05:00
Author
Owner
Copy Link

@NeroYuki commented on GitHub (Apr 22, 2024):

https://ollama.com/library/llama3 Before posting anything do your own research.

@userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text

Are you looking for this

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1

it's this one

<!-- gh-comment-id:2070874532 --> @NeroYuki commented on GitHub (Apr 22, 2024): > > > https://ollama.com/library/llama3 Before posting anything do your own research. > > > > > > @userforsource I'm not referring to llama3, I'm referring to the LLava model which is used for vision to text > > Are you looking for this https://huggingface.co/xtuner/llava-llama-3-8b-v1_1 it's this one
GiteaMirror commented 2026-04-28 09:56:58 -05:00
Author
Owner
Copy Link

@ChildOf7Sins commented on GitHub (Apr 23, 2024):

How to we show support to get this specific model added to Ollama's library? I know there are quite a few model requests.

I would add it myself, but it doesn't stop responding when I try an example. I think it's a problem with my Modelfile, but I'm not sure what I did wrong. :/

`FROM quantized.bin
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token"`

<!-- gh-comment-id:2072959190 --> @ChildOf7Sins commented on GitHub (Apr 23, 2024): How to we show support to get this specific model added to Ollama's library? I know there are quite a few model requests. I would add it myself, but it doesn't stop responding when I try an example. I think it's a problem with my Modelfile, but I'm not sure what I did wrong. :/ `FROM quantized.bin TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>""" PARAMETER stop "<|start_header_id|>" PARAMETER stop "<|end_header_id|>" PARAMETER stop "<|eot_id|>" PARAMETER stop "<|reserved_special_token"`
GiteaMirror commented 2026-04-28 09:57:00 -05:00
Author
Owner
Copy Link

@n2nco commented on GitHub (Apr 25, 2024):

+1

<!-- gh-comment-id:2077641465 --> @n2nco commented on GitHub (Apr 25, 2024): +1
GiteaMirror commented 2026-04-28 09:57:00 -05:00
Author
Owner
Copy Link

@LZHgrla commented on GitHub (Apr 29, 2024):

Hi!
We have released GGUF versions of the models and look forward to someone giving it a try :)

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf

<!-- gh-comment-id:2081993452 --> @LZHgrla commented on GitHub (Apr 29, 2024): Hi! We have released GGUF versions of the models and look forward to someone giving it a try :) https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf
GiteaMirror commented 2026-04-28 09:57:01 -05:00
Author
Owner
Copy Link

@ChildOf7Sins commented on GitHub (Apr 29, 2024):

Thank you! I was able to build it. Was hoping it would work well with Open Interpreter, but it didn't do better than any other local LLM. I'll just have to wait for a open source LAM.

If anyone wants to use llava-llama3 try this:
ollama run childof7sins/llava-llama3-f16

(Hopefully I built and uploaded it correctly 😅)

<!-- gh-comment-id:2083301957 --> @ChildOf7Sins commented on GitHub (Apr 29, 2024): Thank you! I was able to build it. Was hoping it would work well with Open Interpreter, but it didn't do better than any other local LLM. I'll just have to wait for a open source LAM. If anyone wants to use llava-llama3 try this: `ollama run childof7sins/llava-llama3-f16` (Hopefully I built and uploaded it correctly 😅)
GiteaMirror commented 2026-04-28 09:57:01 -05:00
Author
Owner
Copy Link

@Sprinter1999 commented on GitHub (May 2, 2024):

Hi! We have released GGUF versions of the models and look forward to someone giving it a try :)

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf

Oh Thank you Grla : ) I wonder if there is any guidance on conducting supervised-FT on your proposed LLava-llama-3-8b-v1,1, it will help a lot, thank you~

Xuefen

<!-- gh-comment-id:2090542074 --> @Sprinter1999 commented on GitHub (May 2, 2024): > Hi! We have released GGUF versions of the models and look forward to someone giving it a try :) > > https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf Oh Thank you Grla : ) I wonder if there is any guidance on conducting supervised-FT on your proposed LLava-llama-3-8b-v1,1, it will help a lot, thank you~ Xuefen
GiteaMirror commented 2026-04-28 09:57:02 -05:00
Author
Owner
Copy Link

@LZHgrla commented on GitHub (May 6, 2024):

Hi! We have released GGUF versions of the models and look forward to someone giving it a try :)
https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf

Oh Thank you Grla : ) I wonder if there is any guidance on conducting supervised-FT on your proposed LLava-llama-3-8b-v1,1, it will help a lot, thank you~

Xuefen

Hi! You can fine-tune this model by xtuner.
We have released the .pth weight of llava-llama-3-8b-v1.1 (url), and you can set this .pth file as the pretrained_pth for LLaVAModel to initialize your model.

086df69e99/xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/finetune/llava_llama3_8b_instruct_full_clip_vit_large_p14_336_lora_e1_gpu8_internvl_finetune.py (L26)

<!-- gh-comment-id:2095303249 --> @LZHgrla commented on GitHub (May 6, 2024): > > Hi! We have released GGUF versions of the models and look forward to someone giving it a try :) > > https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf > > Oh Thank you Grla : ) I wonder if there is any guidance on conducting supervised-FT on your proposed LLava-llama-3-8b-v1,1, it will help a lot, thank you~ > > Xuefen Hi! You can fine-tune this model by [xtuner](https://github.com/InternLM/xtuner). We have released the `.pth` weight of llava-llama-3-8b-v1.1 ([url](https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-pth)), and you can set this `.pth` file as the `pretrained_pth` for `LLaVAModel` to initialize your model. https://github.com/InternLM/xtuner/blob/086df69e99888e1735d6f1e2b63eb3792693a36a/xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/finetune/llava_llama3_8b_instruct_full_clip_vit_large_p14_336_lora_e1_gpu8_internvl_finetune.py#L26
GiteaMirror commented 2026-04-28 09:57:03 -05:00
Author
Owner
Copy Link

@Sprinter1999 commented on GitHub (May 6, 2024):

Thank you for your quick response after the May Day holidy 👍🏻 I'll have a look!

<!-- gh-comment-id:2095312498 --> @Sprinter1999 commented on GitHub (May 6, 2024): Thank you for your quick response after the May Day holidy 👍🏻 I'll have a look!
GiteaMirror commented 2026-04-28 09:57:03 -05:00
Author
Owner
Copy Link

@PaulCapestany commented on GitHub (May 9, 2024):

@ChildOf7Sins had mentioned "it doesn't stop responding" and I'm getting similar odd behavior (looping/repeating responses, regurgitation of input, chains of emojis, etc) when using https://ollama.com/library/llava-llama3:8b-v1.1-fp16

I'd be curious if anyone has any ideas on what the issue might be (or how to debug the situation)?

cc: @LZHgrla

<!-- gh-comment-id:2102794342 --> @PaulCapestany commented on GitHub (May 9, 2024): @ChildOf7Sins had mentioned "it doesn't stop responding" and I'm getting similar odd behavior (looping/repeating responses, regurgitation of input, chains of emojis, etc) when using https://ollama.com/library/llava-llama3:8b-v1.1-fp16 I'd be curious if anyone has any ideas on what the issue might be (or how to debug the situation)? cc: @LZHgrla
GiteaMirror commented 2026-04-28 09:57:04 -05:00
Author
Owner
Copy Link

@PaulCapestany commented on GitHub (May 9, 2024):

@ChildOf7Sins had mentioned "it doesn't stop responding" and I'm getting similar odd behavior (looping/repeating responses, regurgitation of input, chains of emojis, etc) when using https://ollama.com/library/llava-llama3:8b-v1.1-fp16

I'd be curious if anyone has any ideas on what the issue might be (or how to debug the situation)?

cc: @LZHgrla

Okay, I think I figured it out (following advice from Psynaps on ollama's Discord). It appears the odd behavior may have been due to running an older version of ollama.

Previously (when I was getting aforementioned weird model behavior):

ollama -v
ollama version is 0.1.30

Now everything seems to work perfectly:

brew update && brew upgrade ollama
ollama -v
ollama version is 0.1.34
<!-- gh-comment-id:2102818010 --> @PaulCapestany commented on GitHub (May 9, 2024): > @ChildOf7Sins had mentioned "it doesn't stop responding" and I'm getting similar odd behavior (looping/repeating responses, regurgitation of input, chains of emojis, etc) when using https://ollama.com/library/llava-llama3:8b-v1.1-fp16 > > I'd be curious if anyone has any ideas on what the issue might be (or how to debug the situation)? > > cc: @LZHgrla Okay, I think I figured it out (following [advice from Psynaps on ollama's Discord](https://discord.com/channels/1128867683291627614/1237531135261806612/1237576465994743848)). It appears the odd behavior may have been due to running an older version of ollama. Previously (when I was getting aforementioned weird model behavior): ```shell ollama -v ollama version is 0.1.30 ``` Now everything seems to work perfectly: ```shell brew update && brew upgrade ollama ollama -v ollama version is 0.1.34 ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48877
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?