github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4332] Difference in performance between liuhaotian/llava-v1.6-34b and Ollama's llava:34b-v1.6 #2696

New Issue
Closed
opened 2026-04-12 13:01:07 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 13:01:07 -05:00
Owner
Copy Link

Originally created by @EricWiener on GitHub (May 11, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4332

What is the issue?

When using the demo here I get much better results using their llava:34b-v1.6 than I do via ollama run llava:34b-v1.6 when prompting with the same prompt followed by the image. Example of how I'm prompting:

ollama run llava:34b-v1.6 --verbose
>>> Is there a dog in this picture and if so what is it doing? /data/dog.png

Is there any reason for this and can I somehow match the performance of the demo?

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

0.1.34

Originally created by @EricWiener on GitHub (May 11, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4332 ### What is the issue? When using the demo [here](https://llava.hliu.cc/) I get much better results using their `llava:34b-v1.6` than I do via `ollama run llava:34b-v1.6` when prompting with the same prompt followed by the image. Example of how I'm prompting: ``` ollama run llava:34b-v1.6 --verbose >>> Is there a dog in this picture and if so what is it doing? /data/dog.png ``` Is there any reason for this and can I somehow match the performance of the demo? ### OS Linux ### GPU Nvidia ### CPU _No response_ ### Ollama version 0.1.34
GiteaMirror added the bug label 2026-04-12 13:01:07 -05:00
GiteaMirror commented 2026-04-12 13:01:10 -05:00
Author
Owner
Copy Link

@lizaibeim commented on GitHub (May 13, 2024):

It is not a bug, as the demo uses fp16 precision while the llava:34b-v1.6 is in 4 bit quantization. Thus, the performance is not comparable.

<!-- gh-comment-id:2107387826 --> @lizaibeim commented on GitHub (May 13, 2024): It is not a bug, as the demo uses fp16 precision while the llava:34b-v1.6 is in 4 bit quantization. Thus, the performance is not comparable.
GiteaMirror commented 2026-04-12 13:01:10 -05:00
Author
Owner
Copy Link

@EricWiener commented on GitHub (May 13, 2024):

Thanks for the response @lizaibeim ! Do you know if the demo is fp16 or bf16? I believe the demo might be bf16 but only see fp16 as an option through ollama.

<!-- gh-comment-id:2107675372 --> @EricWiener commented on GitHub (May 13, 2024): Thanks for the response @lizaibeim ! Do you know if the demo is fp16 or bf16? I believe the demo might be bf16 but only see fp16 as an option through ollama.
GiteaMirror commented 2026-04-12 13:01:11 -05:00
Author
Owner
Copy Link

@lizaibeim commented on GitHub (May 13, 2024):

Sorry i couldn't confirm if it is bf16 or fp16. But based on the recommended VRAM from llava - 80G, versus 69G in Ollama, it might be bf16.

<!-- gh-comment-id:2107995333 --> @lizaibeim commented on GitHub (May 13, 2024): Sorry i couldn't confirm if it is bf16 or fp16. But based on the recommended VRAM from [llava](https://github.com/haotian-liu/LLaVA) - 80G, versus 69G in [Ollama](https://ollama.com/library/llava:34b-v1.6-fp16), it might be bf16.
GiteaMirror commented 2026-04-12 13:01:11 -05:00
Author
Owner
Copy Link

@EricWiener commented on GitHub (May 13, 2024):

Thanks @lizaibeim !

<!-- gh-comment-id:2108043934 --> @EricWiener commented on GitHub (May 13, 2024): Thanks @lizaibeim !
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2696
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?