github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #11259] Ollama versus llama.cpp, which is faster? #7420

New Issue
Closed
opened 2026-04-12 19:30:17 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 19:30:17 -05:00
Owner
Copy Link

Originally created by @Bardo-Konrad on GitHub (Jul 1, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11259

So I've been using Ollama for quite some time and what I have not been happy with is the fact that most models just are not on par at all with what cloud models deliver.

But this aside, what I also don't like about it is that it can be quite slow and I have always thought that this is just the way it is. It just is slow. So in addition to the quality not being the best, it just is rather slow even though you're using CUDA or other acceleration like OpenCL, it still is slow and not really convincing.

But then I heard that at least the speed part can be compensated by using Lama CPP. And I was wondering, what are your experiences?

In my experience using Lama CPP, which seems to be the real deal that even Ollama is based on, is not faster. So I tried some experiments and I realized it's not faster. But maybe given that you can tinker much more with it, you can optimize this and have it be much faster than Ollama at the expense of comfort basically.

So you don't just have a simple framework, a simple wrapper around it, but you deal with it directly. Maybe you have to write your own wrapper that is more specialized and then you can get a faster output. AI itself told me, confronted with this question, that Lama CPP was about 1.8 times faster than Ollama. And it was definitely worth using Lama CPP over Ollama, especially when using Ubuntu when using a CPU only, where you are compiling it and thus using all the extensions the CPU has.

I don't know about Windows because I'm normally using Windows with a GPU. But it seems to be rather unclear how the two stand to each other. As I said, according to cloud AI, Ollama is slower than Lama CPP by a factor of 1.8.

Originally created by @Bardo-Konrad on GitHub (Jul 1, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11259 So I've been using Ollama for quite some time and what I have not been happy with is the fact that most models just are not on par at all with what cloud models deliver. But this aside, what I also don't like about it is that it can be quite slow and I have always thought that this is just the way it is. It just is slow. So in addition to the quality not being the best, it just is rather slow even though you're using CUDA or other acceleration like OpenCL, it still is slow and not really convincing. But then I heard that at least the speed part can be compensated by using Lama CPP. And I was wondering, what are your experiences? In my experience using Lama CPP, which seems to be the real deal that even Ollama is based on, is not faster. So I tried some experiments and I realized it's not faster. But maybe given that you can tinker much more with it, you can optimize this and have it be much faster than Ollama at the expense of comfort basically. So you don't just have a simple framework, a simple wrapper around it, but you deal with it directly. Maybe you have to write your own wrapper that is more specialized and then you can get a faster output. AI itself told me, confronted with this question, that Lama CPP was about 1.8 times faster than Ollama. And it was definitely worth using Lama CPP over Ollama, especially when using Ubuntu when using a CPU only, where you are compiling it and thus using all the extensions the CPU has. I don't know about Windows because I'm normally using Windows with a GPU. But it seems to be rather unclear how the two stand to each other. As I said, according to cloud AI, Ollama is slower than Lama CPP by a factor of 1.8.
GiteaMirror commented 2026-04-12 19:30:18 -05:00
Author
Owner
Copy Link

@gcr commented on GitHub (Jul 1, 2025):

hi! ollama uses llama-cpp as its inference engine, so it should be just as fast. please collect your own benchmarks and don't listen to AI here. :)

the way to get faster output is:

  • use smaller models
  • on better hardware
  • with more aggressive quantization

smaller models. number of parameters is important. how does gemma3n:e2b or gemma3:1b feel?

better hardware. are you able to run your model on GPU? how are you verifying that it's running on GPU properly?

more aggressive quantization. some models have different quantizations available, see https://ollama.com/library/llama3.1/tags for example. you can expand "View All" on each model page. this is a direct "speed vs output quality" tradeoff. Q2_K models are smallest/fastest, followed by Q3, Q4, ..., up to 8 and float16 as the largest. Sort by size and try the very smallest and see what you get.

hope that helps!

<!-- gh-comment-id:3025717540 --> @gcr commented on GitHub (Jul 1, 2025): hi! ollama uses llama-cpp as its inference engine, so it should be just as fast. please collect your own benchmarks and don't listen to AI here. :) the way to get faster output is: - use smaller models - on better hardware - with more aggressive quantization **smaller models.** number of parameters is important. how does `gemma3n:e2b` or `gemma3:1b` feel? **better hardware.** are you able to run your model on GPU? how are you verifying that it's running on GPU properly? **more aggressive quantization.** some models have different quantizations available, see https://ollama.com/library/llama3.1/tags for example. you can expand "View All" on each model page. this is a direct "speed vs output quality" tradeoff. Q2_K models are smallest/fastest, followed by Q3, Q4, ..., up to 8 and float16 as the largest. Sort by size and try the very smallest and see what you get. hope that helps!
GiteaMirror commented 2026-04-12 19:30:18 -05:00
Author
Owner
Copy Link

@Bardo-Konrad commented on GitHub (Jul 2, 2025):

Thanks. Here are some sources:

<!-- gh-comment-id:3025895113 --> @Bardo-Konrad commented on GitHub (Jul 2, 2025): Thanks. Here are some sources: - https://www.reddit.com/r/LocalLLaMA/comments/1d05x6v/llamacpp_runs_18_times_faster_than_ollama/ - https://www.arsturn.com/blog/llama-cpp-vs-ollama-which-one-to-choose - https://www.infralovers.com/de/blog/2024-07-09-empowering-local-ai/
GiteaMirror commented 2026-04-12 19:30:19 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Jul 2, 2025):

@gcr Ollama doesn't use llama-cpp for either gemma3n or gemma3. Most new models are implemented directly inside of Ollama. There are some older models which still run on llama.cpp, but the plan is to move away from using it.

@Bardo-Konrad the reason for being "1/18th" the speed (or llama.cpp being "1.8x faster") is almost certainly due to the memory calculation when calculating the graph and running on very constrained hardware. The problem is if even a single layer gets put into system memory instead of on the GPU, the performance will be dramatically impacted. We have a number of improvements we've been rolling out around memory estimation (particularly in the new engine vs. llama.cpp) which we've been working on. I would take the blog posts and comments on r/localllama with a grain of salt, particularly the one from a year ago.

That said, I think the answer of which is faster depends on a lot of factors, so it's difficult to give a concrete answer.

<!-- gh-comment-id:3025972600 --> @pdevine commented on GitHub (Jul 2, 2025): @gcr Ollama doesn't use llama-cpp for either `gemma3n` or `gemma3`. Most new models are implemented directly inside of Ollama. There are some older models which still run on llama.cpp, but the plan is to move away from using it. @Bardo-Konrad the reason for being "1/18th" the speed (or llama.cpp being "1.8x faster") is almost certainly due to the memory calculation when calculating the graph and running on very constrained hardware. The problem is if even a single layer gets put into system memory instead of on the GPU, the performance will be dramatically impacted. We have a number of improvements we've been rolling out around memory estimation (particularly in the new engine vs. llama.cpp) which we've been working on. I would take the blog posts and comments on r/localllama with a grain of salt, particularly the one from a year ago. That said, I think the answer of which is faster depends on a lot of factors, so it's difficult to give a concrete answer.
GiteaMirror commented 2026-04-12 19:30:19 -05:00
Author
Owner
Copy Link

@gcr commented on GitHub (Jul 2, 2025):

Apologies for the misinformation!

<!-- gh-comment-id:3025975001 --> @gcr commented on GitHub (Jul 2, 2025): Apologies for the misinformation!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7420
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?