github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-05 23:53:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #327] Embedding model support #142

New Issue
Closed
opened 2026-04-12 09:40:23 -05:00 by GiteaMirror · 18 comments
GiteaMirror commented 2026-04-12 09:40:23 -05:00
Owner
Copy Link

Originally created by @jmorganca on GitHub (Aug 11, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/327

Originally assigned to: @jmorganca on GitHub.

Add embedding models to use primarily with /api/embeddings

  • instructor-xl
  • bge-large
  • all-MiniLM-L6-v2

See the full leaderboard

Originally created by @jmorganca on GitHub (Aug 11, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/327 Originally assigned to: @jmorganca on GitHub. Add embedding models to use primarily with `/api/embeddings` * `instructor-xl` * `bge-large` * `all-MiniLM-L6-v2` See the full [leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
GiteaMirror added the feature requestmodel labels 2026-04-12 09:40:23 -05:00
GiteaMirror commented 2026-04-12 09:40:24 -05:00
Author
Owner
Copy Link

@brunnolou commented on GitHub (Oct 18, 2023):

Yes, please! Any of these embedding models above text-embedding-ada-002 would be a great addition.

I've tried LLam2 and Mistral model with the /api/embeddings as is, and I'm getting poor-quality similarity scores.
Even with almost identical queries, It fails to retrieve results. Are there some prompting technics to improve the embedding quality?

Anyway, in comparison, I've tried Xenova/gte-small with transformers and it is much faster and yields better results.

<!-- gh-comment-id:1768363892 --> @brunnolou commented on GitHub (Oct 18, 2023): Yes, please! Any of these embedding models above `text-embedding-ada-002` would be a great addition. I've tried LLam2 and Mistral model with the `/api/embeddings` as is, and I'm getting poor-quality similarity scores. Even with almost identical queries, It fails to retrieve results. Are there some prompting technics to improve the embedding quality? Anyway, in comparison, I've tried [Xenova/gte-small](https://huggingface.co/Xenova/gte-small) with transformers and it is much faster and yields better results.
GiteaMirror commented 2026-04-12 09:40:25 -05:00
Author
Owner
Copy Link

@corani commented on GitHub (Nov 17, 2023):

jinaai/jina-embeddings-v2-base-en (and other variants) also look promising.

<!-- gh-comment-id:1816078155 --> @corani commented on GitHub (Nov 17, 2023): `jinaai/jina-embeddings-v2-base-en` (and other variants) also look promising.
GiteaMirror commented 2026-04-12 09:40:25 -05:00
Author
Owner
Copy Link

@sandangel commented on GitHub (Dec 7, 2023):

Hi, is there an update on this issue? I would love to contribute

<!-- gh-comment-id:1844401118 --> @sandangel commented on GitHub (Dec 7, 2023): Hi, is there an update on this issue? I would love to contribute
GiteaMirror commented 2026-04-12 09:40:26 -05:00
Author
Owner
Copy Link

@corani commented on GitHub (Dec 8, 2023):

I've been playing with https://github.com/nlpodyssey/cybertron which is pure Go (but I guess CPU only?) and at least supports all-MiniLM-L6-v2, e5-*-v2, bge-*-en-v1.5 and ember-v1.

I did some testing with the STS-2016 dataset and got the below accuracies compared to llama2 and mistral:instruct (Pearson correlation with the gold answers):

  • Ollama
    • llama2: 0.23431
    • mistral:instruct: 0.5656
  • Cybertron
    • all-MiniLM-L6-v2: 0.80344
    • e5-small-v2: 0.82318
    • e5-base-v2: 0.83845
    • bge-small-en-v1.5: 0.84514
    • bge-base-en-v1.5: 0.85297

So I agree with the previous comment that the embeddings generated by the completion models are pretty bad!

<!-- gh-comment-id:1846841081 --> @corani commented on GitHub (Dec 8, 2023): I've been playing with https://github.com/nlpodyssey/cybertron which is pure Go (but I guess CPU only?) and at least supports `all-MiniLM-L6-v2`, `e5-*-v2`, `bge-*-en-v1.5` and `ember-v1`. I did some testing with the [STS-2016](https://alt.qcri.org/semeval2016/task1/) dataset and got the below accuracies compared to `llama2` and `mistral:instruct` (Pearson correlation with the gold answers): - Ollama - `llama2`: 0.23431 - `mistral:instruct`: 0.5656 - Cybertron - `all-MiniLM-L6-v2`: 0.80344 - `e5-small-v2`: 0.82318 - `e5-base-v2`: 0.83845 - `bge-small-en-v1.5`: 0.84514 - `bge-base-en-v1.5`: 0.85297 So I agree with the previous comment that the embeddings generated by the completion models are pretty bad!
GiteaMirror commented 2026-04-12 09:40:26 -05:00
Author
Owner
Copy Link

@sandangel commented on GitHub (Dec 8, 2023):

That is interesting. For GPU support, I guess we will need to use: https://github.com/skeskinen/bert.cpp ?
I think the implementation would be something similar to llama.cpp?

<!-- gh-comment-id:1846914774 --> @sandangel commented on GitHub (Dec 8, 2023): That is interesting. For GPU support, I guess we will need to use: https://github.com/skeskinen/bert.cpp ? I think the implementation would be something similar to llama.cpp?
GiteaMirror commented 2026-04-12 09:40:27 -05:00
Author
Owner
Copy Link

@sandangel commented on GitHub (Dec 8, 2023):

I found this issue: https://github.com/ggerganov/llama.cpp/issues/2872
I think they plan to implement it in llama.cpp. So maybe we will just need to update the llama.cpp when it's done?

<!-- gh-comment-id:1846988582 --> @sandangel commented on GitHub (Dec 8, 2023): I found this issue: https://github.com/ggerganov/llama.cpp/issues/2872 I think they plan to implement it in llama.cpp. So maybe we will just need to update the llama.cpp when it's done?
GiteaMirror commented 2026-04-12 09:40:27 -05:00
Author
Owner
Copy Link

@sandangel commented on GitHub (Dec 11, 2023):

I also found this https://github.com/ml-explore/mlx-examples/blob/main/bert/README.md, which we can use to run inference on M1 mac. Is it possible to support mlx with Ollama?

<!-- gh-comment-id:1849906676 --> @sandangel commented on GitHub (Dec 11, 2023): I also found this https://github.com/ml-explore/mlx-examples/blob/main/bert/README.md, which we can use to run inference on M1 mac. Is it possible to support `mlx` with Ollama?
GiteaMirror commented 2026-04-12 09:40:27 -05:00
Author
Owner
Copy Link

@CodeWithKyrian commented on GitHub (Dec 29, 2023):

Any update on this or plan to allow Bert Models?

<!-- gh-comment-id:1872103263 --> @CodeWithKyrian commented on GitHub (Dec 29, 2023): Any update on this or plan to allow Bert Models?
GiteaMirror commented 2026-04-12 09:40:28 -05:00
Author
Owner
Copy Link

@tjohnson4 commented on GitHub (Jan 20, 2024):

Any update on this issue?

<!-- gh-comment-id:1901817912 --> @tjohnson4 commented on GitHub (Jan 20, 2024): Any update on this issue?
GiteaMirror commented 2026-04-12 09:40:28 -05:00
Author
Owner
Copy Link

@ymohamed08 commented on GitHub (Jan 20, 2024):

Do you have any updates so far? very interested to contribute

<!-- gh-comment-id:1902256534 --> @ymohamed08 commented on GitHub (Jan 20, 2024): Do you have any updates so far? very interested to contribute
GiteaMirror commented 2026-04-12 09:40:29 -05:00
Author
Owner
Copy Link

@ill-yes commented on GitHub (Feb 2, 2024):

Any updates here?

<!-- gh-comment-id:1923593838 --> @ill-yes commented on GitHub (Feb 2, 2024): Any updates here?
GiteaMirror commented 2026-04-12 09:40:29 -05:00
Author
Owner
Copy Link

@easp commented on GitHub (Feb 3, 2024):

Plans to support BERT models in llama.cpp stalled out when the dev who had assumed the task ended up focusing on something else. In the last few days it looks like the project management artifacts were updated to acknowledge this, so maybe there will be some action soon. Actually, it looks like there has been some activity. Maybe there will be working code soon:
https://github.com/ggerganov/llama.cpp/issues/2872

<!-- gh-comment-id:1925439708 --> @easp commented on GitHub (Feb 3, 2024): Plans to support BERT models in llama.cpp stalled out when the dev who had assumed the task ended up focusing on something else. In the last few days it looks like the project management artifacts were updated to acknowledge this, so maybe there will be some action soon. Actually, it looks like there has been some activity. Maybe there will be working code soon: https://github.com/ggerganov/llama.cpp/issues/2872
GiteaMirror commented 2026-04-12 09:40:30 -05:00
Author
Owner
Copy Link

@AndreBerzun commented on GitHub (Feb 15, 2024):

BERT support was merged 3 days ago into llama.cpp

<!-- gh-comment-id:1945203922 --> @AndreBerzun commented on GitHub (Feb 15, 2024): BERT support was [merged](https://github.com/ggerganov/llama.cpp/pull/5423) 3 days ago into llama.cpp
GiteaMirror commented 2026-04-12 09:40:31 -05:00
Author
Owner
Copy Link

@easp commented on GitHub (Feb 15, 2024):

Looks like there are still kinks being worked out.

<!-- gh-comment-id:1945443776 --> @easp commented on GitHub (Feb 15, 2024): Looks like there are still kinks being worked out.
GiteaMirror commented 2026-04-12 09:40:31 -05:00
Author
Owner
Copy Link

@s-kostyaev commented on GitHub (Feb 15, 2024):

Looks like there are still kinks being worked out.

Link to check the progress https://github.com/ggerganov/llama.cpp/pull/5500

<!-- gh-comment-id:1945511760 --> @s-kostyaev commented on GitHub (Feb 15, 2024): > Looks like there are still kinks being worked out. Link to check the progress https://github.com/ggerganov/llama.cpp/pull/5500
GiteaMirror commented 2026-04-12 09:40:31 -05:00
Author
Owner
Copy Link

@s-kostyaev commented on GitHub (Feb 15, 2024):

Looks like there are still kinks being worked out.

Link to check the progress ggerganov/llama.cpp#5500

It is merged now

<!-- gh-comment-id:1946673048 --> @s-kostyaev commented on GitHub (Feb 15, 2024): > > Looks like there are still kinks being worked out. > > Link to check the progress [ggerganov/llama.cpp#5500](https://github.com/ggerganov/llama.cpp/pull/5500) It is merged now
GiteaMirror commented 2026-04-12 09:40:32 -05:00
Author
Owner
Copy Link

@AndreBerzun commented on GitHub (Feb 19, 2024):

@jmorganca just wanted to follow up and see if this topic is on your roadmap. Since llama.cpp added support for BERT models, this seems like a great low-hanging fruit, no?

Initial support for BERT models has been merged with ggerganov/llama.cpp#5423 and released with b2127. Some kinks related to embedding pooling were fixed with ggerganov/llama.cpp#5500. Batch embedding is supported as well.

There has been a new bug related to the tokenizer implementation but that's it as far as I can tell.

<!-- gh-comment-id:1953250510 --> @AndreBerzun commented on GitHub (Feb 19, 2024): @jmorganca just wanted to follow up and see if this topic is on your roadmap. Since llama.cpp added support for BERT models, this seems like a great low-hanging fruit, no? Initial support for BERT models has been merged with [ggerganov/llama.cpp#5423](https://github.com/ggerganov/llama.cpp/pull/5423) and released with [b2127](https://github.com/ggerganov/llama.cpp/releases/tag/b2127). Some kinks related to embedding pooling were fixed with [ggerganov/llama.cpp#5500](https://github.com/ggerganov/llama.cpp/pull/5500). [Batch embedding](https://github.com/ggerganov/llama.cpp/pull/5466) is supported as well. There has been a new bug related to the [tokenizer implementation](https://github.com/ggerganov/llama.cpp/issues/5496) but that's it as [far as I can tell](https://github.com/ggerganov/llama.cpp/issues?q=is%3Aissue+is%3Aopen+bert).
GiteaMirror commented 2026-04-12 09:40:32 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Feb 20, 2024):

@AndreBerzun it absolutely is – working on it!

<!-- gh-comment-id:1953296819 --> @jmorganca commented on GitHub (Feb 20, 2024): @AndreBerzun it absolutely is – working on it!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#142
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?