github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-05 23:53:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #7219] [CLOSED] FEAT: add rerank support #59053

New Issue
Closed
opened 2026-04-29 13:56:00 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-29 13:56:00 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/7219
Author: @liuy
Created: 10/16/2024
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (3)

  • 2555401 go runner: add rerank support
  • 627fee7 route: add rerank support
  • a144669 enable --reranking flag for rerank handler while starting server

📊 Changes

5 files changed (+275 additions, -3 deletions)

View changed files

📝 api/types.go (+18 -0)
📝 llama/llama.go (+20 -1)
📝 llama/runner/runner.go (+93 -2)
📝 llm/server.go (+68 -0)
📝 server/routes.go (+76 -0)

📄 Description

This patch set is tring to solve #3368, add reranking support in ollama based on the llama.cpp (edc26566), which got reranking support recently.

Basically:
patch 1 - bump llm/llama.cpp to 17bb9280
patch 2 - add rerank support
patch 3 - allow passing extra command to llama server before starting a new llmsever
patch 4 - go runner: add rerank support

TODOs

  1. '--reranking' flag is set and it can only runs rerank model. need to find a way to add '--reranking' flag to the llama server on the fly.
  2. only ext_server is supported, need to rebase llama/* for go runner later

Updates

10.16
patch 3 - allow passing extra command to llama server before starting a new llmsever
solved problem 1.
10.18
rebased on the latest branch main
10.29
add go runner support.
Now both llama ext_server and go runner have rerank support in the same way :)
10.30
removed server.cpp relabed patch as requisted

How to test it

Download the source code

git clone https://github.com/liuy/ollama.git

Prepare the rerank model (Note, llama.cpp only support this model that is top reranking multi-lang model right now)

git clone git@hf.co:BAAI/bge-reranker-v2-m3
python3 ollama/llm/llama.cpp/convert_hf_to_gguf.py bge-reranker-v2-m3/ --outfile bge-reranker-v2-m3-f16.gguf --outtype f16
echo   ' FROM ./bge-reranker-v2-m3-f16.gguf' > modelfile
ollama create bge-reranker-v2-m3 -f modelfile

Compile: make sure you have go, cmake, gcc installed

go generate ./...
go build .

Run the compiled binary

OLLAMA_HOST=127.0.0.1:11435 ./ollama serve #use  port 11435 to avoid mess with running ollama server

Open a new terminal

      curl http://127.0.0.1:11435/api/rerank \
	-H "Content-Type: application/json" \
	-d '{
      "model": "bge-reranker-v2-m3",
      "query": "Organic skincare products for sensitive skin",
      "top_n": 3,
      "documents": [
		"Organic skincare for sensitive skin with aloe vera and chamomile: Imagine the soothing embrace of nature with our organic skincare range, crafted specifically for sensitive skin. Infused with the calming properties of aloe vera and chamomile, each product provides gentle nourishment and protection. Say goodbye to irritation and hello to a glowing, healthy complexion.",
		"New makeup trends focus on bold colors and innovative techniques: Step into the world of cutting-edge beauty with this seasons makeup trends. Bold, vibrant colors and groundbreaking techniques are redefining the art of makeup. From neon eyeliners to holographic highlighters, unleash your creativity and make a statement with every look.",
		"Bio-Hautpflege für empfindliche Haut mit Aloe Vera und Kamille: Erleben Sie die wohltuende Wirkung unserer Bio-Hautpflege, speziell für empfindliche Haut entwickelt. Mit den beruhigenden Eigenschaften von Aloe Vera und Kamille pflegen und schützen unsere Produkte Ihre Haut auf natürliche Weise. Verabschieden Sie sich von Hautirritationen und genießen Sie einen strahlenden Teint.",
		"Neue Make-up-Trends setzen auf kräftige Farben und innovative Techniken: Tauchen Sie ein in die Welt der modernen Schönheit mit den neuesten Make-up-Trends. Kräftige, lebendige Farben und innovative Techniken setzen neue Maßstäbe. Von auffälligen Eyelinern bis hin zu holografischen Highlightern – lassen Sie Ihrer Kreativität freien Lauf und setzen Sie jedes Mal ein Statement.",
		"Cuidado de la piel orgánico para piel sensible con aloe vera y manzanilla: Descubre el poder de la naturaleza con nuestra línea de cuidado de la piel orgánico, diseñada especialmente para pieles sensibles. Enriquecidos con aloe vera y manzanilla, estos productos ofrecen una hidratación y protección suave. Despídete de las irritaciones y saluda a una piel radiante y saludable.",
		"Las nuevas tendencias de maquillaje se centran en colores vivos y técnicas innovadoras: Entra en el fascinante mundo del maquillaje con las tendencias más actuales. Colores vivos y técnicas innovadoras están revolucionando el arte del maquillaje. Desde delineadores neón hasta iluminadores holográficos, desata tu creatividad y destaca en cada look.",
		"针对敏感肌专门设计的天然有机护肤产品：体验由芦荟和洋甘菊提取物带来的自然呵护。我们的护肤产品特别为敏感肌设计，温和滋润，保护您的肌肤不受刺激。让您的肌肤告别不适，迎来健康光彩。",
		"新的化妆趋势注重鲜艳的颜色和创新的技巧：进入化妆艺术的新纪元，本季的化妆趋势以大胆的颜色和创新的技巧为主。无论是霓虹眼线还是全息高光，每一款妆容都能让您脱颖而出，展现独特魅力。",
		"敏感肌のために特別に設計された天然有機スキンケア製品: アロエベラとカモミールのやさしい力で、自然の抱擁を感じてください。敏感肌用に特別に設計された私たちのスキンケア製品は、肌に優しく栄養を与え、保護します。肌トラブルにさようなら、輝く健康な肌にこんにちは。",
		"新しいメイクのトレンドは鮮やかな色と革新的な技術に焦点を当てています: 今シーズンのメイクアップトレンドは、大胆な色彩と革新的な技術に注目しています。ネオンアイライナーからホログラフィックハイライターまで、クリエイティビティを解き放ち、毎回ユニークなルックを演出しましょう。"
      ]
    }' | jq

If you are lucky, you'll get the reranked result in a descendant manner like following:

{
  "model": "bge-reranker-v2-m3",
  "results": [
    {
      "document": "敏感肌のために特別に設計された天然有機スキンケア製品: アロエベラとカモミールのやさしい力で、自然の抱擁を感じてください。 敏感肌用に特別に設計された私たちのスキンケア製品は、肌に優しく栄養を与え、保護します。肌トラブルにさようなら、輝く健康な肌にこんにちは。",
      "relevance_score": 6.4258623
    },
    {
      "document": "Organic skincare for sensitive skin with aloe vera and chamomile: Imagine the soothing embrace of nature with our organic skincare range, crafted specifically for sensitive skin. Infused with the calming properties of aloe vera and chamomile, each product provides gentle nourishment and protection. Say goodbye to irritation and hello to a glowing, healthy complexion.",
      "relevance_score": 6.3774652
    },
    {
      "document": "针对敏感肌专门设计的天然有机护肤产品：体验由芦荟和洋甘菊提取物带来的自然呵护。���们的护肤产品特别为敏感肌设计，温和滋润， 保护您的肌肤不受刺激。让您的肌肤告别不适，迎来健康光彩。",
      "relevance_score": 5.556752
    }
  ]
}

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/7219 **Author:** [@liuy](https://github.com/liuy) **Created:** 10/16/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (3) - [`2555401`](https://github.com/ollama/ollama/commit/2555401c681110044a9ae2a378f203d4c6ce2bd4) go runner: add rerank support - [`627fee7`](https://github.com/ollama/ollama/commit/627fee7fee58b0bb3295fcd1400e9fd63c428e77) route: add rerank support - [`a144669`](https://github.com/ollama/ollama/commit/a144669072e98c5d5d393a0bd0a3533256fd3f57) enable --reranking flag for rerank handler while starting server ### 📊 Changes **5 files changed** (+275 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+18 -0) 📝 `llama/llama.go` (+20 -1) 📝 `llama/runner/runner.go` (+93 -2) 📝 `llm/server.go` (+68 -0) 📝 `server/routes.go` (+76 -0) </details> ### 📄 Description This patch set is tring to solve #3368, add reranking support in ollama based on the llama.cpp (edc26566), which got reranking support recently. Basically: patch 1 - bump llm/llama.cpp to 17bb9280 patch 2 - add rerank support patch 3 - allow passing extra command to llama server before starting a new llmsever patch 4 - go runner: add rerank support ### TODOs 1. ~~**'--reranking' flag is set and it can only runs rerank model. need to find a way to add '--reranking' flag to the llama server on the fly.**~~ 2. ~~only ext_server is supported, need to rebase llama/* for go runner later~~ ### Updates **10.16** patch 3 - allow passing extra command to llama server before starting a new llmsever solved problem 1. **10.18** rebased on the latest branch main **10.29** add go runner support. **Now both llama ext_server and go runner have rerank support in the same way** :) **10.30** removed server.cpp relabed patch as requisted ### How to test it #### Download the source code ``` git clone https://github.com/liuy/ollama.git ```` #### Prepare the rerank model (Note, llama.cpp only support [this](https://huggingface.co/BAAI/bge-reranker-v2-m3) model that is top reranking multi-lang model right now) ``` git clone git@hf.co:BAAI/bge-reranker-v2-m3 python3 ollama/llm/llama.cpp/convert_hf_to_gguf.py bge-reranker-v2-m3/ --outfile bge-reranker-v2-m3-f16.gguf --outtype f16 echo ' FROM ./bge-reranker-v2-m3-f16.gguf' > modelfile ollama create bge-reranker-v2-m3 -f modelfile ``` #### Compile: make sure you have go, cmake, gcc installed ``` go generate ./... go build . ``` #### Run the compiled binary ``` OLLAMA_HOST=127.0.0.1:11435 ./ollama serve #use port 11435 to avoid mess with running ollama server ``` #### Open a new terminal ``` curl http://127.0.0.1:11435/api/rerank \ -H "Content-Type: application/json" \ -d '{ "model": "bge-reranker-v2-m3", "query": "Organic skincare products for sensitive skin", "top_n": 3, "documents": [ "Organic skincare for sensitive skin with aloe vera and chamomile: Imagine the soothing embrace of nature with our organic skincare range, crafted specifically for sensitive skin. Infused with the calming properties of aloe vera and chamomile, each product provides gentle nourishment and protection. Say goodbye to irritation and hello to a glowing, healthy complexion.", "New makeup trends focus on bold colors and innovative techniques: Step into the world of cutting-edge beauty with this seasons makeup trends. Bold, vibrant colors and groundbreaking techniques are redefining the art of makeup. From neon eyeliners to holographic highlighters, unleash your creativity and make a statement with every look.", "Bio-Hautpflege für empfindliche Haut mit Aloe Vera und Kamille: Erleben Sie die wohltuende Wirkung unserer Bio-Hautpflege, speziell für empfindliche Haut entwickelt. Mit den beruhigenden Eigenschaften von Aloe Vera und Kamille pflegen und schützen unsere Produkte Ihre Haut auf natürliche Weise. Verabschieden Sie sich von Hautirritationen und genießen Sie einen strahlenden Teint.", "Neue Make-up-Trends setzen auf kräftige Farben und innovative Techniken: Tauchen Sie ein in die Welt der modernen Schönheit mit den neuesten Make-up-Trends. Kräftige, lebendige Farben und innovative Techniken setzen neue Maßstäbe. Von auffälligen Eyelinern bis hin zu holografischen Highlightern – lassen Sie Ihrer Kreativität freien Lauf und setzen Sie jedes Mal ein Statement.", "Cuidado de la piel orgánico para piel sensible con aloe vera y manzanilla: Descubre el poder de la naturaleza con nuestra línea de cuidado de la piel orgánico, diseñada especialmente para pieles sensibles. Enriquecidos con aloe vera y manzanilla, estos productos ofrecen una hidratación y protección suave. Despídete de las irritaciones y saluda a una piel radiante y saludable.", "Las nuevas tendencias de maquillaje se centran en colores vivos y técnicas innovadoras: Entra en el fascinante mundo del maquillaje con las tendencias más actuales. Colores vivos y técnicas innovadoras están revolucionando el arte del maquillaje. Desde delineadores neón hasta iluminadores holográficos, desata tu creatividad y destaca en cada look.", "针对敏感肌专门设计的天然有机护肤产品：体验由芦荟和洋甘菊提取物带来的自然呵护。我们的护肤产品特别为敏感肌设计，温和滋润，保护您的肌肤不受刺激。让您的肌肤告别不适，迎来健康光彩。", "新的化妆趋势注重鲜艳的颜色和创新的技巧：进入化妆艺术的新纪元，本季的化妆趋势以大胆的颜色和创新的技巧为主。无论是霓虹眼线还是全息高光，每一款妆容都能让您脱颖而出，展现独特魅力。", "敏感肌のために特別に設計された天然有機スキンケア製品: アロエベラとカモミールのやさしい力で、自然の抱擁を感じてください。敏感肌用に特別に設計された私たちのスキンケア製品は、肌に優しく栄養を与え、保護します。肌トラブルにさようなら、輝く健康な肌にこんにちは。", "新しいメイクのトレンドは鮮やかな色と革新的な技術に焦点を当てています: 今シーズンのメイクアップトレンドは、大胆な色彩と革新的な技術に注目しています。ネオンアイライナーからホログラフィックハイライターまで、クリエイティビティを解き放ち、毎回ユニークなルックを演出しましょう。" ] }' | jq ``` #### If you are lucky, you'll get the reranked result in a descendant manner like following: ``` { "model": "bge-reranker-v2-m3", "results": [ { "document": "敏感肌のために特別に設計された天然有機スキンケア製品: アロエベラとカモミールのやさしい力で、自然の抱擁を感じてください。 敏感肌用に特別に設計された私たちのスキンケア製品は、肌に優しく栄養を与え、保護します。肌トラブルにさようなら、輝く健康な肌にこんにちは。", "relevance_score": 6.4258623 }, { "document": "Organic skincare for sensitive skin with aloe vera and chamomile: Imagine the soothing embrace of nature with our organic skincare range, crafted specifically for sensitive skin. Infused with the calming properties of aloe vera and chamomile, each product provides gentle nourishment and protection. Say goodbye to irritation and hello to a glowing, healthy complexion.", "relevance_score": 6.3774652 }, { "document": "针对敏感肌专门设计的天然有机护肤产品：体验由芦荟和洋甘菊提取物带来的自然呵护。���们的护肤产品特别为敏感肌设计，温和滋润， 保护您的肌肤不受刺激。让您的肌肤告别不适，迎来健康光彩。", "relevance_score": 5.556752 } ] } ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-29 13:56:00 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#59053
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?