Reranking models #2075

@CHEFRA commented on GitHub (Sep 10, 2024):

netease-youdao/bce-reranker-base_v1
BAAI/bge-reranker-v2-m3
thx

@CHEFRA commented on GitHub (Sep 10, 2024): netease-youdao/bce-reranker-base_v1 BAAI/bge-reranker-v2-m3 thx

GiteaMirror commented

@applepieiris commented on GitHub (Sep 12, 2024):

I see several issues are related to the reranker models and they are all closed by the authors. Why? Is the reranker model planned? I do not see any reranker models published in your official model library.

@applepieiris commented on GitHub (Sep 12, 2024): I see several issues are related to the reranker models and they are all closed by the authors. Why? Is the reranker model planned? I do not see any reranker models published in your official model library.

GiteaMirror commented

@BradKML commented on GitHub (Sep 12, 2024):

@applepieiris keeping every reranker into one single issue is easier, since they are all just data files (models are just numbers) that needs a little wrapper code to interact with.

@BradKML commented on GitHub (Sep 12, 2024): @applepieiris keeping every reranker into one single issue is easier, since they are all just data files (models are just numbers) that needs a little wrapper code to interact with.

GiteaMirror commented

@vaclcer commented on GitHub (Sep 24, 2024):

llama.cpp is about to get ranking support, better to start working on those endpoints :) https://github.com/ggerganov/llama.cpp/pull/9510#issuecomment-2367526441

@vaclcer commented on GitHub (Sep 24, 2024): llama.cpp is about to get ranking support, better to start working on those endpoints :) https://github.com/ggerganov/llama.cpp/pull/9510#issuecomment-2367526441

GiteaMirror commented

@Bearsaerker commented on GitHub (Sep 28, 2024):

Reranker support is merged, this should be now easy to implement and extremely important for RAG :) https://github.com/ggerganov/llama.cpp/pull/9510#issuecomment-2367526441

@Bearsaerker commented on GitHub (Sep 28, 2024): Reranker support is merged, this should be now easy to implement and extremely important for RAG :) https://github.com/ggerganov/llama.cpp/pull/9510#issuecomment-2367526441

GiteaMirror commented

@lixc123 commented on GitHub (Oct 1, 2024):

Ollama will be more powerful if it supports rerank model

@lixc123 commented on GitHub (Oct 1, 2024): Ollama will be more powerful if it supports rerank model

GiteaMirror commented

@bdqfork commented on GitHub (Oct 14, 2024):

Any update?

@bdqfork commented on GitHub (Oct 14, 2024): Any update?

GiteaMirror commented

@liuy commented on GitHub (Oct 14, 2024):

Hi there. I'm going to add 'reranking' support(almost done) to ollama since llama.cpp has supported reranking days ago.
I am done with 'rerank' support on ollama part (routes.go and server.go) and I'm a little bit confused about how to rebase llama.cpp to the latest commit correctly.

The problem is that there are two spots holding llama.cpp source files, namely:

ollama/llama (seems to me go runner use it and it is NOT a git repo)
ollama/llm/llama.cpp (seems to me ext_server use it and it IS a git repo, a little bit easier to rebase)

so for now, do I need to rebase both 1 and 2 or just 2 is okay? (since go running is under development).
can we just use llm/llama.cpp/examples/server instead of llm/ext_server?

@jmorganca @mxyng Thanks in advance!

@liuy commented on GitHub (Oct 14, 2024): Hi there. I'm going to add 'reranking' support(almost done) to ollama since llama.cpp has supported reranking days ago. I am done with 'rerank' support on ollama part (routes.go and server.go) and I'm a little bit confused about **how to rebase llama.cpp to the latest commit correctly**. The problem is that there are two spots holding llama.cpp source files, namely: 1. ollama/llama (seems to me go runner use it and it is NOT a git repo) 2. ollama/llm/llama.cpp (seems to me ext_server use it and it IS a git repo, a little bit easier to rebase) so for now, do I need to rebase both 1 and 2 or just 2 is okay? (since go running is under development). can we just use llm/llama.cpp/examples/server instead of llm/ext_server? @jmorganca @mxyng Thanks in advance!

GiteaMirror commented

2025-11-12 10:44:16 -06:00

@aceliuchanghong commented on GitHub (Oct 15, 2024):

looking forward of it

@aceliuchanghong commented on GitHub (Oct 15, 2024): looking forward of it

GiteaMirror commented

@jessegross commented on GitHub (Oct 15, 2024):

Hi there. I'm going to add 'reranking' support(almost done) to ollama since llama.cpp has supported reranking days ago. I am done with 'rerank' support on ollama part (routes.go and server.go) and I'm a little bit confused about how to rebase llama.cpp to the latest commit correctly.

The problem is that there are two spots holding llama.cpp source files, namely:

ollama/llama (seems to me go runner use it and it is NOT a git repo)

ollama/llm/llama.cpp (seems to me ext_server use it and it IS a git repo, a little bit easier to rebase)

so for now, do I need to rebase both 1 and 2 or just 2 is okay? (since go running is under development). can we just use llm/llama.cpp/examples/server instead of llm/ext_server?

@jmorganca @mxyng Thanks in advance!

For now, we need to update both. The Go server (#1) is currently feature complete and is just getting some soak time before switching over to it, so we don't want to add new features that will cause it to fall behind.

There is another PR that is in the process of bumping llama.cpp in both of those locations, you should check it out - it might already provide what you are looking for:
https://github.com/ollama/ollama/pull/6760

@jessegross commented on GitHub (Oct 15, 2024): > Hi there. I'm going to add 'reranking' support(almost done) to ollama since llama.cpp has supported reranking days ago. I am done with 'rerank' support on ollama part (routes.go and server.go) and I'm a little bit confused about **how to rebase llama.cpp to the latest commit correctly**. > > The problem is that there are two spots holding llama.cpp source files, namely: > > 1. ollama/llama (seems to me go runner use it and it is NOT a git repo) > 2. ollama/llm/llama.cpp (seems to me ext_server use it and it IS a git repo, a little bit easier to rebase) > > so for now, do I need to rebase both 1 and 2 or just 2 is okay? (since go running is under development). can we just use llm/llama.cpp/examples/server instead of llm/ext_server? > > @jmorganca @mxyng Thanks in advance! For now, we need to update both. The Go server (#1) is currently feature complete and is just getting some soak time before switching over to it, so we don't want to add new features that will cause it to fall behind. There is another PR that is in the process of bumping llama.cpp in both of those locations, you should check it out - it might already provide what you are looking for: https://github.com/ollama/ollama/pull/6760

GiteaMirror commented

2025-11-12 10:44:16 -06:00

@liuy commented on GitHub (Oct 16, 2024):

Hi there, I've made a PR for reranking support #7219 . Reviews and comments are welcomed :)

@liuy commented on GitHub (Oct 16, 2024): Hi there, I've made a PR for reranking support #7219 . Reviews and comments are welcomed :)

GiteaMirror commented

2025-11-12 10:44:17 -06:00

@cruzanstx commented on GitHub (Oct 27, 2024):

any update on this?

@cruzanstx commented on GitHub (Oct 27, 2024): any update on this?

GiteaMirror commented

2025-11-12 10:44:17 -06:00

@liuy commented on GitHub (Oct 29, 2024):

any update on this?

I've finished the patch set, waiting for review. Now both the traditional llama server and the new go runner under the development support reranking, which was required by the ollama reviewer. This slowed down merging a bit.

@liuy commented on GitHub (Oct 29, 2024): > any update on this? I've finished the patch set, waiting for review. Now both the traditional llama server and the new go runner under the development support reranking, which was required by the ollama reviewer. This slowed down merging a bit.

GiteaMirror commented

2025-11-12 10:44:17 -06:00

@orkutmuratyilmaz commented on GitHub (Oct 31, 2024):

maybe it is better to adapt the patch set for 0.4.0 version?

@orkutmuratyilmaz commented on GitHub (Oct 31, 2024): maybe it is better to adapt the patch set for 0.4.0 version?

GiteaMirror commented

2025-11-12 10:44:18 -06:00

@beeant0512 commented on GitHub (Nov 5, 2024):

+1

@beeant0512 commented on GitHub (Nov 5, 2024): +1

GiteaMirror commented

2025-11-12 10:44:18 -06:00

@cleverpig commented on GitHub (Nov 12, 2024):

+1

@cleverpig commented on GitHub (Nov 12, 2024): +1

GiteaMirror commented

2025-11-12 10:44:18 -06:00

@stefan-detter-m commented on GitHub (Nov 15, 2024):

+1

@stefan-detter-m commented on GitHub (Nov 15, 2024): +1

GiteaMirror commented

2025-11-12 10:44:19 -06:00

@netandreus commented on GitHub (Nov 29, 2024):

+1 Please add reranking support.

@netandreus commented on GitHub (Nov 29, 2024): +1 Please add reranking support.

GiteaMirror commented

2025-11-12 10:44:19 -06:00

@Proteusiq commented on GitHub (Nov 30, 2024):

It would be great to close this before the year end 😊 Do we know what the huddle holding this behind?

@Proteusiq commented on GitHub (Nov 30, 2024): It would be great to close this before the year end 😊 Do we know what the huddle holding this behind?

GiteaMirror commented

2025-11-12 10:44:19 -06:00

@netandreus commented on GitHub (Dec 4, 2024):

Guys, please, we are waiting for this!

@netandreus commented on GitHub (Dec 4, 2024): Guys, please, we are waiting for this!

GiteaMirror commented

2025-11-12 10:44:20 -06:00

@Kraego commented on GitHub (Dec 6, 2024):

There is a ongoing PR: https://github.com/ollama/ollama/pull/7219

@Kraego commented on GitHub (Dec 6, 2024): There is a ongoing PR: https://github.com/ollama/ollama/pull/7219

GiteaMirror commented

2025-11-12 10:44:20 -06:00

@netandreus commented on GitHub (Dec 7, 2024):

Check #3749

Also: https://ollama.com/search?q=rerank

Hm, there is a model https://ollama.com/linux6200/bge-reranker-v2-m3
How should I pass documents to it for reranking? May be you know?

@netandreus commented on GitHub (Dec 7, 2024): > Check #3749 > > Also: https://ollama.com/search?q=rerank Hm, there is a model https://ollama.com/linux6200/bge-reranker-v2-m3 How should I pass documents to it for reranking? May be you know?

GiteaMirror commented

2025-11-12 10:44:20 -06:00

@Kisaragi-ng commented on GitHub (Dec 9, 2024):

Check #3749
Also: https://ollama.com/search?q=rerank

Hm, there is a model https://ollama.com/linux6200/bge-reranker-v2-m3 How should I pass documents to it for reranking? May be you know?

ollama's reranking model works in open-webui. there's an settings specific for document embedding, and you can choose reranking model in that options.

@Kisaragi-ng commented on GitHub (Dec 9, 2024): > > Check #3749 > > Also: https://ollama.com/search?q=rerank > > Hm, there is a model https://ollama.com/linux6200/bge-reranker-v2-m3 How should I pass documents to it for reranking? May be you know? ollama's reranking model works in open-webui. there's an settings specific for document embedding, and you can choose reranking model in that options.

GiteaMirror commented

2025-11-12 10:44:21 -06:00

@cope commented on GitHub (Dec 9, 2024):

What's with the thumbs down?
At least comment, so that I know what I did wrong.

@cope commented on GitHub (Dec 9, 2024): What's with the thumbs down? At least comment, so that I know what I did wrong.

GiteaMirror commented

2025-11-12 10:44:21 -06:00

@Enteee commented on GitHub (Dec 9, 2024):

What's with the thumbs down? At least comment, so that I know what I did wrong.

#3749 was closed as duplicate of this issue here
The models you find using "reranking" are actually used for embeddings.

Without api support ollama can not do reranking. See #7219

@Enteee commented on GitHub (Dec 9, 2024): > What's with the thumbs down? At least comment, so that I know what I did wrong. * #3749 was closed as duplicate of this issue here * The models you find using "reranking" are actually used for embeddings. Without api support ollama can not do reranking. See #7219

GiteaMirror commented

2025-11-12 10:44:21 -06:00

@netandreus commented on GitHub (Dec 12, 2024):

Yes, confirmed. We need rerank models support from Ollama.

@netandreus commented on GitHub (Dec 12, 2024): Yes, confirmed. We need rerank models support from Ollama.

GiteaMirror commented

2025-11-12 10:44:22 -06:00

@lrq3000 commented on GitHub (Dec 23, 2024):

Please add support for reranker models.

Also I would like to suggest support for this one, which in my tests works better for long documents with data overshadowing LLM's training data (eg, data that contradicts what an LLM was trained on): maidalun1020/bce-reranker-base_v1

@lrq3000 commented on GitHub (Dec 23, 2024): Please add support for reranker models. Also I would like to suggest support for this one, which in my tests works better for long documents with data overshadowing LLM's training data (eg, data that contradicts what an LLM was trained on): maidalun1020/bce-reranker-base_v1

GiteaMirror commented

2025-11-12 10:44:22 -06:00

@BradKML commented on GitHub (Dec 24, 2024):

I think Ollama already took a stance on this by making this easier (even though having dedicated names would be better than ollama run hf.co/{username}/{repository}:{quantization) https://www.kdnuggets.com/use-almost-any-language-model-locally-with-ollama-and-hugging-face-hub

@BradKML commented on GitHub (Dec 24, 2024): I think Ollama already took a stance on this by making this easier (even though having dedicated names would be better than `ollama run hf.co/{username}/{repository}:{quantization`) https://www.kdnuggets.com/use-almost-any-language-model-locally-with-ollama-and-hugging-face-hub

GiteaMirror commented

@Rakhsan commented on GitHub (Jan 20, 2025):

If rerank models are supported. API docs is the first thing that we need

@Rakhsan commented on GitHub (Jan 20, 2025): If rerank models are supported. API docs is the first thing that we need

GiteaMirror commented

@yuanbaobaoo commented on GitHub (Jan 23, 2025):

many pepole have this requirement, please add support for reranker models and about api docs

@yuanbaobaoo commented on GitHub (Jan 23, 2025): many pepole have this requirement, please add support for reranker models and about api docs

GiteaMirror commented

@Proteusiq commented on GitHub (Jan 23, 2025):

Alternative

Until /rerankendpoint is supported by Ollama, I have opt for llama.cpp to serve reranking models
For example, bgemodel

 llama-server --hf-repo klnstpr/bge-reranker-v2-m3-Q8_0-GGUF --hf-file bge-reranker-v2-m3-q8_0.gguf --port 11435 --reranking --pooling rank

Pointing services to port 11435 will now work in with bge reranker.

curl http://127.0.0.1:11435/v1/rerank \
    -H "Content-Type: application/json" \
    -d '{
        "model": "whatever",
            "query": "What is Corona disease?",
            "top_n": 3,
            "documents": [
                "Corona is a Mexican brand of beer produced by Grupo Modelo in Mexico and exported to markets around the world.",
            "it is a bear",
            "COVID-19 is a contagious illness caused by the a virus SARS-CoV-2."
            ]
    }' | jq

@Proteusiq commented on GitHub (Jan 23, 2025): ### Alternative Until `/rerank`endpoint is supported by Ollama, I have opt for [llama.cpp](https://github.com/ggerganov/llama.cpp) to serve reranking models For example, `bge`model ```sh llama-server --hf-repo klnstpr/bge-reranker-v2-m3-Q8_0-GGUF --hf-file bge-reranker-v2-m3-q8_0.gguf --port 11435 --reranking --pooling rank ``` Pointing services to port `11435` will now work in with bge reranker. ```sh curl http://127.0.0.1:11435/v1/rerank \ -H "Content-Type: application/json" \ -d '{ "model": "whatever", "query": "What is Corona disease?", "top_n": 3, "documents": [ "Corona is a Mexican brand of beer produced by Grupo Modelo in Mexico and exported to markets around the world.", "it is a bear", "COVID-19 is a contagious illness caused by the a virus SARS-CoV-2." ] }' | jq ```

GiteaMirror commented

2025-11-12 10:44:24 -06:00

@Scrazzled commented on GitHub (Feb 15, 2025):

It looks like this is related to feature ticket 7219. The last update there was Jan 14, 2025, saying that work on it is stalled.

@Scrazzled commented on GitHub (Feb 15, 2025): It looks like this is related to feature ticket [7219](https://github.com/ollama/ollama/pull/7219). The last update there was Jan 14, 2025, saying that work on it is stalled.

GiteaMirror commented

@RLinnae commented on GitHub (Mar 14, 2025):

Is there still no support for re-rankers?

@RLinnae commented on GitHub (Mar 14, 2025): Is there still no support for re-rankers?

GiteaMirror commented

2025-11-12 10:44:24 -06:00

@dkker2025 commented on GitHub (Mar 20, 2025):

https://ollama.com/qllama

@dkker2025 commented on GitHub (Mar 20, 2025): https://ollama.com/qllama

GiteaMirror commented

2025-11-12 10:44:25 -06:00

@Proteusiq commented on GitHub (Mar 20, 2025):

https://ollama.com/qllama

We are missing /rerank endpoint 😊. We have plenty of models.

@Proteusiq commented on GitHub (Mar 20, 2025): > https://ollama.com/qllama We are missing `/rerank` endpoint 😊. We have plenty of models.

GiteaMirror commented

2025-11-12 10:44:25 -06:00

@jagga99 commented on GitHub (Mar 20, 2025):

please add reranker support

@jagga99 commented on GitHub (Mar 20, 2025): please add reranker support

GiteaMirror commented

2025-11-12 10:44:26 -06:00

@suifengfengye commented on GitHub (Mar 31, 2025):

Does Ollama support reranking model? When I call localhost:11434/api/rerank, it returns 404

@suifengfengye commented on GitHub (Mar 31, 2025): Does Ollama support reranking model? When I call localhost:11434/api/rerank, it returns 404

GiteaMirror commented

2025-11-12 10:44:26 -06:00

@Proteusiq commented on GitHub (Apr 1, 2025):

Does Ollama support reranking model? When I call localhost:11434/api/rerank, it returns 404

Not yet 😩😢😭. I have opt for llama.cpp as temporary solution. See comment above.

@Proteusiq commented on GitHub (Apr 1, 2025): > Does Ollama support reranking model? When I call localhost:11434/api/rerank, it returns 404 Not yet 😩😢😭. I have opt for llama.cpp as temporary solution. See comment above.

GiteaMirror commented

2025-11-12 10:44:26 -06:00

@xiaomeixw commented on GitHub (Apr 23, 2025):

need rerank！！！！！！

@xiaomeixw commented on GitHub (Apr 23, 2025): need rerank！！！！！！

GiteaMirror commented

2025-11-12 10:44:27 -06:00

@bitons commented on GitHub (May 16, 2025):

We need it

@bitons commented on GitHub (May 16, 2025): We need it

GiteaMirror commented

@Pl8tinium commented on GitHub (May 16, 2025):

also looking forward :)

@Pl8tinium commented on GitHub (May 16, 2025): also looking forward :)

GiteaMirror commented

@victoryangn commented on GitHub (May 19, 2025):

we need it

@victoryangn commented on GitHub (May 19, 2025): we need it

GiteaMirror commented

@mengdeer589 commented on GitHub (May 27, 2025):

we need it

@mengdeer589 commented on GitHub (May 27, 2025): we need it

GiteaMirror commented

@jackywu commented on GitHub (May 31, 2025):

+1

@jackywu commented on GitHub (May 31, 2025): +1

GiteaMirror commented

@DemonJun commented on GitHub (Jun 1, 2025):

+1

@DemonJun commented on GitHub (Jun 1, 2025): +1

GiteaMirror commented

@shidio commented on GitHub (Jun 9, 2025):

+1

@shidio commented on GitHub (Jun 9, 2025): +1

GiteaMirror commented

@x22x22 commented on GitHub (Jun 10, 2025):

+1

@x22x22 commented on GitHub (Jun 10, 2025): +1

GiteaMirror commented

2025-11-12 10:44:31 -06:00

@ZevAlain commented on GitHub (Jun 11, 2025):

we need it

@ZevAlain commented on GitHub (Jun 11, 2025): we need it

GiteaMirror commented

@skaterqiang commented on GitHub (Jun 11, 2025):

we need it

@skaterqiang commented on GitHub (Jun 11, 2025): we need it

GiteaMirror commented

2025-11-12 10:44:31 -06:00

@mihucho commented on GitHub (Jun 12, 2025):

we need it

@mihucho commented on GitHub (Jun 12, 2025): we need it

GiteaMirror commented

@dzy888 commented on GitHub (Jun 12, 2025):

+1

@dzy888 commented on GitHub (Jun 12, 2025): +1

GiteaMirror commented

@FlorinCiobotea commented on GitHub (Jun 12, 2025):

+1

@FlorinCiobotea commented on GitHub (Jun 12, 2025): +1

GiteaMirror commented

@dengcao commented on GitHub (Jun 12, 2025):

仍然不支持重排序模型，有点失望。

@dengcao commented on GitHub (Jun 12, 2025): 仍然不支持重排序模型，有点失望。

GiteaMirror commented

2025-11-12 10:44:33 -06:00

@ghost commented on GitHub (Jun 16, 2025):

Please add support for /API/rerank. Whole project comes to halt, when realizing feature not implemented. So cruicial for good RAG software.

@ghost commented on GitHub (Jun 16, 2025): Please add support for /API/rerank. Whole project comes to halt, when realizing feature not implemented. So cruicial for good RAG software.

GiteaMirror commented

@cshaptx4869 commented on GitHub (Jun 19, 2025):

+1

@cshaptx4869 commented on GitHub (Jun 19, 2025): +1

GiteaMirror commented

2025-11-12 10:44:33 -06:00

@jumuning commented on GitHub (Jun 20, 2025):

+1

@jumuning commented on GitHub (Jun 20, 2025): +1

GiteaMirror commented

2025-11-12 10:44:33 -06:00

@dengcao commented on GitHub (Jun 20, 2025):

2025年6月26日更新说明
·本项目旨在解决Qwen3-Reranker模型无法通过Vllm平台直接部署的问题。

·采用vllm最新的开发版制作了Docker镜像dengcao/vllm-openai : v0.9.2-dev，经测试正常，可放心使用。

·修复了Qwen3-Reranker排序结果可能不准确的问题。

注意：2025年6月26日之前已下载本项目的，请删除对应的docker容器和文件后重新使用此方法部署，即可完美在Vllm上运行Qwen3-Reranker模型。

You can use Vllm to deploy the Qwen3-Reranker large model.There is already a temporary solution:

ModelsCope：
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-8B
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-4B
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-0.6B

GitHub：
https://github.com/dengcao/Qwen3-Reranker-8B
https://github.com/dengcao/Qwen3-Reranker-4B
https://github.com/dengcao/Qwen3-Reranker-0.6B

@dengcao commented on GitHub (Jun 20, 2025): 2025年6月26日更新说明 ·本项目旨在解决Qwen3-Reranker模型无法通过Vllm平台直接部署的问题。 ·采用vllm最新的开发版制作了Docker镜像dengcao/vllm-openai : v0.9.2-dev，经测试正常，可放心使用。 ·修复了Qwen3-Reranker排序结果可能不准确的问题。注意：2025年6月26日之前已下载本项目的，请删除对应的docker容器和文件后重新使用此方法部署，即可完美在Vllm上运行Qwen3-Reranker模型。 You can use Vllm to deploy the Qwen3-Reranker large model.There is already a temporary solution: ModelsCope： https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-8B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-4B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-0.6B GitHub： https://github.com/dengcao/Qwen3-Reranker-8B https://github.com/dengcao/Qwen3-Reranker-4B https://github.com/dengcao/Qwen3-Reranker-0.6B

GiteaMirror commented

2025-11-12 10:44:34 -06:00

@cruzanstx commented on GitHub (Jun 21, 2025):

+1

@cruzanstx commented on GitHub (Jun 21, 2025): +1

GiteaMirror commented

2025-11-12 10:44:34 -06:00

@deerleo commented on GitHub (Jun 30, 2025):

we need this function

@deerleo commented on GitHub (Jun 30, 2025): we need this function

GiteaMirror commented

2025-11-12 10:44:35 -06:00

@lixc123 commented on GitHub (Jul 1, 2025):

This feature is needed urgently!!! It’s 2025, reranker is very important!!

@lixc123 commented on GitHub (Jul 1, 2025): This feature is needed urgently!!! It’s 2025, reranker is very important!!

GiteaMirror commented

2025-11-12 10:44:35 -06:00

@Choogle-Ma commented on GitHub (Jul 7, 2025):

+1 we need it!

@Choogle-Ma commented on GitHub (Jul 7, 2025): +1 we need it!

GiteaMirror commented

2025-11-12 10:44:35 -06:00

@tianlichunhong commented on GitHub (Jul 19, 2025):

2025年6月26日更新说明 ·本项目旨在解决Qwen3-Reranker模型无法通过Vllm平台直接部署的问题。

·采用vllm最新的开发版制作了Docker镜像dengcao/vllm-openai : v0.9.2-dev，经测试正常，可放心使用。

·修复了Qwen3-Reranker排序结果可能不准确的问题。

注意：2025年6月26日之前已下载本项目的，请删除对应的docker容器和文件后重新使用此方法部署，即可完美在Vllm上运行Qwen3-Reranker模型。

You can use Vllm to deploy the Qwen3-Reranker large model.There is already a temporary solution:

ModelsCope： https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-8B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-4B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-0.6B

GitHub： https://github.com/dengcao/Qwen3-Reranker-8B https://github.com/dengcao/Qwen3-Reranker-4B https://github.com/dengcao/Qwen3-Reranker-0.6B

如何使用？我发现ollama这个API接口 http://127.0.0.0:11434/api/rerank 并不存在，我设置了Environment="OLLAMA_NEW_ENGINE=1" 也是不行的。能否给个如何启动ollama对rerank的支持？谢谢！

@tianlichunhong commented on GitHub (Jul 19, 2025): > 2025年6月26日更新说明 ·本项目旨在解决Qwen3-Reranker模型无法通过Vllm平台直接部署的问题。 > > ·采用vllm最新的开发版制作了Docker镜像dengcao/vllm-openai : v0.9.2-dev，经测试正常，可放心使用。 > > ·修复了Qwen3-Reranker排序结果可能不准确的问题。 > > 注意：2025年6月26日之前已下载本项目的，请删除对应的docker容器和文件后重新使用此方法部署，即可完美在Vllm上运行Qwen3-Reranker模型。 > > You can use Vllm to deploy the Qwen3-Reranker large model.There is already a temporary solution: > > ModelsCope： https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-8B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-4B https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-0.6B > > GitHub： https://github.com/dengcao/Qwen3-Reranker-8B https://github.com/dengcao/Qwen3-Reranker-4B https://github.com/dengcao/Qwen3-Reranker-0.6B 如何使用？我发现ollama这个API接口 http://127.0.0.0:11434/api/rerank 并不存在，我设置了Environment="OLLAMA_NEW_ENGINE=1" 也是不行的。能否给个如何启动ollama对rerank的支持？谢谢！

GiteaMirror commented

@AuditAIH commented on GitHub (Jul 21, 2025):

如何使用？我发现ollama这个API接口 http://127.0.0.0:11434/api/rerank 并不存在，我设置了Environment="OLLAMA_NEW_ENGINE=1" 也是不行的。能否给个如何启动ollama对rerank的支持？谢谢！

If you need to run rerank on Ollama, you need to compile it yourself. Currently, enthusiastic contributors have submitted code reviews, which have not yet been merged. For details, please refer to https://github.com/ollama/ollama/pull/11389#issuecomment-3089786702. If you need the compiled Ollama binary files and lib, please refer to the link: https://ollama.com/AuditAid/Qwen3_Reranker

@AuditAIH commented on GitHub (Jul 21, 2025): > 如何使用？我发现ollama这个API接口 http://127.0.0.0:11434/api/rerank 并不存在，我设置了Environment="OLLAMA_NEW_ENGINE=1" 也是不行的。能否给个如何启动ollama对rerank的支持？谢谢！ > If you need to run rerank on Ollama, you need to compile it yourself. Currently, enthusiastic contributors have submitted code reviews, which have not yet been merged. For details, please refer to https://github.com/ollama/ollama/pull/11389#issuecomment-3089786702. If you need the compiled Ollama binary files and lib, please refer to the link: https://ollama.com/AuditAid/Qwen3_Reranker

GiteaMirror commented

@enryteam commented on GitHub (Aug 29, 2025):

+1，reranke very important

@enryteam commented on GitHub (Aug 29, 2025): +1，reranke very important

GiteaMirror commented

@crazyn2 commented on GitHub (Aug 31, 2025):

A PR was submitted, but the manager didn’t merge it. Why is that?

@crazyn2 commented on GitHub (Aug 31, 2025): A PR was submitted, but the manager didn’t merge it. Why is that?

GiteaMirror commented