I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
Fast reranking performance
Actual Behavior:
It doesn't seem like reranking is run on the gpu even when it is available
Description
Bug Summary:
The reranking model doesn't seem to use the gpu resulting in bad performance
Reproduction Details
Steps to Reproduce:
Add a reranking model and use nvidia-smi to check gpu memory. Running RAG with and without the reranker does not change the ammount of gpu memory usage
Originally created by @fireblade2534 on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8784
## Installation Method
Docker
## Environment
- **Open WebUI Version:** V0.5.6
- **Operating System:** Ubuntu
**Confirmation:**
- [ ] I have read and followed all the instructions provided in the README.md.
- [ ] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [ ] I have included the Docker container logs.
- [ ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
Fast reranking performance
## Actual Behavior:
It doesn't seem like reranking is run on the gpu even when it is available
## Description
**Bug Summary:**
The reranking model doesn't seem to use the gpu resulting in bad performance
## Reproduction Details
**Steps to Reproduce:**
Add a reranking model and use nvidia-smi to check gpu memory. Running RAG with and without the reranker does not change the ammount of gpu memory usage
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @fireblade2534 on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8784
Installation Method
Docker
Environment
Open WebUI Version: V0.5.6
Operating System: Ubuntu
Confirmation:
Expected Behavior:
Fast reranking performance
Actual Behavior:
It doesn't seem like reranking is run on the gpu even when it is available
Description
Bug Summary:
The reranking model doesn't seem to use the gpu resulting in bad performance
Reproduction Details
Steps to Reproduce:
Add a reranking model and use nvidia-smi to check gpu memory. Running RAG with and without the reranker does not change the ammount of gpu memory usage