github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-05-07 03:18:23 -05:00
Code Issues 1.3k Packages Projects Releases 126 Wiki Activity

[GH-ISSUE #8784] Rerranking not using gpu #53920

New Issue
Closed
opened 2026-05-05 15:34:19 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-05 15:34:19 -05:00
Owner
Copy Link

Originally created by @fireblade2534 on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8784

Installation Method

Docker

Environment

  • Open WebUI Version: V0.5.6

  • Operating System: Ubuntu

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

Fast reranking performance

Actual Behavior:

It doesn't seem like reranking is run on the gpu even when it is available

Description

Bug Summary:
The reranking model doesn't seem to use the gpu resulting in bad performance

Reproduction Details

Steps to Reproduce:
Add a reranking model and use nvidia-smi to check gpu memory. Running RAG with and without the reranker does not change the ammount of gpu memory usage

Originally created by @fireblade2534 on GitHub (Jan 23, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/8784 ## Installation Method Docker ## Environment - **Open WebUI Version:** V0.5.6 - **Operating System:** Ubuntu **Confirmation:** - [ ] I have read and followed all the instructions provided in the README.md. - [ ] I am on the latest version of both Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. - [ ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: Fast reranking performance ## Actual Behavior: It doesn't seem like reranking is run on the gpu even when it is available ## Description **Bug Summary:** The reranking model doesn't seem to use the gpu resulting in bad performance ## Reproduction Details **Steps to Reproduce:** Add a reranking model and use nvidia-smi to check gpu memory. Running RAG with and without the reranker does not change the ammount of gpu memory usage
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#53920
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?