[PR #13087] [CLOSED] add retrieval model optimizations #23095

New Issue

GiteaMirror · 2026-04-20T04:37:43-05:00

GiteaMirror commented

2026-04-20 04:37:43 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/13087
Author: @chsasank
Created: 4/20/2025
Status: ❌ Closed

Base: main ← Head: sasank

📝 Commits (1)

1001139 add retrieval model optimizations

📊 Changes

2 files changed (+11 additions, -3 deletions)

View changed files

📝 backend/open_webui/retrieval/utils.py (+9 -3)
📝 backend/open_webui/routers/retrieval.py (+2 -0)

📄 Description

Added retrieval model optimizations. torch.cuda.empty_cache() clears up any unnecessary memory occupied by pytorch after inference is complete. Models are loaded in FP16 because that's usually good enough but saves ram by 2x.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/13087 **Author:** [@chsasank](https://github.com/chsasank) **Created:** 4/20/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `sasank` --- ### 📝 Commits (1) - [`1001139`](https://github.com/open-webui/open-webui/commit/1001139910bd07498ad3882fe6ebbc87c2c53651) add retrieval model optimizations ### 📊 Changes **2 files changed** (+11 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/retrieval/utils.py` (+9 -3) 📝 `backend/open_webui/routers/retrieval.py` (+2 -0) </details> ### 📄 Description Added retrieval model optimizations. `torch.cuda.empty_cache()` clears up any unnecessary memory occupied by pytorch after inference is complete. Models are loaded in FP16 because that's usually good enough but saves ram by 2x. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 04:37:43 -05:00

GiteaMirror closed this issue

2026-04-20 04:37:44 -05:00

GiteaMirror referenced this issue

2026-04-20 06:49:30 -05:00

[PR #23095] [CLOSED] fix: prevent infinite loop when tool_calls is empty with finish_reason tool_calls #27023

GiteaMirror referenced this issue

2026-04-25 14:28:32 -05:00

[PR #23095] [CLOSED] fix: prevent infinite loop when tool_calls is empty with finish_reason tool_calls #42653

GiteaMirror referenced this issue

2026-04-30 02:35:15 -05:00

[PR #23095] [CLOSED] fix: prevent infinite loop when tool_calls is empty with finish_reason tool_calls #50071