I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
When RAG stop processed question than its free VRAM. Technically — unloading RAG models for next use to give VRAM for another models. And won't load these models till it actually needed.
In ollama i can set variable «keep-alive» to zero to make sure than models will be dead after use, can we set this similar щ
Actual Behavior:
OpenWebUI eat VRAM on start, then eat another portion VRAM on update models, then eat another VRAM when make RAG request. And never give it back :)
Description
Bug Summary:
RAG not free VRAM after process query and after update embedded/reranking models too.
Reproduction Details
Set embedding model. My option — «ai-forever/ru-en-RoSBERTa».
Set reranking model. My option — «bge-reranker-v2-m3».
Click «save».
See minus 1.5 Gb VRAM.
Click «save» again, minus another 1.5 Gb VRAM.
Click till memory will fill totally.
And this VRAM will not be free in the near future. But if restart container it will be free again.
Similar things happen when i make RAG questions: first use — minus 1.5 Gb, second — minus another 1.5 Gb and so on.
And on start i see that it takes 1.5 Gb VRAM for, i think, the first init of models.
Originally created by @treepoint on GitHub (Oct 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6379
# Bug Report
## Installation Method
Docker, WSL, CUDA image.
## Environment
- **Open WebUI Version:** [e.g., v0.3.32]
- **Operating System:** [e.g., Windows 10]
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
When RAG stop processed question than its free VRAM. Technically — unloading RAG models for next use to give VRAM for another models. And won't load these models till it actually needed.
In ollama i can set variable «keep-alive» to zero to make sure than models will be dead after use, can we set this similar щ
## Actual Behavior:
OpenWebUI eat VRAM on start, then eat another portion VRAM on update models, then eat another VRAM when make RAG request. And never give it back :)
## Description
**Bug Summary:**
RAG not free VRAM after process query and after update embedded/reranking models too.
## Reproduction Details
1. Set embedding model. My option — «ai-forever/ru-en-RoSBERTa».
2. Set reranking model. My option — «bge-reranker-v2-m3».
3. Click «save».
4. See minus 1.5 Gb VRAM.
5. Click «save» again, minus another 1.5 Gb VRAM.
6. Click till memory will fill totally.
And this VRAM will not be free in the near future. But if restart container it will be free again.
Similar things happen when i make RAG questions: first use — minus 1.5 Gb, second — minus another 1.5 Gb and so on.
And on start i see that it takes 1.5 Gb VRAM for, i think, the first init of models.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @treepoint on GitHub (Oct 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6379
Bug Report
Installation Method
Docker, WSL, CUDA image.
Environment
Open WebUI Version: [e.g., v0.3.32]
Operating System: [e.g., Windows 10]
Confirmation:
Expected Behavior:
When RAG stop processed question than its free VRAM. Technically — unloading RAG models for next use to give VRAM for another models. And won't load these models till it actually needed.
In ollama i can set variable «keep-alive» to zero to make sure than models will be dead after use, can we set this similar щ
Actual Behavior:
OpenWebUI eat VRAM on start, then eat another portion VRAM on update models, then eat another VRAM when make RAG request. And never give it back :)
Description
Bug Summary:
RAG not free VRAM after process query and after update embedded/reranking models too.
Reproduction Details
And this VRAM will not be free in the near future. But if restart container it will be free again.
Similar things happen when i make RAG questions: first use — minus 1.5 Gb, second — minus another 1.5 Gb and so on.
And on start i see that it takes 1.5 Gb VRAM for, i think, the first init of models.