mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-27 17:04:57 -05:00
When two pages use two different models, the two models cannot be started at the same time. #888
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#888
Reference in New Issue
Block a user
Originally created by @ALixuhui on GitHub (May 11, 2024).
When two pages use two different models, the two models cannot be started at the same time.
I have some concurrent needs to support different users calling different models,. However, I've noticed that when multiple users chat with different models concurrently, Ollama only keeps one model in the startup state at a time. Whenever a new user starts chatting with a different model, the current model is killed and the new one is restarted. This impacts the response speed. How can I configure Ollama to start and maintain multiple models concurrently, without killing other models, to improve response speed?
@ALixuhui commented on GitHub (May 11, 2024):
Environment="OLLAMA_MAX_LOADED_MODELS=2"
Solved by setting the ollama environment variable