mac: Sentence Transformers and OpenWebUI does not use the MPS/GPU on Apple Silicon #3190

New Issue

2025-11-11T15:25:18-06:00

GiteaMirror commented

2025-11-11 15:25:18 -06:00

Originally created by @richtong on GitHub (Jan 5, 2025).

Discussed in https://github.com/open-webui/open-webui/discussions/8278

^{Originally posted by richtong January 2, 2025}
Ok, I'm using MacOS Sequoia 15.2 with pipx installed open-webui 0.5.3 and the sentence transformers for RAG for both the embed model and the ranker seem to be CPU only.

Ollama runs fine as an MPS (Apple version of CUDA), but not sentence transformers. Is there some variable i can set, I haven't groveled down into the code to look to see how this works. :-)

The fix required is a check for the mps device in env.py, I wrote up a fix and it seems to work

Originally created by @richtong on GitHub (Jan 5, 2025). ### Discussed in https://github.com/open-webui/open-webui/discussions/8278 <div type='discussions-op-text'> <sup>Originally posted by **richtong** January 2, 2025</sup> Ok, I'm using MacOS Sequoia 15.2 with pipx installed open-webui 0.5.3 and the sentence transformers for RAG for both the embed model and the ranker seem to be CPU only. Ollama runs fine as an MPS (Apple version of CUDA), but not sentence transformers. Is there some variable i can set, I haven't groveled down into the code to look to see how this works. :-)</div> The fix required is a check for the mps device in env.py, I wrote up a fix and it seems to work

GiteaMirror closed this issue

2025-11-11 15:25:19 -06:00

GiteaMirror commented

2025-11-11 15:25:20 -06:00