[GH-ISSUE #1510] feat: multi gpu embedding support #99523

New Issue

GiteaMirror · 2026-05-17T21:31:10-05:00

GiteaMirror commented

2026-05-17 21:31:10 -05:00

Originally created by @buroa on GitHub (Apr 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1510

Example: https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/computing-embeddings/computing_embeddings_multi_gpu.py

Originally created by @buroa on GitHub (Apr 11, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/1510 Example: https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/computing-embeddings/computing_embeddings_multi_gpu.py

GiteaMirror closed this issue

2026-05-17 21:31:10 -05:00

GiteaMirror commented

2026-05-17 21:31:10 -05:00

@FarisHijazi commented on GitHub (Mar 19, 2025):

why though, if you actually cared about performance you should be using proper embeddings inference like HF text-embedding and then connecting to it as an OpenAI endpoint, open-webui already supports custom OpenAI endpoints.

Or you could connect everything through litellm which also allows you to configure embedding endpoints and model endpoints all in one place.

@FarisHijazi commented on GitHub (Mar 19, 2025): why though, if you actually cared about performance you should be using proper embeddings inference like [HF text-embedding](https://github.com/huggingface/text-embeddings-inference) and then connecting to it as an OpenAI endpoint, open-webui already supports custom OpenAI endpoints. Or you could connect everything through litellm which also allows you to configure embedding endpoints and model endpoints all in one place.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#99523