mirror of
https://github.com/open-webui/open-webui.git
synced 2026-05-06 02:48:13 -05:00
[GH-ISSUE #1437] feat: Tier 1 support for LocalAI #12497
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#12497
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @lee-b on GitHub (Apr 5, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1437
Originally assigned to: @tjbck on GitHub.
LocalAI is similar to ollama, but aims to be a complete replacement for OpenAI, already including TTS, STT, and image generation (all of which which open webui use). It would be great to have tier 1 support for this.
@lenaxia commented on GitHub (Oct 19, 2024):
What does Tier 1 support entail in this case?
@lee-b commented on GitHub (Oct 19, 2024):
Primarily support for its named models and asking it to pull down new models, as ollama has support for those things, but also support for recognising that it does tts, stt, and image-gen, so not having to know and set the various URL permutations in different tabs and fields in the Open-WebUI admin pages to make it all work. Ideally its features would "just work", when you add the url and tell Open-WebUI that it's a LocalAI server, much like adding an Ollama URL "just works".
However, more broadly, here is what I observe in the UX, and what I propose instead:
Current UX
Still a lot of focus on Ollama here, and conceptual separation of ollama from other providers, despite the renaming from ollama-webui to open-webui, and the addition of generic openai server support, audio (TTS/STT), and image support.
These all need to be configured separately, and ollama is the only service that really feels like it has first class support, with the others being tacked on, with features like support for model names being missing or awkward. For example, with a llama.cpp server, the model name that is currently running always appears, but if you run a different model at the same server, then it breaks all of your derived models in workspace, because the model name has changed.
Now like Ollama, LocalAI supports multiple textgen models, each with their own names, as well as pulling down new models. Additionally, it supports other types of models too, each with names, and tags to identify their use cases / modalities:
AND, unlike ollama, it supports this for other types of models: text to speech, speech to text, image gen. But none of this is directly supported or recognised in Open-WebUI in the same way that Ollama's multiple model support is recognised and used.
Additionally, multimodal models that can do end-to-end speech, speech to text, and text to speech (amongst other modalities such as image to text) are a thing.
So it seems to me that, currently (and understandably) in Open-WebUI, ollama support is currently the only provider truly supported as "tier 1", but that expanding support for other providers would improve both the experience for users, and Open-WebUI's core design.
Proposed UX changes
Genericizing the configuration for a more LocalAI-like approach, via a generic plugin system, would be beneficial, such that:
i) any server plugin can provide 1...N named models (e.g., ollama, LocalAI)
ii) each provided model can have 1...N modality/capability tags (text-gen, text-to-speech, speech-to-text, speech-to-speech, text-embedding, text-to-image, image-to-text, text-to-video, text-only-ocr, ocr-with-table-reflow, etc.)
And so, (with apologies for the quick mock-up) I think that the admin UI should work a bit more like this, both to allow for alternative providers now, and to better place Open-WebUI for all of the multimodal and competing providers stuff that's either here or on its way:
Then then models tab could be a list of all models available, and options to pull down new models. The audio/images tabs would not be cluttered with URL details, but would provide more specialized options, like which voice to use by default, whether to use a dedicated "ocr" type of model or whether using an "image-to-text" model is allowed for images, whether you approve local or remote text-to-speech, etc.