Feature Request: Browser-Based LLM inference #3127

New Issue

GiteaMirror · 2025-11-11T15:23:10-06:00

GiteaMirror commented

2025-11-11 15:23:10 -06:00

Originally created by @gloryknight on GitHub (Dec 27, 2024).

Feature Request

I propose integrating https://github.com/ngxson/wllama, a powerful open-source library for WebAssembly binding for llama.cpp, enabling in-browser LLM inference as a selectable model option within the web UI. This integration would allow users to run GGUF models, directly within their browser.

Local execution could be implemented as a checkbox in the model capabilities (see admin panes->models->click on model).

By leveraging wllama, we can combine the user-friendly interface of open-webui with local inference in the browser. This offers several potential benefits for our users:

Hosting on low end servers: Processing LLM queries locally within the browser could allow hosting of open-webui without any connections to external LLM providers.
Mature interface for local inference: Mature interface of open-webui will enhance experience of users interested in using in-browser inference.
Offline access: Users could continue to utilize LLM capabilities even without an active internet connection.

wllama is available under the permissive MIT License, making it a suitable choice for integration into our existing web application.

Originally created by @gloryknight on GitHub (Dec 27, 2024). # Feature Request I propose integrating https://github.com/ngxson/wllama, a powerful open-source library for WebAssembly binding for llama.cpp, enabling in-browser LLM inference as a selectable model option within the web UI. This integration would allow users to run GGUF models, directly within their browser. Local execution could be implemented as a checkbox in the model capabilities (see admin panes->models->click on model). By leveraging wllama, we can combine the user-friendly interface of open-webui with local inference in the browser. This offers several potential benefits for our users: * **Hosting on low end servers:** Processing LLM queries locally within the browser could allow hosting of open-webui without any connections to external LLM providers. * **Mature interface for local inference:** Mature interface of open-webui will enhance experience of users interested in using in-browser inference. * **Offline access:** Users could continue to utilize LLM capabilities even without an active internet connection. wllama is available under the permissive MIT License, making it a suitable choice for integration into our existing web application.

GiteaMirror closed this issue

2025-11-11 15:23:10 -06:00

GiteaMirror referenced this issue

2025-11-11 17:46:31 -06:00

[PR #3930] [CLOSED] 💄 fix LaTex #8157

GiteaMirror referenced this issue

2026-04-20 03:27:44 -05:00

[PR #3930] [CLOSED] 💄 fix LaTex #21361

Sign in to join this conversation.