mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-17 08:21:12 -05:00
[GH-ISSUE #2970] feat: browser wasm llm #13087
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#13087
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @tjbck on GitHub (Jun 10, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/2970
@justinh-rahb commented on GitHub (Jun 10, 2024):
https://github.com/mlc-ai/web-llm or something like it?
@lukestanley commented on GitHub (Jun 12, 2024):
This is useful if you can only do a limited amount of processing on the server, and where the client has more processing power. WASM approaches to running Llama.cpp exist, as well as web GPU approaches (that only work on certain systems and and certain browser versions). There may be a WASM memory size issue with normal LLM sizes though. Working around it may well be possible though.
@tjbck commented on GitHub (Jun 13, 2024):
@justinh-rahb yep, seems promising!
@BuildBackBuehler commented on GitHub (Jun 21, 2024):
Yeah MLC-LLM is the ship!
https://github.com/open-webui/open-webui/issues/1270
Would love to plug-n-play with it
@sala91 commented on GitHub (Jul 28, 2024):
WASM multi-threading is not good to this day, having been developing on top of it for years. This could work for smaller models tho, e.g. running a tiny model to translate between languages on the fly for better model output.
@justinh-rahb commented on GitHub (Jul 28, 2024):
This is new and potentially useful as well:
https://developer.chrome.com/docs/ai/built-in