mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-16 14:39:31 -05:00
[GH-ISSUE #16936] feat: Embed browser-use/web-ui for immersive browser-based agent interaction in Open WebUI #56767
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#56767
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Varun-SV on GitHub (Aug 26, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/16936
Check Existing Issues
Problem Description
Open WebUI is a powerhouse of self-hosted, offline-friendly AI—packed with pipelines, RAG, voice, Markdown, LLM runner support, and more. Meanwhile, browser-use/web-ui is a slick Gradio-based interface that lets your AI agents actually drive a real browser, complete with persistent sessions and model-aware automation.
But here’s the snag: these two worlds don’t talk. Open WebUI’s agents are brilliant at chat but can’t, say, click a button or navigate a site. Conversely, browser-use’s browsing wizardry lives in its own sandbox. It’s like having a gifted chef (Open WebUI) and a culinary robot (browser-use) in separate kitchens. They belong at the same table—but right now? Parallel simulations.
Desired Solution you'd like
Introduce a Browser UI Plugin for Open WebUI—a modular mashup that brings browser-use/web-ui’s capabilities inside the Open WebUI universe:
Optional Plugin/Module
Add browser-use/web-ui as a plugin within Open WebUI. Users enable it when they want, so there’s no unnecessary bulk.
MCP-based Integration Bridge
Link both systems using the Model Context Protocol (MCP). Let your Open WebUI agents issue natural-language actions—browse, click, screenshot—via browser-use/web-ui machinery.
Interactive Browser Session Panel
Embed a panel or button in the dashboard that spins up a browser session. Users (and agents) can watch, click, capture—all in real-time inside Open WebUI.
Maintain Modular & Secure Design
Keep it optional, gated, and seamless—no runaway browser windows unless you specifically want them. Offer UI guidance and sandboxing for safe usage.
Alternatives Considered
Write custom pipeline that uses playwright directly
Could script browser actions manually—but ends up duplicating browser-use/web-ui functionality (session persistence, UI, VNC) from scratch. More dev effort, less reuse.
Use existing RAG/Web Search features within Open WebUI
Open WebUI already supports web content integration using RAG pipelines or
# URLcommands :contentReference[oaicite:3]{index=3}. But those only fetch page content for the model—they don’t browse or interact live.Leverage Jupyter Notebook integration for browser scripting
With the Jupyter notebook support in Open WebUI, an agent could run Python scripts that control a browser (via Playwright). It’s flexible, but heavyweight, unsafe if misused, and lacks UI cohesion.
Additional Context
Write custom pipeline that uses playwright directly
Could script browser actions manually—but ends up duplicating browser-use/web-ui functionality (session persistence, UI, VNC) from scratch. More dev effort, less reuse.
Use existing RAG/Web Search features within Open WebUI
Open WebUI already supports web content integration using RAG pipelines or
# URLcommands. But those only fetch page content for the model—they don’t browse or interact live.Leverage Jupyter Notebook integration for browser scripting
With the Jupyter notebook support in Open WebUI, an agent could run Python scripts that control a browser (via Playwright). It’s flexible, but heavyweight, unsafe if misused, and lacks UI cohesion.
@flefevre commented on GitHub (Aug 26, 2025):
May I ask a question: in your proposed design, where will run the browser plugin/module?
Will it run in the open Webui server or on the client?