Feature Request: Integrated Browser Panel
Summary
Currently, when an AI agent needs to interact with a website or check results in a browser, it has to spin up an isolated headless browser (e.g. Playwright, Puppeteer, Chromium Headless) in the background — completely detached from the OpenGUI interface. The user cannot see what the AI is doing in the browser, and the AI cannot interact with a visible, shared browser session.
This creates friction: users often have to write system prompts explicitly forbidding the AI from using their real browser and instructing it to use headless-only tools.
Proposed Solution
Integrate a built-in browser panel directly into OpenGUI so that:
- The AI can navigate, click, fill forms, and read pages using the embedded browser — no external headless setup required.
- Results are visible to the user in real-time inside the OpenGUI interface.
- The AI and user share the same browser session — the user can observe and intervene if needed.
- No more workarounds like manually configuring Playwright or Chromium Headless just to give the AI basic web access.
Motivation
Today, a typical setup looks like this (from a user's system prompt):
"Du darfst nicht meinen sichtbaren Browser übernehmen, öffnen oder fernsteuern. Verwende ausschließlich einen isolierten Headless-Browser im Hintergrund, z. B. Playwright, Puppeteer oder Chromium Headless."
("You must not take over, open or remote-control my visible browser. Use only an isolated headless browser in the background, e.g. Playwright, Puppeteer or Chromium Headless.")
This is entirely unnecessary if OpenGUI provides a sandboxed, integrated browser that the AI is allowed to use freely — without touching the user's system browser.
Possible Implementation Approaches
- Embed a Chromium/WebView panel inside the OpenGUI window (e.g. via Electron's webview tag, Tauri's WebView, or CEF).
- Expose a CDP (Chrome DevTools Protocol) endpoint that the AI backend can connect to for automation.
- Show a live preview of the browser state alongside the chat, so users always see what the AI is doing.
- Optionally: allow the user to take over control of the embedded browser at any time.
Benefits
| Today |
With integrated browser |
| AI uses headless browser, user sees nothing |
AI uses embedded browser, user sees everything |
| Complex system prompts needed to restrict browser access |
No restrictions needed — browser is sandboxed by design |
| External tools (Playwright etc.) must be installed |
Browser is built into OpenGUI |
| No shared state between AI and user |
AI and user share the same browser session |
Related
- Similar concept: Cursor with browser preview, Devin with integrated browser agent view
- Could pair nicely with tool-use / function-calling features in OpenGUI
This would be a significant UX improvement and would make OpenGUI much more powerful as an agentic AI interface.
Feature Request: Integrated Browser Panel
Summary
Currently, when an AI agent needs to interact with a website or check results in a browser, it has to spin up an isolated headless browser (e.g. Playwright, Puppeteer, Chromium Headless) in the background — completely detached from the OpenGUI interface. The user cannot see what the AI is doing in the browser, and the AI cannot interact with a visible, shared browser session.
This creates friction: users often have to write system prompts explicitly forbidding the AI from using their real browser and instructing it to use headless-only tools.
Proposed Solution
Integrate a built-in browser panel directly into OpenGUI so that:
Motivation
Today, a typical setup looks like this (from a user's system prompt):
This is entirely unnecessary if OpenGUI provides a sandboxed, integrated browser that the AI is allowed to use freely — without touching the user's system browser.
Possible Implementation Approaches
Benefits
Related
This would be a significant UX improvement and would make OpenGUI much more powerful as an agentic AI interface.