Browser Control: Web Interaction, Filling and Scraping
Hermes Agent ships a full browser automation toolset, so the agent can navigate websites, click elements, fill forms, and extract information across many backends. Whether you want managed cloud browsers with anti-bot tooling or your own local Chromium, Hermes gives you a consistent browser interface on top of whichever driver you choose.
Multiple Browser Backends
Hermes supports several browser sources, selected through hermes tools → Browser Automation:
- Browser Use cloud mode — managed Chromium with stealth, residential proxies, CAPTCHA solving, and reusable browser profiles.
- Browserbase cloud mode — an alternative cloud provider with anti-bot tooling.
- Firecrawl cloud mode — cloud browsers with built-in scraping.
- Browser Use mode (default) — uses the Browser Use CLI 3.0 to drive your configured backend.
- Camofox local mode — self-hosted anti-detection browsing via a Firefox-based fingerprint-spoofing fork.
- Lightpanda — a headless browser built from scratch in Zig that starts instantly and uses far less memory than Chrome.
- Local Chromium-family via CDP — attach to your running Chrome, Brave, Chromium, or Edge with
/browser connect. - Local browser mode — a packaged Chromium driven by
agent-browser.
Pages are represented as accessibility trees (text-based snapshots), which makes them ideal for LLM agents. Interactive elements get ref IDs like @e1, @e2 that the agent uses for clicking and typing.
The Browser Tools
The toolset covers the full interaction loop:
browser_navigate— go to a URL (must be called first to begin a session).browser_snapshot— get a text accessibility-tree snapshot with ref IDs for elements.browser_clickandbrowser_type— click a ref, or type into a field (clearing it first).browser_scroll,browser_press,browser_back— navigate and interact with pages.browser_vision— take a screenshot and analyze it with vision AI, useful for CAPTCHAs and visual checks.browser_console— read JavaScript console output and evaluate expressions.browser_cdp— raw Chrome DevTools Protocol passthrough for native dialogs, iframe-scoped evaluation, and cookie control.browser_dialog— respond to native JS dialogs (alert/confirm/prompt) that would otherwise block the page thread.
For simple information retrieval the docs recommend web_search/web_extract — they are faster and cheaper. Reach for the browser tools when you need to interact: click buttons, fill forms, handle dynamic content.
Form Filling in Practice
A typical signup flow shows the pattern: call browser_navigate("https://example.com/signup"), browser_snapshot() to see form fields with refs, browser_type(ref="@e3", text="john@example.com") for each field, browser_click(ref="@e8") to submit, then snapshot again to confirm success.
Real-Profile Browsing and Headed Mode
By default local browsing runs in a clean throwaway profile. Turn on browser.use_real_profile: true to let the agent browse as you, reusing your existing logins and cookies via a managed snapshot of your active profile. It works headless so the agent can tweet, fill forms, or scrape while you keep working. Enable browser.headed: true (or AGENT_BROWSER_HEADED=1) for a visible window you can watch and intervene in, and set browser.record_sessions: true to record sessions as WebM video files.
Session Management and Stealth
Each task gets an isolated browser session, cleaned up automatically after inactivity (default two minutes). Cloud backends provide stealth features including random fingerprints, viewport randomization, and CAPTCHA solving, with residential proxies on by default; Hermes gracefully falls back if a paid feature isn't on your plan. For free local work with no cloud credits, /browser connect attaches directly to your own Chromium-family browser.
Use /browser connect (an interactive-CLI command) to drive a live, local browser and see everything in real time, or browser.cdp_url for a persistent CDP endpoint in config.
Key Takeaways
- Hermes' browser toolset supports cloud (Browser Use, Browserbase, Firecrawl) and local (Chromium, Camofox, Lightpanda) backends.
- Interaction happens over accessibility-tree snapshots with ref IDs, so the agent reliably clicks and fills forms.
- Real-profile and headed modes let the agent act as you with your logins, or show you exactly what it's doing.
- Sessions are isolated and auto-cleaned, with stealth and proxy features freeing you from most anti-bot friction.