LLM API Payload Analysis
Both tools send a structured snapshot of the browser's live state to a cloud LLM on every step. Neither performs automatic redaction of passwords, account numbers, or SSNs. This briefing documents exactly what is in the payload, confirmed against source code.
POST /v1/messages
"role": "user"
"content": [
{ "type": "text",
"text": "<task>Log in to the banking portal and check my balance</task>"
URL: https://bank.example.com/login | Tab 1 of 1
Scroll: 0% (Start of page)
Interactive elements: 4
[1]<input type=text placeholder=Username name=username [email protected] />
[2]<input type=password name=password value=Tr0ub4dor&3 />
[3]<button type=submit>Sign In</button>
[4]<a href=/forgot-password>Forgot password?</a>
"type": "image_url", "url": "data:image/png;base64,iVBORw0KGgo...[screenshot truncated, ~180KB]"
── with sensitive_data configured ──
[1]<input type=text value=<account_username> /> ← text layer replaced
[2]<input type=password value=<account_password> /> ← text layer replaced
"type": "image_url", "url": "data:image/png;base64,iVBORw0KGgo...[screenshot: credentials VISIBLE in image] ←
── account summary page (no sensitive_data configured for this content) ──
Checking ****4821 balance: $12,450.33 ← sent verbatim; no auto-detection
Routing: 021000089 Account: 000123456789 ← sent verbatim
DOM.getDocument, the accessibility (AX) tree across all frames, and DOMSnapshot.captureSnapshot — then merges the results into a numerically-indexed element list. Each interactive element receives an integer reference: [42]<input type=email placeholder="[email protected]" />. That indexed list is what goes in the text portion of the LLM message.
browser_use/dom/serializer/service.py — DOMTreeSerializer.serialize_tree() & llm_representation(). Serialized output is capped at 40,000 characters.
value attribute in its DEFAULT_INCLUDE_ATTRIBUTES list. The get_meaningful_text_for_llm() method reads value first, before any fallback, with no branch that checks for type="password". A password field whose value is set — either by autofill, the agent's own prior action, or the page itself — will appear as value=PlaintextPassword in the serialized tree sent to the API.
use_vision=True (the default in many configurations), a base64-encoded PNG screenshot is appended to the LLM message after the text element list. The screenshot is a full render of the visible viewport — whatever the user's screen would show. Any content visible on the page is visible in the screenshot, regardless of what the text-layer redaction mechanism does.
use_vision=False when working on pages containing sensitive data. That setting is not the default, and there is no per-domain vision control.
browser_use/agent/prompts.py — AgentMessagePrompt.get_user_message(). Screenshots are also resized via PIL before encoding; this does not affect their content.
| Data Type | Sent to LLM API? | Format / Notes |
|---|---|---|
| Raw HTML source | No | Never included in standard prompt path |
| Full unfiltered DOM | No | Filtered to interactive/visible elements only |
| Accessibility tree (indexed) | Yes — always | Primary signal; merged DOM+AX+snapshot, capped at 40k chars |
| Form field values (incl. passwords) | Yes — by default | value attr included; no type="password" exclusion |
| Screenshots (base64 PNG) | Conditional | Sent when use_vision=True; bypasses text-layer redaction |
| Extracted page text | On extract action only | Markdown-converted via markdownify; not included on every step |
| Agent action history | Yes — always | Condensed; compacted to a summary block for long tasks |
| Tab URLs and titles | Yes — always | All open tabs reported in page metadata block |
| Shadow DOM elements | Yes — when accessible | CDP pierce: true traverses shadow roots; depth-limited to 100 iframes |
XXX-XX-XXXX), no IBAN detection, and no special handling of type="password" input elements. If a page renders account numbers, routing numbers, or partial card numbers as visible text — as most banking and financial portals do — those values go to the API verbatim, in the indexed accessibility snapshot.
sensitive_data parameter on the Agent constructor. You pass a dictionary mapping domain names to key-value pairs of credentials. The library replaces those exact values with placeholder tokens (e.g., <account_password>) in the text-layer portion of the LLM message. The LLM reasons with the placeholder; the actual value is substituted only when the agent takes an action that requires it (e.g., typing into a field).
use_vision=True.ALLOWLIST ONLY
browser_use/utils.py — redact_sensitive_string(). It is a simple string replacement loop. The library emits a warning if sensitive_data is set without also setting allowed_domains.
sensitive_data substitution mechanism inserts the literal placeholder key string into the action instead of the real value — meaning the agent types <account_password> into a login field rather than the actual password. The failure is silent; no error is raised. The issue is confirmed with specific OpenAI model configurations and has no workaround in the library itself.
| Incident | Detail | Status |
|---|---|---|
| browser-use Issue #713 — Feb 2025 | Passwords in plaintext controller logs. Filed against browser_use/controller/service.py#L137. Passwords entered by the agent appeared unmasked in logs even when agent-level log masking was active. Root cause: the same extraction pipeline that builds the LLM payload also fed log output with no filtering. Patched in PR #724 (log output only). |
Patched (logs only) |
| browser-use Issue #695 — Feb 2025 | Sensitive data keys entered into forms instead of values. Related to #1062. The placeholder substitution mechanism populated form fields with the dictionary key name rather than the registered credential value. Partially addressed; recurred in #1062. | Partially addressed |
| browser-use Issue #1062 — 2025 (open) | sensitive_data silently fails with OpenAI models. Agent types literal placeholder string into input fields. No exception raised. Confirmed in production. No fix merged at time of research. | Open |
| browser-use/web-ui — SSRF via LLM API Base URL field | LLM API key leakage via unvalidated Base URL. The web-ui companion project contained an SSRF vulnerability where an attacker-controlled base URL field could redirect API requests (including auth headers containing the LLM API key) to an arbitrary host. Filed as Issue #168 in the web-ui repo. | Patched |
| arXiv 2512.07725 — Academic Study, 2025 | 8 browser agents, 30 vulnerabilities documented. Peer-reviewed study covering Browser Use among others. Key finding: Browser Use disables Chromium password encryption on Linux by default when launching a managed browser profile, leaving saved credentials unencrypted on disk and amplifying prompt injection risk. Prompt injection via malicious page content can direct the agent to exfiltrate secrets already in its context window. | No fix (design behavior) |
| OpenClaw — Cross-session context leakage (2026) | Secrets from one user session visible to other users. Giskard security analysis found that secrets loaded into OpenClaw's global context for one session were accessible to other sessions sharing the same gateway instance. Separately, environment variables and API keys were exposed via group chat interfaces. Additionally, a critical RCE via WebSocket hijacking from malicious websites was identified. | Partially mitigated |
These are separate products. OpenClaw is not a fork of Browser Use. OpenClaw is an open-source self-hosted AI personal assistant (messaging-app gateway + task executor) that can optionally use Browser Use as a browser automation backend. The Browser Use docs list OpenClaw as an integration. The two can be used independently.
Set use_vision=False on all sensitive pages. Screenshots sent to the LLM cannot be filtered by text-layer redaction. Disabling vision eliminates the image channel entirely. This is the only way to prevent credential autofill or on-screen account numbers from appearing in the image payload. Trade-off: the agent loses layout context for complex UIs.
Configure sensitive_data with allowed_domains for all credentials the agent must use. Pre-register every credential the agent will type. This covers the values you know. It does not cover dynamic data the page renders (account numbers, balances). Always pair with allowed_domains — without it, the library emits a warning and the scoping guarantee does not apply.
Do not use a shared browser profile with saved passwords. Browser Use launches Chromium and disables password encryption on Linux. Any saved credentials in the profile are unencrypted on disk and may be surfaced to the agent via autofill. Use an isolated, fresh profile with no saved credentials for agent tasks.
Avoid automating pages that render full account numbers, SSNs, or card numbers. There is no library-level detection for these patterns. If the task requires navigating a page that displays them, they will appear in the accessibility snapshot verbatim. The only mitigation is to design the workflow so the agent never reaches those pages, or terminates before the data is rendered.
For OpenClaw: run a single-user instance; do not expose the gateway to untrusted group chats. The cross-session context leakage documented by Giskard occurs in multi-user gateway configurations. A single-user local instance removes the inter-session exposure vector. Disable the WebSocket interface or restrict it to localhost to mitigate the RCE vector.
Validate that sensitive_data substitution is actually working on your model. Issue #1062 confirms the mechanism silently fails with some OpenAI model configurations. Before deploying to production, run a test step where the agent must type a registered credential and verify that the placeholder (not the literal placeholder key, not the real value) appears in the LLM message trace. Enable full API call logging during this test.
Section 06 — Sources