Skip to content

docs(browser): document the screenshot auto-attach escape hatch - #124

Merged
Alezander9 merged 1 commit into
mainfrom
screenshot-attach-optout
Jul 29, 2026
Merged

docs(browser): document the screenshot auto-attach escape hatch#124
Alezander9 merged 1 commit into
mainfrom
screenshot-attach-optout

Conversation

@Alezander9

@Alezander9 Alezander9 commented Jul 29, 2026

Copy link
Copy Markdown
Member

Problem

browser_execute attaches every Page.captureScreenshot result to the next assistant turn. That default is right — it saves a read call almost every time the agent screenshots something it wants to look at, and we don't want to change it.

But there was no way to capture pixels the agent does not need to see. Two real cases:

  • Bulk capture (e.g. screenshotting many pages for a dataset) attaches every frame to context, with no opt-out.
  • An artifact that only gets written to disk — an og:image card, a thumbnail — is attached even though the agent never looks at it.

Both are rare, which is why this is documentation rather than tool surface. Alternatives considered and rejected in #122: a synthetic attachToContext param on Page.captureScreenshot (teaches the agent a CDP field Chrome has never heard of, and carried the opt-out via WeakSet object identity, which fails silently in the leak direction), and a fourth browser_execute tool parameter (a permanent, eagerly-loaded schema field on every call for a rare need).

Why a documented route works

The screenshot tap subscribes via session.onCallResult(...), which fires only from _call's resolve path (session.ts:252-256) — the command/response channel. CDP events route through a separate listener list (session.ts:273-277) and never reach it.

So pixels delivered as the Page.screencastFrame event are structurally invisible to the tap. No code change, and the agent learns a genuine protocol capability that transfers to any CDP client instead of a BrowserCode-only fiction.

Change

15 added lines in skills/browser-execute/SKILL.md, next to the auto-attach explanation it qualifies. The lead sentence states the rule (response attaches, event does not) so the agent can reason about cases beyond the example.

Validation

Verified against Chrome 150.0.7871.187 with --headless=new — the harder case for compositing — via browser_execute against a throwaway debug-port profile:

  • Static, already-loaded, idle page: startScreencast delivers a frame in 7ms with no navigation, scroll, or re-layout nudge. This was the open question; the documented snippet needs no nudge and no polling loop.
  • Screencast frames produce zero attachments, across every run. Page.captureScreenshot still attaches (control case rendered inline as expected).
  • setDeviceMetricsOverride({ width: 1200, height: 630, deviceScaleFactor: 1 }) yields a PNG whose header reads exactly 1200x630, confirming the viewport is what sets frame size.
  • bun test test/skills.test.ts — 2 pass (skill still materializes with {{SKILLS_DIR}} substituted).
  • Pre-push filtered typecheck passed.

Emulation.setDeviceMetricsOverride is marked Optional: in the snippet on purpose: it reflows the page, so it is wrong as an unconditional step for bulk capture where you want the page as-is.

Protocol details found while testing, deliberately left out of the skill text

Recorded here rather than spending tokens on them in a lazily-loaded doc:

  • screencastFrameAck is not needed for navigation-paced capture — 3 navigations produced 3 frames with zero acks. It is required for continuous capture: on an animating page a 3s window yielded 3 frames unacked vs 178 acked, so Chrome stalls after a couple of unacked frames. The bulk-capture case that motivated this PR is navigation-paced, so the skill omits ack entirely.
  • sessionId on the frame event is not a frame counter, despite the protocol describing it as "Frame number". It is constant within a cast and increments per startScreencast call. Anything acking per-frame by treating it as a frame identity will behave confusingly.
  • Page.screencastFrame is flagged "experimental": true in the protocol. It is the mechanism cloud's own live preview uses, so it is stable in practice.

Zone

Green — packages/bcode-browser/ only, docs-only, no upstream diff, no EXCEPTIONS.md entry needed.

Closes #122.


Summary by cubic

Documented how to opt out of screenshot auto-attach in browser_execute. Use Page.screencastFrame instead of Page.captureScreenshot to capture pixels without attaching (responses attach; events do not), useful for bulk capture and disk-only artifacts.

Written for commit da8b535. Summary will update on new commits.

Review in cubic

`browser_execute` attaches every `Page.captureScreenshot` result to the next
assistant turn, which is the right default — it saves a `read` call almost
every time. But there was no way to capture pixels the agent does not need to
look at: bulk capture, or an artifact that only gets written to disk. A bulk
task attaches every frame to context with no opt-out.

The tap subscribes to `onCallResult`, which fires only from `_call`'s resolve
path — the command-response channel. CDP events route through a separate
listener list, so pixels delivered as `Page.screencastFrame` are structurally
invisible to it. That makes the escape hatch real CDP rather than an invented
parameter, so what the agent learns transfers to any client.

Verified against Chrome 150.0.7871.187 (--headless=new):
- static, already-loaded, idle page: startScreencast delivers a frame in 7ms
  with no navigation, scroll, or re-layout nudge
- screencast frames produce zero attachments; captureScreenshot still attaches
- setDeviceMetricsOverride({ width: 1200, height: 630 }) yields a PNG whose
  header reads exactly 1200x630

Two protocol details found while testing that are deliberately left out of the
skill text, recorded here instead:
- ack is not needed for navigation-paced capture (3 navigations produced 3
  frames unacked). It is required for continuous capture: on an animating
  page, 3s yielded 3 frames unacked vs 178 acked, so Chrome stalls after a
  couple of unacked frames.
- `sessionId` on the frame event is not a frame counter despite the protocol
  describing it as "Frame number" — it is constant within a cast and
  increments per startScreencast.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@Alezander9
Alezander9 merged commit 7f37345 into main Jul 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant