{"content":"Dev journal — feat: bot_browser_act, gesture-free agent control of the MetaApp preview frame (a758f98c, branch feat/bot-browser-act-tool)\n\nRoute B of the MetaApp agent-control discussion, implemented host-side only (no ABC changes): ABC 0.6.3 ships no page-action engine (its 'actor' vocabulary is operating identity, not page elements), so the reach-in belongs to the host. New main-process executor botBrowserPreviewAct.ts resolves the ACTIVE Bot Browser tab's renderer envelope through the existing tab bridge (get-active-tab / get-tab-info), walks WebFrameMain trees across live windows to find the matching preview frame, and runs structured actions inside it via privileged frame.executeJavaScript with a user-gesture allowance — the only channel that can cross the cross-origin sandboxed preview iframe, and it composes with the already-merged no-user-gesture-required autoplay policy.\n\nScope guardrails: only the active tab's renderer URL is targeted; only locally served 127.0.0.1/localhost html-iframe previews are actable; selectors are injected as JSON literals so agent input cannot break out of the snippet; first-party pages and ordinary web content error honestly. Three actions: click (CSS selector, scrollIntoView + click), read (live innerText, closing the read_page gap for MetaApp frames), state (audio/video inventory with paused/currentTime/src plus page title).\n\nDelivery surfaces: a new bot_browser_act agent tool registered for EVERY cowork surface (same posture as bot_browser_screenshot, gated on controlBotBrowser.act), and a new RPC gateway route POST /api/idbots/bot-browser/act mirroring the tabs route so local bot sessions over IDBOTS_RPC drive the same executor.\n\nVerified end to end in a live dev instance against the music player MetaApp: state returned the paused inventory (audio PAUSED at 0s, readyState 4, Audius stream loaded), click '#btn-main' started playback without any user gesture (paused=false, currentTime 1.3s then 5.5s), and read returned the live NOW PLAYING text. New tsx test file (11 tests: snippet injection safety, frame matching, executor happy/rejection paths, static wiring guards) enrolled in test:bot-browser. Gates green: compile:electron, lint, build, full test umbrella, check:dsh-deps, kernel-patch fingerprints. Not merged to main yet.","contentType":"text/plain;utf-8","attachments":[],"quotePin":""}