Client tools and safety
When the Onboarding coach points at a button or fills a field, it isn’t controlling the page from a server. It asks your page to do it, through the browser SDK, and the SDK decides. This page describes that protocol and the rules the SDK enforces. It’s written to be read by a security reviewer.
How a tool call travels
The conversation runs in a LiveKit room. The voice worker calls the page with LiveKit RPC method rydhm.tool, waiting up to 10 seconds (30 seconds for click, because a risky click waits for the person’s answer):
The page answers:
or
- Payloads are JSON, at most 15 KB each way. A larger result is shortened to
{ "truncated": true, "text": "…" }. - Each call answers inside the caller’s timeout, less a second of headroom; no answer means it didn’t happen.
- Only the expert (an agent participant in the room) can call tools. Calls from anyone else are refused.
- The coach never claims an action it didn’t get
okfor. Atool.calledevent (name, ok, duration; never the arguments) is recorded for each call.
Built-in tools
Which ones an expert has is a setting of the expert: Page tools, in a coach’s Settings → Actions & tools and in other experts’ Advanced tab (toolPolicy.clientTools in the API). All of them except look are on for an Onboarding coach and off for other experts. look (screen vision, See charts and icons) is off for every expert until you turn it on (toolPolicy.screenVision: "on"), because it sends a picture of part of the person’s screen to a vision model.
An Onboarding coach also has complete_step, which runs on the worker, not in your page: it marks a guide step done, only after it saw the step happen. Rydhm’s API decides whether a guide is complete; the page’s word is never enough.
Targets
target is resolved in this order:
- A page-map name from Rydhm (“Invite button”): its CSS selector, then its visible-text hint.
- A CSS selector (
#invite,[data-test=invite]). text:<visible text>: the best match among visible, enabled, interactive elements. An exact match beats a prefix, which beats a substring. Hidden elements never match.
A target that doesn’t resolve is answered ok: false and reported as a target.missing event, so the page map shows it as Missing.
Custom tools
Your page adds its own with registerTool(name, { description, parameters }, handler). They’re offered to the model next to the built-ins:
- Names: lowercase letters, digits and
_, starting with a letter, 2 to 41 characters, and not a built-in’s name. - Up to 20 per page; each description up to 1,000 characters and each JSON Schema up to 4 KB.
- The handler gets
(args, { callId, signal })and returns any JSON. Throwing answersok: falsewith your error’s message (up to 500 characters).
WebMCP
WebMCP lets a page declare tools to the browser’s own AI agents. It’s a W3C Community Group draft that still changes, so the SDK feature-detects it and does nothing in a browser without it.
- Your
registerTooltools are WebMCP tools too. When the browser hasdocument.modelContext(ornavigator.modelContext),registerToolalso registers the tool there, and the function it returns unregisters both. - Your WebMCP tools are the coach’s. During a conversation the SDK offers the page’s WebMCP tools to the expert as custom tools, run through the browser’s
executeTool: up to 12, each with a description and an object schema of at most 4 KB. A tool marked destructive asks the person first, like a risky click.
Page context
The browser keeps two participant attributes current, and the worker reads them on every turn:
Safety guarantees
These live in the SDK, in the browser, so text in your page or a prompt injection that reaches the model can’t turn them off.
- Page text is data, never instructions. What
read_screenreturns is marked as page content and reaches the model inside a clearly marked block. fillrefuses password, hidden and file inputs; every paymentautocompletetoken (cc-*,transaction-*);current-password,new-passwordandone-time-code; fields whose name, id, label or placeholder say card, CVV or CVC, expiry, IBAN, routing or account number, SSN, password, PIN, OTP, API key or secret; every field in a form that collects card details; and disabled or read-only fields.- Risky clicks ask first. A click on an element whose text, aria-label, title, value, name, id, test id, link or form action says delete, remove, pay, buy, purchase, checkout, place order, cancel subscription, close account, transfer, refund, withdraw, unsubscribe, upgrade, downgrade, deactivate, disable, reset, revoke, disconnect or uninstall (and a few more), on a link to another website, or on the submit button of a payment or password form, shows a confirmation. It clicks only on Yes; no answer before the coach’s timeout means no click. Add
data-rydhm-confirmto any element to always ask. read_screennever reads the value of password, payment, one-time-code, hidden or private fields (it saysprotected: true).data-rydhm-privateon any element hides it and everything inside it from every tool: never read, clicked, filled or highlighted.navigatestays on your origin.- Nothing is parsed as HTML. The bubble and the spotlight live in Shadow DOM and set every string with
textContent. - Network traffic goes only to the API address, the LiveKit address the API returns, and the expert’s photo (served from Rydhm’s own hosts). No cookies, no external fonts, no third-party requests.
- The publishable key works only from your allowed websites, and only for its one expert. It can start a conversation and report events; it can’t read your data or change any setting.
A good rule for your own tools: anything that spends money, deletes or sends something should ask the person in your UI before it acts, the same way the built-in click does.