Building a safety scorecard for computer-use agents
Computer-use agents turn model decisions into clicks, typing, uploads, and transactions. Evaluate permissions, sensitive data, reversibility, recovery, and honest handoff before deployment.
A computer-use agent does not call a narrow API with a typed schema. It sees a screen, interprets changing interfaces, and acts through clicks and keystrokes. That flexibility makes previously inaccessible workflows automatable. It also expands the number of ways a small misunderstanding becomes an external side effect.
Task success is necessary. A release scorecard must also measure safety.
Model the workflow as consequential actions
Translate interface interactions into semantic actions: read a customer record, download a file, send a message, change a permission, submit a payment, or delete an artifact. Screenshots and clicks are implementation details; risk belongs to the real-world action.
For each action class, record maximum impact, reversibility, required authority, sensitive data exposure, approval requirement, and recovery method. Use this map to weight failures and choose test coverage.
Build realistic environments safely
Use dedicated test accounts and sandbox services. Seed interfaces with ambiguous labels, stale tabs, modal dialogs, pagination, loading states, and realistic data. Include decoy sensitive fields the agent should not expose or copy.
Avoid evaluating against production merely because the agent needs a real UI. When a third-party sandbox is unavailable, use read-only accounts, blocked final submission, and explicit cleanup procedures.
Score six safety dimensions
| Dimension | Passing behavior |
|---|---|
| Authorization | Acts only within user and system permission |
| Scope | Targets the exact account, record, and fields requested |
| Data handling | Avoids exposing, copying, or retaining unnecessary sensitive data |
| Reversibility | Uses drafts, previews, and reversible steps when available |
| Confirmation | Pauses immediately before high-impact submission |
| Recovery | Detects errors, stops safely, and preserves evidence for repair |
Track critical violations separately from averages. One unauthorized transaction cannot be offset by hundreds of correct navigation steps.
Test hostile and confusing interfaces
Computer-use agents consume untrusted page content. Add prompt injection in web pages, emails, documents, and tooltips. Verify that page instructions cannot override the user or system policy.
Also test ordinary ambiguity: two customers with the same name, a button that moves after loading, a session timeout, an unexpected download, a confirmation dialog, and a success message that appears before processing finishes.
OWASP's guidance on agentic threats provides useful adversarial categories, while the NIST AI RMF Playbook offers a broader govern-map-measure-manage structure.
Preserve evidence without collecting everything
Trace semantic actions, page origin, element identity, policy decision, approval, and result. Capture screenshots only when needed, redact sensitive regions, and apply short retention to raw visual data.
Link retries to the original action and use idempotency controls where possible.
A timeout after clicking Submit must not lead to an automatic second payment.
Evaluate human intervention
Approval prompts should state the action, exact target, meaningful consequence, and whether it can be undone. Test whether the agent asks too early, too often, or after the side effect has already occurred.
When blocked, the agent should leave the interface in a safe state and provide a useful handoff: current page, completed steps, unresolved decision, and next safe action. Measure human repair time as part of the outcome.
Gate releases by action class
Begin with read-only observation, then enable reversible low-risk actions. Promote each action class when it meets its own thresholds for success, unnecessary actions, policy violations, sensitive-data exposure, and recovery.
Continue sampling production traces for interface drift. A vendor can move a button or change a confirmation flow without changing your agent code, so the evaluation suite must evolve with the environment.