Computer use / browser use

An AI capability that observes a graphical interface and issues interaction actions such as clicking, typing, scrolling, or navigating.

Computer use operates through the interface presented to a user rather than only through a purpose-built API. It can fill forms, navigate dashboards, or bridge applications when a structured integration is unavailable or omits a needed workflow. The runtime may use screenshots, accessibility trees, browser state, or a combination, so products differ in what the model actually observes.

Builder example

The interface is untrusted input. Pop-ups, changed layouts, prompt injection in page content, hidden state, and timing can alter the chosen action. APIs provide typed operations but can also expose broad or unsafe capabilities. Compare the two paths on reliability, scope, auditability, maintenance, and the exact operation rather than treating either as universally safer.

Common confusion: Computer use is neither universal automation nor merely a last resort. It can be the practical path for cross-application work, yet a stable API is often easier to validate and make idempotent. Hybrid workflows can use visual navigation for one step and structured tools for consequential effects.