HeadsUpAI

Claude Code Desktop now allows AI agents to control macOS interfaces

· Updated

Anthropic released a research preview of computer use for the Claude Code Desktop app on macOS. This feature enables the agent to capture screen data and simulate mouse and keyboard actions like clicking and typing. It allows Claude to operate native applications that lack a command-line interface or API.

Most coding agents are restricted to terminal commands, preventing them from verifying work in graphical environments. By controlling the desktop, the agent can now interact with the iOS simulator or hardware control panels. This shift moves AI from a text-based assistant to an environment-aware operator.

Enable this in settings by granting macOS Accessibility and Screen Recording permissions. The system uses tiered access, restricting Claude to view-only mode for browsers and click-only for IDEs. It is available for Pro and Max subscribers and supports remote task execution through the Dispatch mobile integration.

Felix Rieseberg
Felix Rieseberg
@felixrieseberg
X

Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re https://t.co/tthl6vpID2

1.5kretweets19klikes
View on X

Share this update