Google DeepMind
@GoogleDeepMind
AI co-clinician is our new research initiative to help explore how multimodal agents could better support healthcare workers and patients. 🩺 Here’s a snapshot of our progress 🧵
220retweets1.2klikes
View on XGoogle Previews AI co-clinician Agents With Real Time Multimodal Senses
Google DeepMind· Updated
Google announced the AI co-clinician research initiative, a system of multimodal agents designed to assist doctors and patients through real-time audio and video. By moving beyond text-based chat to eyes, ears, and a voice, the system can guide physical exams and medication reasoning.
Google announced the AI co-clinician research initiative, a system of multimodal agents (AI that processes text, images, and audio together) designed to support healthcare teams. Similar to Gemini 3.1 Flash Live, the system uses live audio and video to interact with patients and clinicians in simulated telemedical settings.
- Clinical accuracy (primary care)
- 97 of 98 queries with zero critical errors
- Performance vs PCPs
- Matched or exceeded in 68 of 140 areas
- Architecture
- Dual-agent (Planner and Talker)
- Core benchmarks
- RxQA and NOHARM
- Availability
- Research initiative and trusted tester program
- Primary models
- Gemini and Project Astra
This initiative addresses healthcare worker shortages through a triadic care model where AI acts as a supervised teammate. A dual-agent architecture uses a Planner to monitor a Talker agent, keeping interactions within clinical boundaries. The AI matched or exceeded primary care performance in 68 of 140 assessed clinical areas.
The system is currently a research project and not intended for medical advice. Google is expanding its trusted tester program to sites in the US, India, and Singapore. You can follow the progress of Google's agentic transformation as the lab refines the system's ability to navigate complex medication reasoning.
Still wondering? A few quick answers below.
AI co-clinician is a research initiative from Google exploring how multimodal AI agents can assist healthcare workers and patients. Unlike text-only systems, it uses real-time audio and video to perceive physical cues and guide clinical tasks. It is designed to work under the authority of a human physician to extend their reach and improve care quality.
The system uses a dual-agent architecture to maintain clinical boundaries. A built-in Planner agent continuously monitors the interaction to verify that the Talker agent stays within safe parameters. This design provides an architectural safeguard during real-time patient conversations, ensuring the AI provides helpful assistance without overstepping its role or providing unsafe medical guidance.
In randomized simulations, AI co-clinician performed at or above primary care physician levels in 68 of 140 assessed areas. However, expert human doctors still outperformed the AI in identifying critical red flags and conducting physical examinations. The research suggests these systems are currently most effective as supportive tools for practitioners rather than replacements for human clinical judgment.
No, AI co-clinician is currently a research project and is not intended for medical advice, diagnosis, or treatment. Google is testing the system through a phased trusted tester program with academic collaborators in the US, India, Australia, New Zealand, Singapore, and UAE. It is not yet available as a commercial product for general clinical or patient use.
The triadic care model is a framework where AI agents act as collaborative teammates alongside patients and doctors. In this setup, the AI assists with the patient's care journey while remaining under the expert clinical supervision and authority of a human physician. This approach aims to amplify a doctor's expertise rather than replacing the essential human-to-human medical relationship.
Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →
