Advancing AMIE towards expert-level audio-visual clinical consultations
Google Research 2 months ago 36
AMIE advanced from text-based clinical reasoning to a real-time video configuration that can observe non-verbal cues, guide physical examination maneuvers, and deliver diagnostic dialogue. In a randomized OSCE study with 100 scenarios, 300 live consultations, and 30 board-certified primary care physicians, AMIE (Video) was rated on par with PCPs across core clinical competencies and higher than both PCPs and AMIE (Text) for physical observation and examination. This shift changes the system from discarding visual and audio information in text chat to using synchronous audio-visual inputs, with results still limited to simulated patient actors and requiring validation on real patients before real-world use.