'Mixed reality' human helps medical students learn to do intimate exams

GAINESVILLE, Fla. — "What brings you in to see me today?"

"Part of my left breast has been painful for awhile."

"Can you lie down so that I can examine you?"

It sounds like a snippet of conversation between doctor and patient. But the doctor, in this recent exchange at the University of Florida campus, was actually an engineering doctoral student — and the patient a "mixed reality human" composed of a life-sized computer avatar on a flat screen and a mannequin with a prosthetic breast.

Intimate procedures such as breast exams, while a routine and critical part of medical care, are notoriously tough to teach. Medical students practice on disembodied prosthetics but have limited opportunities to practice exams on real people — especially patients who have an abnormality. In a collaboration with the Augusta, Ga.-based Medical College of Georgia and three other universities, UF engineers have crafted a solution: a hybrid computer/mannequin that helps train students not only how to correctly perform a breast exam — but also how to talk to, and glean information from, the patient during the procedure.

The project is important because correct examinations and good doctor-patient communication are critical to successful medical treatment, said Benjamin Lok, a UF assistant professor of computer and information sciences and engineering who heads the effort.

"Studies have shown that communication skills are actually a better predictor of outcome than medical skills," Lok said. With the virtual patient, "all of a sudden, students have to not only practice their technique, but they also have to work on their empathy."

The mixed reality human, named Amanda Jones, "talks" to students, and they respond via a computer speech and voice recognition system tailored by doctoral student Aaron Kotranza, Lok and others on the team. Her physical form — a mannequin — is immobile, but her virtual representation, created by the engineers, moves and speaks from a large flat screen above her physical body. Students can also view Jones through a head-mounted display.

The interaction is unscripted, but it follows a typical pattern for a woman's visit and examination — with both verbal and tactile challenges for the medical students.