Amanda Jones is no ordinary woman. Literally. She's virtual and she's helping, along with her Gator Engineer makers, create better doctors.

"What brings you in to see me today?"

"Part of my left breast has been painful for awhile."

"Can you lie down so that I can examine you?"

It sounds like a snippet of conversation between doctor and patient. But
the doctor, in this recent exchange at the University of Florida campus,
was actually an engineering doctoral student -- and the patient a "mixed
reality human" composed of a life-sized computer avatar on a flat screen
and a mannequin with a prosthetic breast.

Intimate procedures such as breast exams, while a routine and critical
part of medical care, are notoriously tough to teach. Medical students
practice on disembodied prosthetics but have limited opportunities to
practice exams on real people -- especially patients who have an
abnormality. In a collaboration with the Augusta, Ga.-based Medical
College of Georgia and three other universities, UF engineers have
crafted a solution: a hybrid computer/mannequin that helps train
students not only how to correctly perform a breast exam -- but also how
to talk to, and glean information from, the patient during the
procedure.

The project is important because correct examinations and good
doctor-patient communication are critical to successful medical
treatment, said Benjamin Lok, a UF assistant professor of computer and
information sciences and engineering who heads the effort.

"Studies have shown that communication skills are actually a better
predictor of outcome than medical skills," Lok said. With the virtual
patient, "all of a sudden, students have to not only practice their
technique, but they also have to work on their empathy."

The mixed reality human, named Amanda Jones, "talks" to students, and
they respond via a computer speech and voice recognition system tailored
by doctoral student Aaron Kotranza, Lok and others on the team. Her
physical form -- a mannequin -- is immobile, but her virtual
representation, created by the engineers, moves and speaks from a large
flat screen above her physical body. Students can also view Jones
through a head-mounted display.

The interaction is unscripted, but it follows a typical pattern for a
woman's visit and examination -- with both verbal and tactile challenges
for the medical students.

The student must tease out Jones' medical history, listen to her
concerns and respond to her questions. Just as in a real exam, this
interaction occurs simultaneously with the physical examination. For
that, the student must use the correct palpitating technique and apply
the proper pressure. Sensors within the prosthetic breast -- developed
by Dr. Carla Pugh at Northwestern University -- provide pressure
information depicted by colors on the virtual breast, guiding students
in the exams. The engineers can program the system to include or exclude
an abnormality -- and the attendant conversation.

It sounds awkward, and to be sure, the speech recognition element has
its hiccups.

But especially for students reared in an era of sophisticated
three-dimensional video games, the system turns out to be surprisingly
convincing. The researchers have tested it on about 100 medical students
so far, all from the Medical College of Georgia, where co-principal
investigator Dr. D. Scott Lind is based. One of their most consistent
and prominent findings: Students do not hesitate to express empathy to
Jones.

"We have found that they will try to comfort the virtual human,"
Kotranza said. "They'll often touch the mannequin in order to comfort
her."

A pilot study has concluded that students who practiced with a mixed
realty human improved their communication skills and their technical
abilities, but more trials are needed to determine whether those skills
persist once the students examine real patients.

That said, it seems obvious that more practice students get, the better
off they will be. Lok said the mixed reality patient is not intended to
replace real volunteers - far from it. But students typically have only
a handful of opportunities with those volunteers before graduating. The
mixed reality patient can add to their training while making it easier
for teachers to help students with both their conversational and medical
techniques.

"What happens if you find something in a woman's breast? How do you talk
to the patient?" Lok asked. "Students have to somehow build their
database of experience."

While the breast exam research continues, the team also intends to
explore other intimate exams. Next in line: prostate exams. Lok and the
students already have prosthetics they intend to couple with a virtual
male patient similar to the breast exam patient. 

The other institutions participating in the project are the University
of Central Florida, the University of Georgia and Northwestern
University. The research, part of a larger effort involving a number of
different virtual patient projects, is supported by grants of about $2.8
million primarily from the National Science Foundation and the National
Institutes of Health.