Jesse Wang believes the best way to develop technology that can fit seamlessly into a doctor’s practice is to bring medical training and technical understanding to the table. To that end, Wang has developed a prototype of a virtual assistant for physicians.
Wang, who is pursing both an MD and a PhD in translational biomedical science at the University of Rochester, recently won the Audience Choice Award at the American College of Physicians’ Innovation Challenge, for his invention. In a “Shark Tank”-style contest, Wang had the opportunity to explain the reasoning behind his device and its use.
“I got amazing feedback from the audience. Many physicians said the problem of too much documentation work is detracting from the joy of practicing medicine,” he says. “They also said they’d be interested in testing the prototype system.”
Wang’s tool, called the Digital Scribe, aims to help physicians focus on patients. It has three main components: a speech-to-text engine; a dialogue framework; and a grammar conversion process. The Digital Scribe is built around the way clinicians talk to patients, using patient-centered communication; elements include summarizing, transitioning and asking open-ended questions.
The technology identifies different speakers in a conversation by using artificial intelligence and machine learning. Language-processing algorithms separate medical information from the conversation and go as far as to converting words in first-, second- and third-person tenses. Integration of these working parts results in a complete documentation of a patient visit.
Wang expects the final version of Digital Scribe will be integrated into electronic records. It will be deployed as a mobile application for iOS and Android as well.
“Clinicians will just select which patient they are seeing and then carry on with the patient visit,” he says. “When they’re finished, they’ll tap a button to end the visit and the Digital Scribe will upload a generated encounter note.”
For those worried about privacy (a question posed by a judge at the contest), Wang says it is a matter of explaining the purpose of the recording, and that the device could be removed if patients are uncomfortable.
“The physician would say something like, ‘This is a new system that will record our conversation to help automatically fill out the paperwork. There are no real people listening to the recording—it’s all computationally interpreted and when we’re done the recording will be deleted. This system will help me spend less time on the computer, so I can focus all my attention on you. If you’re uncomfortable though we can remove the device,’” he says, adding that the ACP Innovation Challenge judge seemed to like the answer.
Scores of reports suggest that the adoption of electronic medical records continues to meet with resistance from physicians. In fact, health care in general has been slow to embrace technology. Wang ties it to a lack of understanding of the practice of medicine.
“Most people developing technology do not really understand what it is like to practice medicine,” he says. “The e-record system, built by software engineers with no medical training, is a prime example. Even having physician consultants to guide the development process is insufficient.”
Even though he has medical training and software development experience, Wang says it is still very difficult for him to communicate to computer scientists what medicine needs and how it’s practiced.
“I could not imagine how difficult it’d be for physician consultants without experience in programming to talk to a team of software engineers,” Wang says. “In addition, medical offices usually tend to be very busy and it’s hard to break routine to incorporate new technology.”
The Digital Scribe, which Wang also terms as an intelligent speech system, could ease some of the burden off physicians while accomplishing documentation at the same time. During a doctor’s visit today, it is not uncommon for a practitioner to take notes on a laptop while listening to a patient, to complete a report. Wang’s tool would complete those notes from the visit through speech recognition.
Wang is no stranger to innovation. His work with CupQ, a literature search engine for clinicians that also uses machine learning, aided in the development of building a speech system.
He has considered starting his own business. First, he plans to test the prototype with physicians.
“When physicians seem satisfied, then I’ll start exploring more on the commercialization side,” Wang says.
It would probably be far better and more efficient for everyone if doctors had reduced loads, less bureaucracy, and more time with patients.
Technology is amazing. Can human agency be digitized in a meaningful way?