Physicians Assessment of IBM Watson Generated Problem List

cancer drug resistance, Genomic Data

Objective: An accurate, comprehensive and up-to-date problem list can help clinicians focus on providing patient-centered care. In this study, we report on physicians’ assessment of IBM Watson generated problem lists and comparison with an existing manually curated problem list in an institution’s EHR system.

Materials and Methods: Fifteen randomly selected, de-identified patient records from a large healthcare system were analyzed using Watson. Ten internal medicine physicians each reviewed five randomly selected patient records and created their own problem lists (P) for each patient record. Then, they evaluated the Watson generated problem lists (W), and rated the overall usefulness of P and W, as well as the existing EHR problem lists (E). The primary outcome was the physicians’ usefulness ratings of the problem lists on a 10-point scale and their pairwise comparisons.

Results: Six out of the 10 invited physicians completed 27 assessments of P, W, and E, consisting of 732 Watson generated problems and 444 problems in the EHR system. As expected, physicians rated their own lists, P, best. However, they rated W higher than E. In 89% of the assessments, Watson identified at least one important problem that the physicians missed. The higher ratings of W relative to E were influenced by the number of problems missing from E.

Conclusion: Cognitive computing systems hold the potential for accurate, problem-list-centered summarization of patient records, leading to increased efficiency, better clinical decision support, and improved quality of patient care.

By: Murthy V. Devarakonda, Neil Mehta, Ching-Huei Tsou, Jennifer L. Liang, Amy S. Nowacki, John Eric Jelovsek

Published in: RC25615 in 2016