How Do Diagnosis Errors Happen? New National Report Sheds Light

Duration: 9:02

The clear conclusion of CRICO’s new analysis of national malpractice data is that diagnostic errors should be among the highest priorities for intervention.

This podcast is an episode of Patient Safety Updates. You can find other episodes and subscribe using the links to the left.

Guest Commenters

• Mark Graber, MD, FACP
• Heather Riah, MBA
• Hardeep Singh, MD, MPH

Transcript

When a medical case goes badly and the patient or family sues the clinician, we can learn from that. But it probably doesn’t tell us enough about what led to the problem. A single case also doesn’t tell hospitals or ambulatory practices with limited resources, where the priorities are for improvement.

In late 2014, the Harvard medical system’s malpractice insurance company, CRICO, used a national pool of 300-thousand malpractice cases to suggest where providers can place their resources in order to reduce the most risk to patients.

CRICO gathers these data through its national comparative benchmarking system with more than a dozen malpractice insurance carriers from across the United States. Its 2014 Annual Benchmarking report, “Malpractice Risks in the Diagnostic Process” draws from more than 23-thousand open and closed malpractice cases from a recent five-year period.

And the clear conclusion of the new analysis is that diagnostic errors should be among the highest priorities for intervention. Between 2008 and 2012, a fifth of the cases alleged failures in diagnosis. Diagnostic errors out-paced errors in obstetrics, and were costlier than surgical cases. Of those diagnosis-related malpractice cases, 57 percent arose in the ambulatory setting, excluding Emergency Department cases. The 2014 benchmarking report goes much further to help direct future interventions. By far, the diagnoses most commonly missed involved cancer. And the benchmarking data indicate that improvement efforts should zero in specifically on differential diagnosis, test interpretation, and follow-up of consults.

“To me, this tells the story that these errors are everywhere.”

Dr. Mark Graber is President of the Society to Improve Diagnosis in Medicine. He is an internist and a researcher at RTI International, a global research institute, and is Professor Emeritus of Medicine at the State University of New York at Stony Brook.

“We used to just think oh, it’s just in the emergency room, which we consider, you know, the Petri dish because it’s such a high-intensity environment with so much uncertainty. But that turns out to be the smallest piece of the pie. So when I show these data, the point I make is these are just all over the place, and so many are in the ambulatory area.”

The cases in CRICO’s database represent more than $25 billion in reserves and losses, involving more than 400 hospitals and 165-thousand physicians. Although the cases come from different malpractice carriers across the country, they are all coded using a shared exhaustive taxonomy of risk management and patient safety codes that were developed at CRICO.

The report organizes diagnoses-related cases by breaking down the diagnostic process into 12 steps, and then dividing them into three categories: 1-Diagnostic Assessment, 2, Testing and results processing, and 3-Follow up and coordination. Dr. Graber says that medicine has to address human error at every step along the way.

“Physicians have a very, in practice, have a very outmoded approach to diagnostic testing. They really believe in these tests and they believe the tests are so accurate and that they understand them. And what I think what we’re learning in analyzing these cases of diagnostic error is that they kind of don’t. They don’t know how to use test results appropriately. They don’t use Bayesian thinking the way they should, and so often they’ve ordered the wrong test or they’ve interpreted it incorrectly

The report shows that the category with the highest percentage of failures—58 percent—is the initial assessment. Among the steps in this category are: taking a history, evaluating symptoms, and establishing a differential diagnosis. The data show that problems with the differential diagnosis command the most attention. In 22 percent of these malpractice cases, no differential diagnosis was established. Only three percent of the cases had atypical presentations.

Dr. Hardeep Singh has done extensive research on medical errors for Baylor Medical College and the Michael E. Debakey VA Center in Houston. In a study of diagnostic errors published in JAMA Internal Medicine in 2013, Dr. Singh and his colleagues noted that over-confidence was a problem. Among the physician study subjects, their measured confidence levels did not match their diagnostic accuracy or the difficulty of the cases. The authors concluded that inflated confidence may prevent doctors from re-thinking a diagnosis that might be incorrect.

“So in our study, for instance, we asked clinicians about these cases. But the issue was, when they most needed help in difficult cases, they actually didn’t ask for it. And we asked them, ‘would you like to get more tests before you come up with your diagnosis?’ And they didn’t want it. And not just tests, by the way. We asked them if they would like to seek referrals, would they like to speak with a colleague, would they like to do more testing or imaging, would they like to consult a clinical decision engine or even a reference material? And they basically said ‘no.’”

The CRICO report moves through the diagnostic process to the second largest category of diagnostic error, Testing and Result Processing. Here the data show that the biggest problem is test interpretation. This was also noted in Dr. Singh’s 2013 study, again linked to measures of physician over-confidence.

According to Dr. Graber, other research has shown that physicians don’t know as much as they think they know about the right test to order, what the results mean, and how to apply the results:

“There’s about 70,000 tests you can order, something like that, and it’s growing by the year with all the genetic tests. And so few people understand these genetic tests. And even without that, I mean hepatitis testing, HIV testing, coagulation testing. Whoever you talk to who’s an expert in those fields will tell you that physicians don’t do a very good job ordering those tests or interpreting them appropriately. So for years we’ve been talking about the lab, well, they screwed up a test—and they do once in a while. But the lab has gotten so much better. Their error rate is down in the, you know, 5 sigma arena there. They’re just doing terrific. The problem these days in lab testing is what they call pre-pre and post-post, those phases, test ordering and test interpretation. That’s where the money is.”

The remaining category in the diagnostic process is follow-up and coordination. According to the CRICO benchmark data report, the step in that category that’s most vulnerable to error is the ordering and management of referrals and consults. Nineteen percent of the cases in the report included such problems as: inappropriate referral, mishandling the consult, or failing to identify who will be following the patient’s problem.

CRICO began compiling and coding its malpractice data from the Harvard system in the 1980s, and added identically coded data from other systems and malpractice carriers a decade later. Heather Riah is Assistant Vice President at CRICO Strategies, which operates the comparative benchmarking system. As with previous CRICO benchmarking data reports on errors in Obstetrics and Surgery, Riah hopes this report on diagnostic errors will help organizations and clinicians focus improvement efforts.

“It is our hope that this gets into the hands of people who are making decisions on what interventions to implement within their setting, people who are actually seeing patients, delivering care. And when they read this, they are able to understand there are errors, there are risks in different environments or different steps in the process of care that they may not have thought of before, and that it impacts how they deliver care.”

Riah says that the specificity of the findings in previous reports has helped institutions find their patient safety targets. One large insurer in California used the benchmark data to identify a problem with communications in a single specialty.

“We had done an analysis for them on their orthopedic claims, and out of that analysis they’ve actually started this new initiative where they used a piece of software to really track all the communications that’s going on with their orthopeds. And they were very excited about being able to use the claims data, look at issues seen within specific cases and then really move forward now to implementing a solution that they’re very excited about and seems to have great promise.”

Additional promising interventions are referenced in the diagnostic error benchmark report itself. “Malpractice Risks in the Diagnostic Process is available online at www.rmf.harvard.edu/diagnosticrisk.