When quality measures go bad

This person is using two clipboards to assess quality measures.
Let me start out by saying that this is a shameless plug for one of my own research projects. Actually, the real credit for this study goes to Erin Quattromani, one of my co-residents, and Emilie Powell, a fellow at my program.

Basically we looked at a national inpatient sample of adults admitted for pneumonia. Erin and Emilie (with help from several others attendings) stratified hospitals according to their performance on the Center for Medicare Studies quality measure for getting appropriate antibiotics in pneumonia patients within 6 hours.

Unsurprisingly for those familiar with the controversy surrounding this quality measure, we didn't find much difference in mortality.

Now there are plenty of limitations and you could pick about the methodology until you were blue in the face. All we really said was:
Hospitals that are the best at getting antibiotics within 6 hours are not hospitals with the lowest inpatient mortality.
Again, pick apart to your heart's content.

That said, I did some of the lit search into how this rule came up and found that its scientific basis is shaky at best. For the best breakdown of this, see the Yu and Wyer paper I cite at the bottom.

Basically there were these two big studies of old, sick Medicare patients publishes in The JAMA and in Archives of Internal Medicine. They showed a trend towards increased survival with early antibiotics.

So there you go: because in 2004 someone showed that 84 year olds with cancer and pneumonia do better with early antibiotics, hospitals get dinged when you don't get them into your otherwise healthy 45 year old male on time.

The truly shocking thing is that there is decent research to suggest that attempted compliance with this silly rule has led to diagnostic errors, overtesting, and (worst of all) administration of antibiotics to patients who didn't need them.

Don't just do something-- stand there (at least until you know the damned diagnosis!).

-MJP

seth says:
One of the key differences between the data that suggest early antibiotics may be good and the CMS rule is that the studies were done with ED diagnoses of pneumonia, whereas CMS dings hospitals for missing the 4 hour window on patients with a discharge diagnosis of pneumonia.  
I can speculate that the patients who are not diagnosed initially with pneumonia may have more complex presentations and therefore might be sicker and more likely to die, but the truth is that no one knows.
Also, the PORT scoring system (or Pneumonia Severity Index) is a great tool to estimate mortality associated with pneumonia, but a lot of studies (and clinicians) use it as an admission criterion, although it has not been prospectively validated as a disposition tool.
-nst

References:
Quattromani E. Powell E. et al. Hospital-reported Data on the Pneumonia Quality Measure ‘‘Time to First Antibiotic Dose’’ Are Not Associated With Inpatient Mortality: Results of a Nationwide Cross-sectional Analysis. Academic Emerg Med. 2011;18:1-8.

Houck PM, Bratzler DW, Nsa W, et al. Timing of antibiotic administration and outcomes for Medicare patients hospitalized with community acquired pneumonia. Arch Intern Med. 2004;164:637-644.

Meehan TP, Fine MJ, Krumholz HM, et al. Quality of care, process, and outcomes in elderly patients with pneumonia. JAMA. 1997;278:2080-2084.

Yu KT, Wyer PC. Evidence behind the 4 hour rule for initiation of antibiotic therapy in CAP. Annals of Emerg Med. 2008;51:651-662.

Comments