Should antibiotics be used to treat acute otitis media in young children?

Bob: In the November 15, 2011, issue of American Family Physician, Andrea, Mark, and I discussed why the conclusion of this high-profile study, which suggests that antibiotics are beneficial in children six to 23 months of age with acute otitis media (AOM), was incorrect. Since our original analysis, we have uncovered more information regarding this study. We believe this additional information is worth knowing when it comes to evaluating clinical trials. First, let's look at what we shared with you last time.

What does this article say?

Bob: This randomized double-blind study included children six to 23 months of age with AOM taking high-dose amoxicillin/clavulanate (Augmentin) or placebo. The authors reported the following: (1) there was no difference between the two groups in time to initial resolution of symptoms; (2) there was a slight improvement in “sustained” resolution of symptoms at seven days; (3) the mean symptoms scores favored antibiotics; and (4) more clinical failures (defined as persistent findings on otoscopy) occurred in the placebo group. The study's conclusion favored antibiotic therapy.

Should we believe this study?

Bob: We need to focus our attention on the four primary outcomes. The first shows no difference in time to initial resolution of symptoms in children who received antibiotics versus those who did not. This end point is what we are most concerned with—when does the child finally stop crying or fussing, or when does the child's fever go away? The second outcome (sustained resolution of symptoms) documents a slightly greater likelihood of two consecutive days of resolution of symptoms with antibiotics compared with placebo. The third outcome was the mean AOM severity of symptoms score (a 14-point scale) at seven days. In the group that received antibiotics, the mean score was 2.79, whereas the mean score in the placebo group was 3.42. These two end points, although statistically significant, are clinically insignificant; do you think a child or the child's parents can tell a 0.63 difference on a 14-point scale? And lastly, the presence of more persistent findings on otoscopy in the placebo group in follow-up has no clinical effect. This is what we call disease-oriented evidence; it is not a patient-oriented outcome that we care about.

Mark: Having four primary outcomes is odd. Clinical trials should be designed to have one predesignated primary outcome, with all other outcomes considered secondary. The reason for this is straightforward—when attempting to assess if an intervention makes a statistically significant difference, you can only look at one outcome. If you look at three, four, or 10 outcomes, you keep increasing the odds that a result will become positive by chance alone. The time-honored analogy is that of flipping a coin and the likelihood of it landing on the heads side 10 times in a row. On the first try, this is unlikely, but if you repetitively flip the coin, you are more likely to accomplish this improbable feat. So, four primary outcomes is not what one would expect to see in a study like this.

Andrea: I think it is important to clarify the difference between primary and secondary outcomes. In large trials, a lot of data can be generated and a lot of subgroups can be analyzed. But, before a trial begins, one primary question is asked, and all the remaining data that are analyzed (the secondary outcomes) should be used to generate thoughts/ideas/hypotheses for future studies.

Bob: That is where ClinicalTrials.gov comes in. Launched in February 2000, this Web site sponsored by the National Institutes of Health is a large registry and database of studies, completed and ongoing, in the United States and around the world. It was created in response to researchers', patients', and policymakers' need to obtain a comprehensive understanding of published and unpublished studies. Currently, the database contains 112,970 trials and includes extensive details on each, such as the planned primary and secondary outcomes.

When you look up this study on the ClincialTrials.gov Web site, you find that there were only three primary outcomes planned and the fourth outcome, otoscopic resolution, was one of many planned secondary outcomes.¹

Mark: Wait a minute—you can't switch a planned secondary outcome and make it a primary outcome just because you like the way the result turned out.

Andrea: What is even more concerning is that the otoscopic findings are only one of 22 secondary outcomes evaluated in this study. It amazes me that a significant number of these findings, the ones that just happen to support placebo, were never reported. The secondary outcomes that demonstrated no difference between placebo and amoxicillin/clavulanate were analgesia requirements in these children; number of needed follow-up visits to a primary care physician; number of visits to the emergency department; missed hours of work by the parents; and parental satisfaction.

It is disconcerting to see studies spun so positively when the original primary outcome, time to resolution of symptoms, demonstrated no improvement with amoxicillin/clavulanate.

Mark: So, no benefit of antibiotics, but all the harm. Twenty-four percent of the children in the antibiotic group developed diarrhea (number needed to harm = 6).

Bob: The shifting of primary outcomes becomes even more perplexing when reviewing the study's original protocol that was submitted to The New England Journal of Medicine (and posted on their Web site). That protocol notes that only one primary outcome (not three or four) was to be studied: time to resolution of symptoms. And, as previously mentioned, there was no difference in this outcome between placebo and amoxicillin/clavulanate.

Mark: With the clear discrepancies between the protocol submitted to The New England Journal of Medicine and ClinicalTrials.gov and what was ultimately published, one has to wonder how it escaped the attention of the editors and the manuscript peer reviewers. Or was this a case of positive-outcome bias that causes reviewers and editors to overlook flaws in a study? This phenomenon of overlooking manuscript errors in studies with positive outcomes was demonstrated in a recent study in which peer reviewers were given two fabricated manuscripts, one with a positive outcome and one with no difference, each with intentionally embedded errors in the manuscript. The reviewers identified more errors in the no-difference manuscript and missed the same errors in the positive-outcome manuscript.²

It is sobering to realize the potential pitfalls that can distort the actual findings of a research study. One other potential source of bias relates to conflicts of interest, which have been found to bias the findings and reporting of research studies, contrary to the common assumption that “the findings speak for themselves.”³

Bob: It is usually easy to identify if authors of a study have such conflicts, because most journals report the authors' conflicts of interest. In this study, the lead author and a colleague each received honoraria from the manufacturer of the drug used in the study.

What should the family physician do?

Andrea: A Cochrane review of 10 studies including children with AOM that compared antibiotics with placebo notes that 16 children need to be treated with antibiotics to prevent one from having ear pain.⁴ A more recent report in the Journal of the American Medical Association reported a number needed to treat of 9 to prevent one child from having ear pain.⁵ But, the gain in pain relief is minimal—relief of pain approximately 12 to 24 hours earlier.

When you add the study we have been discussing, which is overwhelmingly negative, to all the previous studies assessing the effectiveness of antibiotics, most of the data suggest that AOM is a self-limiting condition. Look at figure 2 in the original article; it shows that the placebo group approaches minimal pain and discomfort as quickly as the antibiotic group.

Mark: This study lends credence to why the “wait-and-see” and “backup” approaches to antibiotics for AOM have been so successful.^6–8 You should try to hold off on prescribing antibiotics in these patients.

Bob: Antibiotic stewardship has to become a priority. The reasons are obvious: cost; adverse effects, including increased Clostridium difficile infection rates in children⁹; increased resistance; and an antibiotic development pipeline that is drying up.¹⁰ Antibiotic stewardship for a self-limiting condition such as AOM is a good place to start.