Bob: Because journal clubs are designed to teach participants how to critically analyze a study or journal article, participants may often feel that the medical literature is nothing but a wastebasket of bad studies. To turn the tables, we chose to analyze an article that critiques the guideline development process. It discusses how two reputable organizations, the U.S. Preventive Services Task Force (USPSTF) and the American Academy of Pediatrics (AAP), came to completely different conclusions in their recommendations on lipid screening in children. That will give us a starting point to discuss the topic of guidelines.

What does this article say?

Bob: The family physician is likely to face this clinical dilemma: Should I screen a child/adolescent for hyperlipidemia? In 2008, the AAP recommended screening in children two to 10 years of age with risk factors for cardiovascular disease or a family history of premature cardiovascular disease or hyperlipidemia.¹ Around the same time, the USPSTF noted that there was insufficient evidence to recommend routine screening.² How do two organizations with the same data come to such different conclusions?

Table 1 summarizes the main differences between the two organizations' approaches to developing recommendations.³ The USPSTF has a standardized approach of forming a key question, collecting evidence, assessing and synthesizing data, developing the recommendation, and obtaining an external peer and public review. In this case, the USPSTF starts with the core question: Are there any controlled trials of lipid screening versus no screening with information on long-term health outcomes? Because there are no such studies, they formulated nine other questions looking for indirect evidence. With a lack of adequate indirect evidence, a USPSTF I recommendation (insufficient evidence) was given for lipid screening in children. Although the AAP recommendation lacks this level of rigor, the article notes that previous AAP guidelines have been thorough and organized.

What should we make of this article?

Bob: It should come as no surprise that this article was written by the authors responsible for the USPSTF's childhood lipid screening recommendation. They clearly lay out their process of guideline development and fairly point out differences with the AAP. If they sound a little defensive, I can't blame them after the heat they received with their recent, highly publicized recommendations on screening for breast and prostate cancers.

Mark: I like this article because it points out some of the problems with the deluge of guidelines that are released.

Let's talk about “evidence,” a term that is thrown around far too loosely. The USPSTF's technique of question development followed by a detailed, standardized analysis of the literature to answer a specific question is “best in class.” Far too often, recommendations that are purported as evidence-based are nothing more than a consensus recommendation from a group of experts without any well-defined data to support it. Case in point: A 2009 review of 14 American Heart Association/American College of Cardiology (AHA/ACC) guidelines revealed that only 11 percent were based on multiple randomized trials or meta-analyses (AHA/ACC level A evidence), whereas nearly one-half were based on consensus (AHA/ACC level C evidence).⁴ Simply put, a group of experts expressing their views is not evidence.

Andrea: And just who are these experts involved with guideline development? The USPSTF dismisses experts with conflicts of interest, and so does the United Kingdom's National Institute for Health and Clinical Excellence. This is a no-brainer in my book. However, a review of 17 ACC/AHA guidelines involving 498 contributors revealed that 56 percent of the guidelines had an author with a conflict of interest.⁵ And this ethical dilemma is not limited to ACC/AHA guidelines. A 2011 review of 14 U.S. and Canadian clinical practice guidelines on diabetes mellitus and hyperlipidemia revealed that there was no conflict-of-interest documentation in five of the guidelines (this does not mean there were no conflicts of interest), and of the nine with documentation, 48 percent had authors who disclosed conflicts of interest.⁶ Digging deeper, specialty society guidelines in particular seem to have strong ties to pharmaceutical manufacturers. A 2002 survey of 100 specialty society guidelines revealed that 87 percent had ties to the pharmaceutical industry.⁷

Bob: One has to wonder why an organization that is creating a guideline does not seek conflict-of-interest disclosure or recuse authors that have a conflict of interest. Perhaps the truth is that specialty society organizations themselves may sometimes have a conflict of interest (e.g., receiving income from pharmaceutical companies). This can lead to the organization consciously or unconsciously predetermining the outcome of a guideline. Don't believe this can happen? A British Medical Journal report revealed that a prominent “nonprofit” organization that received a total of $11 million from a pharmaceutical company subsequently placed six members with ties to this company on a nine-person committee that was to determine the role of the company's drug in a guideline.⁸

Many have called for transparency in the clinical practice guideline development process. Currently, final recommendations often appear as if there was unanimity among the contributors' recommendations. This is not always the case. It would be more appropriate to disclose how the committee members voted on the final recommendation, similar to Supreme Court rulings. Wouldn't a five-to-four decision reflect a level of concern (or conflict) that clinicians should be aware of?

Mark: The Institute of Medicine (IOM) has proposed eight standards that should be followed when developing clinical practice guidelines⁹:

• Complete transparency in guideline creation

• Conflict-of-interest disclosure

• Member composition that is multidisciplinary

• Systematic review of the literature that meets IOM standards

• Clear and consistent rating and description of the evidence

• Recommendations that are articulated in detail and in a standard form

• External review by the full spectrum of stakeholders

• Appropriate updating of the guideline

Although these proposed standards are laudable, currently there is no way to enforce them. Consequently, guidelines will continue to remain unregulated in the foreseeable future.

Andrea: I would like to comment on the IOM's recommendation for appropriate updating. A guideline is worthwhile only if it is up to date. An analysis by the RAND Corporation estimates that 90 percent of guidelines are relevant for 3.6 years, and that 50 percent become obsolete after 5.8 years.¹⁰ Here is a good example: In 1996, the AAP made a strong recommendation that children six to 18 months of age with a febrile seizure should have a diagnostic lumbar puncture. This reflected the prevalence of childhood meningitis before pneumococcal vaccines. Over the next couple of years, a number of reports demonstrated the changing epidemiology of the disease and the limited yield of lumbar puncture in these patients.^11–13 However, the AAP didn't change its recommendation until 2011.¹⁴

Bob: A visit to the National Guideline Clearinghouse (http://www.guideline.gov), a product of the Agency for Healthcare Research and Quality, puts this issue into perspective. As of July 12, 2012, there were 2,476 clinical practice guidelines listed. There can be 10, 20, or more guidelines on a single topic. The Web site has a link that allows users to compare various guidelines. Still, there is no documented oversight on how the guidelines were developed, and there are no comments on the quality of the guidelines—any group who wants to create a clinical practice guideline is free to do so.

What should the family physician do?

Andrea: When it comes to the question of lipid screening in children, I would go with the more rigorous, albeit conservative, USPSTF recommendation. There is no evidence that early identification of hyperlipidemia in children changes long-term outcomes. Let's focus our limited health care resources where they may do some good: encouraging physical activity and maintaining an appropriate body weight.

Mark: When it comes to trying to evaluate an organization's clinical practice guideline, I think it is better to side with a primary care organization versus a specialty society. A review of 33 guidelines on hypertension, hyperlipidemia, and cholesterol screening and cardiovascular prevention showed that those from specialty groups tended to be less methodologically sound and recommended more aggressive therapy compared with those from primary care organizations.¹⁵

Bob: I still believe there is a role for well-conceived, impartial guidelines. The IOM has made sound recommendations, and the World Health Organization has also proposed a rigorous 19-step process for guideline development.¹⁶ The American Cancer Society has stated that it will follow the IOM recommendations.¹⁷ Let's see how many other organizations take these lessons to heart.

editor's note: The issue of universal screening for hyperlipidemia in children will be addressed in an upcoming Controversies in Family Medicine pair of pro/con editorials.