In 2000, the World Health Organization (WHO) ranked the overall performance of health care systems of its member nations.¹ It considered responsiveness of the system, fairness of financial contribution, overall health goal attainment, per capita health care expenditure, and overall level of health. Overall, the United States ranked 37th—between Slovenia and Costa Rica. Although we ranked first in responsiveness to patients, we also led in per capita health care expenditure, spending 13.7 percent of our gross domestic product compared with 7.1 percent for Japan (the highest performer for overall health). Figure 1 compares the per capita expenditures of the 30 nations with the highest healthy life expectancies (i.e., the number of healthy years of life).¹

The United States spends 50 to 100 percent more per person than most of our industrialized trading partners, yet we have worse overall health outcomes than most. Underscoring this point, Banks and colleagues demonstrated that at every socioeconomic stratum, British citizens are healthier than Americans.² Although the WHO report can be criticized,^3–6 a report card commissioned by the Commonwealth Fund similarly found room for improvement in the quality of care provided in the United States.⁷

A report by Manuel and colleagues gives us an opportunity to explore a few of the reasons why we spend so much and get so little benefit.⁸ This team compared six different guidelines on the use of statins to prevent death from coronary artery disease. If the most widely cited American guideline was applied to a hypothetical cohort of adults, we would treat twice as many patients as the most efficient guideline (from New Zealand) but mortality rates would not decrease.⁹ The efficiency of the American and international guidelines is summarized in Table 1.⁸ Although the American guideline saved a few more lives than the New Zealand guideline, it did so at great cost. If the least expensive statins cost $200 per year, the drug costs associated with the American guideline would be $198,000 to prevent one death, compared with $108,000 for the New Zealand guideline. This does not include the additional costs associated with subsequent office visits and other expenses for monitoring or managing side effects of treatment, all of which make the American guideline even more expensive. If we wish to understand why a major guideline can recommend treating so many more persons with so little additional benefit, we have to look at the process through which guidelines are developed.

Clinical practice guidelines come in two broad types: consensus guidelines, which are typically based on the opinions of experts; and evidence-based guidelines, which are developed using an explicit process that first identifies the relevant research, then grades the quality of that research, extracting and summarizing the relevant data and identifying strategies that should lead to improved health outcomes. In both cases, however, the devil is in the details. Guidelines that are based on flawed research, that rely on surrogate disease-oriented end points (e.g., cholesterol levels) rather than clinically relevant patient-oriented end points (e.g., survival, symptom control, quality of life), that interpret the evidence inappropriately, or that are created by individuals or groups with substantial financial conf licts of interest are of questionable validity.¹⁰

The very low lipid thresholds recommended in the American guideline were based on extrapolation of data from studies that used different dosages of statins but did not explicitly study the effects of lowering cholesterol to specific target levels. Statins may have effects that prevent cardiovascular events independent of their lipid-lowering effects, and it may be that the dosage of statins is more important than lipid targets. Furthermore, the extraction of the research data lacked an explicit process, and the guideline became widely accepted before it was tested. Finally, the manner in which the guideline was developed created substantial concern at the time it was released. The panel that made the recommendations was composed largely of persons with a financial relationship with statin manufacturers. Interestingly, the acting director of the National Heart, Lung, and Blood Institute denied a public petition to convene a panel composed of persons without conflicts of interest to provide an independent review of the data.^11,12 In fact, a subsequent independent analysis of the studies used to support the U.S. guideline concluded that the existing data do not support titrating lipid therapy to achieve specific lipid targets.¹³

Although guidelines from other countries may be more efficient, a common limitation of all guidelines, however they are created, is that few are tested to see if they actually work. Because guidelines affect the lives of millions of persons, we should make every effort to evaluate their impact in terms of resources and patient-oriented outcomes. This is rarely done, and analyses like that of Manuel and colleagues⁸ are valuable for that reason.

How, then, can family physicians determine if a guideline can be trusted? We suggest that the guideline development process itself must be explicit and transparent. The process must have a clearly defined purpose and target audience. Finally, and most importantly, the members of the development team should be free of financial conflicts of interest. In Table 2 we propose a set of criteria to evaluate guidelines. Despite our concerns about the National Cholesterol Education Program guideline,⁹ an excellent source of high-quality guidelines that include many of these elements can be found at the National Guideline Clearinghouse (http://www.guideline.gov). American physicians should be open to the likelihood that excellent guidelines may come from organizations outside the United States.

We do not suggest that consensus guidelines should be summarily rejected. But given the many threats to the validity of so many current guidelines, we must demand that there be evidence that a guideline is based on careful and unbiased analysis of clinically relevant data, and subjected to a critical evaluation of its own impact.