During the health-care debate, one damning statistic keeps popping up in newspaper columns and letters, on cable television and in politicians' statements: The U.S. ranks 37th in the world in health care.
The trouble is, the ranking is dated and flawed, and has contributed to misconceptions about the quality of the U.S. medical system.
Among all the numbers bandied about in the health-care debate, this ranking stands out as particularly misleading. It is based on a report released nearly a decade ago by the World Health Organization and relies on statistics that are even older and incomplete.
Few people who cite the ranking are aware that some public-health officials were skeptical of the report from the outset. The ranking was faulted because it judges health-care systems for problems -- cultural, behavioral, economic -- that aren't controlled by health care.
"It's a very notorious ranking," says Mark Pearson, head of health for the Organization for Economic Cooperation and Development, the 30-member, Paris-based organization of the world's largest economies. "Health analysts don't like to talk about it in polite company. It's one of those things that we wish would go away."
“ The study is both flawed and dated, but that does not necessarily mean that we are doing better on health care than the study indicates. ” — David Beemer
More recent efforts to rank national health systems have been inconclusive. On measures such as child mortality and life expectancy, the U.S. has slipped since the 2000 rankings. But some researchers say that factors beyond the control of the health-care system are to blame, such as dietary habits. Studies that have attempted to exclude these factors from the equation don't agree on whether the U.S. system looks better or worse.
The WHO ranking was ambitious in its scope, grading each nation's health care on five factors. Two of these were relatively uncontroversial: health level, which is roughly the average healthy lifespan of a nation's residents; and responsiveness, which is a sort of customer-service rating encompassing factors such as the system's speed, choice and quality of amenities. The other three measure inequality in health-care outcomes; responsiveness; and individual spending.
These last three measures struck some analysts as problematic, because a country with unhealthy people could rank above a healthier one where there was a bigger gap between healthy and unhealthy people. It is certainly possible that spreading health care as evenly as possible makes a society healthier, but the rankings struck some health-care researchers as assuming that, rather than demonstrating it.
An even bigger problem was shared by all five of these factors: The underlying data about each nation generally weren't available. So WHO researchers calculated the relationship between those factors and other, available numbers, such as literacy rates and income inequality. Such measures, they argued, were linked closely to health in those countries where fuller health data were available. Even though there was no way to be sure that link held in other countries, they used these literacy and income data to estimate health performance.
Philip Musgrove, the editor-in-chief of the WHO report that accompanied the rankings, calls the figures that resulted from this step "so many made-up numbers," and the result a "nonsense ranking." Dr. Musgrove, an economist who is now deputy editor of the journal Health Affairs, says he was hired to edit the report's text but didn't fully understand the methodology until after the report was released. After he left the WHO, he wrote an article in 2003 for the medical journal Lancet criticizing the rankings as "meaningless."
The objects of his criticism, including Christopher Murray, who oversaw the ranking for the WHO, responded in a letter to the Lancet arguing that WHO "has an obligation to provide the best available evidence in a timely manner to Member States and the scientific community." It also credited the report with achieving its "original intent" of stimulating debate and focus on health systems.
Prof. Murray, now director of the Institute for Health Metrics and Evaluation at the University of Washington, Seattle, says that "the biggest problem was just data" -- or the lack thereof, in many cases. He says the rankings are now "very old," and acknowledges they contained a lot of uncertainty. His institute is seeking to produce its own rankings in the next three years. The data limitations hampering earlier work "are why groups like ours are so focused on trying to get rankings better."
A WHO spokesman says the organization has no plans to update the rankings, and adds, "We would not consider it current."
And yet many people apparently do. The 37th place ranking is often cited in today's overhaul debate, even though, in some ways, the U.S. actually ranked a lot higher. Specifically, it placed 15th overall, based on its performance in the five criteria. But for the most widely publicized form of its rankings, the WHO took the additional step of adjusting for national health expenditures per capita, to calculate each country's health-care bang for its bucks. Because the U.S. ranked first in spending, that adjustment pushed its ranking down to 37th. Dominica, Costa Rica and Morocco ranked 42nd, 45th and 94th before adjusting for spending levels, compared to the U.S.'s No. 15 ranking. After adjustment, all three countries ranked higher than the U.S.
Still, people often claim that the 37th-place ranking refers to quality or outcomes. High spending rates pushed the ranking down but didn't degrade the quality of care. Among those who have recently failed to make that distinction in published comments are Colorado Rep. Diana DeGette; Iowa Democratic Sen. Tom Harkin; and Margaret E. O'Kane, president of the National Committee for Quality Assurance, an advocacy group.
Representatives for Ms. DeGette and Mr. Harkin didn't respond to requests for comment. A spokeswoman for the National Committee for Quality Assurance said, "WHO is a respected organization. ...We have no reason to believe it is inaccurate, and we would never knowingly misrepresent or misuse another organization's data."
The flawed WHO report shouldn't obscure that the U.S. is lagging its peers in some major barometers of public health. For instance, the U.S. slipped from 18th to 24th in male life expectancy from 2000 to 2009, according to the United Nations, and from 28th to 35th in female life expectancy. Its rankings in preventing male and female under-5 mortality also fell, and placed in the 30s.
But even such analyses, more limited in scope than the WHO's effort, face similar problems: How to differentiate between the quality of the medical system and other factors, such as diet, exercise and violent-crime rates.
Some think that if the U.S. health-care system isn't responsible for troubling outcomes, trying to fix it doesn't provide the best return on investment.
"We might get more bang for the buck by setting aside some of our health-care money to support novel approaches to improve nutrition, education, exercise or public safety," says Alan Garber, an economist and professor of medicine at Stanford University. "Not every health problem has a medical solution."
Nor can everything be ranked -- especially health-care systems. "I think it's a fool's errand," says Dr. Musgrove.
http://online.wsj.com/article/SB125608054324397621.html
