Risk Calculator for Cholesterol Appears Flawed
By GINA KOLATA
Available at http://www.nytimes.com/2013/
Last week, the nation’s leading heart organizations released a sweeping new set of guidelines for lowering cholesterol, along with an online calculator meant to help doctors assess risks and treatment options. But, in a major embarrassment to the health groups, the calculator appears to greatly overestimate risk, so much so that it could mistakenly suggest that millions more people are candidates for statin drugs.
The apparent problem prompted one leading cardiologist, a past president of the American College of Cardiology, to call on Sunday for a halt to the implementation of the new guidelines.
“It’s stunning,” said the cardiologist, Dr. Steven Nissen, chief of cardiovascular medicine at the Cleveland Clinic. “We need a pause to further evaluate this approach before it is implemented on a widespread basis.”
The controversy set off turmoil at the annual meeting of the American Heart Association, which started this weekend in Dallas. After an emergency session on Saturday night, the two organizations that published the guidelines — the American Heart Association and the American College of Cardiology — said that while the calculator was not perfect, it was a major step forward, and that the guidelines already say patients and doctors should discuss treatment options rather than blindly follow a calculator.
Dr. Sidney Smith, the executive chairman of the guideline committee, said the associations would examine the flaws found in the calculator and determine if changes were needed. “We need to see if the concerns raised are substantive,” he said in a telephone interview on Sunday. “Do there need to be changes?”
The problems were identified by two Harvard Medical School professors whose findings will be published Tuesday in a commentary in The Lancet, a major medical journal. The professors, Dr. Paul M. Ridker and Dr. Nancy Cook, had pointed out the problems a year earlier when the National Institutes of Health’s National Heart, Lung, and Blood Institute, which originally was developing the guidelines, sent a draft to each professor independently to review. Both reported back that the calculator was not working among the populations it was tested on by the guideline makers.
That was unfortunate because the committee thought the researchers had been given the professors’ responses, said Dr. Donald Lloyd-Jones, co-chairman of the guidelines task force and chairman of the department of preventive medicine at Northwestern University.
Drs. Ridker and Cook saw the final guidelines and risk calculator on Tuesday at 4 p.m., when a news embargo was lifted, and saw that the problems remained.
On Saturday night, members of the association and the college of cardiology held a hastily called closed-door meeting with Dr. Ridker, who directs the Center for Cardiovascular Disease Prevention at Brigham and Women’s Hospital in Boston. He showed them his data and pointed out the problem. On Sunday, officials from the organizations struggled with how to respond.
Other experts said there has not been a real appreciation of the difficulties with this and other risk calculators. “I don’t think people have a good idea of what needs to be done,” said Dr. Michael Blaha, director of clinical research at the Ciccarone Center for the Prevention of Heart Disease at Johns Hopkins University, who was not associated with forming the new guidelines.
Dr. Blaha said the problem might have stemmed from the fact that the calculator uses as reference points data collected more than a decade ago, when more people smoked and had strokes and heart attacks earlier in life. For example, the guideline makers used data from studies in the 1990s to determine how various risk factors like cholesterol levels and blood pressure led to actual heart attacks and strokes over a decade of observation.
But people have changed in the past few decades, Dr. Blaha said. Among other things, there is no longer such a big gap between women’s risks and those of men at a given age. And people get heart attacks and strokes at older ages.
“The cohorts were from a different era,” Dr. Blaha said.
This week, after they saw the guidelines and the calculator, Dr. Ridker and Dr. Cook evaluated it using three large studies that involved thousands of people and continued for at least a decade. They knew the subjects’ characteristics at the start — their ages, whether they smoked, their cholesterol levels, their blood pressures. Then they asked how many had heart attacks or strokes in the next 10 years and how many would the risk calculator predict.
The answer was that the calculator overpredicted risk by 75 to 150 percent, depending on the population. A man whose risk was 4 percent, for example, might show up as having an 8 percent risk. With a 4 percent risk, he would not warrant treatment — the guidelines that say treatment is advised for those with at least a 7.5 percent risk and that treatment can be considered for those whose risk is 5 percent.
“Miscalibration to this extent should be reconciled and addressed before these new prediction models are widely implemented,” Dr. Ridker and Dr. Cook wrote in The Lancet. “If real, such systematic overestimation of risk will lead to considerable overprescription.”
In a response on Sunday, Dr. Smith of the guidelines committee said the concerns raised by Dr. Cook and Dr. Ridker “merit attention.”
But, he continued, “a lot of people put a lot of thought into how can we identify people who can benefit from therapy.” Further, said Dr. Smith, who is also a professor of medicine at the University of North Carolina and a past president of the American Heart Association, “What we have come forward with represents the best efforts of people who have been working for five years.”
The chairmen of the guidelines panel said they believed the three populations Dr. Ridker and Dr. Cook examined were unusually healthy and so their heart attack and stroke rates might be lower than expected.
Asked to comment on the situation on Sunday, some doctors said they worried that, with many people already leery of statins, the public would lose its trust in the guidelines or the heart associations.
“We’re surrounded by a real disaster in terms of credibility,” said Dr. Peter Libby, the chairman of the department of cardiovascular medicine at Brigham and Women’s Hospital.
What are patients and doctors to do? On Sunday, there seemed to be no firm answers, except that those at the highest risk, like people who have had a heart attack or have diabetes, should take statins.
The guideline developers said they were not totally surprised by the problems with the calculator.
“We recognize a potential for overestimates, especially at the high end of risk,” said Dr. David Goff, the dean of the University of Colorado School of Public Health and the co-chairman of the guidelines’ risk assessment working group.
Last year, not long after it received the assessments from Dr. Ridker and Dr. Cook, the National Heart, Lung and Blood Institute removed itself from the development of the guidelines, saying that was not its mission. The institute handed responsibility to the American Heart Association and the American College of Cardiology.
Dr. Michael Lauer, the director of the division of cardiovascular sciences at the institute, said on Sunday that it had received many reviews and sent them to the other groups, together with the responses of the guidelines’ authors.
Some doctors who tested the calculator with hypothetical patients wondered if they should trust the results.
Dr. Nissen entered the figures for a 60-year-old African-American man with no risk factors — total cholesterol of 150, HDL (the good cholesterol) of 45, systolic blood pressure of 125 — who was not a diabetic or a smoker. He ended up with a 10-year risk of 7.5 percent, meaning he should be taking cholesterol-lowering statins despite being in a seemingly low-risk group.
Dr. Nissen also calculated the figures for a healthy white man, age 60, and also got a risk factor of 7.5 percent.
“Something is terribly wrong,” Dr. Nissen said. Using the calculator’s results, he said, “your average healthy Joe gets treated, virtually every African-American man over 65 gets treated.”
