The Rest of the Story

On CBS MarketWatch, we learn that the stocks of antidepressant drug makers went up after the FDA met to formulate recommendations for the use of their products in children and adolescents.  In this post, I comment on the meaning of the statements made by the FDA, try to explain why some panel members contradicted others, and get into the issue of risk-benefit decisions.  Although the recent FDA meeting only addressed the risk-benefit question in children and adolescents, the same concepts apply to adults. 


From the "full story" that the first story refers to:


Yesterday, I stated that the FDA had clarified the situation. I guess I spoke too soon: now we hear that some panel members say that the medications can help, but one says that they don't work. How is it that different experts looking at the same data could derive seemingly opposite conclusions?

To understand this, it is necessary to understand how the results of antidepressant drug studies are evaluated. During the study, the patients are assessed using numerical rating scales. The score is supposed to correlate with the severity of the depression. An example is the HAM-D, or Hamilton Rating Scale for Depression. The drug company Glaxo kindly posted a copy on the 'net. The HAM-D is used commonly in the USA. European studies tend to use the MADRAS (Montgomery-Asberg Depression Rating Scale). The patients then are divided into two groups. One groups gets active drug; the other groups get placebos.

The scales are administered periodically throughout the study. At the end of the study, a patient is considered to be a "responder" if the final score is less than 50% of the initial score. Invariably, some patients will get better; some will get worse. This will be true no matter what drug is given, or even if the patient is given placebo.

In order to determine if the drug "works," the initial scores are added up, and the arithmetic mean is calculated. Then the mean final score is calculated. This is done separately for the patients given active drug and those given placebo. The a statistical analysis is done. If there is a greater drop in the mean scores for the drug-treated patients than for the placebo-treated patients, AND if the difference is statistically significant, then the drug could be said to be effective.

Note that you might get a statistically significant difference even if the HAM-D scores only go down by an average of one point. For that reason, the mere presence of a statistically significant difference is not enough for doctors to want to prescribe a drug. Therefore, additional analysis of the data is performed. The percentage of patients who are responders is calculated for each group. In order for an antidepressant to be considered truly effective, there has to be a significant difference in the rates of drug response. This is a way of assuring that the effect size is clinically meaningful.

Those readers who are familiar with statistics will see that a great deal of information is lost, if all you look at is the presence or absence of a statistically significant difference in the rate of response between the two groups.

One problem with the statistical analysis of antidepressant study data is that there is always a large placebo response rate. That is, usually you get a response in 30 to 40% of the patients who take placebo.

Now, to illustrate a point, let's assume that a study is done, and that 35% of the patients in the placebo group are responders, and 60% in the drug group are responders. Let us say that an analysis of the data shows that there is a 10% chance that that the difference could be due to random variation. That would not meet the traditional criterion for statistical significance, yet it is clear that, on average, the patients who got the drug did better than those who did not.

One way to interpret the data would be to say that the drug helped about 25% of the patients, based upon the idea that 35% would have gotten better anyway, but the rest of the 60% who got better, got better because of the drug.

The panel members who stated that the drugs can work were looking at it that way.

Another way to interpret the data would be to say that the usual standard for accepting a study is to accept it as a success only if it is statistically significant at the 95% confidence level. Since the studies did not show that, one could conclude that the drugs don't work. Same data; different conclusion.

Of course, if the drug never caused adverse effects, most people probable would accept a 25% chance of response, if that is the best that medical science had to offer. But because the studies show that there is a 2 or 3% chance of the drug causing suicide-related behavior, the situation becomes much more complex. Add to that the fact that untreated depressed patients also have a risk of suicide and suicide-related behaviors, and the risk-benefit decision becomes murky.

Traditionally, the US government has tried to leave these kinds of risk-benefit decisions to the doctors and their patients (or parents, in the case of children.) The government lets people over the age of 18 smoke cigarettes, so long as there is a warning label. It lets people over the age of 16 drive a car, so long as they have passed a test that assures they know the risks they are taking. It does not let people inject cocaine into their veins, and a warning label does not make a difference. Thus, the role of the FDA is to decide which risks should be left to the individuals involved, and which risks it simply will not allow. That is what the recent FDA committee meeting set out to decide.

In this case, they took the middle ground: it is OK to let people take the risk, so long as the warnings are made clear. They also added a lukewarm acknowledgment that the drugs might be effective, at least for some people, to an extent that some reasonable people might decide that the chance of a benefit is worth the risk. The stock market responded to this lukewarm endorsement, and the drug company stocks went up.

Of course, the US government traditionally has allowed people to take risks in the stock market.