Diet Modification Trials: Notes on Study Design

The other day, my internet meanderings brought me back to a review of fat modification trials conducted by the Cochrane collaboration. This is a not-for-profit group known for its rigorous meta-analyses.

They selected 27 studies that reduced saturated fat or total fat (in some cases along with increased PUFA), and fit several inclusion criteria. The results:

There was no significant effect on total mortality (rate ratio 0.98, 95% CI 0.86 to 1.12), a trend towards protection form cardiovascular mortality (rate ratio 0.91, 95% CI 0.77 to 1.07), and significant protection from cardiovascular events (rate ratio 0.84, 95% CI 0.72 to 0.99). The latter became non-significant on sensitivity analysis.

Trials where participants were involved for more than 2 years showed significant reductions in the rate of cardiovascular events and a suggestion of protection from total mortality ["suggestion" = not statistically significant]. The degree of protection from cardiovascular events appeared similar in high and low risk groups, but was statistically significant only in the former.

"Sensitivity analysis" is a statistical method that allows investigators to take into account the limitations of their model, in this case, the way in which they aggregated the studies' data. Another way of putting their findings is that they found no significant effect of fat modification on mortality or cardiovascular mortality, and they found a reduction in non-fatal "cardiovascular events" (more on this phrase later) only in a subset of the data.

I'll be the first to admit the meta-analysis isn't perfect. They cast too wide a net, not allowing them to distinguish the effect of reducing total fat from the effect of reducing saturated fat. They lumped both together, which from a practical standpoint isn't actually a problem because both sets of studies show essentially the same thing: zilch. But it's still not the best way to conduct a meta-analysis. They also omitted the Sydney Diet-Heart study for mysterious reasons, which was a five year randomized trial that found an increase in mortality in volunteers substituting vegetable oils for animal fat. Then there's the conclusion, which boggles the mind:

Lifestyle advice to all those at high risk of cardiovascular disease (especially where statins are unavailable or rationed), and to lower risk population groups, should continue to include permanent reduction of dietary saturated fat and partial replacement by unsaturates.

Are these the same people who wrote the results section? I don't understand how they arrived at that conclusion from their own results.

In any case, this brings me to my main point. There are two types of outcomes you can measure in these trials: "hard endpoints" and "soft endpoints". Hard endpoints are not subjective. The hardest endpoint is death. Either you're dead or you aren't; there's no room for interpretation there. A bit less hard is death from a particular cause, such as heart attack. In that case, you're definitely dead, but the physician has to guess what you died of. That involves some judgment on the part of the physician and can introduce bias, depending on the study design. The softest endpoints are non-fatal events like angina, bypasses and stents. These depend on the judgment of both the physician and the patient, and are the most susceptible to bias.

The gold standard for controlled trials is a design known as "double-blind", where neither the participant nor the physician knows which group the participant is in. This design eliminates bias from both the participant and the physician side, allowing correction for the placebo effect and subtle bias in diagnosis. This is easy to do for drug trials, where placebo pills look just like the drug. But it's more difficult to pull off in a diet trial, where the patient knows what foods he's eating. Still, it can be done by giving participants similar-looking margarines containing either saturated or polyunsaturated fats, or sometimes by controlling diets in an institutionalized setting.

There have been three double-blind trials comparing the incidence of heart attack and/or mortality in volunteers eating either saturated animal fat or polyunsaturated vegetable fat: the 1968 National Diet-Heart trial (2 years), the 1969 Los Angeles Veterans' Administration trial (8 years), and the 1989 Minnesota Coronary Survey trial (4.5 years). The two studies that reported total mortality found no significant difference between groups. Two out of three found no difference in heart attack deaths. Of the two that reported on non-fatal cardiovascular endpoints, one found a significant difference. The V.A. trial was the only one to find a significant difference in heart attack deaths (18% decrease) and non-fatal events. There were significantly more heavy smokers in the animal fat arm of the V.A. trial, which was an unfortunate consequence of the participant randomization process. So that result is difficult to interpret.

The three double-blind diet trials, with the least potential for bias, really give no support to the idea that saturated/animal fat contributes to cardiovascular disease. As the participants were already eating a diet high in omega-6 to begin with, there is also no detectable effect of increasing omega-6 on cardiovascular health.

Many of the trials of this nature have been "single-blinded", where the participant knows which group he's in, but the physician doesn't. In this case, all endpoints involving death will be unbiased, because the physician deciding the diagnosis is not influenced by knowing what group the patient is in. However, soft, non-fatal events such as angina are still highly susceptible to the placebo effect. This is because they depend on the subjective judgment of the patient, who knows which group he's in.

I think it's interesting to note that very few dietary fat modification trials have found reductions in total mortality, which is the hardest endpoint and the least susceptible to bias. This is reflected in the Cochrane collaboration's findings. However, a number of the non-blinded and single-blinded studies have found differences in non-fatal cardiovascular events, sometimes creating absurd results. For example, in the 1966 Anti-Coronary Club trial, participants in the vegetable oil group had a significant reduction in non-fatal cardiovascular events, but a massive increase in cardiovascular deaths and total mortality. The former result could result from a placebo effect, due to the non-blinded nature of the trial.

The fact that the Cochrane review found statistically significant benefits of fat modification in soft, non-fatal endpoints (for a portion of the data set), but not endpoints involving death, suggests to me that what we're seeing may actually be a placebo effect resulting from the fact that patients were not blinded in the majority of these trials.

The only "fat modification" intervention that consistently reduces total and cardiovascular mortality is omega-3 fat supplementation, ideally in combination with omega-6 restriction. This is supported by the results of the DART trial, the Lyon Diet-Heart trial, the ISIS trial and the the GISSI-prevenzione trial.