It is generally agreed that health care should be based on scientific evidence. The controlled randomized trial is a cornerstone of evidence based medicine. It is therefore deeply worrying that the reporting of randomized trials is frequently biased. Boutron and co-authors presented data earlier this year in JAMA on the occurrence and strategies of spin in prospective randomized trials (1). Spin was defined as specific reporting that could distort the interpretation of results and mislead readers. Papers with statistically non significant primary end-points were selected. In 68% of the reviewed papers spin was identified in the abstract and in 61% in the main text. A common strategy of spin was to downplay major end-points and instead focus on selected, secondary end-points. Another was to claim equivalence or comparable effectiveness to an active comparator when p > 0.05 (but never when a placebo control had been used).

The association between study quality, competing interests, authors’ conclusions and the risk of spin has been investigated in several studies (2–6). A consistent finding has been that industrial funding is more often associated with unwarranted positive conclusions regarding the test treatment and with a lower concordance between results and conclusions than non-profit funding. The association, if any, between funding and publication in high impact factor journals has also been studied (6). Curiously, journals with a high impact factor published more papers with for profit funding, irrespective of quality or size of studies. Hence, editorial bias is another matter of concern (7).

It may be of interest to review two cardiovascular papers against this background (8, 9). The papers have much in common. Both have the same first author, same topic (coronary artery bypass surgery vs. percutanous coronary intervention), both have industrial funding, both are prospective randomized trials with the same major end-point, by and large the results are the same, and both were published in a journal (the same) with high impact factor.

The first paper was published in 2001(8). Patients undergoing percutanous coronary stenting, PCI, (n = 600) or bypass surgery, CABG, (n = 605) were followed up for one year. Two thirds of the patients had 2-vessel disease, the rest had 3-vessel disease, the mean ejection fraction was 60% and <20% had diabetes. So the patients had a relatively small to moderate risk of cardiovascular death.

The major end-point, used for power calculations, was the combined end-point freedom from death, stroke, transient ischemic attacks, myocardial infarction or repeated revascularization. A significant difference to the advantage of bypass surgery was found (87.8 vs. 73.8 %, p < 0.001). Yet, no mention of this was made in the conclusion sections of neither the abstract- nor the main text. Instead, it was concluded that “…coronary stenting for multivessel disease is less expensive than bypass surgery and offers the same protection against death, stroke and myocardial infarction.”

This is, to put it mildly, a dubious statement. The statistical power to detect differences in mortality was calculated (by us) to 5%, i.e. if a difference had existed (in either direction) it carried a 95% risk of being missed. To obtain a power of 80% it would require more than 3000 patients (instead of 600) in each arm to detect a difference of 50%.

Furthermore, as the patients had a low to moderate risk of cardiovascular death, CABG is not expected to have a prognostic effect in the majority of these patients. So, even if an adequately powered study had been performed, and if it had shown no significant difference, this would not indicate that the two treatments were equally protective (rather that they both were inactive).

Another objective of revascularization is to relieve angina. After twelve months 89.5% of CABG patients were free of angina compared to 78.9% in the PCI patients (p < 0.001). The finding was not mentioned in the conclusion sections of abstract or main text.

Clearly, the study fulfills the criteria of spin. In the study by Boutron, the level of spin was estimated. The current study could have been classified as “high”. It is remarkable that the distorted interpretation of results passed unnoticed by the peer review process.

The second study was published in 2009 (9). In the randomized trial 903 and 897 patients underwent PCI and CABG respectively. The patients were of a higher risk category than in the first study (only 3-vessel and left main lesions). The same major end-point was used (albeit as percentage of events instead of freedom from events) for power calculations. Again, a significant difference to the advantage of CABG was found (rate of events: 17.8% vs. 12.4%, p = 0.002, for PCI and CABG resp.). In contrast to the first study, the major finding was reported, impeccably, in the conclusions sections of both abstract and main text. Sad to say, this does not imply that the paper is free from spin. In the results section it is claimed that the rates of death and myocardial infarction were similar between the two groups. The statement is not justified by the results.

If anything, they hint at CABG superiority. All cause relative risk of death within a year (PCI/CABG) given in a table was 1.24 (0.78–1.98) p = 0.37 (if <1 risk is lower with PCI, if >1 lower with CABG). The statistical power for this endpoint was 15%, i.e. there is an 85% risk of missing a true difference, if present. As the confidence interval indicates, an almost doubled risk with PCI would be compatible with the results. Furthermore, the figures for cardiac death were 1.75(0.99–3.08), p = 0.05 and for myocardial infarction 1.46(0.92–2.33), p = 0.11. So, again the claim that “not significant” indicates similarity is not backed by data and represents spin.

The 3-year follow data from the second study are being released. Hopefully, the longer follow-up with an increased number of events will provide more conclusive evidence on these important, but unresolved issues. The data are welcome, in particular if presented free from spin.

There are differences between the two papers. First, they were published 8 years apart. Second, spin is less severe in the second paper. It is impossible to say if this is by chance, or if it reflects a change in attitude, either among authors, or editors, or both. If it is a change, possibly inspired by the on-going debate, this would be encouraging. But as long as the phenomenon is not rooted out, it seriously threatens the implementation of a health care which is based on scientific evidence.

As this review makes obvious, the peer-review system of scientific journals has failed as gate-keeper. The load of responsibility will lie more heavily on editors and the peer-review system. However, before the actual implementation of evidence based medicine into clinical practice, at least two hurdles remain. First, the making of guide-lines is open to biased interpretation of literature. It must be secured that the reviewing staff has no conflicting interests whatsoever, economical or professional.

Finally, the compliance to guideline recommendations may be the most difficult obstacle, as complained by late Lars Werkö, once head of the Swedish governmental agency for health care assessment. Difficult or not, all aspects have to be addressed successfully for the sake of patient safety and efficient use of limited resources.