How many calories should I eat?
“Investigators want to go overboard to make their studies look positive,” said Ian Tannock, senior author of the study that appeared in the Annals of Oncology.
In two-thirds of the 164 studies that Tannock and his colleagues scrutinized, that meant not listing serious side effects, whether of chemotherapy, radiation or surgery, in the paper’s abstract. Such abstracts summarize the findings, and run a few hundred words. That’s important, said Tannock, at Princess Margaret Hospital in Toronto, because doctors have little time to read. “Most of us are so damn busy, we only read the abstract and skim the tables and figures,” he added.
In fact, a fifth of the studies didn’t include serious side effects in results tables, and about a third failed to mention them in either the abstract or the discussion section.
Most surprising, said Tannock, was that in a third of the studies, if the treatment didn’t work as well as one might hope, researchers moved the goalposts, reporting results that weren’t what the study was originally designed to test. Often, those so-called “secondary endpoints” may be less important and meaningful. There is a difference, for example, between showing people lived longer overall, or simply lived longer without their cancers coming back. Cancer research is not the only area where some researchers are concerned.
In November, a group of cardiology journal editors urged authors to watch their language when describing their results, and a month earlier two pediatrics researchers warned of “spin and boasting” in their field’s journals.
Tannock, whose team analyzed reports of late-stage trials of the kind used by the U.S. Food and Drug Administration to decide whether to approve drugs, said researchers “gain more influence with positive studies.”
There are also various pressures on researchers to make their results look better than they really are, he added, including from the drug companies that often sponsor trials. But in his study, who paid for a study didn’t have any relationship with how the results were presented. Scientists may also spin their results to increase their chances of publishing in the top journals surveyed by the study. Such marquee publications can improve the chances for tenure, promotion and grants.
One of the journals whose studies Tannock and his colleagues looked at, the New England Journal of Medicine, declined to comment, saying they don’t typically comment on other studies.