“Black Hole” of medical research–Negative Results Don’t get Published – JAMA, WSJ
Sat, 5 Jun 2004
“No one likes bad news. But in science, there is an ethical imperative to publish research, even if the findings aren’t what the researcher had hoped for.”
The New York Times, which tends to report even preliminary, unpublished and invalidated “positive” findings favoring drugs, on its front page, has not seen fit to report significant systemic “bad findings,” such as an Oxford University report published in the current Journal of the American Medical Association. The report provides evidence of systemic manipulation and fraudulent research reports.
Dr An-Wen Chan and an Oxford team examined 100 clinical trials and the published reports of the findings. Their analysis shows that the system of testing and reporting about drugs and surgical procedures has been corrupted by pharmaceutical industry influence. Even those studies that were not directly funded by pharmaceutical companies were found to be biased. That is because the scientists conducting these trials have ongoing financial ties to drug companies. “The most worrying aspect is that over 50 per cent of the outcomes found by the trials weren’t reported – and so can’t be included in the reviews used to assess different treatments.”
The JAMA report notes that 80% of industry sponsored trials conducted at universities such as Yale, Hopkins, and others, are reported as positive, reflecting bias.
“In almost two-thirds of these cases, the results omitted concerns over potential harmful effects. Independent researchers were just as prone to bias as those funded by industry. Crucial information, from the intensity of pain to survival rates, was either downgraded in importance or omitted from the published report.”
The Wall Street Journal also addressed the “Black Hole” of unreported scientific findings: a Finnish analysis of the cardiovascular risk from hormone-replacement therapy found that only 23 of some 200 trials submitted to regulatory authorities were published. Kay Dickersin, an epidemiologist at Brown University, found that only 30% to 50% of all studies, regardless of funding source, are eventually published.
That means that 50% to 70% of clinical trial results are negative – but the public never knows about it, nor that the products tested, may possibly be dangerous.
The medical research community and its self-regulating system is dysfunctional; failing to protect the public from hazardous drugs and medical procedures, and tainting the credibility of the scientific literature with fraudulently manipulated findings. To fill that “black hole” in medicine, New York State Attorney General Eliot Spitzer has filed a suit against GlaxoSmithKline, charging the company with fraudulently concealing the health hazards of its antidepressant, Paxil (paroxetine). As the reports in JAMA and the WSJ demonstrate, that charge is likely to be applicable to the drug industry at large.
Contact: Vera Hassner Sharav
Researchers ignore ‘inconvenient’ drug trial results
By Robert Matthews, Science Correspondent
The Sunday Telegraph (UK national newspaper)
Scientists are routinely cherry-picking the results of clinical trials so that they can present the findings that they want, a study by academics at Oxford University shows.
The research, which assessed the published results of more than 100 scientific trials, also found that inconvenient findings were often not disclosed to the public. In several cases, the stated purpose of the trial was altered as it progressed so that acceptable findings, rather than inconvenient results, could be published.
The manipulation, which contravenes official guidelines on reporting medical research, was uncovered by academics at Oxford University, led by Dr An-Wen Chan, a researcher on clinical medicine.
Dr Chan warned that the findings called into question the National Health Service’s evidence-based approach to developing medicine, in which clinical trials are used to determine whether to introduce new treatments.
“The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols,” said the team. “Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention.”
Suspicion about the reliability of published medical research, which has been increasing for some time, has been prompted by concern over the influence of drugs-company funding.
A recent study at the Yale School of Medicine showed that 80 per cent of clinical trials backed by drug manufacturers reported positive findings – compared to 50 per cent of those carried out by independent academics.
Other studies have shown evidence of a bias against unclear trial results being published in academic journals, and of positive results being repeatedly published – giving the impression that a drug is far more effective than it really is.
The Oxford team’s findings, which are published in the latest edition of the Journal of the American Medical Association, are based on an assessment of the original paperwork from more than 100 trials of medical techniques ranging from drug to surgical methods.
It compared the supposed aims of the trials with what was finally published and found that in half of the studies, results that would have given a more accurate picture of the effectiveness of the treatment being studied were not fully reported.
In almost two-thirds of these cases, the results omitted concerns over potential harmful effects. Independent researchers were just as prone to bias as those funded by industry.
Crucial information, from the intensity of pain to survival rates, was either downgraded in importance or omitted from the published report.
In more than half of the trials examined, discrepancies were found between the original aims of the study and those finally reported. Such changes are in direct contravention of official guidelines on trials. Despite this, not one trial report made clear that the original aims of the research had been altered.
When contacted by the Oxford team, almost 90 per cent of the research teams denied that they had failed to report everything, despite evidence to the contrary. Dr Doug Altman, a professor of statistics in medicine at the Institute of Health Sciences, Oxford, and a member of the research team, said: “All trials should be published honestly and transparently, and this study shows neither is happening.
The most worrying aspect is that over 50 per cent of the outcomes found by the trials weren’t reported – and so can’t be included in the reviews used to assess different treatments. This has serious implications for the reliability of the recommendations made to the National Health Service.”
Dr Altman said that there was no evidence that the cherry-picking of results was because of fraudulent intent, and claimed instead that it was because of pressure from journals to present findings on medical advances.
“There is a lot of pressure from medical journals to publish positive findings and to keep the length of papers down, which can lead to negative results being omitted,” he said.
Dr Altman added, however, that this did not explain why the original design aims of trials were altered so often. He called for the original design aims and findings of all clinical trials to be made publicly available, to prevent changes going undetected.
Other leading authorities on medical research expressed dismay at the findings of the study. Sir Iain Chalmers, the founder of the Cochrane Collaboration, which publishes reviews of clinical evidence for doctors worldwide said: “In the days when no one paid any attention to scientific evidence, this would not have been a worry, but now people are investing huge amounts in research, these skeletons are coming out of the cupboard.”
The Association of the British Pharmaceutical Industry said that it agreed that researchers had a duty to reveal any changes made to the original trial design, and supported full disclosure – but only when the trials had been completed.
Richard Ley, a spokesman for the association, said: “There can be commercially confidential information in trial protocols, and with a new drug costing £500 million and 10 years to produce, that is an important issue.”
A spokesman for the National Institute for Clinical Excellence, which provides the NHS with recommendations on best practice, said that it was already taking steps to avoid the problems of biased reporting of results. “Our appraisal committee considers all of the evidence – including an independent systematic review of the manufacturer’s submission – and other literature, which is designed to take into account possible bias in the reporting of trial outcomes.”
23 April 2004: Drug firms ‘hid risks to children’
The WALL STREET JOURNAL
Medical Research Has ‘Black Hole’
Negative Results Often Fail To Get Published in Journals;
Some Blame Drug Industry
By SCOTT HENSLEY and LEILA ABBOUD
June 4, 2004; Page B3
The controversy over unpublished medical studies showing negative results of the antidepressant Paxil is exposing a black hole in medical research where as many as half of studies, especially ones with negative results, remain unpublished and languish in dusty file cabinets.
Despite compelling ethical and scientific reasons for the publication of negative results from clinical studies of drugs, the information frequently doesn’t become public, researchers say.
“It’s been pretty well established that publication bias is associated with industry funding,” says Kay Dickersin, an epidemiologist at Brown University. And those industry-sponsored papers that appear in print tend to favor the sponsor. But the problem is broad: Only 30% to 50% of all studies, regardless of funding source, are eventually published, she says.
She cited a paper from last year that found only one of 37 clinical trials of nonsteroidal anti-inflammatory drugs reported to FDA had been published. Likewise a Finnish analysis of the cardiovascular risk from hormone-replacement therapy found that only 23 of some 200 trials submitted to regulatory authorities were publicly available.
No one likes bad news. But in science, there is an ethical imperative to publish research, even if the findings aren’t what the researcher had hoped for.
The issue came to a head this week when New York State Attorney General Eliot Spitzer sued GlaxoSmithKline PLC.
In a novel tack, he alleged that the British drug maker committed fraud by not publishing studies suggesting its drug Paxil was ineffective in treating depression in children and adolescents, and might lead to an increased risk of suicide. Glaxo denies withholding the information and argues that it is impractical to publish all studies.
Drug-industry funding has been blamed for a bias toward publication of favorable results. The “overwhelming majority” of drug researchers relies on the industry for direct or indirect financial support, says Muhammad Mamdani, a clinical pharmacist at the Institute for Clinical Evaluative Sciences in Toronto. Last week Dr. Mamdani published a study in the Lancet suggesting that Merck & Co.’s painkiller Vioxx carries a higher risk for congestive heart failure than alternative pain relievers. The study was funded by the Canadian government, not industry. Merck cautioned that Dr. Mamdani’s retrospective study design precluded the establishment of cause and effect between Vioxx and an increase in congestive heart failure.
The paper took more than two years to make it into print. Dr. Mamdani says it was rejected by the prestigious Journal of the American Medical Association because perceived methodological limitations might compromise findings that could influence common clinical practice. As a matter of policy JAMA editors don’t comment on rejected manuscripts.
In their defense, journal editors say the shortcomings in the way studies are conducted is the most common reason for papers to be rejected. “The decision to publish is based on the quality of the study,” declares Phil Fontanarosa, executive editor of JAMA. He says JAMA and other journals have taken action against the recognized bias toward publication of positive results. “Over the past five years, I think that if you look through major journals you do see studies with negative results appearing more and more frequently.”
Jeffrey Drazen, editor in chief of the New England Journal of Medicine, says that many drug trials are designed to show that one medicine isn’t any worse than another. While this approach reduces the number of patients and money needed to test a medicine, it often undercuts the ability of investigators to draw valid conclusions about side effects or other questions.
Some physicians acknowledge the ethical imperatives and seek contractual freedom to publish results of studies funded by drug makers. “I won’t do a clinical trial in which the contract doesn’t give me the right to publish,” said Steven E. Nissen, a cardiologist at the Cleveland Clinic who has led high-profile studies of cholesterol-lowering and hypertension drugs.
In 2002 the Pharmaceutical Research and Manufacturers of America, a trade group, agreed on a set of voluntary rules governing the communication of clinical-trial results. The members, all drug makers, pledged to release “meaningful study results, regardless of the outcome” and said they “will not suppress or veto publications.”
Abbott Laboratories, based in North Chicago, Illinois, says it adheres to the PhRMA principles and encourages the scientists it works with to publish. Spokeswomen at Pfizer Inc. and Merck say their publication policies rely on predetermined study criteria. Both pointed to publication of negative results of company drugs from studies they sponsored as evidence they aren’t tying the hands of researchers.
Eli Lilly & Co. has a set of internal rules on disclosure of clinical-trial results, which were approved by the board and in which employees are trained. “We will publish all medical research, good or bad, that has relevance to patients, prescribers, and payers,” says Alan Breier, Lilly’s chief medical officer.
Glaxo CEO: Litigation Is ‘Getting Out of Control’
Copyright © 2004 Dow Jones & Company, Inc. All Rights Reserved
FAIR USE NOTICE: This may contain copyrighted (© ) material the use of which has not always been specifically authorized by the copyright owner. Such material is made available for educational purposes, to advance understanding of human rights, democracy, scientific, moral, ethical, and social justice issues, etc. It is believed that this constitutes a ‘fair use’ of any such copyrighted material as provided for in Title 17 U.S.C. section 107 of the US Copyright Law. This material is distributed without profit.