Wouldst that New York Times science reporters would learn how to critically
report about science and medicine as does Dr. Lawrence Altman.
"For Science’s Gatekeepers, a Credibilty Gap," is an unflinching, sobering
appraisal of science journal editors’ performance–they don’t make the
grade.
"The flurry of episodes has led many people to ask why authors, editors and independent
expert reviewers all failed to detect the problems before publication."
This persuasively argued critique goes beyond the headlines and the
self-serving rationalizations of the fraternity of journal editors who demur
when asked why they don’t establish meaningful review mechanisms to prevent
fraudulent articles from passing peer review. Altman lifts the curtain to
reveal that the scientific fraternity has vested interests in maintaining
the illusion that "passing peer review is the scientific equivalent of the
Good Housekeeping seal of approval."
Instead, Altman lets lay readers in on the intricate interwoven conflicting
interests–financial and political–that the system protects.
"They include economic pressures for journals to avoid investigating
suspected errors; the desire to avoid displeasing the authors and the
experts who review manuscripts; and the fear that angry scientists will
withhold the manuscripts that are the lifeline of the journals, putting them
out of business.By promoting the sanctity of peer review and using it to
justify a number of their actions in recent years, journals have added to
their enormous power. The release of news about scientific and medical
findings is among the most tightly managed in country. Journals control when
the public learns about findings from taxpayer-supported research by setting
dates when the research can be published. They also impose severe
restrictions on what authors can say publicly, even before they submit a
manuscript, and they have penalized authors for infractions by refusing to
publish their papers."
Scientists, academic institutions, the pharmaceutical industry, and the
journals are all invested in "keeping up appearances" but not the integrity
of science. Journal editors recoil at suggestions that the reviewing process for
scientific reports itself be tested in accordance with scientific methods.
They reject conducting random audits “like those used to monitor quality
control in medicine,” citing costs and “the potential for creating
distrust.”
Such excuses betray an underlying elitism among this elitist fraternity that
has still not woken up to the fact that the products of its endeavors are
tainted—just as the blockbuster drugs that have enriched all those involved,
are defective.
Altman gently but surely disabuses lay persons of their illusion:
"A widespread belief among nonscientists is that journal editors and their
reviewers check authors’ research firsthand and even repeat the research. In
fact, journal editors do not routinely examine authors’ scientific
notebooks. Instead, they rely on peer reviewers’ criticisms, which are based
on the information submitted by the authors."
Whether intentionally or not, Altman underscores the inferior performance of
U.S. journal editors compared to British journal, by giving the last word to
Dr. Richard Smith, former (longtime) editor of the BMJ (British Medical
Journal) and Dr. Richard Horton, editor of The Lancet.
"Journals have devolved into information-laundering operations for the
pharmaceutical industry…journals rely on revenues from industry
advertisements. But because journals also profit handsomely by selling drug
companies reprints of articles reporting findings from large clinical trials
involving their products, editors may ‘face a frighteningly stark conflict
of interest’ in deciding whether to publish such a study."
Contact: Vera Hassner Sharav
veracare@ahrp.org
http://www.nytimes.com/2006/05/02/health/02docs.html?_r=1&oref=slogin&pagewanted=all
The New York Times
The Doctor’s World
For Science’s Gatekeepers, a Credibility Gap
By LAWRENCE K. ALTMAN, M.D.
Published: May 2, 2006
Recent disclosures of fraudulent or flawed studies in medical and scientific
journals have called into question as never before the merits of their
peer-review system.
The system is based on journals inviting independent experts to critique
submitted manuscripts. The stated aim is to weed out sloppy and bad
research, ensuring the integrity of what it has published.
Because findings published in peer-reviewed journals affect patient care,
public policy and the authors’ academic promotions, journal editors contend
that new scientific information should be published in a peer-reviewed
journal before it is presented to doctors and the public.
That message, however, has created a widespread misimpression that passing
peer review is the scientific equivalent of the Good Housekeeping seal of
approval.
Virtually every major scientific and medical journal has been humbled
recently by publishing findings that are later discredited. The flurry of
episodes has led many people to ask why authors, editors and independent
expert reviewers all failed to detect the problems before publication.
The publication process is complex. Many factors can allow error, even
fraud, to slip through. They include economic pressures for journals to
avoid investigating suspected errors; the desire to avoid displeasing the
authors and the experts who review manuscripts; and the fear that angry
scientists will withhold the manuscripts that are the lifeline of the
journals, putting them out of business.By promoting the sanctity of peer
review and using it to justify a number of their actions in recent years,
journals have added to their enormous power.
The release of news about scientific and medical findings is among the most
tightly managed in country. Journals control when the public learns about
findings from taxpayer-supported research by setting dates when the research
can be published. They also impose severe restrictions on what authors can
say publicly, even before they submit a manuscript, and they have penalized
authors for infractions by refusing to publish their papers. Exceptions are
made for scientific meetings and health emergencies.
But many authors have still withheld information for fear that journals
would pull their papers for an infraction. Increasingly, journals and
authors’ institutions also send out news releases ahead of time about a
peer-reviewed discovery so that reports from news organizations coincide
with a journal’s date of issue.
A barrage of news reports can follow. But often the news release is sent
without the full paper, so reports may be based only on the spin created by
a journal or an institution.
Journal editors say publicity about corrections and retractions distorts and
erodes confidence in science, which is an honorable business. Editors also
say they are gatekeepers, not detectives, and that even though peer review
is not intended to detect fraud, it catches flawed research and improves the
quality of the thousands of published papers.
However, even the system’s most ardent supporters acknowledge that peer
review does not eliminate mediocre and inferior papers and has never passed
the very test for which it is used. Studies have found that journals publish
findings based on sloppy statistics. If peer review were a drug, it would
never be marketed, say critics, including journal editors.
None of the recent flawed studies have been as humiliating as an article in
1972 in the journal Pediatrics that labeled sudden infant death syndrome a
hereditary disorder, when, in the case examined, the real cause was murder.
Twenty-three years later, the mother was convicted of smothering her five
children. Scientific naïveté surely contributed to the false conclusion, but
a forensic pathologist was not one of the reviewers. The faulty research in
part prompted the National Institutes of Health to spend millions of dollars
on a wrong line of research.
Fraud, flawed articles and corrections have haunted general interest news
organizations. But such problems are far more embarrassing for scientific
journals because of their claims for the superiority of their system of
editing.
A widespread belief among nonscientists is that journal editors and their
reviewers check authors’ research firsthand and even repeat the research. In
fact, journal editors do not routinely examine authors’ scientific
notebooks. Instead, they rely on peer reviewers’ criticisms, which are based
on the information submitted by the authors.
While editors and reviewers may ask authors for more information, journals
and their invited experts examine raw data only under the most unusual
circumstances.
In that respect, journal editors are like newspaper editors, who check the
content of reporters’ copy for facts and internal inconsistencies but
generally not their notes. Still, journal editors have refused to call peer
review what many others say it is — a form of vetting or technical editing.
In spot checks, many scientists and nonscientists said they believed that
editors decided what to publish by counting reviewers’ votes. But journal
editors say that they are not tally clerks and that decisions to publish are
theirs, not the reviewers’.
Editors say they have accepted a number of papers that reviewers have
harshly criticized as unworthy of publication and have rejected many that
received high plaudits.
Many nonscientists perceive reviewers to be impartial. But the reviewers,
called independent experts, in fact are often competitors of the authors of
the papers they scrutinize, raising potential conflicts of interest.
Except when gaffes are publicized, there is little scrutiny of the quality
of what journals publish.
Journals have rejected calls to make the process scientific by conducting
random audits like those used to monitor quality control in medicine. The
costs and the potential for creating distrust are the most commonly cited
reasons for not auditing.
In defending themselves, journal editors often shift blame to the authors
and excuse themselves and their peer reviewers.
Journals seldom investigate frauds that they have published, contending that
they are not investigative bodies and that they could not afford the costs.
Instead, the journals say that the investigations are up to the accused
authors’ employers and agencies that financed the research.
Editors also insist that science corrects its errors. But corrections often
require whistle-blowers or prodding by lawyers. Editors at The New England
Journal of Medicine said they would not have learned about a problem that
led them to publish two letters of concern about omission of data concerning
the arthritis drug Vioxx unless lawyers for the drug’s manufacturer, Merck,
had asked them questions in depositions. Fraud has also slipped through in
part because editors have long been loath to question the authors.
"A request from an editor for primary data to support the honesty of an
author’s findings in a manuscript under review would probably poison the air
and make civil discourse between authors and editors even more difficult
than it is now," Dr. Arnold S. Relman wrote in 1983. At the time, he was
editor of The New England Journal of Medicine, and it had published a
fraudulent paper.
Fraud is a substantial problem, and the attitude toward it has changed
little over the years, other editors say. Some journals fail to retract
known cases of fraud for fear of lawsuits.
Journals have no widely accepted way to retract papers, said Donald Kennedy,
editor in chief of Science, after the it retracted two papers by the South
Korean researcher Dr. Hwang Woo Suk , who fabricated evidence that he had
cloned human cells.
In the April 18 issue of Annals of Internal Medicine, its editor, Dr. Harold
C. Sox, wrote about lessons learned after the journal retracted an article
on menopause by Dr. Eric Poehlman of the University of Vermont.
When an author is found to have fabricated data in one paper, scientists
rarely examine all of that author’s publications, so the scientific
literature may be more polluted than believed, Dr. Sox said.
Dr. Sox and other scientists have documented that invalid work is not
effectively purged from the scientific literature because the authors of new
papers continue to cite retracted ones.
When journals try to retract discredited papers, Dr. Sox said, the process
is slow, and the system used to inform readers faulty. Authors often use
euphemisms instead of the words "fabrication" or "research misconduct," and
finding published retractions can be costly because some affected journals
charge readers a fee to visit their Web sites to learn about them, Dr. Sox
said.
Despite its flaws, scientists favor the system in part because they need to
publish or perish. The institutions where the scientists work and the
private and government agencies that pay for their grants seek publicity in
their eagerness to show financial backers results for their efforts.
The public and many scientists tend to overlook the journals’ economic
benefits that stem from linking their embargo policies to peer review. Some
journals are owned by private for-profit companies, while others are owned
by professional societies that rely on income from the journals. The costs
of running journals are low because authors and reviewers are generally not
paid.
A few journals that not long ago measured profits in the tens of thousands
of dollars a year now make millions, according to at least three editors who
agreed to discuss finances only if granted anonymity, because they were not
authorized to speak about finances.
Any influential system that profits from taxpayer-financed research should
be held publicly accountable for how the revenues are spent. Journals
generally decline to disclose such data.
Although editors of some journals say they demand statements from their
editing staff members that they have no financial conflicts of interest,
there is no way to be sure. At least one editor of a leading American
journal had to resign because of conflicts of interest with industry.
Journals have devolved into information-laundering operations for the
pharmaceutical industry, say Dr. Richard Smith, the former editor of BMJ,
the British medical journal, and Dr. Richard Horton, the editor of The
Lancet, also based in Britain.
The journals rely on revenues from industry advertisements. But because
journals also profit handsomely by selling drug companies reprints of
articles reporting findings from large clinical trials involving their
products, editors may "face a frighteningly stark conflict of interest" in
deciding whether to publish such a study, Dr. Smith said.
FAIR USE NOTICE: This may contain copyrighted (© ) material the use of which
has not always been specifically authorized by the copyright owner. Such
material is made available for educational purposes, to advance
understanding of human rights, democracy, scientific, moral, ethical, and
social justice issues, etc. It is believed that this constitutes a ‘fair
use’ of any such copyrighted material as provided for in Title 17 U.S.C.
section 107 of the US Copyright Law. This material is distributed without
profit.