Blog

Reviewing Peer Review

A recent editorial in Science discusses the value of peer review, along with the increasing stresses on it, due to the growth of scientific publishing.

The editorial states that "peer review is fundamental to scientific progress, and the achievements of science in the last century are an endorsement of its value ... Despite its successes, peer review attracts its share of criticism. Reviewers can exhibit bias or only support expected, pedestrian results ... Reviews improve most papers, some dramatically so ... But peer review is under increasing stress, in ways that are perhaps not fully appreciated".

The article emphasizes that "the way scientists and research institutions are evaluated needs revision" and that "inappropriately high value is placed on publication in certain journals. Increased competition for the limited slots in these preferred journals exacerbates the natural aggravations of peer review experienced by authors".

The article goes on to state that "Efforts like the Faculty of 1000, where experts scan a large set of biology journals and select the best contributions wherever published, can be very helpful. Such efforts can reduce the pressures that many group leaders feel from young scientists, who often place undue emphasis on publishing in a few high-profile journals--where the criteria used for evaluation may not match their research, no matter how valuable the contribution".

At F1000, we agree -- articles from smaller, less widely read journals can be just important. The F1000 Factor provides an alternative measure to the impact factor and is based on the opinion of eminent members of the medical community, irrespective of what journal the article comes from.

Reference:
Alberts et al., Science 2008, 321:15, DOI:10.1126/science.1162115

Save to del.icio.us | Email this

Posted by Adie Chan on Tuesday Jul 22, 2008, Category: Information Overload | Permalink

The misuse of citation statistics

"While it is incorrect to say that the impact factor gives no information about individual papers in a journal, the information is surprisingly vague and can be dramatically misleading" says a recent report published by the Joint Committee on Quantitative Assessment of Research {1}.

We loved this highly informative report detailing how statistics such as the impact factor and h-index are misused, not well understood and not well studied. Robert Adler and his colleagues aim to point out the limitations of using citation statistics, as well as how to make better use of them.

This superb article also calls into question the accuracy, objectivity and simplicity of using such statistics as the sole measure of research quality:

"We do not dismiss citation statistics as a tool for assessing the quality of research-citation data and statistics can provide some valuable information ... But citation data provide only a limited and incomplete view of research quality, and the statistics derived from citation data are sometimes poorly understood and misused. Research is too important to measure its value with only a single coarse tool."

We agree - that's why the F1000 Factor is so useful. It provides an alternative measure to the impact factor and is based on the opinion of eminent members of the medical community, irrespective of what journal the article comes from.

The Adler et al paper is receiving a lot of attention and is yet more evidence adding to the ever-growing base that our reliance on, and trust of, such statistics may need to be re-assessed.

Read more about the F1000 Factor here.
 

References:

1. http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf 

Save to del.icio.us | Email this

Posted by Sean Porazinski on Wednesday Jul 02, 2008, Category: Impact factor versus... | Permalink

« Previous month (Jun 2008) | Main | Next month (Aug 2008) »