Ed Witten's rating is 110, Marvin Cohen's, 94 and Stephen W. Hawking is characterised by 62. These are the results if the h-index is applied to the publications of some of the world's most renowned physicists. Jorge Hirsch, himself a physicist at the University of California in San Diego, did just this when he proposed this index as a new measure for the scientific performance of researchers this summer (1)(2).

110 means that Ed Witten has 110 publications to his name, each of which has been cited 110 times. Or to put it more abstractly, a scientist has an index h, if h of each of his or her total number of publications (N) has been cited at least h times and the other (N-h) publications have had fewer than h citations. The index thus collates the number of publications with the frequency of pertinent citations.

Robust against outliers

It is in this collation that Lutz Bornmann, postdoctoral researcher at the ETH Professorship for Social Psychology and Research on Higher Education(3), sees this as the great advantage of this new ranking method because it is more robust against outliers. A frequently cited, single publication is thus not accorded the same weight, anymore and neither is a body of more sparsely cited publications. Moreover, according to Bornmann, the h-index is very easy to calculate.

Bornmann has carried out a first survey of the measure outside physics. He applied the h-index to a group of more than 400 biomedical researchers, all holding a PhD-degree, who had applied for research fellowships from the Boehringer Ingelheim Fonds (BIF) in the 1990s (4)(5). An analysis of the publications (around 1,600) before application for a research fellowship and counting the number of citations until the end of 2001 (around 61,000) reveals that on average, the successful applicants had a higher h-index than applicants who had not been successful. An h-index assessment therefore led to the same distribution as that carried out by experts in a selection committee of BIF.

A ranking of physicists can be carried out with the h-index. Using this method, Ed Witten is ahead of Marvin Cohen and Stephen W. Hawking (from left to right).

Cuation is indicated for the time being in comparing different fields

As a raw measurement, the h-index was a quality indicator, considers Bornmann. Nevertheless, he advises caution in statng index-numbers for specific scientific fields, for the time being. This is what Hirsch had done by classifying physicists after 20 years with the h-index of 40 as "outstanding scientists likely to be found only at the major research laboratories“. According to the ETH researcher what must also be taken into account is that the data-bases used to determine citation frequency–the database most commonly used for this is "Thomson ISI Web of Science“–should be used with caution. Another consideration is that neither the frequency of citation nor publication depended solely on scientific quality but also on other factors.

In order to check how close the method comes to the quality of scientists, Bornmann wants to carry out further evaluations of the h-index. He points to studies in which research colleagues have developed an extended version of the h-index, the so-called hI-index, which should enable a comparison between diverse scientific fields. If such a method proves trustworthy it could then be used, for example, to carry out the delicate job of an internal ranking of a university's academics.