The Perils of Affiliation

Five days ago I noted an open access paper  that analysed the new list of highly cited researchers published by Thomson Reuters (TR) and discussed the disproportionate number of secondary affiliations to a single institution, King Abdulaziz University (KAU) of Jeddah, Saudi Arabia.

The significance of this is that the number of highly cited researchers is an indicator in the Academic Ranking of World Universities (ARWU) produced by Shanghai Jiao Tong University, contributing 20% of the total score. If all affiliations given in the list are counted, including secondary affiliations, then KAU will get an extremely high score for this indicator in the forthcoming rankings and will do very well in the overall rankings.

Times Higher Education (THE) has now noticed what is going on. An article by Paul Jump reports on the findings  by Lutz Bornmann of the Max Planck Society  and Johann Bauer of the Max Planck Institute.

The THE article has been republished in Inside Higher Ed and there is more discussion in University World News by Yves Gingras of the University of Quebec.

That THE should publish such an article is a little surprising. The researchers who have listed KAU as a secondary affiliation will probably not stop with the the TR highly cited researchers' list -- if they do it will be a major scandal -- but will also include it in papers published in the highly regarded journals indexed in the Web of Science database. That could include those multi-"authored", hyper-cited publications in fields such as astronomy, genetics and particle physics that have propelled institutions such as Moscow State Engineering Physics Institute and Panjab University to high places in the Citations indicator in the THE World University Rankings.

KAU has already received a  high score for citations in the THE 2013 world rankings, one that is disproportionate to its very modest score for  the Research: Volume, Reputation and Income indicator. It is not impossible that its secondary affiliations in the old highly cited list will start popping up in the"author" list in one or two of those  Multi Author (or contributor?) Publications in this year's rankings as might those on the new lists in the years to come.

If so, one wonders whether it is fair to single out ARWU for criticism. However, TR has announced that  In the new highly cited list  that they  excluded physics papers with more than 30 institutional addresses. 

"The methodology described above was applied to all ESI fields with the exception of Physics. The relative large number of Highly Cited Papers in Physics dealing with high- energy experiments typically carried hundreds of author names. Using the whole counting method produced a list of high-energy physicists only and excluded those working in other subfields. For example, the number of Highly Cited Papers required for inclusion in Physics, using the standard methodology, turned out to be a remarkable 63. So, as an expedient, it was decided to eliminate from consideration any paper with more than 30 institutional addresses. This removed 436 out of 10,373 Highly Cited Papers in physics and the problem of overweighting to high-energy physics. An analysis without these papers produced a list in which the threshold for number of Highly Cited Papers was 14. It also produced a ranking in which the 2010 Nobel Prize winner in Physics Andre Geim of the University of Manchester appeared first, with 40 Highly Cited Papers. Fields of physics other than high-energy alone now appear as represented by the scientists selected."

If TR does that for this year's rankings as well they will save THE some embarrassment, although Panjab University may be wondering why its "excellence" has suddenly disappeared. We shall have to wait and see what happens.

There is an important point about the lists and their role in the rankings that needs to be made. The count of affiliations of KAU to in the arxiv paper is probably too high. The authors count the number of primary affiliations, then the total number of affiliations and also the fractionated total so that if a researcher has three affiliations then each counts as one third of an affiliation. 

However, as indicated in their methodology section , ARWU have surveyed the current highly cited researchers to determine how their affiliations and where there was no response assign secondary affiliations a 16% fraction, which was the percentage given by those who did respond to the survey. This would lead to a total for KAU less than that suggested by the paper although still implausibly high and well above the scores for the other indicators.