The most interesting thing about THE's experimental African ranking is the use of fractionalised counting of citations. This means that the total number of citations is divided by the number of institutions contributing to a publication. Previously, the method used in THE rankings was to assign the total citations of a paper to all of the institutions that contributed just as though each one had been the only contributor. This has produced some very questionable results with universities that were excellent but very specialised or just generally unproductive scoring remarkably high scores for citations. Panjab university, Tokyo Metropolitan University, Scuola Normale Superiore Pisa, Federico Santa Maria Technical University and Moscow State Engineering Physics Institute have all had moments of glory in the THE rankings because of citation scores that were dramatically higher than their scores for research or any other indicator or group of indicators.
The new method, if applied generally, is likely to see a significant reduction in the scores given to such universities. We can estimate what might happen by looking at the four universities that are included in both the African pilot ranking and last year's world rankings, Cape Town, Witwatersrand, Stellenbosch and Universite Marrakesh Cadi Ayyad, Morocco.
In the world rankings these universities received scores of 86.6, 67.3, 45.6 and 83 respectively. The score of 83 for Universite Cadi Ayyad resulted very largely from its contributions to several publications from the Large Hadron Collider project, one of which has been cited over 2,000 times, a low overall output of papers and the "regional modification" that gave a big boost to low scoring countries. The scores for the three South African universities reflected a larger total output and citations over a broad range of disciplines.
In the African pilot ranking the scores for citations were Cape Town 90.90, Witwatersrand 99.76, Stellenbosch 95.48 and Cadi Ayyad 78.61. The high scores for the South African institutions reflect a much lower mean score than in the world rankings.
The fall in Cadi Ayyad's citation score from 3.6 points below Cape Town to 21.3 below and its falling behind Stellenbosch and Witwatersrand presumably reflect the impact of fractionalised counting.
This suggests that if fractionalised counting is used in the coming World University Rankings many small or specialised institutions will suffer and there will be a lot of reshuffling.
No comments:
Post a Comment