Energetic and Proactive Leadership or a Defective Indicator?

It is now normal for university administrators to use global rankings to market their institutions, reward themselves and, all too often, justify demands for more public funding. Unfortunately, some ranking agencies are more than happy to go along with this.

One example of the misuse of rankings is from July of last year  when the Vice-Chancellor of the University of the West Indies, Sir Hilary Beckles, proclaimed a triple first  in the 2020 Times Higher Education (THE) rankings.

The original  target was to be be in the top 3% of ranked universities by the end of the current strategic planning cycle. The Vice-Chancellor was very happy that as a result of  the work of "an energetic and proactive leadership team of campus principals and others" the university was in the top 1% of the THE Latin America and Caribbean rankings and the top 1% of golden age universities, and was the only ranked university in the Caribbean.

Another article has just appeared lauding the achievements of the university. The Vice-Chancellor has asserted that "(t)he Times Higher Education informed us that what we have achieved is quite spectacular. That many universities had taken 30 years to achieve what we have achieved in a mere three years."

If this is an accurate report of what THE said then the magazine is being very irresponsible. Universities are complex structures, and cannot be turned around with just a few million dollars or a few dozen highly cited researchers. Without drastic restructuring or amalgamation, such a remarkable change is almost invariably the result of methodological changes or methodological defects. The latter is the case here.

UWI 's performance might appear  quite impressive but it seems that we have another case of  THE seeing things that nobody else can. THE is not the only global university ranking. In fact, it is in some important respects not a good one. Let's take a look at some other rankings. UWI is not ranked in Leiden Ranking, the Shanghai Academic Ranking of World Universities, the US News Best Global Universities, or the QS World University Rankings.

It does make an appearance in University Ranking by Academic Performance (URAP) published by Middle East Technical University where it is ranked 1622th and it is 1938th in the Center for World University Rankings.

But in the THE World University Rankings the university is in the top 600 and it is 18th in Latin America and the Caribbean. This is because of a single defective indicator.

UWI has an apparently healthy 81.1 in the THE world rankings citations indicator, supposedly a measure of research influence or impact, which at first sight seems odd since it has a very low score, 10.4, for research. In the Latin American rankings, where competition is less severe, that becomes  93.6 for citations and 75.8 for research. How could a university do so brilliantly for research influence when it has a poor reputation for research, doesn't publish many papers, and has little research income?

What has happened is that UWI has benefitted from THE's peculiar citations indicator that does not use fractional counting for papers with less than a hundred authors, plus hyper-normalisation, plus a country bonus for being located in a low performing country, resulting in a few multi-author, multi-cited papers pushing universities into positions that they could never achieve anywhere else.

UWI has recently contributed to a few papers with a large number of contributors and many citations in genetics and medicine in top journals including Lancet, Science, Nature, and the International Journal of Surgery. Five of these papers are linked with the Global Burden of Disease Study.

UWI is  to be congratulated for having a number of affiliated scientists who are taking part in cutting edge projects. Nonetheless, it is true that these have an impact only because of the technical defects of the THE rankings. In the last few years a succession of unlikely places -- Anglia Ruskin, Peradeniya, Reykjavik, Aswan, Brighton and Sussex Medical School, Durban University of Technology, have risen to the top of the citations charts in the THE rankings.

They have done so nowhere else. The exalted status of UWI in the THE WUR and its various offshoots is due to an eccentric methodology. If THE reforms that methodology, something that they have been talking about for a long time, or abandons the world rankings, the university will once again be consigned to the outer darkness of the unranked.