Observations on the Indian Ranking Boycott

Seven Indian Institutes of Technology (IITs) -- Delhi, Bombay, Guwahati, Kanpur, Kharagpur, Madras, and Roorkee -- have announced that they will be boycotting this year's Times Higher Education (THE) World University Rankings. The move has been coming for some time. Indian universities have not performed well in most rankings but they have done especially badly in THE's.

Take a look at the at the latest THE world rankings and the performance of three elite institutions. IIT Delhi (IITD) and IIT Bombay (IITB) are in the 401-500 band, and the Indian Institute of Science (IISc) is 301-350.

It is noticeable that these three all do much better in the QS world rankings where IIT Delhi is 182nd, IIT Bombay 152nd, and IISc  184th. That no doubt explains why these Institutes are boycotting THE but still engaging with QS. 

It should be pointed out that with regard to research THE probably  treats the Institutes better than they deserve. The Shanghai rankings, which are concerned only with research, have IITD and IITB in the 701-800 band, and IISc at 401-500. In the US News Best Global Universities IITD is 654th, ITB 513th, and IISc 530th.

The dissatisfaction with THE is understandable. Indeed it might be surprising that the IITs have taken so long to take action. They complain about transparency and the parameters. They have a point, in fact several points. The THE rankings are uniquely opaque: they combine eleven indicators into three clusters so it is impossible for a reader to figure out exactly why a university is doing so well or so badly for teaching or research. THE's income and international metrics, three of each, also work against Indian universities.

It is, however, noticeable that a few Indian universities have done surprisingly well in the THE world rankings: IIT Ropar and IIT Indore are in the top 400 and IIT Gandhinagar in the top 600 thanks to high scores for citations. IIT Ropar is credited with a score of 100, making it fourth in the world behind those giants of research impact: Aswan University, Brandeis University, and Brighton and Sussex Medical School. 

Regular readers of this blog will know what is coming next. IIT Ropar has contributed to 15 papers related to the multi-author and hugely cited Global Burden of Disease Study (GBDS), which is slightly less than 1.5% of its total papers over the relevant period but well over 40% of citations. 

It would be relatively simple for the mutinous seven to recruit one or two researchers involved in the GBDS and in a few years -- assuming the current methodology or something like it continues -- they too would be getting near "perfect" scores for citations and heading for top three hundred spots.

They may, however, have judged that the THE methodology is going to be changed sooner or later -- now looking like a little bit later -- or that aiming for the QS reputation surveys is more cost effective. Or perhaps they were simply unaware of exactly how to get a good score in the THE rankings.

It is sad that the Indian debate over ranking has largely been limited to comparisons between THE and QS. There are other rankings that are technically better in some ways and are certainly better suited to Indian circumstances. The Round University Ranking which has 20 indicators and a balanced weighting has IISC in 62nd place with extremely good scores for financial sustainability and doctoral students.

The boycott is long overdue. If it leads to a more critical and sceptical approach to ranking then it may do everybody a lot of good.