The Ranking Effect

The observer effect in science refers to the changes that a phenomenon undergoes as a result of being observed.

Similarly, in the world of university ranking, indicators that may once have been useful measures of quality sometimes become less so once they are included in the global rankings.

Counting highly cited researchers might have been a good measure of research quality but its value was greatly reduced once someone found out how easy it was to recruit adjunct researchers and to get them to list their new "employer" as an affiliation.

International students and international faculty could also be markers of quality but not so much if universities are deliberately recruiting under-qualified staff and students just to boost their rankings score.

Or, as Irish writer Eoin O'Malley aptly puts it in the online magazine Village , "As Goodhart’s Law warns us: when a measure becomes a target, it ceases to be a good measure".

O'Malley argues that reputation surveys are little more than an exercise in name reputation, that Nobel awards do not measure anything important (although I should point out that Trinity College Dublin would not get any credit for Samuel Beckett since the Shanghai Rankings do not count literature awards), and that the major criteria used by rankers do not measure anything of interest to students.

Irish universities have experienced a disproportionate impact from the methodological changes introduced by QS and Times Higher Education towards the end of last year. I suspect that Dr O'Malley's criticism will have a receptive audience.