Comments on the THE Reputation Rankings

 From Kris Olds at GlobalHigherEd

The 2012 Times Higher Education (THE) World Reputation Rankings were released at 00.01 on 15 March by Times Higher Education via its website. It was intensely promoted via Twitter by the ‘Energizer Bunny’ of rankings, Phil Baty, and will be circulated in hard copy format to the magazine’s subscribers. As someone who thinks there are more cons than pros related to the rankings phenomenon, I could not resist examining the outcome, of course! See below and to the right for a screen grab of the Top 20, with Harvard demolishing the others in the reputation standings.

I do have to give Phil Baty and his colleagues at Times Higher Education and Thomson Reuters credit for enhancing the reputation rankings methodology. Each year their methodology gets better and better.

From Alex Usher at Higher Education Strategy Associates

There actually is a respectable argument to be made for polling academics about “best” universities. Gero Federkeil of the Centrum für Hochschulentwicklung in Gütersloh noted a few years ago that if you ask professors which institution in their country is “the best” in their field of study, you get a .8 correlation with scholarly publication output. Why bother with tedious mucking around with bibliometrics when a survey can get you the same thing?

Two reasons, actually. One is that there’s no evidence this effect carries over to the international arena (could you name the best Chinese university in your discipline?) and second is that there’s no evidence it carries over beyond an academic’s field of study (could you name the best Canadian university for mechanical engineering?).

So, while the Times makes a big deal about having a globally-balanced sample frame of academics (and of having translated the instrument into nine languages), the fact that it doesn’t bother to tell us who actually answered the questionnaire is a problem. Does the fact that McGill and UBC do better on this survey than on more quantitatively-oriented research measures have to do with abnormally high participation rates among Canadian academics? Does the fact that Waterloo fell out of the top 100 have to do with the fact that fewer computer scientists, engineers and mathematicians responded this year? In neither case can we know for sure.