Baloney

The economist David Blanchflower has dismissed the QS rankings as "a load of old baloney".

Much of what he says is sensible, indeed obvious. But not entirely.

"This ranking is complete rubbish and nobody should place any credence in it."

A bit too strong. The QS rankings are not too bad in parts, having improved over the last few years, and are moderately accurate about sorting out universities within a country or region. I doubt that anyone seriously thinks that Cambridge is the best university in the world unless we start counting May balls and punting on the Cam but it is quite reasonable that it is better than Oxford or Durham. Similarly, I wonder if anyone could argue that it is rubbish that Tokyo is the best university in Japan or Cape Town in Africa.

"It is unclear whether having more foreign students and faculty should even have a positive rank; less is probably better."

Students yes, but if nothing else more international faculty does mean that a university is recruiting from a larger pool of talent.

Blanchflower does not mention the academic and employer surveys both of which are flawed but do provide another dimension of assessment or the faculty student ratio which is very crude but might have a slightly closer relationship to teaching quality than the number of alumni who received Nobel prizes decades ago.

He then goes on to compare the QS rankings unfavorably with the Shanghai rankings (That actually is Shanghai Jiao Tong University not what he calls the University of Shanghai). I would certainly agree with most of what he says here but I think that we should remember that flawed as they are the QS rankings do, unlike the Shanghai index, give some recognition to excellence in the arts and humanities, make some attempt to assess teaching and  provide a basis for discriminating among those universities without Nobel prize winners or Fields medalists.

Finally, I would love to see if Blanchflower has any comments on last year's THE-Thomson Reuters rankings which put Alexandria, Bilkent and Hong Kong Baptist University among the world's research superpowers.

Well Done, QS

 
QS have just indicated that they have excluded self-citations from their citations per faculty indicator in this year's World University Rankings. This is a very positive move that will remove some of the distortions that have crept into this indicator over the last few years. It would have been even better if they had excluded citations within journals and within institutions. Maybe next year.

It will be interesting to see if Times Higher Education and Thomson Reuters do the same with their rankings in October.It would not be very difficult and it might help to exclude Alexandria University and a few others from an undeserved place in the world's top universities for research impact.

(By the way Karolinska Institute is not in the US)


Although it may not make very much difference at the very top of this indicator, it seems that some places have suffered severely and others have benefited  from the change. According to the QS intelligence unit:

• Of all of the institutions we looked at the institution with the largest absolute number of self-citations, by some margin, is Harvard with over 93,000 representing 12.9% of their overall citations count
• The top five institutions producing over 3,000 papers, in terms of proportion of self-citations are all in Eastern Europe – St Petersburg State University, Czech Technical University, Warsaw University of Technology, Babes-Bolyai University and Lomonosov Moscow State University
• The top five in terms of the difference in citations per paper when self-citations are excluded are Caltech, Rockefeller, UC Santa Cruz, ENS Lyon and the University of Hawaii
• And the top 10 in terms of the difference in citations per faculty when self-citations are included are:

#	Institution	Country
1	California Institute of Technology (Caltech)	United States
2	Rockefeller University	United States
3	Stanford University	United States
4	Gwangju Institute of Science and Technology (GIST)	South Korea
5	Karolinska Institute	United States
6	Princeton University	United States
7	Leiden University	Netherlands
8	Harvard University	United States
9	University of California, San Diego (UCSD)	United States
10	University of California, San Francisco (UCSF)	United States

The Best University in the World
Update 8/9/2011 -- some comments added

For many people the most interesting thing about the QS rankings is the battle for the top place. The Shanghai rankings put Harvard in first place year after year and no doubt will do so for the next few decades. QS when it was in partnership with Times Higher Education also routinely put Harvard first. This is scarcely surprising since the research prowess of Cambridge has steadily declined in recent years. Still, Cambridge, Oxford and two London colleges did quite well mainly because they got high scores for international faculty and students and for the academic survey (not surprising since a disproportionate number of responses came from the UK, Australia and New Zealand) but not well enough to get over their not very distinguished research record.

Last year, however, Cambridge squeezed past Harvard. This was not because of the  academic and employer surveys. That remained at 100 for both places. What happened was that between 2009 and 2010 Cambridge's score for citations per faculty increased from 89 to 93. This would be a fine achievement if it represented a real improvement. Unfortunately, almost every university with scores above 60 for this indicator in 2009 went up by a similar margin in 2010 while universities with scores below 50 slumped. Evidently, there was a new method of converting raw scores. Perhaps a mathematician out there can help.

And this year?

Cambridge and Harvard are both at 100 for the academic and employer surveys just like last year. (Note that although Harvard does better than Cambridge in both surveys they get the same reported score of 100).


For the faculty student ratio Harvard narrowed the gap a little from 3 to 2.5 points. In citations per faculty Cambridge slipped a bit by 0.3 points. However, Cambridge pulled further ahead on international students and faculty.

Basically, from 2004 to 2009 Harvard reigned supreme because its obvious superiority in research was more than enough to offset the advantages Cambridge enjoyed with regard to internationalisation (small country and policies favouring international students), faculty student ratio (counting non-teaching research staff) and the academic survey (disproportionate responses from the UK and Commonwealth). But this year and last the change in the method of converting the raw scores for citations per faculty artificially boosted Cambridge's overall scores.

So, is Cambridge really the world's top university?

The THE-TR Rankings

The THE-TR World University Rankings will be published on October 6th.

There will be some changes. The weighting given to the citations indicator will be slightly reduced to 30% and internationalisation gets 7.5% instead of 5%.

There will be some tweaking of the citations indicator to avoid a repeat of the Alexandria and other anomalies. Let's hope it works.

In the research indicator there will be a reduction in the weighting given to the survey and public research income as a percentage of research income will be removed.

There will, unfortunately, be a slight increase in the percentage given of international students and a decline in that for international faculty.