How Much Instability?

Phil Baty in Times Higher Education writes about fluctuations in the old THE-QS rankings

"Magazines that compile league tables have an interest in instability - playing around with their methodologies to ensure rankings remain newsworthy.

This was the argument made by Alice Gast, president of Lehigh University, Pennsylvania, at the Lord Dearing memorial conference at the University of Nottingham this month.

She has a point. Dramatic movements in the league tables make the news and generate interest - helpful for the circulation figures.

But too much movement raises questions about credibility: everyone knows that it takes more than 12 months for an 800-year-old university to lose its status, or for a young pretender to ascend the heights. "

The THE-QS rankings were famous for their yearly fluctuations. This of course helped to make them much more popular than the reliable but boring Shanghai rankings (unless you were prepared to spend a few hours cutting and pasting the indicator scores of universities in the 300s and 400s into an Excel file and then they could be interesting). The rises and falls resulted from changes in methodology, errors, correction of errors and inconsistent application of guidelines.

Still, there are cases when universities undergo serious restructuring or pour massive funds into research or recruit administrators of the highest calibre and these developments should be reflected in any valid index. Rankings that do not show some upward movement by, say, the Hong Kong University of Science and Technology or King Abdullah University of Science and Technology ought to be considered suspect

Equally, it is striking that the major rankings contain elements, the THE-QS academic opinion survey, the Nobel laureates in the Shanghai rankings, even eleven year old publications and citations in the Taiwan rankings, that disguise the steady relative decline of Oxford and Cambridge over the last two decades.

We shall have to wait until 2011 to see if the new THE ranking will avoid the suspicious fluctuations of the THE-QS rankings and also be sensitive to genuine changes in international higher education.