New Rankings on the Way

Times Higher Education have announced that their new rankings will be published on September 16th and have revealed the outline of their methodology.

The rankings will include five groups of indicators as follows:

A new broad category, called "Teaching - the learning environment", will be
given a weighting of 30 per cent.

Using five separate indicators, this category will use data on an institution's income, staff-student ratios and undergraduate-postgraduate mix, as well as the results of the first-ever global academic reputation survey examining the quality of teaching.

A further 30 per cent of the final rankings score will be based on another new indicator, "Research - volume, income and reputation".

This category will use four separate indicators, including data on research income, research output (measured by publications in leading peer-reviewed journals) and the results of the academic reputation survey relating to research.

The highest-weighted category is "Citations - research influence".

This category will examine a university's research influence, measured by the number of times its published work is cited in other academics' papers.

Based on the 12,000 journals indexed by Thomson Reuters' Web of Science, and taken over a five-year period, the citations data will be normalised to take account
of different volumes of citations between disciplines.

Reflecting the high levels of correlation between citations data and research excellence, this category will be given a weighting of 32.5 per cent.

A fourth category, "International mix - staff and students", will use data on the proportion of international staff and students on campus. This indicator will be given a 5 per cent weighting.

Knowledge transfer activities will be reflected in "Industry income - innovation", a new category worth 2.5 per cent of the total rankings score. This will be based on just one measure in 2010 - research income from industry.

There is still a lot apparently left undecided such as the distribution of indicators within the groups and exactly what faculty will count for scaling. In general, though, the broad outlines of the new ranking look promising with the exception of the large weighting -- nearly one third -- assigned to a single indication, citations. Certainly citations are a good measure of research impact and more difficult to manipulate than some others but putting so much emphasis on just one indicator will be a problem for face validity and will also amplify any data entry errors should they occur.

Finally, I wonder if it is a good idea to refer to the "seventh annual survey". Wouldn't it better to start all over again with the First THE Rankings?