A Survey from Thomson Reuters

A survey was conducted recently for Thomson Reuters to provide input for the forthcoming Times Higher Education World University Rankings. The results of the survey can be accessed at the Global Institutional Profiles Project set up by Thomson Reuters.

The results of the survey are important since they might provide a clue to what the new ranking will look like.

There were 350 respondents from the “global academic community” This is apparently more than the numbers that answered similar surveys by QS but it does not seem very large especially when THE has raised justified concern about the low and possibly unrepresentative numbers participating in the THE-QS survey of academic opinion.

Of those 350, 107 were from the UK (31%), 90 from the US, 30 from Australia, nine from Canada and seven from New Zealand. Thirty three were from the rest of Europe, 32 from Asia and 42 from the others, i.e. not from North America, Europe, Asia and Australasia. With nearly a third of the respondents coming from the UK and over two thirds from just five English speaking countries this is a distinctly Anglo-Saxon-centric affair.

The first question was the level of familiarity with various rankings. The THE ranking was the one with which the largest number of respondents were familiar. This is a slightly odd result since from 2004 to 2009 THE published rankings under the name THE (S) - QS World University Rankings. The Times Higher Education World University Rankings (minus QS) have yet to appear. No doubt, THE would claim that it was they who published the ranking and can retrospectively rename them if they wish.

Summarizing the responses to the survey, it seems that the respondents believe that university rankings
• are useful
• have methodological problems
• are biased
• encourage the manipulation of data
• encourage a focus on numerical comparisons
• use data that is not transparent or reproducible
• do not not include appropriate metrics
• favour research institutions

Among the information that respondents need or would like to have are

• Publications and citations
• Research awards
• Patents
• Faculty student ratio
• Faculty activity ratios (teaching income/research grants/publications per staff)
• Number of faculty by gender, international, ethnicity or race
• Number of graduate programs and degrees
• Collaboration
• Community engagement
• Perceptions of researchers, employers, alumni and community

Some of this, community engagement for instance, is too vague to be useful. Other items contradict the stated objectives of the developing ranking system: including patents and research collaboration in a general ranking would add more bias in favour of the natural and applied sciences. Others betray the American or European concerns of the respondents: alumni have little significance outside the USA. It is also noticeable that nobody seems interested in student perceptions.