A Bit More About the THE Survey

Thomson Reuters have released a bit more information about the reputational survey they recently conducted for the 2010 Times Higher Education World University Rankings.

They managed to get 13,388 responses. This is quite a lot less than the original target of 25,000 although it is higher than the 9,000 plus respondents to the 2009 THE-QS rankings. This means that QS, who are preparing their own rankings, now have an opportunity to boost the numbers of their respondents by using the usual devices -- reminders, extended deadlines, a chance to win an iPad instead of a Blackberry and so on . Thomson Reuters may have made a mistake by closing their survey so early.

Still, numbers are not everything. Thomson Reuters can claim that their survey, which uses the ISI database of authors published in reputable academic journals, targets people who know something about research. The QS survey, on the other hand, consists merely of those who have managed to get on the mailing list of World Scientific.

Thomson Reuters have also provided some information about the regional and disciplinary distributions of their respondents. The largest group is from the Americas. While most disciplinary clusters are well represented, there is a very small number from the arts and humanities. Respondents spend slightly more than half their time doing research and slightly less than a third teaching.

Is this really enough? It would be interesting to know how many forms were sent out and what the response rate was. Also, how far back in time did Thomson Reuters go in collecting respondents? If they went back five or ten years many respondents might have retired or lost interest in research since publishing.

It also would be helpful if more information were given about the geographical distribution of the survey. One notable absurdity of the THE-QS surveys of 2004-2009 was the marked bias in favor of particular countries – more respondents from Indonesia than from Germany, more from the UK plus Australia than from the US, more from Ireland (just the Republic?) than from Russia. Thomson Reuters have probably overcome these biases but have new ones emerged? Has there been an adequate response from Southeast Asia outside Singapore? Have Russia and Central Asia and the Middle East outside Israel been affected by the omission of Russian and Arabic from the list of languages in which the forms can be completed?

It is good that Thomson Reuters have released some information but if they are to fulfill their promise of greater transparency more is needed.