Preliminary "Results" from Thomson Reuters

Thomas Reuters have announced what they call preliminary results of the reputational survey of academic researchers that will be one indicator in the forthcoming Times Higher Education international university rankings. The survey closed on May 2nd.

Thomson Reuters have not said how many survey forms were distributed nor how many responses they have received.

They say that they have received "thousands of responses from every corner of the world" and that they have achieved "an excellent breadth of results across six subject areas".

In an article in Forbes, Jonathan Adams, director of research evaluation at Thomson Reuters. says

"We're particularly pleased with the number of responses from the Asia Pacific region. As other surveys have been criticized for over-representing North America and Europe, we took particular care to better balance regional representation."

So far there has been nothing that QS, who conducted the much and very justly condemned academic survey for the THE-QS rankings of 2004-2009, have not said. Indeed, QS weighted their survey precisely so that one third of responses were from the Asia Pacific region.

The problem with the old THE-QS survey was that the UK and Australia were overrepresented in comparison with the United States and Japan and that within the three super-regions of the world there were marked variations in the number of reponses from country to country.

Thomson Reuters say that "To help control for language and translation bias, the Academic Reputation Survey was offered in eight languages: Japanese, Simplified Chinese, Spanish, French, German, Brazilian Portuguese, European Portuguese and English."

If you going to send out the survey in two kinds of Portuguese, then not sending them in the two languages of the United Nations, Arabic and Russian, seems a little odd.

I am also wondering about about the distribution of the survey. I know of several people who took part in the QS survey last year and before but who, despite reasonable publication records in ISI-indexed journals, did not receive a survey form this time.

Thomson Reuters claim that their survey is superior to that of QS. Perhaps, it is: they have, for a start, made progress by asking questions about teaching. But we need more information than has been released so far to be convinced. It should be quite easy, at least, to release the total number of responses and the total response rate and those for individual countries and regions.