The Best Universities for Technology?

The Times Higher Education Supplement (THES) have published a list of the supposed top 100 universities in the world in the field of technology. The list purports to be based on opinion of experts in the field. However, like the ranking for science, it cannot be considered valid. First, let us compare the top 20 universities according to peer review and then the top 20 according to the data provided by THES for citations per paper, a reasonable measure of the quality of research.

First, the peer review:

1. MIT
2. Berkeley
3. Indian Institutes of Technology (all of them)
4. Imperial College London
5. Stanford
6. Cambridge
7. Tokyo
8. National University of Singapore
9. Caltech
10. Carnegie-Mellon
11. Oxford
12. ETH Zurich
13. Delft University of Technology
14. Tsing Hua
15. Nanyang Technological University
16. Melbourne
17. Hong Kong University of science and Technology
18. Tokyo Institute of Technology
19. New South Wales
20. Beijing (Peking University)

Now, the top twenty ranked according to citations per paper:

1. Caltech
2. Harvard
3. Yale
4. Stanford
5. Berkeley
6. University of California at Santa Barbara
7. Princeton
8. Technical University of Denmark
9. University of California at San Diego
10. MIT
11. Oxford
12. University of Pennsylvania
13. Pennsylvania State University
14. Cornell
15. Johns Hopkins
16. Boston
17. Northwestern
18. Columbia
19. Washington (St. Louis)
20. Technion (Israel)

Notice that the Indian Institutes of Technology, Tokyo, National University of Singapore, Nanyang Technological University, Tsing Hua, Melbourne, New South Wales and Beijing are not ranked in the top 20 according to quality of published research. Admittedly, it is possible that in this field a substantial amount of research consists of unpublished reports for state organizations or private companies but this would surely be more likely to affect American rather than Asian or Australian universities.

Looking a bit more closely at some of the universities in the top twenty for technology according to the peer review, we find that, when ranked for citations per paper, Tokyo is in 59th place, National University of Singapore 70th, Tsing Hua 86th, Indian Institutes of Technology 88th, Melbourne 35th, New South Wales 71st, and Beijing 76th. Even Cambridge, sixth in the peer review, falls to 29th.

Again, there are a large number of institutions that did not even produce enough papers to be worth counting, raising the question of how they could be sufficiently well known for there to be peers to vote for them. This is the list:

Indian Institutes of Technology
Korean Advanced Institute of Science and Technology
Tokyo Institute of Technology
Auckland
Royal Institute of Technology Sweden
Indian Institutes of Management
Queensland University of Technology
Adelaide
Sydney Technological University
Chulalongkorn
RMIT
Fudan
Nanjing

Once again there is a very clear pattern of the peer review massively favoring Asian and Australasian universities. Once again, I can see no other explanation than an overrepresentation of these regions, and a somewhat less glaring one of Europe, in the survey of peers combined with questions that allow or encourage respondents to nominate universities from their own regions or countries.

It is also rather disturbing that once again Cambridge does so much better on the peer review than on citations. Is it possible that THES and QS are manipulating the peer review to create an artificial race for supremacy – “Best of British Closing in on Uncle Sam’s finest”. Would it be cynical to suspect that next year Cambridge and Harvard will be in a circulation-boosting race for the number one position?

According to citations per faculty Harvard was 4th for science, second for technology and 6th for biomedicine while Cambridge was 19th, 29th and 9th.

For the peer review, Cambridge was first for science, 6th for technology and first for biomedicine. Harvard was 4th, 23rd and second.

Overall, there is no significant relationship between the peer review and research quality as measured by citations per paper. The correlation between the two is .169, which is statistically insignificant. For the few Asian universities that produced enough research to be counted, the correlation is .009, effectively no better than chance.

At the risk of being boringly repetitive, it is becoming clearer and clearer that that the THES rankings, especially the peer review component, are devoid of validity.