Responses to the QS Academic Survey

QS has published the percentage of responses to this year's academic survey (which is about the best universities for research) from different countries. 

The table below combines the percentages in the surveys of 2007, 2013 and 2018 (for the "2019" rankings) ranked by the percentages for 2013. The data for 2009 and 2013 are from a previous post.

There are some interesting things here. The UK has more responses than France and Germany combined. 

There are more responses from Malaysia than from China.

There are more responses from Kazakhstan than from India.

There are almost as many responses from Australia, Canada and New Zealand as there are from the USA.

The number of responses from these countries has risen by a percentage point or more since 2013: Russia, Malaysia, Iraq, Kazakhstan.

The number of responses from these countries has fallen by a percentage point or more since 2013: USA, Brazil, Italy, Germany, Hungary. 


Table: Percentage of responses to QS academic survey.

country	2007	2013	2018
USA	10.0	17.4	8.5
UK	5.6	6.5	7.0
Brazil	1.1	6.3	3.3
Italy	3.3	4.7	3.5
Germany	3.0	3.8	2.5
Canada	4.0	3.4	3.3
Australia	3.5	3.2	4.0
France	2.9	2.4	2.0
Japan	1.9	2.9	3.5
Spain	2.3	2.7	3.1
Mexico	0.8	2.6	2.3
Hungary	--	2.0	0.9
Russia	0.7	1.7	4.0
India	3.5	1.7	2.6
Chile	--	1.7	2.0
Ireland	1.5	1.6	0.9
Malaysia	3.2	1.5	4.6
Belgium	2.6	1.4	0.7
Hong Kong	1.9	1.4	1.5
Taiwan	0.7	1.3	2.0
Netherlands	0.6	1.2	0.9
New Zealand	4.1	1.2	1.0
Singapore	2.5	1.2	0.8
China	1.6	1.1	1.8
Portugal	0.9	1.1	1.0
Colombia	--	1.1	1.6
Argentina	0.7	1.0	0.8
South Africa	0.7	1.0	0.8
Denmark	1.2	0.9	0.5
Sweden	1.7	0.9	0.8
Switzerland	1.5	0.8	0.7
Austria	1.3	0.8	0.5
Turkey	1.1	0.7	0.8
Indonesia	1.2	0.5	0.9
Philippines	1.8	0.3	0.5
Iraq	--	0.2	1.4
Kazakhstan	--	0.9	3.0
South Korea	?	?	4.0

Ranking rankings: Crass materialism

As the number and scope of university rankings increase it is time to start thinking about how to rank the rankers.

Indicators for global rankings might include number of universities ranked (Webometrics in 1st place), number of indicators (Round University Ranking), bias, and stability.

There could also be an indicator for crass materialism. Here is a candidate for first place. CNBC quotes a report from Wealth-X (supposedly downloadable, good luck) and lists the top ten universities, all in the USA, for billionaires. Apparently, the ranking also includes universities outside the US.

1.  Harvard
2.  Stanford
3.  Pennsylvania
4.  Columbia
5.  MIT
6.  Cornell
7.  Yale
8= Southern California
8= Chicago
10 Michigan.

Where did the top data scientists study?

The website efinancialcareers has a list of the top twenty data scientists in finance and banking. This looks like a subjective list and another writer might come up with a different set of experts. Even so it is quite interesting.

Their degrees are mainly in things like engineering, computer science and maths. There is only one each in business, economics and finance.

The institutions where they studied are:
Stanford (three)
University College London (three)
Institut Polytechnique de Grenoble
Oxford
Leonard Stern School of Business, New York University
University of Mexico
Universite Paris Dauphine
Ecole Polytechnique
Rensselaer Polytechnic Institute (RPI)
California State University
Indian Institute of Science
Johns Hopkins University
Institute of Management Development and Research, India
University of Illinois
University of Pittsburgh
Indian Institute of Technology.

Harvard, MIT and Cambridge are absent but there are three Indian Institutes,  three French schools and some non-Ivy US places like RPI and the Universities of Pittsburgh and Illinois.

Why are US universities doing so well in the THE reputation rankings?

For the last couple of years the higher education media has tried to present any blip in the fortunes of UK universities as one of the malign effects of Brexit, whose toxic rays are unlimited by space, time or logic. Similarly, if anything unpleasant happens to US institutions, it is often linked to the evil spell of the great orange devil, who is scaring away international students, preventing the recruitment of the scientific elites of the world, or even being insufficiently credulous of the latest settled science.

So what is the explanation for the remarkable renaissance of US higher education apparently revealed by the THE reputation survey published today?

Is Trump working his magic to make American colleges great again?

UCLA is up four places, Carnegie Mellon seven, Cornell six, University of Washington six, Pennsylvania three. In contrast, several European and Asian institutions have fallen, University College London and the University of Kyoto by two places, Munich by seven, and Moscow State University by three.

In the previous post I noted that this year's survey had seen an increased response from engineering and computer science and a reduced one from the social sciences and the arts and humanities. As expected, LSE has tumbled five places and Oxford has fallen one place. Surprisingly, Caltech has fallen as well.

Some schools that are strong in engineering, such as Nanyang Technological University and Georgia Institute of Technology, have done well but I do not know if that is a full explanation for  the success of US universities.

I suspect that US administrators have learned that influencing reputation is easier than maintaining scientific and intellectual standards and that a gap is emerging between perceptions and actual achievements.

It will be interesting to see if these results are confirmed by the reputation indicators included in the QS, Best Global Universities, and the Round University Rankings

The THE reputation rankings

THE have just published details of their reputation rankings which will be published on May 30th, just ahead, no doubt coincidentally, of the QS World University Rankings.

The number of responses has gone down a bit, from 10,566 last year to 10,162, possibly reflecting growing survey fatigue among academics.

In surveys of this kind the distribution of responses is crucial. The more responses from engineers the better for universities in Asia. The more from scholars in the humanities the better for  Western Europe. I have noted in a previous blog that the fortunes of Oxford in this ranking are tied to the percentage of responses from the arts and humanities.

This year there have been modest or small reductions in the percentage of responses from the clinical and health sciences, the life sciences, the social sciences, education and psychology and  large ones for business and economics and the arts and humanities.

The number of responses in engineering and computer science has increased considerably.

It is likely that this year places like Caltech and Nanyang Technological University will do better while Oxford and LSE will suffer. It will be interesting to see if THE claim that this is all the fault of Brexit, an anti-feminist reaction to Oxford's appointment of a female vice-chancellor or government Scrooges turning off the funding tap.

         

	2017  %	2018  %
Physical science	14.6	15.6
Clinical and health	14.5	13.2
Life sciences	13.3	12.8
Business and economics	13.1	9
engineering	12.7	18.1
Arts and humanities	12.5	7.5
Social sciences	8.9	7.6
Computer science	4.2	10.4
Education	2.6	2.5
Psychology	2.6	2.3
Law	0.9	1.0
North America	22	22
Asia Pacific	33	32
Western Europe	25	26
Eastern Europe	11	11
Latin America	5	5
Middle East	3	3
Africa	2	2

Getting ready for the next World's Smartest Rankings

As the world waits for the coming round of global rankings -- will Harvard still be number one in the Shanghai rankings? -- I am starting to update my list of smart rankings.

One of favorites was 'the Campus Squirrel Listings." A candidate for inclusion in the next edition is 'The Top 10 Colleges for Dog Lovers'

Number one in the USA is Stephens College in Columbia, Missouri.

Alberta ousts Savitribai Phule Pune in latest edition of innovative ranking

The Fortunate 500 rankings use a sophisticated and innovative methodology to rank global universities. The 2018 edition puts the University of Alberta in first place in the world displacing Savitribai Phule Pune University which has mysteriously dropped out of the top 500.

The top British university is the University of Reading and number one in the USA is Caldwell University.

None of these universities have made any official comment.

These rankings have received almost no interest in the international media. Perhaps the rankers should start announcing the results at a prestigious summit in a spectacular setting along with networking brunches and masterclasses. I have selected a venue for them using an equally sophisticated methodology, North Korea. Perhaps the summit could be combined with with the groundbreaking ceremony for the Pyongyang Trump Tower.

Ranking Insights from Russia

The ranking industry is expanding and new rankings appear all the time. Most global rankings measure research publications and citations. Others try to add to the mix indicators that might have something to do with teaching and learning. There is now a  ranking that tries to capture various third missions.

The Round University Rankings published in Russia are in the tradition  of  holistic rankings. They give a 40 % weighting to research, 40 % to teaching, 10% to international diversity and 10% to financial sustainability. Each group contains five equally weighted indicators. The data is derived from Clarivate Analytics who also contribute to the US News Best Global Universities Rankings.

These rankings are similar to the THE rankings in that they attempt to assess quality rather than quantity but they have 20 indicators instead of 13 and assign sensible weightings. Unfortunately, they receive only a fraction of the attention given to the THE rankings.

They are, however, very  valuable since they dig deeper into the data than  other global rankings. They also show that there is a downside to measures of quality and that data submitted directly by institutions should be  treated with caution and perhaps scepticism.

Here are the top universities for each of the RUR indicators.

Teaching
Academic staff per students: VIB (Flemish Institute of Biotechnology), Belgium
Academic staff per bachelor degrees awarded: University of Valladolid, Spain
Doctoral degrees per academic staff: Kurdistan University of Medical Science, Iran
Doctoral degrees per bachelor degrees awarded:  Jawaharlal Nehru University, India
World teaching reputation  Harvard University, USA.

Research
Citations per academic and research staff: Harvard
Doctoral degrees per admitted PhD: Al Farabi Kazakh National University
Normalised citation impact: Rockefeller University, USA
Share of international co-authored papers: Free University of Berlin
World research reputation: Harvard.

International diversity
Share of international academic staff: American University of Sharjah, UAE
Share of international students: American University of Sharjah
Share of international co-authored papers: Innopolis University, Russia
Reputation outside region: Voronezh State Technical University, Russai
International Level: EPF Lausanne, Switzerland.

Financial sustainability:
Institutional income per academic staff: Universidade Federal Do Ceara, Brazil
Institutional income per student: Rockefeller University
Papers per research income:  Novosibersk State University of Economics and Management, Russia
Research income per academic and research staff: Istanbul Technical University, Turkey
Research income per institutional income: A C Camargo Cancer Center, Brazil.

There are some surprising results here. The most obvious is Voronezh State Technical University which is first for reputation outside its region (Asia, Europe and so on), even though its overall scores for reputation and for international diversity are very low. The other top universities for this metric are just what you would expect, Harvard, MIT, Stanford, Oxford and so on. I wonder whether there is some sort of bug in the survey procedure, perhaps something like the university's supporters being assigned to Asia and therefore out of region. The university is also in second place in the world for papers per research income despite very low scores for the other research indicators.

There are other oddities such as Novosibersk State University of Economics and Management placed first for papers per research income and Universidade Federal Do Ceara for institutional income per academic staff. These may result from anomalies in the procedures for reporting and analysing data, possibly including problems in collecting data on income and staff.

It also seems that medical schools and specialist or predominantly postgraduate institutions such as Rockefeller University, the Kurdistan University of Medical Science, Jawarhalal Nehru University and VIB have a big advantage with these indicators since they tend to have favourable faculty student ratios, sometimes boosted by large numbers of clinical and research only staff, and a large proportion of doctoral students.

Jawaharlal Nehru University is a mainly postgraduate university so a high placing for academic staff per bachelor degrees awarded is not unexpected although I am surprised that it is ahead of Yale and Princeton. I must admit that the third place here for the University of Baghdad needs some explanation.

The indicator doctoral degrees per admitted PhD might identify universities that do a good job of selection and training and get large numbers of doctoral candidates through the system. Or perhaps it identifies universities where doctoral programmes are so lacking in rigour that nearly everybody can get their degree once admitted. The top ten of this indicator includes De Montfort University, Shakarim University, Kingston University, and the University of Westminster, none of which are famous for research excellence across the range of disciplines.

Measures of international diversity have become a staple of global rankings since they are fairly easy to collect. The problem is that international orientation may have something do with quality but it may also simply be a necessary attribute of being in a small country next to larger countries with the same or similar language and culture. The top ten for the international student indicators includes the Central European University and the American university of Sharjah. For international faculty it includes the University of Macau and Qatar University.

To conclude, these indicators suggest that self submitted institutional data should be used sparingly and that data from third party sources may be preferable. Also, while ranking by quality instead of quantity is sometimes advisable it also means that anomalies and outliers are more likely to appear.

World Top 20 Project

The International Rankings Expert Group (IREG) has produced an inventory of international rankings that is testimony to the enormous interest in comparing and classifying universities around the world.

In addition to those rankings that were included there are several "also rans", rankings that were not counted because they included only one indicator, had been published only once or provided insufficient  information about methodology .

One of  these is the World Top 20 Project whose Executive Director and founder is Albert N Mitchell II. The website claims to rank 500 universities according to seven criteria and to use data from institutional databases and educational publications to construct eight regional rankings. The scores are then compared with those from the US News Best Global Universities, the THE World University Rankings, the QS World University Rankings, and the Center for World University Rankings to select the global top twenty.

The top five universities in each region are listed. Most seem quite sensible -- Cape Town is first in Africa, Tokyo in Europe, Harvard in North America and Cambridge in Europe -- but there are no Mexican universities in the top five in central America.

It is interesting that this site has included CWUR rather than the Shanghai ARWU in the big four world rankings. Could this be the start of a new trend?

There is no information about ranks or scores for the various indicators or details about the sources of data. It is also difficult to see how information about things like career development facilities, disability access and low-income outreach could be collected from the universities mentioned. 

Unless further information about sources and methods appears, it seems that there is no need to discuss these rankings any further

At last. A Ranking With Cambridge at the Bottom

Cambridge usually does well in national and global rankings. The most recent ARWU from Shanghai puts it in third place and although it does less well in other rankings it always seems to be in the top twenty. It has suffered at the hands of the citations indicator in the THE world  rankings which seem to think that Anglia Ruskin University, formerly the Cambridgeshire College of Arts and Technology, has a greater global research impact but nobody takes that seriously.

So it is a surprise to find  an article in the Guardian about a ranking from the Higher Education Policy Institute ( HEPI) in the UK that actually puts Cambridge at the bottom and the University of Hull at the top. Near the bottom are others in the Russell group, Oxford, Bristol and LSE.

At the top we find Edge Hill, Cardiff Metropolitan and, of course, Anglia Ruskin Universities.

The ranking was part of a report written for HEPI by Iain Martin, vice-chancellor of Anglia Ruskin University, that supposedly rates universities for fair access, that is having a student intake that mirrors society as a whole. It compares the percentage of participation in higher education of school leavers in local authority areas with the percentage admitted by specific universities. Universities have a high rank if they draw students from areas where relatively few school leavers go to university. The rationale is the claim that learning outcomes are improved when people of diverse backgrounds study together.

It is noticeable that there several Scottish universities clustered at the bottom even though Scotland has a free tuition policy (not  for the English of course) that was supposed to guarantee fair access.

This rankings looks like an inversion of the ranking of UK universities according to average entry tariff, ie 'A' level grades, and a similar inversion of most global rankings based on research or reputation. 

Cambridge and other Russell Group universities have been under increasing pressure to relax entry standards and indiscriminately  recruit more low income students and those from historically unrepresented groups. It seems that they are slowly giving way to the pressure and that as academic standards erode they will be gradually eclipsed by the rising universities of East Asia.