Rankings and the Threat from the East

Recently, we have heard a lot about global university rankings' responsibilities. Some have drawn attention to the increasing number of universities included in the rankings or the new rankings that allow universities to showcase the remarkable and interesting things they have been doing for society or the environment. There are claims that the well-known rankers are promoting global equity by including many more African and Asia universities.

Perhaps. But it seems that some rankings, particularly the two big-name ones, THE and QS, have another function, which is to downplay the rise of Chinese and maybe other Asian institutions and maintain the dominant position of the elite schools of the Global North. 

The table below shows the number of universities included in the top 100 universities in various global rankings. The table is arranged in ascending order according to the number of Mainland Chinese universities and refers to the most recent edition. 

Chinese universities are apparently uninterested in the rankings that supposedly assess contributions or commitment to the environment, sustainability, or equity. There are none in the top 100 of the new QS Sustainability Rankings or the GreenMetric Rankings and only one in the THE Impact Rankings. On the other hand, China does very well in Nature Index and in Leiden Rankings Publications and Publications in the top 1% of journals and fairly well in the URAP, Scimago, and National Taiwan University rankings. In short, China does best in those rankings that emphasize recent achievements in research in STEM subjects. 

The UK does best in rankings that include a substantial weighting for reputation, internationalization,  or activity related to sustainability and much less well in research-based rankings. 

The USA hasn't really bothered with the GreenMetric and THE Impact rankings. Its best performance is in UniRank, which is a measure of web activity, and Webometrics, which is half web activity, CWUR, which includes faculty and alumni achievement, and US News Best Global Universities, which has a strong reputation element. It is not so good in Nature Index, URAP, and NTU, which are research-based. 

It seems, to simplify a bit, that British and American universities benefit from indicators that measure or try to measure resources, reputation, web presence, and international activity, Chinese and some other Asian institutions are rapidly moving ahead in research and innovation.

Table: Number of Universities in the Top 100

Ranking	Country of publication	USA	UK	Mainland China
QS Sustainability	UK	12	28	0
UniRank	Australia	75	7	0
GreenMetric	Indonesia	3	3	1
THE Impact	UK	6	25	1
USN Global	USA	41	11	4
MosIUR	Russia	41	15	4
QS World	UK	27	17	5
CWUR	UAE	50	9	6
GEURS	France	18	8	6
Webometrics	Spain	53	9	6
THE World	UK	36	11	7
RUR	Georgia	38	11	7
ARWU	China	38	8	10
NTU	Taiwan	36	10	14
Scimago- universities	Spain	39	7	24
URAP	Turkiye	28	6	23
Leiden P 1%	Netherlands	39	8	28
Leiden P	Netherlands	31	6	36
Nature Index -  academic	UK	37	5	35

Yet another example of the misuse of rankings

The proliferation of rankings has led to universities selectively quoting metrics in attempts to boost prestige, student applications, and state support. A recent example is Brunel University's claim that it is the joint most international university in the UK and fourth most international in the world.

This is based on the International Outlook pillar in the most recent edition of the Times Higher Education (THE) world rankings.

THE is not the only ranking with an internationalisation indicator. Let's take a look at the others.

In the QS world rankings Brunel is 9th in the UK for International Faculty, joint 12th for International Students, and 36th for International Research Network,

In the latest URAP (at the time of writing) it is 34th in England for International Collaboration.

In Round University Rankings, Brunel is 9th for International academic staff in the UK, 17th for international students, and 22nd for International Level.

In Leiden Ranking it is joint 6th in the UK for International Collaboration.

I don't want to denigrate Brunel in any way but the claim that it is the most international university in the UK is misleading and should be withdrawn or at least accompanied by a very big *.

Global Subject Rankings: The Case of Computer Science

Three ranking agencies have recently released the latest editions of their subject rankings: Times Higher Education, Shanghai Ranking, and Round University Rankings.  

QS, URAP, and National Taiwan University also published subject rankings earlier in the year. The US News global rankings announced last year can be filtered for subject. The methods are different and consequently the results are also rather different. It is instructive to focus on the results for a specific field, computer science and on two universities, Oxford and Tsinghua. Note that the scope of the rankings is sometimes different.

 

1.   Times Higher Education has published rankings of eleven broad subjects using the same indicators as in their world rankings, Teaching, Research Environment, Research Quality, International Outlook, and Industry: Income and Patents, but with different weightings. For example, Teaching has a weighting of 28% for the Engineering rankings and Industry: Income and Patents 8%, while for Arts and Humanities the weightings are 37.5% and 3% respectively.

These rankings continued to be led by the traditional Anglo-American elite. Harvard is in first place for three subjects, Stanford, MIT, and Oxford in two each and Berkeley and Caltech in one each.

The top five for Computer Science are:

1.    University of Oxford

2.    Stanford University

3.    Massachusetts Institute of Technology

4.    Carnegie Mellon University

5.    ETH Zurich.

Tsinghua is 13^th.

 

2.   The Shanghai subject rankings are based on these metrics: influential journal publications, category normalised citation impact, international collaboration, papers in Top Journals or Top Conferences, and faculty winning significant academic awards.

According to these rankings China is now dominant in Engineering subjects. Chinese universities lead in fifteen subjects although Harvard, MIT and Northwestern University lead for seven subjects. The Natural Sciences, Medical Sciences, and Social Sciences are still largely the preserve of American and European universities.

Excellence in the Life Sciences appears to be divided between the USA and China. The top positions in Biology, Human Biology, Agriculture, and Veterinary Science are held respectively by Harvard, University of California San Francisco, Northwest Agriculture and Forestry University, and Nanjing Agricultural University.

The top five for Computer Science and Engineering are:

1.    Massachusetts Institute of Technology

2.    Stanford University

3.    Tsinghua University

4.    Carnegie Mellon University

5.    University of California Berkeley.

Oxford is 9^th.

 

3.  The Round University Rankings (RUR), now published from Tbilisi, Georgia, are derived from 20 metrics grouped in 5 clusters, Teaching, Research, International Diversity, and Financial Sustainability. The same methodology is used for rankings in six broad fields. Here, Harvard is in first place for Medical Sciences, Social Sciences, and Technical Sciences, Caltech for Life Sciences, and University of Pennsylvania for Humanities.

RUR’s narrow subject rankings, published for the first time, use different criteria related to publications and citations: Number of Papers, Number of Citations, Citations per Paper, Number of Citing Papers, and Number of Highly Cited Papers. In these rankings, first place goes to twelve universities in the USA, eight in Mainland China, three in Singapore, and one each in Hong Kong, France, and the UK.

 The top five for Computer Science are:

1.    National University of Singapore

2.    Nanyang Technological University

3.    Massachusetts Institute of Technology

4.    Huazhong University of Science and Technology

5.    University of Electronic Science and Technology of China.

Tsinghua is 10^th.  Oxford is 47^th.

 

4.   The QS World University Rankings by Subject are based on five indicators: Academic reputation, Employer reputation, Research citations per paper, H-index and International research network.  At the top they are mostly led by the usual suspects, MIT, Harvard, Stanford, Oxford, and Cambridge.

The top five for Computer Science and Information Systems

1.    Massachusetts Institute of Technology

2.    Carnegie Mellon University

3.    Stanford University

4.    University of California Berkeley

5.    University of Oxford.

Tsinghua is 15^th.

 

5.   University Ranking by Academic Performance (URAP) is produced by a research group at the Middle East Technical University, Ankara, and is based on publications, citations, and international collaboration. Last July it published rankings of 78 subjects.  

 The top five for Information and Computing Sciences were:

1.    Tsinghua University

2.    University of Electronic Science and Technology of China

3.   Nanyang Technological University

4.   National University of Singapore

5.   Xidian University

Oxford is 19th

 

6.    The US News Best Global Universities can be filtered by subject. They are based on publications, citations and research reputation.

The top five for Computer Science in 2022 were:

1.   Tsinghua University

2.   Stanford University

3.    Massachusetts Institute of Technology

4.    Carnegie Mellon University

5.   University of California Berkeley

Oxford was 11^th.

 

7.    The National Taiwan University Rankings are based on articles, citations, highly cited papers, and H-index.

The top five for Computer Science are:

1.    Nanyang Technological University

2.    Tsinghua University

3.    University of Electronic Science and Technology of China

4.   National University of Singapore

5.    Xidian University

Oxford is 111^th

 

So, Tsinghua is ahead of Oxford for computer science and related fields in the Shanghai Rankings, the Round University Rankings, URAP, the US News Best Global Universities, and the National Taiwan University Rankings. These rankings are entirely or mainly based on research publications and citations. Oxford is ahead of Tsinghua in both the QS and THE subject rankings. The contrast between the THE and the Taiwan rankings is especially striking.

 

 

 

 

 

Observations on the THE Arab University Rankings

Times Higher Education (THE) has just announced the third edition of its Arab University Rankings. There has been a churning of universities with many falling and many rising. Once again, this volatility seems largely the result of methodology changes and only in part any genuine decline or progress. 

The rankings are led by King Abdullah University of Science and Technology (KAUST) in Saudi Arabia, which makes sense from the point of view of high impact research, although it does no undergraduate teaching. After that we have Khalifa University, UAE, Qatar University, King Fahd University of Petroleum and Minerals, Saudi Arabia, and the University of Sharjah, UAE.

THE has introduced a raft of changes in its World University Rankings, including adding patents as a metric, tweaking the internationalisation pillar to help larger countries, and including three new measures of citations. 

They have added more changes to the Arab University Rankings. The weighting given to the teaching and research surveys has been trimmed. Field Normalised Citation Impact has been removed altogether leaving the three new metrics for research impact: Research Strength, Research Excellence, and Research Influence. Within the International Outlook pillar there is now a 2% weighting for inter-Arab collaboration. The Society pillar, unlike the world rankings,  does not include patents and  it gives a 4%. weight to participation and performance in THE's Impact Ranking.

It is always advisable to look at the specific metric ranks for any ranking, especially THE. For this year's ranking we have: Research Quality; KAUST, International Outlook; Gulf Medical University, UAE, Research Environment; KAUST, Teaching; Beirut Arab University, Society; KAUST. 

There are some interesting things about this year's rankings. To start, there is a noticeable improvement in the ranks of universities in the United Arab Emirates. There are now six UAE universities in the top 25 compared with four last year and three in 2021.

Some Emirati universities have done particularly well, Khalifa University in Abu Dhabi has risen from fifth place to second and  Abu Dhabi University from 39th to 9th. 

The results were announced this year at the THE MENA summit which this year was held at the campus of New York University Abu Dhabi. 

That meeting also saw a number of awards going to UAE institutions, including Abu Dhabi University for International Strategy of the year. Gulf Medical University in Ajman for outstanding support for students, New York University Abu Dhabi for Research Project  STEM, American University in Dubai for  Teaching and Learning Strategy.

 A few years ago I noticed that THE was holding conferences  where they would announce results that appeared to favour the host countries. Thus in February 2015 THE held a MENA summit in Qatar with a "snapshot" single metric ranking that put Texas A & M Qatar in first place and UAE University 11th. The next MENA meeting was in January 2016 in Al Ain, UAE where in a ranking that used the WUR metrics, Texas A and M Qatar disappeared and UAEU rose to fifth place.

Another example. In February 2016 at a conference held at the Hong Kong University of Science and Technology, THE introduced a new methodology for its Asian rankings that dethroned the University of Tokyo  as the top Asian university and placed it below universities in Kong Kong, Singapore, and Mainland China.

In contrast, the number of Egyptian universities in the top 25 has fallen from six to two , Mansoura University and the American University in Cairo. Last year's front runner King Abdulaziz University, Saudi Arabia, has fallen to 15th place.

So the holding of a summit in Abu Dhabi and a new methodology coincided with a significant improvement for UAE in general and a very significant improvement for two Abu Dhabi universities. Plus NYU Abu Dhabi, currently unranked, received an award. Perhaps this is just a coincidence or perhaps such a turnover in a single year reflects  real changes, which the new methodology accurately detects. But cynics may wonder a little.

There has been a lot of discussion recently about conflict of interest in the ranking business. It is likely that questions will be asked about a new methodology so conveniently helping institutions in the summit host country.

How Dare They? HE Sector Reacts to the King's Speech

 Occasionally there are moments when a few casual words suddenly illuminate things that have been obscured. 

One such moment was when the Vice-Chancellor of the University of Oxford expressed her embarrassment  that one of the university's alumni had failed to express appropriate deference towards experts. It would seem that the a major function of higher education is not the encouragement of critical thought but the acceptance of anything that has been approved by academic experts. This was especially ironic since the claim was made at a summit held by Times Higher Education, whose expertise in the field of university ranking is somewhat questionable. 

The recent King's Speech contained a bland announcement that the government would "ensure young people have the knowledge and skills to succeed" by combining technical and academic qualifications.  Also, the government will attempt to reduce enrollment in "poor quality university degrees and increase the number undertaking high quality apprenticeships." 

It is not unlikely that any such initiatives will fail to get off the ground or will crash soon after take off and even if implemented they would probably not be very effective or even effective at all.

Research Professional News, however, reports that industry insiders are incensed that the government has dared to say anything that could be considered critical of British universities. Diana Beech, Chief Executive of the London Higher Group of Institutions said "it is beyond belief that the UK government would even contemplate asking His Majesty the King to speak negatively of the national asset that is our world-leading higher education and research sector."

The King is not speaking negatively of the entire sector. He is talking about proposed efforts to improve the sector. And surely the "brightest and the best" of the world are more likely to come to Britain if they think there are efforts to bring about positive change.

It seems that the academic establishment wants everybody to pretend that there is nothing wrong with British universities. That, in the long run is not going to do anyone any good.

Crisis, conflict and global rankings

Just published in the Journal of Adult Learning, Knowledge and Innovation

Crisis, conflict and global rankings

Read here

Journal of Adult Learning, Knowledge and Innovation

Volume/Issue: 

Accepted Manuscript / Online First

Abstract

Global university rankings have always been associated with international political and economic conflicts. Even before the COVID-19 pandemic there were signs that scientific and academic globalism was breaking down. The pandemic, the various measures taken to combat it, and military and ideological conflicts have led to the breakdown of international academic cooperation, the formation of very different research complexes, and the development of new regional ranking systems.

Two Decades of Rankings: Rising and Falling in ARWU

 Most rankings are of little value for identifying trends over more that a couple of years. Changes in methodology, and sometimes a lack of access to old editions make year on year comparisons difficult or impossible. The Shanghai Rankings, aka ARWU, have maintained a generally consistent methodology over two decades and publish data going back to the founding year of 2003.

So it is possible to use ARWU to look for  patterns in the  world's research and higher education landscape. Here are some "winners" and "losers", based on the number of universities in the ARWU top 500 in 2004, when Shanghai changed the initial methodology to include the social sciences and 2023. This is far from a perfect measure; for a start this ranking does not takes no account of the humanities and relies to much on old Nobel and Fields laureates. Even so it does give us some idea of the shift in the academic world's centre of gravity.

Rising

Australia from 14 in 2004 to 24 in 2023 (and from 2 in the top 100 to 7)

Brazil from 4 to 5

China from 16 to 98 (and from zero in the top 100 to 11)

Malaysia from zero to one

New Zealand from 3 to 4

Saudi Arabia from zero to 6

Singapore 2 in the top 500 in 2004 and 2023 (but rising from zero in the top 100 in 2004 to 2 in 2023)

South Korea from 8 to 11

Falling

Canada from 23 to 18 

France from 22 to 18

Germany from 43 to 31

India from 3 to 1

Israel from 7 to 5 (but rising from 1 to 3 in the top 100)

Italy from 24 to 16

Japan from 36 to 12 (and from 5 in the top 100 to 2)

Switzerland from 8 to 7

United Kingdom from 42 to 38 (and from 11 in the top 100 to 8)

United States from 170 to 120 (and from 51 in the top 100 to 38)

The last two decades have seen a massive increase in the research capabilities of universities in Australia, China, South Korea, and Singapore. The rest of Asia, including Japan and India, has stagnated or even fallen relatively and perhaps absolutely.

The biggest losers are the USA, UK, and Germany although Canada, France, Italy and Switzerland have also not done so well.

More recently, Saudi Arabia has noticeably improved and may soon be followed by other Middle eastern states.

                                

The New QS Methodology: Academic Snakes and Ladders

The ranking season is under way. The latest edition of the QS world rankings  has just been announced and we have already seen the publication of the latest tables from Leiden Ranking, CWUR, RUR, uniRank and the THE Impact Rankings plus the THE Asian and Young Universities and  Sub-Saharan Africa rankings. Forgive me if I've missed anything.

Each of these tells us something about the current geopolitics of higher education and science and the way in which they are reflected in the global ranking business. 

The QS rankings have a new methodology, which makes it quite different from previous editions. Nonetheless, the media have been full of universities celebrating their remarkable achievements as they have soared, surged, risen, ascended in the rankings. No doubt, there will be a few promotions and bonuses.

It is in the nature of ranking that places are finite and if some  universities suddenly surge then others will fall. It seems that the general pattern of the QS rankings is that Canadian, Australian, and American universities are rising and Chinese, Korean, and Indian Universities are falling. Russian and Ukrainian universities are also falling although that might be for other reasons.

QS have reduced the weighting of their academic survey from 40% to 30% and faculty student ratio from 20% to 10%. The weighting for the employer survey has increased from 10% to 15% and there are three new indicators, Sustainability, Employment Outcomes, and International Research Network.

QS claim that the new methodology "reflects the collective intelligence of the sector, and the changing priorities of students." If that is so, then the collective intelligence is very localised. The new methodology puts a heavy fist on the scales in favour of Western universities and against Asia.  

The revised methodology works against universities that have acquired a good reputation for research or recruited a large and permanent faculty. It favours those that have mobilised their alumni network for the employer survey and are enthusiastic participants in the sustainability movement. 

As a result the leading Chinese institutions have taken a tumble. Peking has fallen 5 places to 17th, Tsinghua 11 to 25th, and Fudan 16 to 50th. 

Other national and regional flagships have tumbled, Seoul National University from 29th to 41st, the Indian Institute of Science from 115th to 225th, the University of Tokyo from from 23rd to 28th, an the University of Hong Kong from 21st to 26th.

In contrast, the University of British Columbia, McGill University, the University of  Toronto, the University of Melbourne, the University of Sydney, the University of Cape Town, Witwatersrand University, Trinity College Dublin, University College Dublin, University of California Berkeley and UCLA are climbing the ladders. For the moment at any rate.

 

World Economic Forum Declares that Africa is Rising

The World Economic Forum (WEF),  an organization of the global economic elite, has published a report by Phil Baty, currently Chief Global Affairs Officer of Times Higher Education (THE), that proclaims that African universities are "surging in the world rankings" and that this is a highly positive development. This is an irresponsible claim that has scant relationship to reality. 

The report refers to a claim in 2012 by Max Price, then Vice-Chancellor of the University of Cape Town, that African universities needed to rise to the challenge of global university rankings. According to Baty, African universities are now successfully competing in the rankings game and rising to the top.

And just what is the evidence for this? Well, in 2012 there were four African universities in the THE World University Rankings, In 2022 there were 97. An impressive and remarkable achievement indeed if we are talking about same  rankings. But they are not the same.

In 2012 the THE rankings consisted of 402 universities. By 2022 they had expanded to 2,345 including "reporters", of which 1500+ were formally ranked. It would be truly amazing if any region had failed to improve its representation.

The real comparison, of course, is with the numbers in the top 400 in both years and here there is no sign of any African surging. In 2012 there were three South African universities in the top 400. The University of the Witwatersrand and Stellenbosch University were in the 251-275 band and in the 2022-2023 rankings they were in the 251-375.  The University of Cape Town was 103rd in 2012 and 160th in 2022-2023, which might be cause for concern if this was a ranking that had a rigorous and stable methodology but that is not the case for THE. 

The fourth African university in the top 400 in 2012 was the University of Alexandria in the 301-350 band. By 2022 it had dropped to the 801-1000 band. This was a university on a downward spiral from its magical moment of ranking glory in 2010 when it was ranked 147th in the world overall and 4th in the world for citations as a result of a spurious affiliation claim by a serial self-publisher and  self-citer, who was involved in a libel case with Nature.

By 2022-2023 another African university, the University of Cape Coast in Ghana, had entered the top 400. This was not a testimony to any kind of achievement. It was simply the result of the university taking part in the Gates-funded Global Burden of Disease Study (GBDS). Because of a flawed methodology it is possible for a university with a few papers in the study, which typically have hundreds of authors and citations, and a small number of total publications to rack up scores of 80, 90, or even 100  for this indicator. 

There is then no evidence of a surge of any kind, not even a bit of a trickle.

That brings us to the assertion that Nigeria has, followed by Egypt, posted the biggest gains in the citations indicator, which purports to measure research impact or research quality or something, and has therefore achieved excellent progress. 

THE is being entirely too modest here. It could have used the indicator to celebrate the extraordinary accomplishment of a range of African institutions and countries that have surged in the rankings with 90 + scores for citations, an incredible accomplishment that contrasts with very low scores for Research, which includes publications, expenditure and reputation. In fact, if this indicator is taken seriously a number of African universities have now outpaced reputable research universities in North America and China. 

These research influencers, according to THE, include Jimma University, Ethiopia, Damietta University, Egypt, Muhimbili University of Health and Allied Sciences, Tanzania, Aswan University, Egypt, University of Lagos, Nigeria, the University of Zambia, and Kafrelsheikh University, Egypt.

Again, this has nothing to do with excellence or teamwork or transformative practices or any other current managerial shibboleth. It is largely the result of contributing a single researcher or a few to the hundreds of "authors" of GBDS papers in The Lancet and other prestigious journals and collecting credit for thousands of citations.

The cruellest aspect of this is that THE have announced that they are finally getting around to a partial revamping of the world rankings methodology this year. If THE do go ahead it is very likely -- not certainly because the whole process is so complex and opaque -- that these universities will go tumbling down the rankings and we shall probably see leaders across the continent under fire for their gross incompetence.

It is strange that an organisation that supposedly represents the best minds of the corporate world should adopt THE as the sole arbiter of African excellence. It is not the only global ranking and in fact it is probably the worst for Africa. It emphasises income, assessed by three separate indicators, self-submitted data which diverts the unacknowledged  labour of talented and motivated faculty, and reputation, and privileges postgraduate programmes. 

Perhaps also, at the risk of committing heresy in the first degree, the quality of higher education should not be Africa's highest priority. The latest edition of the Progress in International Reading Literacy Study (PIRLS) shows that Morocco, Egypt, and South Africa do very poorly with regard to fourth grade literacy. For South Africa and Morocco, the situation revealed by the 2019 Trends in Mathematics and Science (TIMSS) was little better, although they did come out ahead of Pakistan and the Philippines. Surely, this is as crucial for the future of Africa as the funding of doctoral programmes.