This is an updating and revision of a post from a few days ago
There appears to be no end to the craze for university rankings.
The media in many parts of the world show almost as much interest in global
university rankings as in the Olympics or the World Cup. They are now used to
set requirements for immigration, chose research collaborators, external
examiners, international partners and for marketing, public relations, and
recruitment.
Pakistan has not escaped the craze although it was perhaps a
bit slower than some other places. Recently, we have seen headlines announcing
that ten Pakistani universities are included in the latest Times Higher Education (THE) Asian rankings and highlighting the
achievement of Quaid-i-Azam University (QAU) in Islamabad reaching the top 100.
Rankings are unavoidable and sometimes they have beneficial
results. The first publication of the research-based Shanghai rankings in 2003,
for example, was a salutary shock to continental European universities and a
clear demonstration of how far China had to go to catch up with the West in the
natural sciences. But rankings do need to be treated with caution especially
when ranking metrics are badly and obviously flawed.
THE note that there are now ten Pakistani universities in the
Asian rankings and one, QAU, in 79th place, which would appear to be
evidence of academic progress.
Unfortunately, Pakistani universities, especially QAU, do very
much better in the THE rankings than in others. QAU is in the 401-500 band in
the THE world rankings, which use the same indicators as the Asian rankings.
But in the QS World University Rankings it is in the 650-700 band. It does not
even get into the 800 ranked universities In the Shanghai rankings, the 903 in
the Leiden Ranking, or the 763 in the Russian Round University Rankings. In the
University Ranking by Academic Performance, published in Ankara, it is 605th,
in the Center for World University Rankings list 870th.
How can we explain QAU’s success in the THE world and Asian rankings,
one that is so much greater than any other ranking? It is in large part the
result of a flawed methodology.
Take a look at the scores that QAU got in the THE rankings.
In all cases the top scoring university gets 100.
For Teaching, combining five indicators, it was 25.7 which is
not very good. For international outlook it was 42.1. Since QAU has very few
international staff or students this mediocre score is very probably the result
of a high score for international collaboration.
For research income from industry it was 31.8. This is
probably an estimate since exactly the same score is given for four other
Pakistani universities.
Now we come to something very odd. QAU’s research score was
1.3. It was the lowest of the 350 universities in the Asian rankings, very much
lower than the next worse, Ibaraki University in Japan with 6.6. The research score is composed of research
reputation, publications per faculty and research income per faculty. This
probably means that QAU’s score for research reputation was zero or close to
zero.
In contrast, QAU’s score of 81.2 for research impact measured
by citations is among the best in Asia. Indeed, in this respect it would appear
to be truly world class with a better score than Monash University, the Chinese
University of Hong Kong, the University of Bologna or the University of
Nottingham.
How is it being possible that QAU could be 7th in
Asia for research impact but 350th for research?
The answer is that THE’s research impact indicator is extremely
misleading. It does not simply measure the number of citations but the number
of citations in over 300 fields, five years of publication and up to six years
of citations. This means that a few highly cited papers in a strategic
discipline at a strategic time can have a disproportionate effect on the impact
score especially if the total number of papers is low.
Added to this is THE’s regional modification which means that
the citation impact score of a university is divided by the square root of the
score of the whole country in which they university is located. That means that
the score of universities in the top scoring country remain the same but that of
all the others goes up, the worse the country the bigger the increase. The
effect of this is to give a big boost to countries like Pakistan. THE used to apply this bonus to all of the citations indicator
but now only to 50%.
Then we have to consider how THE deals with mega-papers
mainly in physics and medicine, those with hundred even thousands of authors
and hundreds and thousands of citations.
Until the world rankings of 2015-16 THE treated every single
author of such papers as though he or she were the only author of the papers.
Then they stopped counting citations to these papers and then in 2016-17 they
awarded each institution a minimum 5% for citations.
The effect of the citations metric has been to make a mockery of
the THE Asian and world rankings. A succession of unlikely places has been propelled
to the top of the indicator because of contributions to mega-papers or because
of a few or even a single prolific author combined with a low overall number of
papers. We have seen Alexandria University, Anglia Ruskin University,
Moscow State Engineering Physics Institute, Tokyo Metropolitan University rise
to the top of this indicator. In last year’s Asian rankings, Veltech University
in India appeared to be first for research impact.
QAU has been involved in the Large Hadron Collider (LHC) project,
which produces papers with hundreds or thousands of authors and hundreds or
thousands of citations, and has provided authors for several papers. One 2012
paper derived from this project received 4094 citations so that QU would be
credited with 205 citations just for this paper.
In addition to this QAU employs an extremely productive
mathematician, Tasawar Hayat, who is among the world’s elite of researchers in
Clarivate Analytics list of Highly Cited Researchers where his primary
affiliation is King Abdulaziz University in Saudi Arabia and QAU is his
secondary affiliation. Professor Hayat is extremely prolific: in 2017 alone, he
was author or co-author of 384 scientific documents, articles, reviews, notes
and so on.
There is nothing wrong with QAU taking part in the LHC
project and I am unable to comment on the quality of his research. It
should, however, be understood that if Professor Hayat left QAU or QAU
withdrew from the LHC project or THE changed its methodology then QAU could
suffer a dramatic fall in the rankings similar to those suffered by some
Japanese, Turkish or Korean universities in recent years. This is an
achievement built on desperately weak foundations.
It would be very unwise to use these rankings as evidence for
the excellence of QAU or any other university.
