In case you are wondering why the Indian media get so excited about the THE and QS rankings and not about those that are just as good or better such as Leiden Ranking, RUR or Nature Index, see this from the University Grants Commission.
Note that it says "any time" and that only the Big Three rankings count for getting Assistant Professor jobs.
"NEW DELHI: University Grants Commission (UGC) has come up with, UGC Regulations 2018, which exempts PhD candidates from having NET qualification for direct recruitment to Assistant Professor post. This new draft regulation is known as Minimum Qualifications for Appointment of Teachers and Other Academic Staff in Universities and Colleges and Measures for the Maintenance of Standards in Higher Education. Further the Commission has also listed 'Ph.D degree from a university/ institution with a ranking in top 500 in the World University ranking (at any time) by Quacquarelli Symonds (QS), the Times Higher Education (THE) and Academic Ranking of World Universities (ARWU) of the Shanghai Jiao Tong University (Shanghai),' as one of the criteria for Assistant Professor appointment."
Discussion and analysis of international university rankings and topics related to the quality of higher education. Anyone wishing to contact Richard Holmes without worrying about ending up in comments can go to rjholmes2000@yahoo.com
Saturday, February 10, 2018
Thursday, February 08, 2018
Playing the Rankings Game in Pakistan
This article by Pervez Hoodbhoy from October 2016 is worth reading:
"A recently released report by Thomson-Reuters, a Canada based multinational media firm, says, “In the last decade, Pakistan’s scientific research productivity has increased by more than 4 times, from approximately 2000 articles per year in 2006 to more than 9000 articles in 2015. During this time, the number of Highly Cited Papers (HCPs) featuring Pakistan based authors increased tenfold from 9 articles in 2006 to 98 in 2015.”
This puts Pakistan well ahead of Brazil, Russia, India, and China in terms of HCPs. As the reader surely knows, every citation is an acknowledgement by other researchers of important research or useful new findings. The more citations a researcher earns, the more impact he/she is supposed to have had upon that field. Research evaluations, through multiple pathways, count for 50-70 percent of a university’s ranking (if not more).
If Thomson-Reuters has it right, then Pakistanis should be overjoyed. India has been beaten hollow. Better still, two of the world’s supposedly most advanced countries–Russia and China–are way behind. This steroid propelled growth means Pakistan will overtake America in just a decade or two.
But just a little analysis shows something is amiss. Surely a four-fold increase in scientific productivity must have some obvious manifestations. Does one see science laboratories in Pakistani universities four times busier? Are there four times as many seminars presenting new results? Does one hear animated discussions on scientific topics four times more frequently?
Nothing’s visible. Academic activity on Pakistani campuses might be unchanged or perhaps even less today, but is certainly not higher than ten years ago. So where–and why–are the authors of the HCP’s hiding? Could it be that these hugely prolific researchers are too bashful to present their results in departmental seminars or public lectures? The answer is not too difficult to guess."
Should Pakistan Celebrate the Latest THE Asian Rankings?
This is an updating and revision of a post from a few days ago
There appears to be no end to the craze for university rankings.
The media in many parts of the world show almost as much interest in global
university rankings as in the Olympics or the World Cup. They are now used to
set requirements for immigration, chose research collaborators, external
examiners, international partners and for marketing, public relations, and
recruitment.
Pakistan has not escaped the craze although it was perhaps a
bit slower than some other places. Recently, we have seen headlines announcing
that ten Pakistani universities are included in the latest Times Higher Education (THE) Asian rankings and highlighting the
achievement of Quaid-i-Azam University (QAU) in Islamabad reaching the top 100.
Rankings are unavoidable and sometimes they have beneficial
results. The first publication of the research-based Shanghai rankings in 2003,
for example, was a salutary shock to continental European universities and a
clear demonstration of how far China had to go to catch up with the West in the
natural sciences. But rankings do need to be treated with caution especially
when ranking metrics are badly and obviously flawed.
THE note that there are now ten Pakistani universities in the
Asian rankings and one, QAU, in 79th place, which would appear to be
evidence of academic progress.
Unfortunately, Pakistani universities, especially QAU, do very
much better in the THE rankings than in others. QAU is in the 401-500 band in
the THE world rankings, which use the same indicators as the Asian rankings.
But in the QS World University Rankings it is in the 650-700 band. It does not
even get into the 800 ranked universities In the Shanghai rankings, the 903 in
the Leiden Ranking, or the 763 in the Russian Round University Rankings. In the
University Ranking by Academic Performance, published in Ankara, it is 605th,
in the Center for World University Rankings list 870th.
How can we explain QAU’s success in the THE world and Asian rankings,
one that is so much greater than any other ranking? It is in large part the
result of a flawed methodology.
Take a look at the scores that QAU got in the THE rankings.
In all cases the top scoring university gets 100.
For Teaching, combining five indicators, it was 25.7 which is
not very good. For international outlook it was 42.1. Since QAU has very few
international staff or students this mediocre score is very probably the result
of a high score for international collaboration.
For research income from industry it was 31.8. This is
probably an estimate since exactly the same score is given for four other
Pakistani universities.
Now we come to something very odd. QAU’s research score was
1.3. It was the lowest of the 350 universities in the Asian rankings, very much
lower than the next worse, Ibaraki University in Japan with 6.6. The research score is composed of research
reputation, publications per faculty and research income per faculty. This
probably means that QAU’s score for research reputation was zero or close to
zero.
In contrast, QAU’s score of 81.2 for research impact measured
by citations is among the best in Asia. Indeed, in this respect it would appear
to be truly world class with a better score than Monash University, the Chinese
University of Hong Kong, the University of Bologna or the University of
Nottingham.
How is it being possible that QAU could be 7th in
Asia for research impact but 350th for research?
The answer is that THE’s research impact indicator is extremely
misleading. It does not simply measure the number of citations but the number
of citations in over 300 fields, five years of publication and up to six years
of citations. This means that a few highly cited papers in a strategic
discipline at a strategic time can have a disproportionate effect on the impact
score especially if the total number of papers is low.
Added to this is THE’s regional modification which means that
the citation impact score of a university is divided by the square root of the
score of the whole country in which they university is located. That means that
the score of universities in the top scoring country remain the same but that of
all the others goes up, the worse the country the bigger the increase. The
effect of this is to give a big boost to countries like Pakistan. THE used to apply this bonus to all of the citations indicator
but now only to 50%.
Then we have to consider how THE deals with mega-papers
mainly in physics and medicine, those with hundred even thousands of authors
and hundreds and thousands of citations.
Until the world rankings of 2015-16 THE treated every single
author of such papers as though he or she were the only author of the papers.
Then they stopped counting citations to these papers and then in 2016-17 they
awarded each institution a minimum 5% for citations.
The effect of the citations metric has been to make a mockery of
the THE Asian and world rankings. A succession of unlikely places has been propelled
to the top of the indicator because of contributions to mega-papers or because
of a few or even a single prolific author combined with a low overall number of
papers. We have seen Alexandria University, Anglia Ruskin University,
Moscow State Engineering Physics Institute, Tokyo Metropolitan University rise
to the top of this indicator. In last year’s Asian rankings, Veltech University
in India appeared to be first for research impact.
QAU has been involved in the Large Hadron Collider (LHC) project,
which produces papers with hundreds or thousands of authors and hundreds or
thousands of citations, and has provided authors for several papers. One 2012
paper derived from this project received 4094 citations so that QU would be
credited with 205 citations just for this paper.
In addition to this QAU employs an extremely productive
mathematician, Tasawar Hayat, who is among the world’s elite of researchers in
Clarivate Analytics list of Highly Cited Researchers where his primary
affiliation is King Abdulaziz University in Saudi Arabia and QAU is his
secondary affiliation. Professor Hayat is extremely prolific: in 2017 alone, he
was author or co-author of 384 scientific documents, articles, reviews, notes
and so on.
There is nothing wrong with QAU taking part in the LHC
project and I am unable to comment on the quality of his research. It
should, however, be understood that if Professor Hayat left QAU or QAU
withdrew from the LHC project or THE changed its methodology then QAU could
suffer a dramatic fall in the rankings similar to those suffered by some
Japanese, Turkish or Korean universities in recent years. This is an
achievement built on desperately weak foundations.
It would be very unwise to use these rankings as evidence for
the excellence of QAU or any other university.
Subscribe to:
Posts (Atom)