Monday, June 02, 2014

What should India do about the rankings?

India seems to be suffering from ranking fever. This is a serious problem that periodically sweeps across countries with the national media echoing statements from university heads and bureaucrats about being in the top one hundred or two hundred of something in the next few years or passionate claims that rankings do not reflect the needs of local society or that the uniquely transformative features of this or that institution -- how dare they ignore our sensitivity training or sustainability programs! --  are not recognised by the rankers.

There is  now a lot of debate about which is the best university in India and also about why Indian institutions, especially the sometimes lauded Indian Institutes of Technology (IITs), have a modest impact on the international rankings.

So what do the various rankings say about the quality of Indian universities (counting the IITs and other Institutes)? Starting with Webometrics, which measures the Internet presence of universities,  first place in India goes to IIT Bombay, 517th in the world, followed by IIT Madras, The Indian Institute of Science (IISc) in Bangalore, IIT Kanpur and the University of Delhi.

Moving on to research based rankings, only one Indian university is ranked in  Shanghai Jiao Tong University's Academic Ranking of World Universities (ARWU) top 500, and that is the IISc in the 301-400 band.

Scimago Institutional Rankings in their 2013 default list of number of publications, also puts IISc in first place in India, followed by IIT Kharagur, IIT Delhi, the University of Delhi, and IIT Madras.

The Leiden Ranking has the IISc in first place for number of publications although IIT Roorkee is first for publications in high quality journals.

Looking at the research-only rankings then, the best bet for top place would be the IISc which is ranked first in India by ARWU, Scimago, and, for number of publications, by Leiden Ranking, although for quality of research the IITs at Roorkee, Delhi and Guwahati perform comparatively well.

Moving on to rankings that attempt to assess factors other than research, we find that in the most recent QS World and Asian University Rankings first place in India goes to IIT Delhi with IIT Bombay second and IIT Kanpur third.

Last year's Times Higher Education world rankings produced an unusual result. Panjab University (PU) was ranked in the 226-250 band well ahead of the IITs Delhi, Kanpur, Kharagpur and Roorkee in the 350 - 400. In this case, Panjab university's feat was entirely due to its massive score for citations, 84.7 compared to IIT Delhi's 38.5, a score that was in stark contrast to a very poor 14 for research.

The main reason for PU's whopping score for citations appears to be that a few of its physicists are involved in the Large Hadron Collider project, which involves more than 2000 physicists in more than 150 research centers and 37 countries and consequently produces a huge number of citations. PU gets the credit for all of those citations even though its contribution to the cited papers is extremely small.

This only works because the overall number of papers produced is low. Hundreds or even thousands of citations are of little incremental value if they are spread out over thousands or tens of thousands of papers.

It would be unwise for other Indian universities to emulate PU's approach to get into the THE rankings. For one thing they would have to keep total publications low. For another, they may find that highly cited researchers might be a tempting target for universities in the US or Australia. And it does not work for any other ranking.

It is noticeable that The IISc is not included in the QS or THE rankings, presumably as a result of its own choice.

Should India's universities try to improve their ranking performance? Perhaps, but it would be better if they focused on improving their research performance, admissions policies, administration and selection processes. And here there is a much bigger problem for India, the utterly dismal performance of the country's school system.

In 2009, students from Tamil Nadu and Himachel Pradesh, which do better than the Indian average on social and economic development measures, took the PISA test. They were just ahead of the lowest ranked Kirgystan.

Riaz Haq writes:

"In Tamil Nadu, only 17% of students were estimated to possess proficiency in reading that is at or above the baseline needed to be effective and productive in life. In Himachal Pradesh, this level is 11%. “This compares to 81% of students performing at or above the baseline level in reading in the OECD countries, on an average,” said the study. 
The average Indian child taking part in PISA2009+ is 40 to 50 points behind the worst students in the economic superstars. Even the best performers in Tamil Nadu and Himachal Pradesh - the top 5 percent who India will need in science and technology to complete globally - were almost 100 points behind the average child in Singapore and 83 points behind the average Korean - and a staggering 250 points behind the best in the best.
The average child in HP & TN is right at the level of the worst OECD or American students (only 1.5 or 7.5 points ahead). Contrary to President Obama's oft-expressed concerns about American students ability to compete with their Indian counterparts, the average 15-year-old Indian placed in an American school would be among the weakest students in the classroom, says Lant Pritchett of Harvard University. Even the best TN/HP students are 24 points behind the average American 15 year old."

If this does not change, there is very little that anyone can do to improve the status of India's universities, apart from importing large numbers of Finnish, Chinese or Korean students, teachers and researchers.









Thursday, May 22, 2014

The Twitter Rankings

Coldlime, an Internet marketing company has produced rankings of British and American universities according to Twitter activity.

The five most influential universities in the UK are:

1.   Plymouth
2.   Glasgow
3.   Birmingham
4.   East Anglia
5.   Swansea

The top five in the US are:

1.   Texas at Austin
2.   Wisconsin at Madison
3.   Indiana at Bloomington
4.   Harvard
5.   Florida

Update on Alexandria University

I recently wrote that Alexandria University might be making a comeback. There is some more evidence that this might be the case. The Leiden Ranking now includes 750 universities and Alexandria is there. For most subjects and settings it does not do particularly well but there do seem to be some widely cited publications in Earth and Environmental Sciences plus, of course, a high score for citations in Maths, Computer Science and Engineering. There are not enough papers in Cognitive Sciences and Social Sciences to be ranked.

I received a comment on the previous Alexandria post from "Nadia" that appeared to praise Dr El Naschie and also made apparently defamatory remarks about a British judge and at least two Egyptian public figures. Since this blog has a policy of not publishing possibly libellous comments, it was not approved. However, if "Nadia" should wish to send a comment that is comprehensible and does not contain personal attacks, it will be published. It would help if she or he could verify her identity.

Sunday, May 11, 2014

Global Elite Threatened by New Rivals?

The Centre for Science and Technology Studies (CWTS) at Leiden University has produced its annual Leiden Ranking. I hope to write something more general in a couple of days but for the moment here is an elaboration on a brilliant headline by Julie Hare in The Australian.

UTS kicks Harvard's butt in ranking

Well, sort of.  CWTS provide data for nine different indicators and seven different subject groups and there are settings for size and fractional counting. In addition you can vary the number of papers required for inclusion. 

Try clicking on Medical Sciences and Impact Indicator PP (proportion of articles in the top 10% of journals), keeping the default settings for size-independence and fractional counting and making sure the threshold stays at 100 papers. First is Rockefeller University which had 305 papers in Medical Sciences over a four year period, of which a third were in the top 10% of journals. University of Technology Sydney (UTS) had 115 papers and 22 % of those are in the top 10% of journals, that is 6.325 (remember this is fractional counting) papers a year.

Poor Harvard is fifth with 11,958 papers in Medical Sciences of which 2,523.138 are in the top tenth. But that is only 21.1 % .

Change the threshold to 500 papers and UTS disappears. Stop using fractional counting and it goes down to 20th. Uncheck "calculate size-independent indicators" and it sinks to 438th.

There is nothing wrong with presenting data in this much detail: in fact it can be very valuable. There are very probably a few very able researchers at UTS and it is helpful that they have been uncovered. But it would not be a good idea if research grants were to flow to UTS or students to apply there because of a handful of excellent papers a year in one subject.

So, if you manipulate the settings and parameters, there is a good chance that all sorts of institutions will display a pocket of excellence or two. For citations in Cognitive Sciences the University of Puerto Rico (102 papers) is ahead of University College London. For citations in Earth and Environmental Sciences Universiti Sains Malaysia is ahead of Rice. For high quality papers in Maths, Computer Science and Engineering Universiti Malaya beats Yale and for citations Universiti Kebangsaan Malaysia is fourth in the world, although it disappears if the threshold is moved to 500 papers.

Again, this could be useful information but it would be dangerous to assume that these universities now present a major threat to Oxbridge and the Ivy League.

And yes, Alexandria University is there in 211th place for citations (check for size-independent calculations, check for fractional counting, set the threshold at 100 papers) for Maths, Computer Science and Engineering.