Global Elite Threatened by New Rivals?

The Centre for Science and Technology Studies (CWTS) at Leiden University has produced its annual Leiden Ranking. I hope to write something more general in a couple of days but for the moment here is an elaboration on a brilliant headline by Julie Hare in The Australian.

UTS kicks Harvard's butt in ranking

Well, sort of.  CWTS provide data for nine different indicators and seven different subject groups and there are settings for size and fractional counting. In addition you can vary the number of papers required for inclusion. 

Try clicking on Medical Sciences and Impact Indicator PP (proportion of articles in the top 10% of journals), keeping the default settings for size-independence and fractional counting and making sure the threshold stays at 100 papers. First is Rockefeller University which had 305 papers in Medical Sciences over a four year period, of which a third were in the top 10% of journals. University of Technology Sydney (UTS) had 115 papers and 22 % of those are in the top 10% of journals, that is 6.325 (remember this is fractional counting) papers a year.

Poor Harvard is fifth with 11,958 papers in Medical Sciences of which 2,523.138 are in the top tenth. But that is only 21.1 % .

Change the threshold to 500 papers and UTS disappears. Stop using fractional counting and it goes down to 20th. Uncheck "calculate size-independent indicators" and it sinks to 438th.

There is nothing wrong with presenting data in this much detail: in fact it can be very valuable. There are very probably a few very able researchers at UTS and it is helpful that they have been uncovered. But it would not be a good idea if research grants were to flow to UTS or students to apply there because of a handful of excellent papers a year in one subject.

So, if you manipulate the settings and parameters, there is a good chance that all sorts of institutions will display a pocket of excellence or two. For citations in Cognitive Sciences the University of Puerto Rico (102 papers) is ahead of University College London. For citations in Earth and Environmental Sciences Universiti Sains Malaysia is ahead of Rice. For high quality papers in Maths, Computer Science and Engineering Universiti Malaya beats Yale and for citations Universiti Kebangsaan Malaysia is fourth in the world, although it disappears if the threshold is moved to 500 papers.

Again, this could be useful information but it would be dangerous to assume that these universities now present a major threat to Oxbridge and the Ivy League.

And yes, Alexandria University is there in 211th place for citations (check for size-independent calculations, check for fractional counting, set the threshold at 100 papers) for Maths, Computer Science and Engineering.