Rankings Uproar in Singapore

Singapore has been doing very well in the global university rankings lately. In the recent QS world rankings the National University of Singapore (NUS) and Nanyang Technological University (NTU) are 11th and 12th respectively and 23rd and 51st in the Times Higher Education  (THE) rankings.

Their performance in other rankings is good but not so remarkable. In the Shanghai rankings NUS is 85th and NTU 96th. In the Round University Rankings, which combine teaching and research indicators, they are 50th and 73rd. In the CWUR rankings, which attempt to measure student and faculty, they are 103rd and 173rd. In the CWTS Leiden Ranking publications count they are 34th and 66th.

The universities have performed well across a broad range of indicators and in nearly all of the international rankings. They do, however, perform much better  in the QS and THE rankings than in others. 

There are now at least 45 international rankings of various kinds, including global, regional, subject, system and business school rankings. See the IREG Inventory of International Rankings which already needs updating.  Singapore usually ignores these except for the QS and the THE world and regional rankings. The NUS website proudly displays the ranks in the QS and THE rankings and mentions Reuters Top 75 Asian Innovative Universities but there is nothing about the others.

Singapore's success is the the result of a genuine drive for excellence in research powered by a lot of money and rigorous selection but there also seems to be a careful and perhaps excessive cultivation of links with the UK rankers.

A few years ago, for example, World Scientific Publishing, which used to provide a database for the THES-QS academic survey, published a book co-authored by a NTU academic on how to succeed in the IELTS exam. It was entitled Top the IELTS: Opening the Gates to Top QS-ranked Universities.

There have been complaints that Singapore has become obsessed with rankings, that local researchers and teachers are marginalised, and that the humanities and social sciences are neglected.

An article in the Singapore magazine Today reported that there had been a high and damaging turnover of faculty  in the humanities and social sciences. They were oppressed by demands for publications and citations, key performance indicators which were often changed without warning, an emphasis on the hard sciences and a damaging pursuit of glory in the rankings. Several faculty have departed and this has supposedly had an adverse effect on departments in the humanities and social sciences.

I sympathise with scholars outsider the harder sciences who have to deal with bureaucrats unfamiliar with the publication and citation practices of various disciples. I recall once attending an interview  with an Asian university for a job teaching English where a panel of engineers and administrators wanted to know why my publications were so few. First on the list was a book of 214 pages which would be the equivalent of 20 papers in the natural sciences. It was not co-authored which would make it the equivalent in bulk of about eighty natural science papers with four authors apiece. Next was an article that was one of the most frequently cited paper in the field of genre analysis. But this was ignored. Numbers were the only thing that mattered.

But it seems that the two leading Singapore universities have not in fact neglected the disciplines outside the STEM subjects. In the QS arts and humanities ranking NUS is18th in the world and NTU 61st. In THE's they are 32nd and in the 101-125 band. 

It is also not entirely correct to suggest that the rankings generally discriminate against the social sciences and humanities. Both QS and THE now use normalisation to make sure that citation and publication counts and other metrics give due weight to those disciplines. It is certainly true that the Shanghai rankings do not count scholarship in the humanities  but they do not seem to get much publicity in Singapore.

The big problem with Singapore's  approach to rankings is that it is too concerned with the QS rankings with their emphasis on reputation surveys and international orientation and the THE rankings which also have reputation and international indicators and three different measures of income. This has resulted in ranking successes that even Singaporeans found difficult to believe. Does NTU really have an academic reputation greater than that of Johns Hopkins, Duke, and King's College London? Meanwhile other rankings that are more stable and reliable are simply ignored.

The Today article has been withdrawn apparently for legal reasons. There may be genuine concerns about defamation but it seems that that someone is a bit heavy handed. This may be self defeating since the dissident academics are unlikely to get very much public sympathy. One complained that his door had been defaced while he was on sabbatical. Another took to his bed for days at a time because of the frustration of dealing with the bureaucracy.

There has been more debate since. Linda Lim and Pang Eng Fong, emeriti of Michigan State University and Singapore Management University, argue in the South China Morning Post that the emphasis on rankings is damaging to Singapore because it discourages academics from doing research that is relevant to social policy. They argue that citations are a key metric in the rankings and that the top journals favor research that is of theoretical significance in STEM subjects rather than local research in the humanities and social sciences. 

This seems exaggerated. Citations count for 30% of the THE rankings, which is probably too much and 30% of QS's. Both of then, and other rankings, now have processes that reduce the
he bias to the natural sciences in citations, publications and reputation surveys. In fact QS have claimed that the weighting given to its academic reputation indicator was precisely to give a level playing field to the humanities and social sciences.

They refer to Teo You Yenn a researcher at NTU who has written a book for a commercial publisher that has received a lot of attention but is unlikely to advance her career.

Dr Teo, however, is an Associate Professor and has a very respectable publishing record of articles in leading journals on family, migration, inequality, gender, and poverty in Singapore. Some are highly cited, although not as much as papers in medical research or particle physics. It seems that a focus on elite journals and rankings has done nothing to stop research on social policy issues.

The state and the universities are unlikely to be swayed from their current policy. It would, however, be advisable for them to think about their focus on the QS and THE rankings. Reputation, financial and international indicators are the backbone of Singapore's ranking success. But they can be easily emulated by other countries with supportive governments and the help of benchmarking and reputation management schemes. 

Are Turkish Universities Declining? More misuse of Rankings

Sorry to get repetitive.

Another article has appeared offering the Times Higher Education  (THE) world rankings as evidence for the decline of national universities.

Matin Gurcan in Al-Monitor argues that Turkish universities are in decline for several reasons, including uncontrolled boutique private universities, excessive government control, academic purges, lack of training for research staff and rampant nepotism.

We have been here before. But are Turkish universities really in decline?

The main evidence offered is that there are fewer Turkish universities at the higher levels of the THE rankings. The other rankings that are now available are ignored.

It is typical of the current state of higher education journalism that many commentators seem unaware that there are now many university rankings and that some of them are as valid and accurate as THE's if not better. The ascendancy of the THE is largely a creation of a lazy and compliant media.

Turkish universities have certainly fallen in the THE rankings.

In 2014 there were six Turkish universities in the world's top 500 and four in the top 200. Leading the pack was Middle East Technical University (METU) in 85th place, up from the 221-250 band in 2013

A year later there were four in the top 500 and none in the top 200. METU was in the 500-600 band.

Nepotism, purges, lack of training were not the cause. They were as relevant here as goodness was to Mae West's diamonds. What happened was that in 2015 THE made a number of changes to the methodology of its flagship citations indicator. The country adjustment which favoured universities in countries with low citation counts was reduced. There was a switch from Web of Science to Scopus as  the data source. Citations to mega-papers such as those emanating from the CERN projects, with thousands of contributors and thousands were no longer counted.

Some Turkish universities were heavily over-invested in the CERN project, which took them to an unusually high position in 2014. In 2015 they went crashing down the THE rankings largely as a result of the methodological adjustments. 

Other rankings such as URAP and National Taiwan University show that Turkish universities, especially METU, have declined but not nearly as much or as quickly as the THE rankings appear to show. 

In the Round University Rankings there were seven Turkish universities in the top 500 in 2014, six in 2015, and seven in 2018, METU was 375th in 2014, 308th in 2015, and  437th in 2018: a significantly decline but much less than the one recorded by THE.

Meanwhile the US News Best Global Universities rankings show three Turkish universities, including METU, in the top 500.

I do not dispute that Turkish universities have problems or the significance of the trends mentioned by Matin Gurcan.  The evidence of the rankings is that they are declining at least in comparison with other universities especially in Asia. The THE world rankings are not, however, a good source of evidence.