More lamentation from Dublin

Rankings have become a major weapon in the struggle of universities around the world to get their fair share or what they think is their fair share of public money. The Times Higher Education (THE) world and regional rankings are especially useful in this regard. They have a well known brand name, occasionally confused with the "Times of London", and sponsor prestigious summits at which rankers, political leaders and university heads wallow together in a warm bath of mutual flattery.

In addition, the THE rankings are highly volatile with significant methodological changes in 2011, 2015 and 2016. Another source of instability is the growing number of ranked universities. The scores used for calculating the various indicators in these rankings are not raw but standardised scores derived from means and standard deviations. So if there is an influx of new universities then mean scores are likely to change and consequently the processed scores of those above or below the mean.

The THE rankings can be interpreted to provide useful arguments whatever happens. If Western universities rise that is a sign of authentic excellence but one that is threatened by reduced funding, restrictions on foreign students and researchers, and reputations sullied by xenophobic electorates. If they fall that means of course that those threats have materialised.

The QS rankings are also sometimes unstable, having made significant methodological changes in 2015 and giving a 50% weighting to very subjective reputation indicators.

Irish universities seem to be especially fond of using these rankings as a ploy to gain public favour and largess. In 2015 Ireland's top university, Trinity College Dublin (TCD), fell seven places in the QS world rankings and 22 places in THE's.

TCD announced of course that government cuts had a lot do with it. The Dean of Research said:

“Notwithstanding these combined achievements the cuts in funding and increased investments made by our global competition, continue to have a direct impact on the rankings. Trinity is battling against intense international competition, particularly from Asian universities and from certain European countries where governments are investing heavily in higher education. The continued reduction in government investment in Irish universities has impacted negatively on the international standing of our universities and our ability to compete in a global arena.”“Trinity’s top 100 position globally and top 30 in Europe is remarkable in the context of its reduced income. Trinity’s annual budget per academic is 45% lower than that of the average university in the world top 200.  It is to the credit of Trinity’s dedicated teaching and research staff that the University continues to maintain its global position against such challenges.”

“As a knowledge economy we need an excellent competitive education system.  Trinity remains a world leading research-intensive university and the knowledge and innovation created are critical for the economic development of Ireland.”

I pointed out in 2015 that TCD had been steadily rising in the Shanghai ARWU rankings since 2004, especially in the Publications indicator (papers in the Science Citation Index and the Social Science Citation Index) and PCP (productivity per capita, that is the combined indicator scores divided by the number of faculty). This year, TCD's publication score again went up very slightly from 31 to 31.1 and the PCP quite significantly from 19 to 20.8, compared to top scores of 100 for Harvard and Caltech respectively.

University College Dublin has also continued to do well in the Shanghai rankings with the publications score rising this year from 34.1 (27.3 in 2004) to 34.2 and PCP from 18.0 (8.1 in 2014) to 18.1.

The Shanghai rankings are famous for not counting the arts and humanities or trying to measure anything related to teaching. The RUR rankings from Russia are based on Thomson Reuters data, also used by THE until two years ago and they do include publications in the humanities and teaching-related metrics. They have 12 out of the 13 indicators in the THE world rankings, plus eight others, but with a sensible weighting, for example 8% instead of 30% for field normalised citations.

The RUR rankings show that TCD rose from 174th overall in 2010 to 102nd in 2016. (193rd to 67th for research).

University College Dublin  (UCD) rose from 213th overall to 195th (157th to 69th for research) although some Irish universities, NUI Galway, NUI Maynooth, University College Cork, and Dublin City University, have fallen.

Nonetheless TCD decided in March of this year to develop a rankings strategy  aimed at QS and THE with a Rankings Steering Group chaired by the Provost. The competence and knowledge displayed by such groups and committees often have little relationship to the status and salaries of its members and that appears to be the case for TCD.

It seems that there was a misplaced decimal point in the financial data submitted to THE for the 2016 rankings and that would have left TCD with a lower rating than it deserved and so it has withdrawn from the rankings until the error is corrected.

If TCD cannot find an administrator or a statistician to check things like that it really has no business asking for taxpayers' money. I suspect that decimal points are not misplaced -- or if they are it is to the right rather than the left --  in submissions for grants or subsidies.

This raises the question of whether the THE checking procedures are adequate. I was under the impression that if there was a change of 20% then red flags would start waving. For THE to allow a large change in reported income and therefore at least one, maybe two or three, income indicators sounds rather odd. What about that unique game changing audit?

Meanwhile UCD, 176th in the THE rankings last year, has dropped out of the top 200 altogether.

The  QS  rankings were also bad news for Ireland. Every university fell except for NUI Galway and there were none in the top 100.

But has there in fact been any real decline in the quality of TCD and UCD?

The evidence of RUR and the Shanghai rankings is that the two main universities are steadily improving or at least  holding their own, especially with regard to research. Possibly less highly regarded places like NUI Galway and NUI Maynooth are struggling but that could be fairly easy to remedy.

The Irish Universities Association issued a statement:

'The continued slide of the Irish Universities in the QS World University Rankings should be greeted with alarm. Strenuous efforts on the part of the universities has resulted in strong performance on some measures in the rankings such as those relating to research citations and internationalisation of the staff and student cohort. Unfortunately, this good work is being undermined by the negative impact of underfunding on key indicators such as the student:faculty ratio. The latter is highly influential in scoring in the QS rankings.

It would also appear likely that almost a decade of austerity is spilling over into the reputational component of the rankings, with consequent negative repercussions. IUA Chief Executive, Ned Costello said: “we can no longer hide from the corrosive effect which years of cutbacks are having on our higher education system. At a time when we are more dependent than ever on the talent of our people for our economic future, we simply must invest in our universities. An immediate injection of funding is required in the upcoming Budget and Estimates to fund more lecturers, deliver smaller group teaching and restore quality in our system.” '

The decline of TCD and and UCD in the QS and THE rankings cannot reasonably be attributed to any real deficiencies on the part of those universities. A decline in the number of lecturers would have a negative effect on the faculty student metric but would help indicators scaled for faculty size. The alleged decline is largely a consequence of methodological changes and adjustments, the instability resulting from the influx of new universities and growing ranking sophistication in other places.

It is a shame that researchers and scholars should collude with those rankings that show them in a bad light while ignoring more stable and less biased ones that show a continuing and genuine improvement especially in research.