Should New Zealand Worry about the Rankings?

The Ministry of Education in New Zealand has just published a report by Warren Smart on the performance of the country's universities in the three best known international rankings. The report, which is unusually detailed and insightful, suggests that the eight universities -- Auckland, Otago, Canterbury, Victoria University of Wellington, Massey, Waikato, Auckland University of Technology and Lincoln  --  have a mixed record with regard to the Shanghai rankings and the Times Higher Education (THE) -- Thomson Reuters World University Rankings. Some are falling, some are stable and some are rising.

But things are a bit different when it comes to the QS World University Rankings. There the report finds a steady and general decline both overall and on nearly all of the component indicators. According to the New Zealand Herald this means that New Zealand is losing the race against Asia.

However, looking at the indicators one by one it is difficult to see any consistent and pervasive decline, whether absolute or relative.

Academic Survey

It is true that scores for the academic survey fell between 2007 and 2013 but one reason for this could be that the percentage of responses from New Zealand fell dramatically from 4.1% in 2007 to 1.2% in 2013 (see University Ranking Watch 20th February). This probably reflects the shift from a survey based on the subscription lists of World Scientific, a Singapore- based academic publishing company, to one with several sources, including a sign up facility.

Employer survey

In 2011 QS reported that there had been an enthusiastic response to the employer opinion survey from Latin America and it was found necessary to cap the scores of several universities where there had been a disproportionate response. One consequence of this was that the overall mean for this indicator rose dramatically so that universities received much lower scores in that year for the same number of responses. QS seems to have rectified the situation so that scores for New Zealand universities -- and many others -- recovered to some extent  in 2012 and 2013.

Citations per faculty and faculty student ratio

From 2007 to 2010 or 2011 scores fell for the citations per faculty indicator but have risen since then. The report notes that "the recent improvement in the citations per faculty score by New Zealand universities had not been matched by an increase in their academic reputations score, despite the academic reputation survey being focused on perceptions of research performance."

This apparent contradiction might be reconciled by the declining number of survey respondents from New Zealand noted above. Also, we should not forget the number on the bottom. A fall in the recorded number of faculty could have the same result as an increase in citations. It is interesting that  while the score for faculty student ratio for five  universities -- Auckland , Canterbury, Otago, Victoria University of Wellington and Waikato -- went down from 2010 to 2012, the score for citations per faculty went up. Both changes could result from an a decline in the number of faculty submitted by universities or recorded by QS. In only one case, Massey, did both scores rise. There was insufficient data for the other two universities.

International Faculty and International Students

The scores for international faculty have always been high and are likely to remain so. The scores for international students have been slipping but this indicator counts for only 5% of the total weighting.

New Zealand universities might benefit from looking at the process of submission of data to QS. Have they submitted lists of potential survey respondents? Are they aware of the definitions of faculty, students, international and so on? That might be more productive than worrying about a deep malaise in the tertiary sector.

And perhaps New Zealand salt producers could send out free packets every time the media have anxiety attacks about the rankings.