The First IREG Audit 

QS is the first ranking organisation to get the seal of approval from the International Ranking Experts Group (IREG) for its World, Asian and Latin American rankings. 

The IREG audit process would appear on the surface to be quite rigorous. Take a look at the audit manual. There are a  number of criteria some of which sound quite daunting but are not really so. For example, Criterion 8 says:

 "If rankings are using composite indicators the weights of the individual indicators have to be published. Changes in weights over time should be limited and due to methodological or conception-related considerations."

Fair enough, but there is nothing about how weighting should be distributed across the indicators in the first place. Forty per cent for the academic survey in the QS rankings?

Some indicators are obvious -- providing a contact address. Others are so vague that they mean very little -- organisational measures that enhance the credibility of rankings.

The basic principle of the audit is that ranking organisations are given scores ranging from 1 (not sufficient/not applied) to 6 (distinguished) for the various criteria, with a double weighting for core criteria. The maximum score is 180 and 

"On the bases of the assessment scale described

above, the threshold for a positive audit decision will

be 60 per cent of the maximum total score. This

means the average score on the individual criteria

has to be slightly higher than “adequate”. In order

to establish the IREG Ranking Audit as a quality

label none of the core criteria must be assessed

with a score lower than three."

So a positive result could mean that an organisation is distinguished in everything. It could also mean that it is on average slightly higher than adequate. It would be interesting to know which applies to QS.

I do not know whether the auditors had any criticisms to make. If not it is difficult to see the point of the exercise. If they did it would be nice to know what they were.

QS are to be commended for submitting to the audit although it probably was not very searching but it still seems that the ranking world needs more and better monitoring and observation.