International Rankings Expert Group Conference in Kazakhstan

I have just returned from the International Rankings Expert Group’s fourth conference in Astana, Kazakstan. There were some positive developments at the conference but also a few disappointments.

Starting with the negative aspects, there seems to be a global trend to the proliferation of national rankings which are increasingly and unnecessarily detailed and which impose a serious burden on teachers and researchers.. A case in point is the new ranking produced for Kazakhstan which includes just about every variable imaginable from "the number of Dissertation Councils" to "the availabilty of medical centers, sport halls, preventoriums, recreation zones". Very few at the conference seemed aware of the backwash effect of the rankings boom as universities outside the top 500 create their own rankings or compete for irrelevant awards, medals or certificates. Drudges in the periphery of the world university system now face an endless round of form filling, office tidying, meetings, committees and professional development activities which make teaching difficult and genuine research, as opposed to research-like behaviour, close to impossible.

The European Union ranking project was presented in some detail but I suspect is going to make little impact since it appears largely concerned with making fine distinctions between the research capabilities of faculties and departments.

There was a presentation about the Lisbon project which proposed to ignore research altogether and measure teaching excellence. This is an interesting idea but it seems to miss two important points. One reason for emphasizing the measurement of research is that the qualities required for research, general cognitive ability, reading and writing skills, conscientiousness and interest in a subject also correlate to some extent with teaching ability, however that is measured. Also, the assumption that learning is dependent on teaching which in turn must be regulated by a centralized bureaucracy is surely false, at least for the more able students

Positive developments include a trend towards personalized rankings where consumers assign their own weighting to indicators. There is an interesting project under way in Taiwan.

Richard Vedder introduced a ranking that has the merit of being based largely on publicly accessible data. The basic idea is excellent but there are some issues to be dealt with. Using RateMyProfessors is not a bad way to assess the quality of teaching but to be really valid there needs to be some adjustment for the grades awarded by the instructor. Using the American Who’s Who is also potentially interesting – and could well be applied internationally -- but there are of course obvious issues of bias.

He also gave a presentation without using PowerPoint. I must remember that next time I fill in a form about my innovative teaching methodology.

One measure presented was to create an IREG seal of approval. The logo is ready. I am not sure though whether this is going to be effective.

Overall, the conference has strengthened my conviction that if ranking is to be done it should not be by administrators or businesses but by universities themselves.