Congratulations to Malaysian Universities

Note: corrections have been made to an earlier draft. Some of the figures for 2006 have been changed.

No, I am not being sarcastic. Information just released by QS, the consultants who prepare the data for the THES-QS rankings, shows that Malaysian universities have improved quite significantly in some respects over the last year.

QS have now published detailed information on the top 400 universities in the 2007 rankings. This confirms what I had suspected, namely that there has been no real decline in the quality of any Malaysian university and that the apparent fall in the positions of Universiti Malaya (UM), Universiti Kebangsaan Malaysia (UKM), Universiti Sains Malaysia (USM) and Universiti Putra Malaysia (UPM) is largely the result of nothing more than a change in methodology.

There is no point in comparing the scores in 2006 and 207 for the various components because of changes in methods this year. Basically, the introduction of Z scores means that any such comparison has no meaning. Last year, for example, UM was given a score of 1 for citations and 14 this year. That does not mean anything since the mean score among the top 400 universities for citations was 9 in 2006 and 66 in 2007. To measure genuine change it is necessary to look at the relative positions of the universities.

I have calculated the position of the Malaysian universities on the various criteria in 2006 and 2007 . In both years I have looked only at the top 400 since information for the universities below the 400th place in 2007 is not currently available. .

In 2006, University Malay was 90th for the "peer review", 238th for recruiter rating, 274th for student faculty ratio , 245th for international faculty, 308th for international students and 342th for citations per faculty. UM managed to get into the top 200 because the score for "peer review" was given a much larger weighting than any other criterion.

This year , UM was 131st for the "peer review", 159th for recruiter rating, 261st for student faculty ratio , 146th for international faculty, 241st for international students and 377th for citations per faculty.

Thus, if QS are to be believed, UM has improved its standing with local employers, recruited more teaching staff, and increased the numbers of international faculty and students, all relative to other universities . It did slightly worse on citations per faculty.

The only serious blemish was that UM did rather worse on the "peer review", perhaps because as QS has suggested, respondents were no longer allowed to vote for their own institutions.

So, how could UM suffer such a catastrophic fall?

The answer lies in the in the use of Z scores. To summarise a Z score is constructed when the population mean is subtracted from the raw score, divided by the standard deviation and then normalised,.

The effect of this is to squash scores together at the top and not at the bottom as was previously the case. To go back to the scores for citations, in 2006 UM got a score of 1 for citations, which was quite a bit below average. In 2007, because in the introduction of Z scores, the average was much higher. So UM got 1 for citations in 2006 and Peking ("Beijing" then) got 2. This year UM got 14 and Peking got 53. So Peking got an extra 39 points instead of one.

The switch from ISI to Scopus may also had had some effect but probably not all that much.

Similarly, we find that UPM improved its position for two criteria and USM and UKM for 3 each. All suffered a decline on the "peer review".

UM's fall in the "peer review" section did not make a dramatic difference. Had UM remained in 90th place it would have made a difference of only ten 10 points, 76 instead of 66, for that section.

UM 's supposed tumble happened solely because universities that are doing a bit more research are now getting a lot more points than before.

There has been no decline. Maybe Malaysian universities are not improving fast enough but that is quite a different thing from what the rankings appear to show and what is causing so much anxiety among Malaysian commentators.