Wednesday, September 19, 2012

New Canadian Research Rankings

Higher Education Strategy Associates recently published their Canadian Research Rankings, which are based on the award of grants and H-indexes. I am not sure about counting grants since it is likely that the skills needed to lobby for grants and those needed to actually do research are not always the same.

The rankings do not include medical research.

The top five for science and engineering are:

1.  University of British Columbia
2.  Montreal
3.  Toronto -- St. George
4.  Ottawa
5.  McGill

The top five for  social sciences and humanities are:

1.  University of British Columbia
2.  Mcgill
3.  Toronto -- St George
4.  Alberta
5.  Guelph


These rankings, like the Times Higher Education (THE) World University Rankings are based on field normalisation. In other words they do not simple count the number of grants and H-index scores but compare them with the average for the field. The rationale for this is that there are enormous differences between disciplines so that it would, for example, be unfair to compare a physicist who has won a grant of 10,000 dollars, which is below average for physics, with an education researcher who has won a similar award, which is above average for education. Equally, does it make sense to rank a physicist with an average H-index for physics well above a linguist with an average one for linguistics?

Here are the average grants for various fields:

biological engineering 84,327
physics  42,913
linguistics 13,147
history 6,417
education 5,733

Here are the average H-indexes for discipline clusters:

science  10.6
social sciences 5.2
humanities 2.3


HESA (and THE ) do have a point. But there are problems. One is that as we drill down to smaller units of analysis there is a greater risk of outliers. So a single large grant or a single much cited author in a field with few grants or citations could have a disproportionate impact.

The other is that field normalisation implies that all disciplines are equal. But is that in fact the case?




6 comments:

gutscheine zum ausdrucken said...

very good post

Bridget | Software Apps said...

Thanks for sharing a valuable post. For sure, this can guide people in learning more about the disciplines in which they are interested in.

Anonymous said...

Yes, you nailed the problem. For instance, the University of Rimouski was in the top ten in Canada for engineering and science, yet the University has no researchers in most science departments and an academic staff of under 400. The issue is that they have big research money in marine science.

The idea that Montreal is better at research than UofT st-george is laughable. They are not even comparable.

Anonymous said...

Also, it appears that they are only looking at grants from NSERC or SSHRC.they are excluding other sources of research income. This doesnt make any sense at all. Engineering departments and CS departments often get huge research money from non NSERC sources. They have effectively removed these alternate sources of research income. It is not at all surprising that Montreal and Ottawa do very well in these rankings. Montreal gets a lot of money from NSERC because it is a french institution while Ottawa also gets a lot of money because it is the capital's best University.

For instance, in mathematics, many of Canada's finest professors have prestigious american and international research grants such as the sloan fellowship. Why would these people get NSERC funding?

Anonymous said...

I have to add something else that makes no sense in this ranking. You cant just normalise everything by faculty count. Take two departments of similar quality, one twice as big as the other. Do you think NSERC will give twice as much money to the bigger one? Of course not. There is nonlinearity at play, and nobody in the rankings game seems to understand this.

Anonymous said...

Very awesome blog !! I couldnt have wrote this any better than you if I tried super hard hehe!! I like your style too!! it's very unique & refreshing…