Forbes Magazine has an article by Willard Dix that ranks US ranking sites. The ranking is informal without specifying indicators but the author does give us an idea of what he thinks a good ranking should do.
Here are the top five of thirteen:
1. US News: America's Best Colleges
2. Money magazine: Best Colleges Ranking
3. Forbes: America's Top Colleges
4. Kiplinger's Best College Values
5. Washington Monthly: College Guide and Rankings.
Reading through the comments it is possible to get an idea of the criteria of a good ranking. Rankings should contain a lot of information, they should be comprehensive and include a large number of institutions, they should provide data that helps prospective students and stakeholders, they should be published for several years, if they use surveys they should have a lot of respondents, they should have face validity (a list with a "revolutionary algorithm" that puts non-Ivy places at the top is in 13th place).
Sunday, September 02, 2018
Friday, August 24, 2018
Why is Australia doing well in the Shanghai rankings?
I am feeling a bit embarrassed. In a recent post I wrote about the Shanghai Rankings (ARWU) being a bit boring (which is good) because university ranks usually do not change very much. But then I noticed that a couple of Australian universities did very well in the latest rankings. One of them, the Australian National University (ANU), has risen a spectacular (for ARWU) 31 places over last year. The Financial Review says that "[u]niversity scientific research has boosted the position of two Australian universities in a global ranking of higher education providers."
The ranking is ARWU and the rise in the ranking is linked to the economic contribution of Australian universities, especially those in the Group of Eight.
So how well did Australian universities do? The top performer, as in previous years, is the University of Melbourne, which went up a spot to 38th place. Two other universities went up a lot in a very un-Shanghainese way, ANU, already mentioned, from 69th to 38th place and the University of Sydney from to 83rd to 68th
The University of Queensland was unchanged in 55th place while Monash fell from 78th to 91st and the University of Western Australia from 91st to 93rd.
How did ANU and Sydney do it? The ANU scores for Nobel and Fields awards were unchanged. Publications were up a bit and papers in Nature and Science down a bit.
What made the difference was the score for highly cited researchers, derived from lists kept by Clarivate Analytics, which rose from 15.4 to 23.5, a difference of 8.1 or, after weighting, 1.62 points of the overall score. The difference in total scores between 2017 and 2018 was 1.9 so those highly cited researchers made up most of the difference.
In 2016 ANU had two researchers in the list, which was used for the 2017 rankings. One was also on the 2017 list, used in 2018. In 2017 there were six ANU highly cited researchers, one from the previous year and one who had moved from MIT. The other four were long serving ANU researchers.
Let's be clear. ANU has not been handing out unusual contracts or poaching from other institutions. It has grown its own researchers and should be congratulated.
But using an indicator where a single researcher can lift a top 100 university seven or eight places is an invitation to perverse consequences. ARWU should consider whether it is time to explore other measures of research impact.
The improved scores for the University of Sydney resulted from an increase between 2016 and 2017 in the number of articles published in the Science Citation Index Expanded and the Social Science Citation Index.
The ranking is ARWU and the rise in the ranking is linked to the economic contribution of Australian universities, especially those in the Group of Eight.
So how well did Australian universities do? The top performer, as in previous years, is the University of Melbourne, which went up a spot to 38th place. Two other universities went up a lot in a very un-Shanghainese way, ANU, already mentioned, from 69th to 38th place and the University of Sydney from to 83rd to 68th
The University of Queensland was unchanged in 55th place while Monash fell from 78th to 91st and the University of Western Australia from 91st to 93rd.
How did ANU and Sydney do it? The ANU scores for Nobel and Fields awards were unchanged. Publications were up a bit and papers in Nature and Science down a bit.
What made the difference was the score for highly cited researchers, derived from lists kept by Clarivate Analytics, which rose from 15.4 to 23.5, a difference of 8.1 or, after weighting, 1.62 points of the overall score. The difference in total scores between 2017 and 2018 was 1.9 so those highly cited researchers made up most of the difference.
In 2016 ANU had two researchers in the list, which was used for the 2017 rankings. One was also on the 2017 list, used in 2018. In 2017 there were six ANU highly cited researchers, one from the previous year and one who had moved from MIT. The other four were long serving ANU researchers.
Let's be clear. ANU has not been handing out unusual contracts or poaching from other institutions. It has grown its own researchers and should be congratulated.
But using an indicator where a single researcher can lift a top 100 university seven or eight places is an invitation to perverse consequences. ARWU should consider whether it is time to explore other measures of research impact.
The improved scores for the University of Sydney resulted from an increase between 2016 and 2017 in the number of articles published in the Science Citation Index Expanded and the Social Science Citation Index.
Saturday, August 18, 2018
Who Cares About University rankings?
A paper by Ludo Waltman and Nees Jan van Eck asks what users of the Leiden Ranking are interested in. There's some interesting stuff but for now I just want to look at where the users come from.
The top ten countries where visitors originate are:
1. USA
2. Australia
3. Netherlands
4. UK
5. Turkey
6. Iran
7. South Korea
8. France
9. Germany
10. Denmark.
The authors consider the number of visitors from Australia, Turkey, Iran and South Korea to be "quite remarkable."
Let's look at other signs of interest in rankings. Here are the top countries for respondents to the 2018 QS academic survey:
1. USA
2. UK
3. Malaysia
4= Australia
4= South Korea
4= Russia
7= Italy
7= Japan
9= Brazil
9= Canada
And here are the top ten countries for visitors to this blog:
1. USA
2. UK
3. Russia
4. France
5. Germany
6. Ukraine
7. Canada
8. Malaysia
9. Australia
10. Singapore.
The three countries on all three lists are UK, USA and Australia. The countries on two lists are South Korea, Russia, Malaysia, France, Germany and Canada.
https://www.cwts.nl/blog?article=n-r2s2a4&title=what-are-users-of-the-cwts-leiden-ranking-interested-in
http://rankingwatch.blogspot.com/2018/06/responses-to-qs.html
The top ten countries where visitors originate are:
1. USA
2. Australia
3. Netherlands
4. UK
5. Turkey
6. Iran
7. South Korea
8. France
9. Germany
10. Denmark.
The authors consider the number of visitors from Australia, Turkey, Iran and South Korea to be "quite remarkable."
Let's look at other signs of interest in rankings. Here are the top countries for respondents to the 2018 QS academic survey:
1. USA
2. UK
3. Malaysia
4= Australia
4= South Korea
4= Russia
7= Italy
7= Japan
9= Brazil
9= Canada
And here are the top ten countries for visitors to this blog:
1. USA
2. UK
3. Russia
4. France
5. Germany
6. Ukraine
7. Canada
8. Malaysia
9. Australia
10. Singapore.
The three countries on all three lists are UK, USA and Australia. The countries on two lists are South Korea, Russia, Malaysia, France, Germany and Canada.
https://www.cwts.nl/blog?article=n-r2s2a4&title=what-are-users-of-the-cwts-leiden-ranking-interested-in
http://rankingwatch.blogspot.com/2018/06/responses-to-qs.html
Subscribe to:
Posts (Atom)