Asian University Rankings
QS Quacquarelli Symonds has come out with a ranking of the top 200 Asian universities. Here is the top ten.
1. University of Hong Kong
2. Chinese University of Hong Kong
3. University of Tokyo
4. Hong Kong University of Science and Technology
5. Kyoto University
6. Osaka University
7. Korean Advanced Institute of Science and Technology
8. Seoul National University
9. Tokyo Institute of Technology
10. National University of Singapore and Peking University
There are also rankings by disciplinary cluster and by indicator.
For every single disciplinary cluster, the University of Tokyo, not the University of Hong Kong is top. How strange.
For the indicators, the National University of Singapore is first for Employer Review and International Students, Tokyo University for Academic Peer Review, College of Medicine at Pochon Cha University (Korea) for faculty student ratio [I’m wondering about that as well], Gwangju Institute of Science and Technology for papers per faculty, Yokohama City University for Citations per Paper, Hong Kong University of Science and Technology for International Faculty, Kansai Gaidai for Inbound Exchange Students and City University of Hong Kong for Outbound Exchange Students.
These rankings seem to be a shrewd marketing move. Universities that have no chance of getting anywhere in the World University Rankings will now be able to boast that they came in the top 50 Asian universities for outbound exchange students or top 100 for citations per paper. A glance at the indicator rankings, for example, shows some Malaysian universities that one would not have thought had any chance of being in any sort of ranking. On the other hand, these rankings have been able to identify rising stars such as the Multi Media University.
There are two methodological innovations, both of which are questionable. They need to be discussed since this regional ranking could be a tryout for the global rankings. The first is the addition of two further measures of internationalization, inbound and outbound exchange students.
If internationalization is going to be a criterion, then having more measures might be a good idea. However, it is time to consider whether internationalization is actually a valid indicator of quality. Measures of internationalization do not correlate very well if at all with any other indicator and they also give an unfair advantage to the European Union and Hong Kong.
If we want to measure faculty quality, which internationalization supposedly underlies, a better method might be calculate the percentage of a random sample of teaching and research staff on university web pages who obtained degrees from the top 100 universities (on the Shanghai rankings?).
However, since QS get a lot of their bread and butter from facilitating students moving across national boundaries we are unlikely to see the end of this indicator.
The addition of number of inbound and outbound exchange students might also be very easily manipulated. If it were included in the world rankings it is likely that we will see universities setting up branch campuses a few miles away across some increasingly irrelevant frontier and then moving everybody there for their second year and calling them exchange students. So we might expect to see Queens University Belfast setting up a branch in Dundalk in the Irish Republic or the National University of Singapore in Johore in Malaysia and so on.
The other innovation is that research is measured by citations per paper, which measures the average impact of papers, and papers per faculty which measures the quantity of research in a very basic sense. This represents an improvement over the previous policy of using a single indicator. However, the problem remains that both are based on the Scopus database which aims to be as inclusive as possible. Scopus is an excellent research tool but inclusion in its database is an indicator of quality only in the broadest sense. To be credible, QS should consider finding some measure of research that measures genuine excellence.
These rankings have some surprises, the most noticeable and one lacking in face validity, is that the University of Hong Kong and not the University of Tokyo is the top university in Asia. Or perhaps this should not really be a surprise. Tokyo actually outperforms Hong Kong on all indicators except the internationalization ones and is ahead in all of the disciplinary rankings. Again, a lot of South Korean universities do very well.
It is good that QS are prepared to experiment with different indicators but the methodological innovations of these rankings do not seem to help very much.
Saturday, May 30, 2009
Wednesday, May 13, 2009
Tuesday, May 12, 2009
Don't They Teach Geography Any More?
The QS Asian University Rankings are now out. I hope to comment in a while. For the time being. I've noticed that in the "International Students Review" (I'm not sure that it's a review but never mind), Jawaharlal Nehru University is listed as being in South Korea? I'm wondering how long it will remain there?
The QS Asian University Rankings are now out. I hope to comment in a while. For the time being. I've noticed that in the "International Students Review" (I'm not sure that it's a review but never mind), Jawaharlal Nehru University is listed as being in South Korea? I'm wondering how long it will remain there?
Asian University Rankings
QS will shortly release their Top 100 Asian Universities Rankings. It seems from the bits and pieces released so far that Australia, the Pacific and South West Asia are not included. There appear to be two innovations -- a trial run for the global rankings? -- namely counting student exchanges and including citations per paper as a measure of quality of research.
QS will shortly release their Top 100 Asian Universities Rankings. It seems from the bits and pieces released so far that Australia, the Pacific and South West Asia are not included. There appear to be two innovations -- a trial run for the global rankings? -- namely counting student exchanges and including citations per paper as a measure of quality of research.
Monday, May 11, 2009
The Man with the Midas Touch
Those who are familiar with Malaysian gymnastics – admittedly not a large group -- will know who I am talking about.
There is a gymnastics coach in Malaysia who has achieved remarkable results with the women’s team of a very small state. For the last few years this state has won gold after gold at national competition. Hence the title bestowed by the Malaysian press. There does not seem to be anything special about the state – the men’s team has never done very much. Nor is there anything unusual about his training methods apart from their rigour and his intolerance for poor performance.
There must then be something about his selection methods. The state is so small that it does not have a gymnastics association and so this coach is free to select anyone he wants for training.
Recently, there was an opportunity to see just what these selection methods might be. The coach has now moved up to the national team and held a selection for young gymnasts, one of whom was my daughter, to train at the national sports centre.
The selection was quite revealing. The gymnasts were required to run, jump, do a bridge, do as many chin-ups as they could and a few simple exercises. Their height and weight were measured. There was no interest in their competition records, team spirit, motivation, leadership qualities or ability to respond to adversity.
At the end there was an interesting moment. The gymnasts were told to line up with their parents behind. There was much scurrying into position as people assumed they were being summoned for a group photograph, something without which no Malaysian event of any sort is complete. But after surveying the lineup, the coach said thank you and waved everyone away. What he was in fact doing was checking to see what the gymnasts would look like in a few years
So that was the secret of the man with the Midas touch. Assessment of basic physical skills and characteristics and reference to inherited traits. It was as though someone selected for elite universities by a simple test of general intelligence and a check on parental academic performance.
So here are two proposed experiments. Asian universities – and others – should scrap the proliferating complex of personality tests, language tests, co-curricular activities, interviews to test for leadership, politeness, sensitivity, appearance, interest and so on, profiles, course work and essays and just test for general cognitive ability.
The second experiment is that this coach and others should take heed of the global consensus and introduce personality tests, interviews and the whole paraphernalia of holistic assessment to choose future athletes.
I have a horrible feeling that the first will never happen but that one day the second will.
Those who are familiar with Malaysian gymnastics – admittedly not a large group -- will know who I am talking about.
There is a gymnastics coach in Malaysia who has achieved remarkable results with the women’s team of a very small state. For the last few years this state has won gold after gold at national competition. Hence the title bestowed by the Malaysian press. There does not seem to be anything special about the state – the men’s team has never done very much. Nor is there anything unusual about his training methods apart from their rigour and his intolerance for poor performance.
There must then be something about his selection methods. The state is so small that it does not have a gymnastics association and so this coach is free to select anyone he wants for training.
Recently, there was an opportunity to see just what these selection methods might be. The coach has now moved up to the national team and held a selection for young gymnasts, one of whom was my daughter, to train at the national sports centre.
The selection was quite revealing. The gymnasts were required to run, jump, do a bridge, do as many chin-ups as they could and a few simple exercises. Their height and weight were measured. There was no interest in their competition records, team spirit, motivation, leadership qualities or ability to respond to adversity.
At the end there was an interesting moment. The gymnasts were told to line up with their parents behind. There was much scurrying into position as people assumed they were being summoned for a group photograph, something without which no Malaysian event of any sort is complete. But after surveying the lineup, the coach said thank you and waved everyone away. What he was in fact doing was checking to see what the gymnasts would look like in a few years
So that was the secret of the man with the Midas touch. Assessment of basic physical skills and characteristics and reference to inherited traits. It was as though someone selected for elite universities by a simple test of general intelligence and a check on parental academic performance.
So here are two proposed experiments. Asian universities – and others – should scrap the proliferating complex of personality tests, language tests, co-curricular activities, interviews to test for leadership, politeness, sensitivity, appearance, interest and so on, profiles, course work and essays and just test for general cognitive ability.
The second experiment is that this coach and others should take heed of the global consensus and introduce personality tests, interviews and the whole paraphernalia of holistic assessment to choose future athletes.
I have a horrible feeling that the first will never happen but that one day the second will.
Comments on an Article by Andrew Oswald
I have already referred to an article by Andrew Oswald (14/12/2007) that contained some acerbic comments on claims to excellence by British universities derived from their showing on the THE-QS rankings.
He recently had an article in the Independent suggesting three measures that would allow universities to rise in the rankings. I will skip the first and third and just look at the second. He proposes that universities should be free to pay the market rate for highly rated researchers and offers Dartmouth College, which built up a top economics faculty by paying outstanding researchers appropriate salaries.
Dr Oswald is quite right but a lot of people are going to be depressed after reading his article. How can universities in Africa, Latin America and Asia get anywhere if they cannot afford to pay Ivy League level salaries? Is there anything that a university without a big pot of money can do?
The answer is that there is. In Moneyball Michael Lewis described how the Oakland As baseball team performed dramatically well even though they had only a comparatively small amount of money with which to buy players. They did it by simply ignoring the intuitions of talent scouts and looking at crude statistical data.
Basically, Oakland did the equivalent of a university scrapping search committees, interviews, personality tests, references and looking at the research done by applicants. Or choosing undergraduates by testing cognitive skills, literacy and numeracy rather than holistic assessment of leadership, communication skills, response to adversity, community involvement and so on.
At least one American university has done something like this. George Mason University has built up an excellent economics department by recruiting academics specializing in unfashionable fields that were undervalued by the academic marketplace.
Asian universities and others might consider systematically recruiting researchers whose personal characteristics and choice of unpopular research topics put them at a disadvantage when applying for academic positions. They might end up with a collection of unpleasant eccentrics but they might also see their ranking scores inching upwards.
I have already referred to an article by Andrew Oswald (14/12/2007) that contained some acerbic comments on claims to excellence by British universities derived from their showing on the THE-QS rankings.
He recently had an article in the Independent suggesting three measures that would allow universities to rise in the rankings. I will skip the first and third and just look at the second. He proposes that universities should be free to pay the market rate for highly rated researchers and offers Dartmouth College, which built up a top economics faculty by paying outstanding researchers appropriate salaries.
Dr Oswald is quite right but a lot of people are going to be depressed after reading his article. How can universities in Africa, Latin America and Asia get anywhere if they cannot afford to pay Ivy League level salaries? Is there anything that a university without a big pot of money can do?
The answer is that there is. In Moneyball Michael Lewis described how the Oakland As baseball team performed dramatically well even though they had only a comparatively small amount of money with which to buy players. They did it by simply ignoring the intuitions of talent scouts and looking at crude statistical data.
Basically, Oakland did the equivalent of a university scrapping search committees, interviews, personality tests, references and looking at the research done by applicants. Or choosing undergraduates by testing cognitive skills, literacy and numeracy rather than holistic assessment of leadership, communication skills, response to adversity, community involvement and so on.
At least one American university has done something like this. George Mason University has built up an excellent economics department by recruiting academics specializing in unfashionable fields that were undervalued by the academic marketplace.
Asian universities and others might consider systematically recruiting researchers whose personal characteristics and choice of unpopular research topics put them at a disadvantage when applying for academic positions. They might end up with a collection of unpleasant eccentrics but they might also see their ranking scores inching upwards.
Sunday, March 29, 2009
The Globalization of Cheating
There is a fascinating story in the Chronicle of Higher Education about the growth of the essay writing business. It seems that around the world there are shadowy companies employing dozens or hundreds of writers churning out essays for university students at every level.
There are interviews with some of the professional essay writers. Two of them are Americans with law degrees. Another is a Nigerian with a master's degree from the University of Lagos. Many others appear to be from the Philippines and India.
Customers of the essay mills include a doctoral student in aerospace engineering at MIT and graduate students at Northern Kentucky University, James Madison University and the University of Southern Mississippi.
Nobody seems to be asking what is wrong with selection for American and British universities when there are thousands of students who cannot do the academic work required while there are people without degrees who are able to produce acceptable work for relatively trivial wages. Some of the writers for the paper mills do have degrees. With their obvious and marketable research and writing skills shouldn't they be in doctoral programmes or academic appointments?
Is it possible that the trend towards holistic admissions in the US and the dumbing down of A-levels in the UK have something to do with it? If students are admitted to university on the basis of leadership, social skills displayed at interviews, participating in community service, overcoming adversity and the writing (by whom ?) of admission essays rather than the cognitive abilities and background knowledge necessary to do academic work then it would seem that the essay mill business is essential to keep the system going.
I have a suggestion for any university that wants to improve student and faculty quality within a short period. Find an essay mill operator, appoint him as admissions and recruitment officer and give scholarships to the essay writers who do not have degrees and faculty positions to those who do.
There is a fascinating story in the Chronicle of Higher Education about the growth of the essay writing business. It seems that around the world there are shadowy companies employing dozens or hundreds of writers churning out essays for university students at every level.
'The writers for essay mills are anonymous and often poorly paid. Some of them crank out 10 or more essays a week, hundreds over the course of a year. They earn anywhere from a few dollars to $40 per page, depending on the company and the subject. Some of the freelancers have graduate degrees and can write smooth, A-level prose. Others have no college degree and limited English skills.
James Robbins is one of the good ones. Mr. Robbins, now 30, started
working for essay mills to help pay his way through Lamar University, in Beaumont, Tex. He continued after graduation and, for a time, ran his own company under the name Mr. Essay. What he's discovered, after writing hundreds of academic papers, is that he has a knack for the form: He's fast, and his papers consistently earn high marks. "I can knock out 10 pages in an hour," he says. "Ten pages is nothing."
His most recent gig was for Essay Writers. His clients have included students from top colleges like the University of Pennsylvania, and he's written short freshman-comp papers along with longer, more sophisticated fare. Like all freelancers for Essay Writers, Mr. Robbins
logs in to a password-protected Web site that gives him access to the company's orders. If he finds an assignment that's to his liking, he clicks the "Take Order" button. "I took one on Christological topics in the second and third centuries," he remembers. "I didn't even know what that meant. I had to look it up on Wikipedia." '
There are interviews with some of the professional essay writers. Two of them are Americans with law degrees. Another is a Nigerian with a master's degree from the University of Lagos. Many others appear to be from the Philippines and India.
Customers of the essay mills include a doctoral student in aerospace engineering at MIT and graduate students at Northern Kentucky University, James Madison University and the University of Southern Mississippi.
Nobody seems to be asking what is wrong with selection for American and British universities when there are thousands of students who cannot do the academic work required while there are people without degrees who are able to produce acceptable work for relatively trivial wages. Some of the writers for the paper mills do have degrees. With their obvious and marketable research and writing skills shouldn't they be in doctoral programmes or academic appointments?
Is it possible that the trend towards holistic admissions in the US and the dumbing down of A-levels in the UK have something to do with it? If students are admitted to university on the basis of leadership, social skills displayed at interviews, participating in community service, overcoming adversity and the writing (by whom ?) of admission essays rather than the cognitive abilities and background knowledge necessary to do academic work then it would seem that the essay mill business is essential to keep the system going.
I have a suggestion for any university that wants to improve student and faculty quality within a short period. Find an essay mill operator, appoint him as admissions and recruitment officer and give scholarships to the essay writers who do not have degrees and faculty positions to those who do.
Sunday, March 08, 2009
Best Places for Postdocs
The Scientist has published its annual report on the best places for postdocs. A certain amount of scepticism is in order. There are dramatic changes in position from 2008 and 2009 -- for example, this year's number 1 in the US, the Whitehead Institute for Biomedical Research, was number 14 last year -- that suggest a certain amount of caution is needed.
Nonetheless, the results are interesting. First, in the US the tables are dominated by non-university institutions. Is it possible that politicisation and declining academic standards are beginning to have a noticeable effect on the quality of American university research?
Second, in the international category the top three positions are held by German, Danish and Dutch institutions. The only English university in the international top ten is York. What happened to Imperial College, Oxford and Cambridge?
The Scientist has published its annual report on the best places for postdocs. A certain amount of scepticism is in order. There are dramatic changes in position from 2008 and 2009 -- for example, this year's number 1 in the US, the Whitehead Institute for Biomedical Research, was number 14 last year -- that suggest a certain amount of caution is needed.
Nonetheless, the results are interesting. First, in the US the tables are dominated by non-university institutions. Is it possible that politicisation and declining academic standards are beginning to have a noticeable effect on the quality of American university research?
Second, in the international category the top three positions are held by German, Danish and Dutch institutions. The only English university in the international top ten is York. What happened to Imperial College, Oxford and Cambridge?
Thursday, March 05, 2009
Interesting New Blog
Go here.
International Education Blogs ~ A New Blogging Project by David Comp
International Education Blogs is a new blogging project I started today as an effort to bring all of the blogs on the web that touch on international education issues into one central location. Essentially it will be a blog roll with a few postings. Please bear with me as I learn the technology. This initial launch will have two different blog lists for you to review.The first list of blogs will be those of people who are blogging on the field/state of international education and related matters.The second list of blogs will be those of students and others who are currently blogging from a foreign location.
Wednesday, February 25, 2009
Semantic Shift Watch
Inside Higher Ed has a story about Sonoma State University that starts
Since when is a university administration whose senior members are from one ethnic group decidedly diverse?
Inside Higher Ed has a story about Sonoma State University that starts
A faculty report has stirred some racial tensions at Sonoma State University, following claims from its author that the institution’s
administration has deliberately targeted those from higher-income families as potential students for the past decade. In this process, the report claims that the university has become the “whitest” public institution in California, effectively preferring white students to minorities in an admission practice that it deems “reverse affirmative action.”
One aspect of Sonoma State that is decidedly diverse is the administration, where the president, provost and director of admissions – all criticized in the report – are Latino. The
professor who brought forth this report, however, is white.
Since when is a university administration whose senior members are from one ethnic group decidedly diverse?
Friday, February 20, 2009
A Note on the THE-QS Employer Review
Of all the components of the THE-QS World University Rankings the employer review is probably the least noticed. Universities in South East Asia and elsewhere, for example, are going to great lengths to recruit international students to boost their performance in the rankings but there seems to be no comparable effort to do better in the employer review.
It might be worth looking at the structure of the review for 2008. First of all, the distribution by industry (2008) seems very unrepresentative with a disproportionate number of respondents drawn from financial services and banking, consulting and professional services, manufacturing and engineering and IT and computer services in that order.
The distribution by country (2006-8) is even more skewed. Take a look at the list of employer review resonses by country from the QS topuniversities site. The UK and Australia between them have more respondents than the USA. Mexico has more than China, Greece more than Germany, Singapore more than Japan, Ireland more than France and Romania more than any country in the Middle East, including Israel, Turkey, Iran and the Gulf states.
Incidentally, since the banking sector has been so incompetent at choosing whom to lend money to, is it a good idea to allow it to have such a big say in evaluating universities?
United States
346
United Kingdom
269
Australia
178
Mexico
75
Netherlands
75
Singapore
74
Russia
69
India
64
Argentina
60
Greece
59
Germany
56
Hong Kong
50
Philippines
45
Ireland
41
Malaysia
38
Canada
37
Japan
37
France
36
New Zealand
36
South Korea
32
Italy
29
Chile
28
Spain
27
Venezuela
27
China
25
Denmark
23
Thailand
23
Switzerland
22
Belgium
19
South Africa
19
Ukraine
18
Taiwan
17
Czech Republic
16
Romania
15
Other
354
Of all the components of the THE-QS World University Rankings the employer review is probably the least noticed. Universities in South East Asia and elsewhere, for example, are going to great lengths to recruit international students to boost their performance in the rankings but there seems to be no comparable effort to do better in the employer review.
It might be worth looking at the structure of the review for 2008. First of all, the distribution by industry (2008) seems very unrepresentative with a disproportionate number of respondents drawn from financial services and banking, consulting and professional services, manufacturing and engineering and IT and computer services in that order.
The distribution by country (2006-8) is even more skewed. Take a look at the list of employer review resonses by country from the QS topuniversities site. The UK and Australia between them have more respondents than the USA. Mexico has more than China, Greece more than Germany, Singapore more than Japan, Ireland more than France and Romania more than any country in the Middle East, including Israel, Turkey, Iran and the Gulf states.
Incidentally, since the banking sector has been so incompetent at choosing whom to lend money to, is it a good idea to allow it to have such a big say in evaluating universities?
United States
346
United Kingdom
269
Australia
178
Mexico
75
Netherlands
75
Singapore
74
Russia
69
India
64
Argentina
60
Greece
59
Germany
56
Hong Kong
50
Philippines
45
Ireland
41
Malaysia
38
Canada
37
Japan
37
France
36
New Zealand
36
South Korea
32
Italy
29
Chile
28
Spain
27
Venezuela
27
China
25
Denmark
23
Thailand
23
Switzerland
22
Belgium
19
South Africa
19
Ukraine
18
Taiwan
17
Czech Republic
16
Romania
15
Other
354
Sunday, February 01, 2009
A Bit More About VU Amsterdam and the Previous Post
I have just remembered seeing this item at the bottom of the 2007 rankings on the QS site.
It seems then that in 2007 QS did not count medical faculty but they did in 2008, resulting in a rise from 40 to 84 in the student- faculty ratio score. Secondly, in 2007 they did not count research if the affiliation did not clearly identify VU Amsterdam but they did in 2008, producing a rise in the citations score from1 to 38.
I have just remembered seeing this item at the bottom of the 2007 rankings on the QS site.
Vrije Universiteit AMSTERDAM
The data supplied by VU Amsterdam did not include faculty numbers for the VU Medical Center. Using 2006 citations data as a benchmark, it appears that the mapping of the citations database did not return the expected number of papers and citations - perhaps due to a
volume of research being published under institution names not easily
identifiable as being part of VU Amsterdam. The resolution of these issues would certainly result in a higher ranking for the university and improved performance in the Peer and Recruiter Reviews would suggest that VU Amsterdam may have maintained its Top 200 position.
It seems then that in 2007 QS did not count medical faculty but they did in 2008, resulting in a rise from 40 to 84 in the student- faculty ratio score. Secondly, in 2007 they did not count research if the affiliation did not clearly identify VU Amsterdam but they did in 2008, producing a rise in the citations score from1 to 38.
Saturday, January 31, 2009
No Way But Up
Researchtrends, a newsletter published by Scopus, has an interesting item on the THE-QS World University Rankings. It suggests that the rankings can be used to assess the recent research performance of countries as well as institutions.
This is not a bad idea in principle but researchtrends has failed to examine the rankings closely enough. Its writer observed that universities in some countries have improved their position quite dramatically. Two Indian institutions in the top 200 have enjoyed a net change in rank of 248 between 2007 and 2008, eleven from the Netherlands a change of 230 and seven Swiss universities one of 217 while three Israeli institutions rose 194 places between them. Researchtrends describe the Indian achievement as "astonishing" and "testament to the continued development of research in India. "
The writer concludes
Looking at the scores on the citations per faculty part of the rankings, we find that The Indian Institute of Technology (IIT) Bombay had exactly 1 point in 2007 and 43 in 2008, apparently a truly remarkable increase in research or rather citations of research.
The IIT Delhi went up from 1 to 47, the Free University of Amsterdam from 1 to 38, Technion Israel Institute of Technology from 1 to 79 and the Ecole Polytechnique Federale EPF) Lausanne from 29 to 77.
A couple of American universities also apparently enjoyed spectacular rises in the citations score, Washington University in St. Louis (WUSL), already discussed in this blog, and Stony Brook University from 1 to 75.
In reality, there has of course been no spectacular increase in research or citations. What has happened is that in 2007 QS had problems with identifying certain universities or got them confused with others. They, or THE, took a while to decide whether there was one IIT or several and almost certainly got confused between WUSL and the University of Washington. One reader of the blog has suggested they got the Free University of Amsterdam mixed up with the University of Amsterdam. Something similar could have happened with the EPF Lausanne.
So, there may have been an improvement in research productivity over the last few years in India, Switzerland, Israel and the Netherlands. But that is nothing to do with the ascent of some universities from these countries in the rankings. That is testament only to errors committed by QS or -- let us give them some credit -- the correction of errors.
Researchtrends, a newsletter published by Scopus, has an interesting item on the THE-QS World University Rankings. It suggests that the rankings can be used to assess the recent research performance of countries as well as institutions.
This is not a bad idea in principle but researchtrends has failed to examine the rankings closely enough. Its writer observed that universities in some countries have improved their position quite dramatically. Two Indian institutions in the top 200 have enjoyed a net change in rank of 248 between 2007 and 2008, eleven from the Netherlands a change of 230 and seven Swiss universities one of 217 while three Israeli institutions rose 194 places between them. Researchtrends describe the Indian achievement as "astonishing" and "testament to the continued development of research in India. "
The writer concludes
"This suggests that national improvements in ranking may be at least
partially the result of individual universities taking a more strategic
approach: targeting international publications, aided by bibliometric tools and building and promoting library collections"
Are they?
Looking at the scores on the citations per faculty part of the rankings, we find that The Indian Institute of Technology (IIT) Bombay had exactly 1 point in 2007 and 43 in 2008, apparently a truly remarkable increase in research or rather citations of research.
The IIT Delhi went up from 1 to 47, the Free University of Amsterdam from 1 to 38, Technion Israel Institute of Technology from 1 to 79 and the Ecole Polytechnique Federale EPF) Lausanne from 29 to 77.
A couple of American universities also apparently enjoyed spectacular rises in the citations score, Washington University in St. Louis (WUSL), already discussed in this blog, and Stony Brook University from 1 to 75.
In reality, there has of course been no spectacular increase in research or citations. What has happened is that in 2007 QS had problems with identifying certain universities or got them confused with others. They, or THE, took a while to decide whether there was one IIT or several and almost certainly got confused between WUSL and the University of Washington. One reader of the blog has suggested they got the Free University of Amsterdam mixed up with the University of Amsterdam. Something similar could have happened with the EPF Lausanne.
So, there may have been an improvement in research productivity over the last few years in India, Switzerland, Israel and the Netherlands. But that is nothing to do with the ascent of some universities from these countries in the rankings. That is testament only to errors committed by QS or -- let us give them some credit -- the correction of errors.
Thursday, January 22, 2009
Top Universities Guide
I have finally forced myself to buy and read the second edition of QS's book, retitled Top Universities Guide.
The first edition, which was called Guide to the World's Top Universities, was an unqualified disaster, full of errors the worst of which was getting every single student faculty ratio wrong as a result of somebody moving every university three rows down while copying data. Others included putting the Technical University of Munich in the profiles twice in positions 82 and 98 and listing an "Official University of California, Riverside". It also started to fall apart within a few days.
The new edition, however, tuned out to be a pleasant surprise. The previous errors seem to have been corrected and there do not appear to be any new ones. There is a new dignified purple cover and so far not a single page has fallen out.
This does not mean that the THE-QS rankings are faultless. Far from it. The peer review remains extremely biased, the international components are largely meaningless and the student faculty ratio is misleading and easily manipulable. Still, the book does show that in a technical sense, QS has improved quite a bit recently.
The change in title should be noted. There may be another reason, but this would appear to be a clever attempt to distance QS and the authors from the first edition without admitting that there was anything wrong with it.
And I wonder why Blackwell is no longer distributing the book.
I have finally forced myself to buy and read the second edition of QS's book, retitled Top Universities Guide.
The first edition, which was called Guide to the World's Top Universities, was an unqualified disaster, full of errors the worst of which was getting every single student faculty ratio wrong as a result of somebody moving every university three rows down while copying data. Others included putting the Technical University of Munich in the profiles twice in positions 82 and 98 and listing an "Official University of California, Riverside". It also started to fall apart within a few days.
The new edition, however, tuned out to be a pleasant surprise. The previous errors seem to have been corrected and there do not appear to be any new ones. There is a new dignified purple cover and so far not a single page has fallen out.
This does not mean that the THE-QS rankings are faultless. Far from it. The peer review remains extremely biased, the international components are largely meaningless and the student faculty ratio is misleading and easily manipulable. Still, the book does show that in a technical sense, QS has improved quite a bit recently.
The change in title should be noted. There may be another reason, but this would appear to be a clever attempt to distance QS and the authors from the first edition without admitting that there was anything wrong with it.
And I wonder why Blackwell is no longer distributing the book.
Sunday, December 21, 2008
A Must for University Ranking Fans
University rankings have been sprouting up all over the place recently. Laura Milligan at LearningXL has an annotated list of "100 Free College Rankings". It is very Americocentric -- neither the Shanghai nor the THE-QS rankings are there -- but there is something for nearly everybody. The strangest one is probably the Campus Squirrel Listings. The best, for those who are interested, include Berkeley, Mary Baldwin College and Kansas State University.
University rankings have been sprouting up all over the place recently. Laura Milligan at LearningXL has an annotated list of "100 Free College Rankings". It is very Americocentric -- neither the Shanghai nor the THE-QS rankings are there -- but there is something for nearly everybody. The strangest one is probably the Campus Squirrel Listings. The best, for those who are interested, include Berkeley, Mary Baldwin College and Kansas State University.
Friday, November 28, 2008
Comment on the THE-QS Academic Survey
QS have published some information on the academic survey that makes up 40 % of their World University Rankings. The table listing repondents by country is especially interesting. Here are some highlights
United States 638
United Kingdom 563
Australia 286
Inonesia 228
Malaysia 180
Philippones 201
France 125
Belgium 124
Germany 182
China 116
Kong Kong 100
Japan 96
Ireland 78
Thailand 43
Russia 41
So there are almost as many respondents from the UK as from the USA. There are more from Indonesia than from Germany. There are more from Thailand than from Russia. There are more from Hong Kong than from Japan.
Overall, there is pronounced bias in favour of the UK and Ireland, Southeast Asia and Australasia and against the USA, Japan and Southwest Asia.
QS have published some information on the academic survey that makes up 40 % of their World University Rankings. The table listing repondents by country is especially interesting. Here are some highlights
United States 638
United Kingdom 563
Australia 286
Inonesia 228
Malaysia 180
Philippones 201
France 125
Belgium 124
Germany 182
China 116
Kong Kong 100
Japan 96
Ireland 78
Thailand 43
Russia 41
So there are almost as many respondents from the UK as from the USA. There are more from Indonesia than from Germany. There are more from Thailand than from Russia. There are more from Hong Kong than from Japan.
Overall, there is pronounced bias in favour of the UK and Ireland, Southeast Asia and Australasia and against the USA, Japan and Southwest Asia.
Thursday, October 23, 2008
What does 100 on the academic survey mean?
This year 21 universities got a score of 100 on the academic survey section of the THE-QS rankings. A look at the subject rankings, which are combined with equal weighting to form the total score for this indicator, shows that such a perfect score can mean many different things.
Harvard scored 1oo in Arts and Humanities, 100 in Life Sciences and Biomedicine, 96.1 in Natural Sciences, 100 in Social Sciences and 59.6 in Engineering and IT.
The Australian National University scored 74 in Arts and Humanities, 46 .9 in Life Sciences and Biomedicine, 66.1 in Natural Sciences, 71.4 in Social sciences and 49.9 in Engineering and IT.
Peking University scored 56.4 for Arts and humanities, 56.9 for Life sciences and Biomedicine, 73 for Natural Sciences, 57.8 for Social Sciences, and 39.2 for Engineering and IT.
This year 21 universities got a score of 100 on the academic survey section of the THE-QS rankings. A look at the subject rankings, which are combined with equal weighting to form the total score for this indicator, shows that such a perfect score can mean many different things.
Harvard scored 1oo in Arts and Humanities, 100 in Life Sciences and Biomedicine, 96.1 in Natural Sciences, 100 in Social Sciences and 59.6 in Engineering and IT.
The Australian National University scored 74 in Arts and Humanities, 46 .9 in Life Sciences and Biomedicine, 66.1 in Natural Sciences, 71.4 in Social sciences and 49.9 in Engineering and IT.
Peking University scored 56.4 for Arts and humanities, 56.9 for Life sciences and Biomedicine, 73 for Natural Sciences, 57.8 for Social Sciences, and 39.2 for Engineering and IT.
Friday, October 17, 2008
Farewell, Alabama
In 2007 the University of Alabama was listed by THE-QS as the fifth best research university in the world, as measured by citations per faculty. This year it is not even even in the top100. What happened?
Did all those researchers go on strike?
Or is just that that last year the number of faculty was underestimated and this year the mistake was corrected?
In 2007 the University of Alabama was listed by THE-QS as the fifth best research university in the world, as measured by citations per faculty. This year it is not even even in the top100. What happened?
Did all those researchers go on strike?
Or is just that that last year the number of faculty was underestimated and this year the mistake was corrected?
Tuesday, October 14, 2008
Ratio Inflation or Genuine Improvement?
QS have helpfully provided the means and standard deviations for their ranking indicators. For student faculty the mean is 0.09, which works out as a mean of 11.11 students per faculty.
But looking at the data for 2006, the mean number of students per faculty (using the scores provided by QS and cross-checking with the data on their website) was 16.44 students per faculty (N = 531). There has, it would seem, been a very substantial improvement on this indicator in only two years.
Is this a real improvement?
QS have helpfully provided the means and standard deviations for their ranking indicators. For student faculty the mean is 0.09, which works out as a mean of 11.11 students per faculty.
But looking at the data for 2006, the mean number of students per faculty (using the scores provided by QS and cross-checking with the data on their website) was 16.44 students per faculty (N = 531). There has, it would seem, been a very substantial improvement on this indicator in only two years.
Is this a real improvement?
Saturday, October 11, 2008
Comment on a THE Editorial and Comments
Times Higher Education has published an editorial on the 2008 rankings that deserves comment. It says:
I accept that so far this year the rankings have not been disfigured by the sort of spectacular errors that have occurred in the past but I would feel more confident about those enormous efforts if THE and QS owned up to their past errors, such as putting Duke in top place for student faculty ratio by counting undergraduate students as faculty, and indicated exactly what they are doing to stop similar mistakes from occurring again.
The editorial continues:
After proclaiming its lack of bias THE concludes:
According to THE's citations per faculty indicator the research impact of Cambridge academics is less than that of 35 other institutions including Resselaer Polytechnic Institute, Tufts, McMaster, Tel Aviv, UC Davis, Minnesota, Leiden, Emory, Toronto, Kyoto, Brown and ETH Zurch. Cambridge probably is not quite that bad -- there may be problems with the faculty side of the equation that are causing distortions -- but it does look as though the academic survey is in part an attempt to cover up the steady decline of British higher education and research.
There is a comment on the editorial by bgc:
This is perhaps over-dramatic. The THE- QS rankings do seem to be improving in some respects. However, a reply by Martin Ince, editor of the rankings is rather unfortunate:
Whether the ability to sign on to a mailing list makes one an academic expert is debatable. And telling experts in scientometrics about your nice pie charts does you no good at all.
Times Higher Education has published an editorial on the 2008 rankings that deserves comment. It says:
This is the fifth year we have published the rankings and the methodology
has remained unchanged for the past two. Along with our partners in the venture,
Quacquarelli Symonds, we make enormous efforts to ensure that our
quality-control processes and anti-cheating mechanisms are as robust as
possible.
I accept that so far this year the rankings have not been disfigured by the sort of spectacular errors that have occurred in the past but I would feel more confident about those enormous efforts if THE and QS owned up to their past errors, such as putting Duke in top place for student faculty ratio by counting undergraduate students as faculty, and indicated exactly what they are doing to stop similar mistakes from occurring again.
The editorial continues:
This is fallacious. The validity of any survey depends on how representative the respondents are of the larger population, not how many they are. If QS continued to report a response rate of around 3 percent they must expect continued criticism.We try to ensure that the results are produced with a large amount of data. For example, there were more than 6,000 participants in the academic survey alone, producing an average of 20 responses per head. That is a staggering 120,000 data points, making it the largest known survey of university quality.
After proclaiming its lack of bias THE concludes:
For 2008, we congratulate Harvard University for its success
in topping the rankings yet again. However, it is worth remembering that its
endowment now totals more than $35 billion (£19 billion), roughly equivalent to
the total income received by the entire UK sector last year. By that measure,
the UK, with its 29 institutions in the Top 200 (and four in the Top 10), can
stand proud on the world stage.
According to THE's citations per faculty indicator the research impact of Cambridge academics is less than that of 35 other institutions including Resselaer Polytechnic Institute, Tufts, McMaster, Tel Aviv, UC Davis, Minnesota, Leiden, Emory, Toronto, Kyoto, Brown and ETH Zurch. Cambridge probably is not quite that bad -- there may be problems with the faculty side of the equation that are causing distortions -- but it does look as though the academic survey is in part an attempt to cover up the steady decline of British higher education and research.
There is a comment on the editorial by bgc:
The Times uses a non-transparent, undefined opinion
survey for most of their weightings - presumably this is what leads to such
nonsensical results.
Anyone who knows anything about international HE would
realize that the Times ranking lack basic validity.
For goodness sake,
sort-out the ranking methodology before next year. Or please stop inflicting
this annual embarrassment on those of us who practice scientometrics and try to
use objective methods of educational evaluation.
This is perhaps over-dramatic. The THE- QS rankings do seem to be improving in some respects. However, a reply by Martin Ince, editor of the rankings is rather unfortunate:
By contrast, we have measures relating to teaching,
globalisation and employability, and our research indicators cover the full
range of subjects. We set out exactly who and where our respondents are - there
is a nice pie chart in today's paper. These expert academics provide us with
about 126,000 data points (20 per person for 6,300 people) and make up the
biggest and best survey of university quality.
Whether the ability to sign on to a mailing list makes one an academic expert is debatable. And telling experts in scientometrics about your nice pie charts does you no good at all.
Friday, October 10, 2008
Not What It used To Be
The THE-QS rankings seem to be getting better. So far this year, no outrageous errors like ranking a non existent university (Beijing University), turning Malaysian ethnic minorities into foreigners or giving Washington University in St Louis a near zero for research, have surfaced.
But comparing the scores for the academic survey with those for citations for research, both of which are supposed to measure research quality, suggests that the former has is a large and systematic bias.
Here is a list of universities whose score on the academic survey exceeds their score on the citations per faculty by forty points or more.
Mcgill
Sydney
Melbourne
New York University
Monash
Trinity College Dublin
Peking
Seoul National University
Tsinghua
Auckland
London School of Economics
Warwick
Nanyang Technological University Singapore
University College Dublin
Fudan
Humboldt University Berlin
Nanjing
Shanghai Jiao Tong
Aotonomous National University of Mexico
Chulalongkorn University Thailand
Waseda
Lomonosov State University Moscow
Bologna
Sao Paulo
Beunos Aires
Maybe LSE and NYU can be explained by excellence in subjects that produce few publications or citations. But is it not possible that there is a pronounced geographical bias in the survey?
The THE-QS rankings seem to be getting better. So far this year, no outrageous errors like ranking a non existent university (Beijing University), turning Malaysian ethnic minorities into foreigners or giving Washington University in St Louis a near zero for research, have surfaced.
But comparing the scores for the academic survey with those for citations for research, both of which are supposed to measure research quality, suggests that the former has is a large and systematic bias.
Here is a list of universities whose score on the academic survey exceeds their score on the citations per faculty by forty points or more.
Mcgill
Sydney
Melbourne
New York University
Monash
Trinity College Dublin
Peking
Seoul National University
Tsinghua
Auckland
London School of Economics
Warwick
Nanyang Technological University Singapore
University College Dublin
Fudan
Humboldt University Berlin
Nanjing
Shanghai Jiao Tong
Aotonomous National University of Mexico
Chulalongkorn University Thailand
Waseda
Lomonosov State University Moscow
Bologna
Sao Paulo
Beunos Aires
Maybe LSE and NYU can be explained by excellence in subjects that produce few publications or citations. But is it not possible that there is a pronounced geographical bias in the survey?
Thursday, October 09, 2008
THE-QS 2008 Update
This year there has been only one methodological change, namely the separation of the lists in the academic survey section into international and domestic sections and then their recombination. This would probably work against universities that receive a lot of votes from their own countries and might explain why Hong Kong, Peking and several Australian universities have fallen quite a bit.
Also, it is likely that the geographical spread of the academic and employer surveys has expanded and that this has benefitted universities in Latin America, Africa and India.
The biggest change in the top 100 is that Washington University in St Louis has risen to 60th place from 161st in 2007. This, presumably, is because it is now getting a realistic score for citations per faculty instead of the 1 it got in 2007, when QS seem to have confused it with the University of Washington. I am a little bit suspicious though about these two places being next to each other in this year's ranking.
The University of Hong Kong has fallen from 18th to 26th, Peking from 36th to 50th, Nanyang from 69th to 77th, Melbourne from 21st to 38th and Macquarie from 168th to 183rd (I wonder what Dr. Schwartz will say about that.)
On the other hand, the National Autonomous University of Mexico has risen from 192nd to 150th, the Indian Institute of Technology Delhi from 307th to 154th and Chulalongkorn from 223rd to 166th.
One oddity that I've noticed is that Stony Brook University, which is an autonomous university centre of the State University of new York, has risen dramatically to 127th place from 224th, while the other three centres at Binghamton, Buffalo and Albany which are of equal or better quality do not even get into QS's initial list.
This year there has been only one methodological change, namely the separation of the lists in the academic survey section into international and domestic sections and then their recombination. This would probably work against universities that receive a lot of votes from their own countries and might explain why Hong Kong, Peking and several Australian universities have fallen quite a bit.
Also, it is likely that the geographical spread of the academic and employer surveys has expanded and that this has benefitted universities in Latin America, Africa and India.
The biggest change in the top 100 is that Washington University in St Louis has risen to 60th place from 161st in 2007. This, presumably, is because it is now getting a realistic score for citations per faculty instead of the 1 it got in 2007, when QS seem to have confused it with the University of Washington. I am a little bit suspicious though about these two places being next to each other in this year's ranking.
The University of Hong Kong has fallen from 18th to 26th, Peking from 36th to 50th, Nanyang from 69th to 77th, Melbourne from 21st to 38th and Macquarie from 168th to 183rd (I wonder what Dr. Schwartz will say about that.)
On the other hand, the National Autonomous University of Mexico has risen from 192nd to 150th, the Indian Institute of Technology Delhi from 307th to 154th and Chulalongkorn from 223rd to 166th.
One oddity that I've noticed is that Stony Brook University, which is an autonomous university centre of the State University of new York, has risen dramatically to 127th place from 224th, while the other three centres at Binghamton, Buffalo and Albany which are of equal or better quality do not even get into QS's initial list.
The THE-QS Top 200: 2008
The QS topuniversities site has listed the top 200 universities in their 2008 rankings. I hope to make a few comments in a little while.
The QS topuniversities site has listed the top 200 universities in their 2008 rankings. I hope to make a few comments in a little while.
Wednesday, October 08, 2008
Sarah Palin and the University of Idaho
One of the more distasteful aspects of the US presidential election is the obsession in some quarters with Sara Palin's IQ. See here for example. Some have suggested that she is not particularly bright because she graduated from the University of Idaho and not from Columbia or Harvard.
May I point out that the University of Idaho, although it is not Harvard or Columbia, is ranked in the 400s in the Shanghai Jiao Tong rankings, which puts it way above most institutions in the world and in the United States.
May I also point out that Palin, unlike Joe Biden, has not been accused of plagiarism. Also, if we are going to be snobby about universities, one also wonders why Biden, if he were so clever, would plagiarise from Neil Kinnock who had to take his final exams twice at a university that also wasn't Harvard or Columbia
One of the more distasteful aspects of the US presidential election is the obsession in some quarters with Sara Palin's IQ. See here for example. Some have suggested that she is not particularly bright because she graduated from the University of Idaho and not from Columbia or Harvard.
May I point out that the University of Idaho, although it is not Harvard or Columbia, is ranked in the 400s in the Shanghai Jiao Tong rankings, which puts it way above most institutions in the world and in the United States.
May I also point out that Palin, unlike Joe Biden, has not been accused of plagiarism. Also, if we are going to be snobby about universities, one also wonders why Biden, if he were so clever, would plagiarise from Neil Kinnock who had to take his final exams twice at a university that also wasn't Harvard or Columbia
Thursday, September 18, 2008
Sour Grapes Watch
There is a definite smell of very sour grapes coming out of Imperial College London with the news that University College London has moved ahead in the Shanghai Jiao Tong University rankings, rising from 26th to 22th. Imperial has fallen from 23rd to 27th, a trivial distance by THES-QS standards. This account is from London Student, an online newspaper.
There are a few errors here. The ARWU categories are explained in the methodology section. Not all academics and employers recognise the Times (actually Times Higher Education, not the same thing) as the standard, especially in the USA. It is stretching things a bit to refer to the experts in the THE-QS "peer review". To be a reviewer requires no greater expertise than the ability to sign on to the mailing list of World Scientific, an academic publishing company that has -- this is no doubt completely irrelevant -- an interest in Imperial College Press.
There is a definite smell of very sour grapes coming out of Imperial College London with the news that University College London has moved ahead in the Shanghai Jiao Tong University rankings, rising from 26th to 22th. Imperial has fallen from 23rd to 27th, a trivial distance by THES-QS standards. This account is from London Student, an online newspaper.
Jovan Nedic, Editor-in-Chief of Imperial’s student paper Felix, said: “I mean
just look at the [ARWU] website, that alone tells you that this ranking is a
joke. None of the scoring categories are explained and the ones that are obvious
are not the best indication on what university is all about. It calls itself an
academic ranking, so why is a score on Alumni important?”This ranking is nothing
more than a joke and I’m surprised that London Student are even bothering to
look at it.”As to whether the new rankings meant UCL was ‘better‘, he continued:
“Academics and employers alike all recognise the Times one as the standard, only
those results will mean anything to Imperial students, until then, UCL and the
other 21 above Imperial can (in case they don’t get that, I mean all the ones
that are between 6th and 27th) can only dream to be better!”The difference
between criteria used in university rankings is a long-standing bone of
contention. While ARWU gives higher scores to universities whose staff and
students win Nobel Prizes, The Times bases 40 per cent of its scoring on
researchers peer-review - where experts are asked to list institutions they
think are top in their area.
There are a few errors here. The ARWU categories are explained in the methodology section. Not all academics and employers recognise the Times (actually Times Higher Education, not the same thing) as the standard, especially in the USA. It is stretching things a bit to refer to the experts in the THE-QS "peer review". To be a reviewer requires no greater expertise than the ability to sign on to the mailing list of World Scientific, an academic publishing company that has -- this is no doubt completely irrelevant -- an interest in Imperial College Press.
Another University Ranking
This one is from the Global Language Monitor. which "has ranked the nation’s [USA] colleges and universities according their appearance in the global print and electronic media, as well as on the Internet and throughout the Blogosphere. "
The top ten universities are:
1. Harvard University
2. Columbia University
3. University of Michigan, Ann Arbor
4. University of California, Berkeley
5. Stanford University
6. University of Chicago
7. University of Wisconsin, Madison
8. Yale University
9. Princeton University
10. Cornell University
The top ten colleges are:
1. Colorado College
2. Williams College
3. University of Richmond
4. Middlebury College
5. Wellesley College
6. Bucknell University
7 . Amherst College
8. Oberlin College
9. Vassar College
10. Pomona College
This one is from the Global Language Monitor. which "has ranked the nation’s [USA] colleges and universities according their appearance in the global print and electronic media, as well as on the Internet and throughout the Blogosphere. "
The top ten universities are:
1. Harvard University
2. Columbia University
3. University of Michigan, Ann Arbor
4. University of California, Berkeley
5. Stanford University
6. University of Chicago
7. University of Wisconsin, Madison
8. Yale University
9. Princeton University
10. Cornell University
The top ten colleges are:
1. Colorado College
2. Williams College
3. University of Richmond
4. Middlebury College
5. Wellesley College
6. Bucknell University
7 . Amherst College
8. Oberlin College
9. Vassar College
10. Pomona College
Tuesday, September 16, 2008
University Research Expenditure Rankings
There is an interesting ranking of US universities by the National Science Foundation according to total research and development expenditure.
Here are the top ten:
1. Johns Hopkins
2. Wisconsin (Madison)
3. UCLA
4. Minnesota
5. UC San Francisco
6. Washington
7. UC San Diego
8. Stanford
9. University of Pennsylvania
10. Duke
But does spending a lot of money necessarily lead to a high level of research productivity?
There is an interesting ranking of US universities by the National Science Foundation according to total research and development expenditure.
Here are the top ten:
1. Johns Hopkins
2. Wisconsin (Madison)
3. UCLA
4. Minnesota
5. UC San Francisco
6. Washington
7. UC San Diego
8. Stanford
9. University of Pennsylvania
10. Duke
But does spending a lot of money necessarily lead to a high level of research productivity?
Thursday, September 04, 2008
Plagiarism Multiple Standards Watch
A couple of recent news stories illustrate the multiple standards regarding plagiarism. Allison Routman, an undergraduate student, was kicked off a university of Virginia educational cruise ship for copying precisely three phrases from Wikipedia into a film review. The exact phrases were "when the Germans attacked the Soviet Union during Operation Barbarossa", "German speaking minority outside of Germany" and "who had been released from a concentration camp".
Assuming that the student has given a full and accurate account -- we have not heard from her instructors -- this would seem far too draconian. Would we consider a researcher who wrote "recent research has clearly shown that" to be plagiarising?. That phase gets 748 results from a Google Search.
Meanwhile , a new ranking by Forbes has put the University of Southern Illinois at Carbondale in 489th place among US colleges. The president of the university, Glenn Poshard, had plagiarised a substantial part of his doctoral dissertation written for the university's Department of Higher Education but a committee found that the plagiarism was inadvertent and he just had to revise the offending sections. Neither Poshard nor the university seem to have suffered much from the affair.
Next, Joe Biden has been picked as Barack Obama's running mate. Biden is well known for his plagiarism of a speech by Neil Kinnock during his presidential campaign and that was not his first offense. Still it has not hurt very much in the long run.
Incidentally, one wonders why Biden bothered to copy such a ridiculous passage . Kinnock said
A couple of recent news stories illustrate the multiple standards regarding plagiarism. Allison Routman, an undergraduate student, was kicked off a university of Virginia educational cruise ship for copying precisely three phrases from Wikipedia into a film review. The exact phrases were "when the Germans attacked the Soviet Union during Operation Barbarossa", "German speaking minority outside of Germany" and "who had been released from a concentration camp".
Assuming that the student has given a full and accurate account -- we have not heard from her instructors -- this would seem far too draconian. Would we consider a researcher who wrote "recent research has clearly shown that" to be plagiarising?. That phase gets 748 results from a Google Search.
Meanwhile , a new ranking by Forbes has put the University of Southern Illinois at Carbondale in 489th place among US colleges. The president of the university, Glenn Poshard, had plagiarised a substantial part of his doctoral dissertation written for the university's Department of Higher Education but a committee found that the plagiarism was inadvertent and he just had to revise the offending sections. Neither Poshard nor the university seem to have suffered much from the affair.
Next, Joe Biden has been picked as Barack Obama's running mate. Biden is well known for his plagiarism of a speech by Neil Kinnock during his presidential campaign and that was not his first offense. Still it has not hurt very much in the long run.
Incidentally, one wonders why Biden bothered to copy such a ridiculous passage . Kinnock said
Why am I the first Kinnock in a thousand generations to be able to get
to university? Why is Glenys the first woman in her family in a thousand
generations to be able to get to university? Was it because our predecessors
were thick? Does anybody really think that they didn't get what we had because
they didn't have the talent or the strength or the endurance or the commitment?
Of course not. It was because there was no platform upon which they could
stand.
Well, for 977 of those thousand generations it was because there were no universities in Scotland for anyone to go to.
Sunday, August 17, 2008
The Shanghai Rankings 2008
Shanghai Jiao Tong University (SJTU) has just released their rankings for 2008. Compared to the THE-QS rankings, public response, especially in Asia and Australia, has been slight. This is largely because ascent and descent within the Shanghai index is minimal, a tribute to their reliability. In contrast, the THE-QS rankings, with their changes in methodology and frequent errors, arouse almost as much interest as a country's performance in the Olympics.
Still, it is instructive to check how well various universities do on the different components of the Shanghai rankings.
The current top ten are as follows:
1. Harvard
2. Stanford
3. Berkeley
4. Cambridge
5. MIT
6. Caltech
7. Columbia
8. Princeton
9. Chicago
10. Oxford
Stanford goes down to number 8
Berkeley goes down to 11
Cambridge goes down to 23
MIT is down at 34
Caltech tumbles to 86
Columbia is down just a bit at 10
Princeton crashes to 120
Chicago falls to 72
Oxford goes down to 18
If this category represents current research output then it looks as though some American universities and Oxbridge have entered a period of decline. Of course, Caltech and MIT may suffer from the PUB category including social science research but would that explain why Princeton and Chicago are now apparently producing a relatively small amount of research?
Shanghai Jiao Tong University (SJTU) has just released their rankings for 2008. Compared to the THE-QS rankings, public response, especially in Asia and Australia, has been slight. This is largely because ascent and descent within the Shanghai index is minimal, a tribute to their reliability. In contrast, the THE-QS rankings, with their changes in methodology and frequent errors, arouse almost as much interest as a country's performance in the Olympics.
Still, it is instructive to check how well various universities do on the different components of the Shanghai rankings.
The current top ten are as follows:
1. Harvard
2. Stanford
3. Berkeley
4. Cambridge
5. MIT
6. Caltech
7. Columbia
8. Princeton
9. Chicago
10. Oxford
The Shanghai index includes two categories based on Nobel prizes and Fields medals. These measure the quality of research that might have been produced decades ago. Looking at the other criteria gives a rather different picture of current research.
It is interesting to see what happens to these ten if we rank them according to SJTU's PUB category, the total number of articles indexed in the Science Citation Index-Expanded (SCIE) and Social Science Citation Index (SSCI) in 2007. The SSCI gets a double weighting.
Harvard remains at number 1Stanford goes down to number 8
Berkeley goes down to 11
Cambridge goes down to 23
MIT is down at 34
Caltech tumbles to 86
Columbia is down just a bit at 10
Princeton crashes to 120
Chicago falls to 72
Oxford goes down to 18
If this category represents current research output then it looks as though some American universities and Oxbridge have entered a period of decline. Of course, Caltech and MIT may suffer from the PUB category including social science research but would that explain why Princeton and Chicago are now apparently producing a relatively small amount of research?
The top ten for PUB is
1. Harvard
2. Tokyo
3. Toronto
4. University of Michigan
5. UCLA
6. University of Washington
7. Stanford
8. Kyoto
9. Columbia
10. Berkeley
Sunday, August 10, 2008
Are Things Improving at QS?
I think it worth quoting from a recent comment.
There are signs that QS is making a commendable effort to avoid the errors that have been so prevalent in previous rankings. Still it is rather disconcerting that thousands of faculty have turned into hundreds, especially since it is not altogether impossible that some universities might conveniently forget to correct an error that might be to their advantage.
So, I was wondering how common simple basic errors are in the QS rankings. I have been looking at QS's topuniversities site and checking the number of students listed in the descriptions for each university, comparing the number of undergraduates and postgraduates with the total of all students. Here are the results just for the Universities beginning with A.
For these twelve universities no problems were noticed: Aarhus, Aberdeen, Aberystwyth, Antwerp, Arizona State University, Athens, Aston, Amsterdam, Adelaide, Australian National University, Austral, Vrije Universiteit Amsterdam. In some cases there were minor discrepancies but not enough to cause concern.
About two weeks ago there were, in three cases, discrepancies between the number of total students and the combined numbers of undergraduates and post graduates: University of Arizona (more combined undergraduates and post graduates than total students), University of Auckland (number of postgraduates and total students the same) and Athens University of Economics and Business (more undergraduate international students than total international students) .
In the above three cases, at the time of writing the errors have been corrected with new entries.
There were however three cases where the errors at the time of writing had not been corrected. These are:
University of Arkansas
Full Time Equivalent (FTE) undergraduates. 20,416.
FTE graduate/postgraduate students 4,163.
Total FTE students 15,182.
Over 9,000 students "missing" from the total.
(correction: not the University of Arizona as was indicated in an earlier version of this post)
University of Alabama
FTE undergraduates 33, 986
FTE graduate/postgraduate students 8,291.
FTE total of 19,651 students.
Over 22,000 students “missing” from the total.
University of Alberta
FTE undergraduates 29,178.
FTE graduate/postgraduate students 5,419.
Total FTE total students 32,341.
About 2,500 students “missing” from the total.
I suspect that the problem with these three schools is that the totals of students were complied and entered separately and that the data for undergraduates and postgraduates included students in branch campuses, professional schools and/or research institutes and the data for total students did not. It will be interesting to see whether these errors will be corrected and whether new ones will emerge.
I think it worth quoting from a recent comment.
Just wanted to add a note about submitting data to THES based on my experience
here at a large Australian university. We forwarded our submission earlier this
year and now we have received a query on some of our numbers - a check just to
confirm if they are correct. And I can understand why they would need to be
checked. The numbers we submitted for staff (in the thousands) seem to have
changed to a number in the very low hundreds. Also, comments (made by us) that
were attached to specific sections have, in the past come back with typos... I
think this indicates that there is greater scope for human error in the
compilation of the data, even at such an early and relatively uncomplicated
phase of the data gathering process...
There are signs that QS is making a commendable effort to avoid the errors that have been so prevalent in previous rankings. Still it is rather disconcerting that thousands of faculty have turned into hundreds, especially since it is not altogether impossible that some universities might conveniently forget to correct an error that might be to their advantage.
So, I was wondering how common simple basic errors are in the QS rankings. I have been looking at QS's topuniversities site and checking the number of students listed in the descriptions for each university, comparing the number of undergraduates and postgraduates with the total of all students. Here are the results just for the Universities beginning with A.
For these twelve universities no problems were noticed: Aarhus, Aberdeen, Aberystwyth, Antwerp, Arizona State University, Athens, Aston, Amsterdam, Adelaide, Australian National University, Austral, Vrije Universiteit Amsterdam. In some cases there were minor discrepancies but not enough to cause concern.
About two weeks ago there were, in three cases, discrepancies between the number of total students and the combined numbers of undergraduates and post graduates: University of Arizona (more combined undergraduates and post graduates than total students), University of Auckland (number of postgraduates and total students the same) and Athens University of Economics and Business (more undergraduate international students than total international students) .
In the above three cases, at the time of writing the errors have been corrected with new entries.
There were however three cases where the errors at the time of writing had not been corrected. These are:
University of Arkansas
Full Time Equivalent (FTE) undergraduates. 20,416.
FTE graduate/postgraduate students 4,163.
Total FTE students 15,182.
Over 9,000 students "missing" from the total.
(correction: not the University of Arizona as was indicated in an earlier version of this post)
University of Alabama
FTE undergraduates 33, 986
FTE graduate/postgraduate students 8,291.
FTE total of 19,651 students.
Over 22,000 students “missing” from the total.
University of Alberta
FTE undergraduates 29,178.
FTE graduate/postgraduate students 5,419.
Total FTE total students 32,341.
About 2,500 students “missing” from the total.
I suspect that the problem with these three schools is that the totals of students were complied and entered separately and that the data for undergraduates and postgraduates included students in branch campuses, professional schools and/or research institutes and the data for total students did not. It will be interesting to see whether these errors will be corrected and whether new ones will emerge.
Tuesday, July 29, 2008
Ranking Fever Sweeps America
The US college scene is buzzing about the latest ranking news. No, its not the impending THE-QS rankings but Princeton Review's list of top party schools. See here, here, here, here, here, here, here, here, here and here.
It seems that US colleges are more concerned about being in the top 20 party schools or, in a few cases, in the top 20 stone-cold sober schools than their position in the THE-Qs rankings. And who is to say that they they are wrong.
The US college scene is buzzing about the latest ranking news. No, its not the impending THE-QS rankings but Princeton Review's list of top party schools. See here, here, here, here, here, here, here, here, here and here.
It seems that US colleges are more concerned about being in the top 20 party schools or, in a few cases, in the top 20 stone-cold sober schools than their position in the THE-Qs rankings. And who is to say that they they are wrong.
The Princeton Review Rankings
The Princeton Review's Annual College Rankings, based on a survey of 120,000 US students, is now out. There are 62 lists and there seems to be something for everyone. Here are some of the top schools.
Middlebury College -- best professors
Wheaton College -- tastiest campus food
Loyola College -- best dorms
Yale -- most beautiful campus
Northeastern University -- best career and job placement service
Stanford -- best classroom experience
Texas A and M (College Station) -- most conservative students
Occidental College -- most liberal students
University of Florida (Gainesville) -- top party school
Brigham Young University -- top stone-cold sober school
City University of new York -- most diverse student body
University of Maryland at College Part -- best athletic facilities
The Princeton Review's Annual College Rankings, based on a survey of 120,000 US students, is now out. There are 62 lists and there seems to be something for everyone. Here are some of the top schools.
Middlebury College -- best professors
Wheaton College -- tastiest campus food
Loyola College -- best dorms
Yale -- most beautiful campus
Northeastern University -- best career and job placement service
Stanford -- best classroom experience
Texas A and M (College Station) -- most conservative students
Occidental College -- most liberal students
University of Florida (Gainesville) -- top party school
Brigham Young University -- top stone-cold sober school
City University of new York -- most diverse student body
University of Maryland at College Part -- best athletic facilities
Monday, July 28, 2008
Another University Ranking
The Ecole des Mines de Paris has produced a new ranking based on the number of corporate leaders trained by universities. Its website reports:
It is good to be reminded that universities do other things and not just research. Still this is a very limited measure of excellence.
Here are the top ten
1. Harvard Univ
2. Tokyo Univ
3 Stanford Univ
4 Ecole Polytechnique Paris
5 École des Hautes Études Commerciales Paris
6 Univ Pennsylvania
7 Massachusetts Inst Tech (MIT)
8 Sciences Po - Paris
9 ENA Paris
10 Ecole des Mines de Paris
There are five French schools in the top ten which may say something about the excellence of these institutions or perhaps something about French cultural introversion. Only four foreign institutions have contributed to the training of 37 French CEOs. For the 38 British CEOs the corresponding figure is 22, including Gettysburg College, the US Naval Academy, University College Cork, Utrecht, Witwatersrand and, of course, the Ecole des Mines.
I noticed that in at least two cases, Strode's College in Surrey and Malay College in Malaysia, secondary schools were listed as diploma-granting institutions.
The Ecole des Mines de Paris has produced a new ranking based on the number of corporate leaders trained by universities. Its website reports:
The École des Mines de Paris ranking is based on a quite different
criterion: the number of alumni holding a Chief Executive Officer position (CEO)
in one of the 500 leading worldwide companies as of the date of the Shanghai
ranking 2006. This criterion is aimed at being the equivalent among companies of
the criterion for alumni who have been awarded the Nobel Prize or the Fields
Medal, as the numbers involved are similar.
We have chosen to identify the
500 leading enterprises on the basis of the "Global Fortune 500", based on the
criterion of the published annual turnover and conducted by Fortune
magazine.
It is good to be reminded that universities do other things and not just research. Still this is a very limited measure of excellence.
Here are the top ten
1. Harvard Univ
2. Tokyo Univ
3 Stanford Univ
4 Ecole Polytechnique Paris
5 École des Hautes Études Commerciales Paris
6 Univ Pennsylvania
7 Massachusetts Inst Tech (MIT)
8 Sciences Po - Paris
9 ENA Paris
10 Ecole des Mines de Paris
There are five French schools in the top ten which may say something about the excellence of these institutions or perhaps something about French cultural introversion. Only four foreign institutions have contributed to the training of 37 French CEOs. For the 38 British CEOs the corresponding figure is 22, including Gettysburg College, the US Naval Academy, University College Cork, Utrecht, Witwatersrand and, of course, the Ecole des Mines.
I noticed that in at least two cases, Strode's College in Surrey and Malay College in Malaysia, secondary schools were listed as diploma-granting institutions.
Monday, July 21, 2008
Another Use of Rankings
BBC News reports that graduates of lower ranking universities are more dishonest than those from institutions further up the ranking ladder.
"Analysis of 3,876 financial service job applications found embellishments on the forms of 43% of applicants from the UK's lowest ranking universities.
Only 14% of applicants from the top 20 UK universities were found to have fibbed in their applications.
The survey was commissioned by a pre-employment screening firm, Powerchex.
Its managing director Alexandra Kelly said: "What this survey says is that graduates from lesser-known universities may feel the need to alter their background to compete."
There is no definitive ranking of universities. For the survey the researchers at the Shell technology and enterprise programme used the Times Online 2009 ranking. "
Given the amount of soft marking and numbers massaging that British universities do to get a good place in the rankings, is it posible that graduates of the good ones do not need to embellish their CVs because their universities do all the necessary embellishment for them?
BBC News reports that graduates of lower ranking universities are more dishonest than those from institutions further up the ranking ladder.
"Analysis of 3,876 financial service job applications found embellishments on the forms of 43% of applicants from the UK's lowest ranking universities.
Only 14% of applicants from the top 20 UK universities were found to have fibbed in their applications.
The survey was commissioned by a pre-employment screening firm, Powerchex.
Its managing director Alexandra Kelly said: "What this survey says is that graduates from lesser-known universities may feel the need to alter their background to compete."
There is no definitive ranking of universities. For the survey the researchers at the Shell technology and enterprise programme used the Times Online 2009 ranking. "
Given the amount of soft marking and numbers massaging that British universities do to get a good place in the rankings, is it posible that graduates of the good ones do not need to embellish their CVs because their universities do all the necessary embellishment for them?
Friday, July 18, 2008
France Wants a New Ranking System
One consequence of the ranking craze is the proliferation of "minor leagues" as universities and countries that do badly on prominent rankings encourage the creation of more and more indexes and league tables on which they hope to better.
The French Senate is now proposing a new ranking system for European universities. It is not happy with the attention given to the Shanghai Jiao Tong index where French institutions do not perform very well.
The report on the euractiv site continues:
"France's key bone of contention with the Shanghai index is that the number of citations of a institution's scientific research is used as a ranking factor. Paris says this works against countries that do not publish in English.
Senator Joël Bourdin, the rapporteur on the report, argues that the intrinsic value of the Shanghai ranking is highly questionable ["très discutable"] and owes its interest only to its "mobilising effect".
The highest-ranking French university in the 2006 Shanghai ranking was Paris VI in 39th place, while American universities occupy more than half of the top 100. The only European universities in the top 10 are the UK's Universities of Cambridge and Oxford.
While French Higher Education Minister Valerie Pécresse has said she wishes to use the French EU Presidency to "lay down the foundations of common European criteria" for university classification, Bourdin argues that France should already go ahead and develop its own national classification system, as an EU-level process would take years. "
One wonders why the French do not simply use the citations per faculty section of the THE-QS rankings which shows the Ecole Normale Superieure in 4th place just ahead of the University of Alabama (5th), Pohang University of Science and Technology (11th), University of Helsinki (17th), Hong Kong University of Science and Technology (20th). On this measure the ecole does better than Princeton, Harvard, Cornell and Columbia.
The report can viewed here.
One consequence of the ranking craze is the proliferation of "minor leagues" as universities and countries that do badly on prominent rankings encourage the creation of more and more indexes and league tables on which they hope to better.
The French Senate is now proposing a new ranking system for European universities. It is not happy with the attention given to the Shanghai Jiao Tong index where French institutions do not perform very well.
The report on the euractiv site continues:
"France's key bone of contention with the Shanghai index is that the number of citations of a institution's scientific research is used as a ranking factor. Paris says this works against countries that do not publish in English.
Senator Joël Bourdin, the rapporteur on the report, argues that the intrinsic value of the Shanghai ranking is highly questionable ["très discutable"] and owes its interest only to its "mobilising effect".
The highest-ranking French university in the 2006 Shanghai ranking was Paris VI in 39th place, while American universities occupy more than half of the top 100. The only European universities in the top 10 are the UK's Universities of Cambridge and Oxford.
While French Higher Education Minister Valerie Pécresse has said she wishes to use the French EU Presidency to "lay down the foundations of common European criteria" for university classification, Bourdin argues that France should already go ahead and develop its own national classification system, as an EU-level process would take years. "
One wonders why the French do not simply use the citations per faculty section of the THE-QS rankings which shows the Ecole Normale Superieure in 4th place just ahead of the University of Alabama (5th), Pohang University of Science and Technology (11th), University of Helsinki (17th), Hong Kong University of Science and Technology (20th). On this measure the ecole does better than Princeton, Harvard, Cornell and Columbia.
The report can viewed here.
Descending into Farce
A report in the Independent (UK) provides further evidence of the destructive effects of the obsession with rankings.
"The university degree classification system is "descending into farce", the chairman of the Commons Select Committee on Universities has said.
Phil Willis was speaking as MPs questioned Peter Williams, the chief executive of the Quality Assurance Agency (QAA), the higher education watchdog, on degree standards. "An individual institution can award as many firsts as it wants, provided it satisfies its own criteria on what is a first," Mr Willis said.
It followed comments from Professor Geoffrey Alderman, the former head of quality at the University of London, which were reported in The Independent, that lecturers had been told to "mark softly" to ensure enough first-class degree passes were awarded to win a high ranking in league tables. He also alleged universities were turning a "blind eye" to plagiarism by international students because they were dependent on income from their fees. "
A report in the Independent (UK) provides further evidence of the destructive effects of the obsession with rankings.
"The university degree classification system is "descending into farce", the chairman of the Commons Select Committee on Universities has said.
Phil Willis was speaking as MPs questioned Peter Williams, the chief executive of the Quality Assurance Agency (QAA), the higher education watchdog, on degree standards. "An individual institution can award as many firsts as it wants, provided it satisfies its own criteria on what is a first," Mr Willis said.
It followed comments from Professor Geoffrey Alderman, the former head of quality at the University of London, which were reported in The Independent, that lecturers had been told to "mark softly" to ensure enough first-class degree passes were awarded to win a high ranking in league tables. He also alleged universities were turning a "blind eye" to plagiarism by international students because they were dependent on income from their fees. "
Wednesday, July 16, 2008
Comments on Ben Sowter's Comments
This is in reply to comments by Ben Sowter, head of research at QS, on my post of July 10th.
First of all, I was mistaken about the BlackBerry. At the end of the survey there is a box to tick if respondents want a chance to win one.
Next, I am glad to learn that Ben is a frequent reader of this blog. Thank you also for the comment that "Much of your analysis is detailed and shrewd. In fact, whilst I have declined to comment in the past we have kept an eye on your blog and, in some areas, chosen to make improvements based on your observations".
The clarification that the current survey is of prior respondents only and that later e-mails will also be sent to World Scientific subscribers is also welcome.
As for wondering why Malaysian universities were not on the first list, the introductory messages says that "You'll notice some slight differences to the survey this year - mainly in that we have included a lot more universities - as a result there are two questions about universities - one asking about those around the world, and a second asking about your own country specifically". I assumed that "around the world" meant that every country in the world would be included. The information should have read "around the world excluding your your own country". Still, I accept that I jumped to an unwarranted conclusion.
I think that there are serious questions remaining about the implications of this change. If QS are going to use only the responses to the first list when calculating the score for the academic opinion criterion then this would surely mean a noticeable reduction in the number of votes for universities that got a large number of responses from other places in that country. I suspect that this would have a disproportionate effect on American and Japanese universities. Wouldn't it also have a relatively beneficial impact on British universities with lots of graduates in the Commonwealth and Australia with graduates in South East Asia and China, countries with a lot of names in the World Scientific database?
Ben's comment that the purpose of having two lists is so that "we can do some more analysis on response dynamics and, potentially, more effectively counteract country biases" suggests that this might be the case since the implication is that the second list will not be used to construct the scores for the "peer review".
Is it possible that such a change would give Cambridge, Oxford and Imperial College London a further advantage over Harvard and push one of the former three into the number one spot?
This is in reply to comments by Ben Sowter, head of research at QS, on my post of July 10th.
First of all, I was mistaken about the BlackBerry. At the end of the survey there is a box to tick if respondents want a chance to win one.
Next, I am glad to learn that Ben is a frequent reader of this blog. Thank you also for the comment that "Much of your analysis is detailed and shrewd. In fact, whilst I have declined to comment in the past we have kept an eye on your blog and, in some areas, chosen to make improvements based on your observations".
The clarification that the current survey is of prior respondents only and that later e-mails will also be sent to World Scientific subscribers is also welcome.
As for wondering why Malaysian universities were not on the first list, the introductory messages says that "You'll notice some slight differences to the survey this year - mainly in that we have included a lot more universities - as a result there are two questions about universities - one asking about those around the world, and a second asking about your own country specifically". I assumed that "around the world" meant that every country in the world would be included. The information should have read "around the world excluding your your own country". Still, I accept that I jumped to an unwarranted conclusion.
I think that there are serious questions remaining about the implications of this change. If QS are going to use only the responses to the first list when calculating the score for the academic opinion criterion then this would surely mean a noticeable reduction in the number of votes for universities that got a large number of responses from other places in that country. I suspect that this would have a disproportionate effect on American and Japanese universities. Wouldn't it also have a relatively beneficial impact on British universities with lots of graduates in the Commonwealth and Australia with graduates in South East Asia and China, countries with a lot of names in the World Scientific database?
Ben's comment that the purpose of having two lists is so that "we can do some more analysis on response dynamics and, potentially, more effectively counteract country biases" suggests that this might be the case since the implication is that the second list will not be used to construct the scores for the "peer review".
Is it possible that such a change would give Cambridge, Oxford and Imperial College London a further advantage over Harvard and push one of the former three into the number one spot?
Comments on "What's Happened to Malaysia?"
I recently posted on what I thought was the removal of Malaysian universities from the THES-QS survey of academic opinion.
It appears, as comments on the post and my own fiddling about with the survey indicate, that what has happened is that the survey is divided into two parts: the first in which all universities in the country where the respondent works are excluded and a second in which only those universities -- minus the one where the respondent works -- are presented.
It seems that I have been unfair to QS on this occasion. There are, however, some issues about the survey that I will discuss in a little while.
Two comments refer to the absence on the State University of New York from the list. In fact, Stony Brook is there but not the other three campuses at Binghamton, Albany and Buffalo.
I recently posted on what I thought was the removal of Malaysian universities from the THES-QS survey of academic opinion.
It appears, as comments on the post and my own fiddling about with the survey indicate, that what has happened is that the survey is divided into two parts: the first in which all universities in the country where the respondent works are excluded and a second in which only those universities -- minus the one where the respondent works -- are presented.
It seems that I have been unfair to QS on this occasion. There are, however, some issues about the survey that I will discuss in a little while.
Two comments refer to the absence on the State University of New York from the list. In fact, Stony Brook is there but not the other three campuses at Binghamton, Albany and Buffalo.
Thursday, July 10, 2008
How to get into the Top 50
Malaysia has declared that it wants to get a couple of local universities into the world's top 50. This obviously means the THES-QS and not the Shanghai Jiao Tong index.
So what should a Malaysian university do to get into the 50 on the THES-QS ranking? I assume that the likeliest candidate is Universiti Malaya (UM) and that it should try to equal the score of the University of Auckland, which was 50th in last year's index.
In 2007, Auckland got a total score of 77.5 made up of 95 for the "peer review", 83 for the employer review, 38 for student faculty ratio, 61 for citations per faculty, 100 for international faculty, and 99 for international students.
UM got a total of 49.4 made up 66 for the "peer review", 66 for the employer review, 38 for the student faculty ratio, 14 for citations per faculty, 63 for international faculty and 41 for international students.
So what does UM have to do to get a score equal to Auckland's?
- First of all , it should make sure that UM is actually included in the survey of academic opinion (see previous post). If it is not, then UM will probably not even make it into the top 500. Then it has to improve its score on this criterion by about half (the use of Z scores means that we can't be more exact than this) .
- Then the score on the employer review would have to increase by about a third.
- No need to worry about the student faculty ratio. UM is as good as Auckland.
- The number of citations per faculty would have to improve four-fold.
- The proportion of international students would have to increase by two thirds.
- The proportion of international students would have to more than double.
I doubt that it is worth trying to do better on the "peer review" and the "employer review" since these are opaque and biased. Increasing the number of international faculty and students would probably cause more trouble than its worth without strict controls over quality.
Could the citations per faculty be improved? This is not totally impossible. Over ten years Cardiff University managed to triple total research output. According to an interview with the Star, noted in an earlier post, this is how it was done.
To encourage productivity, Prof Smith switched the promotion system from a quota-based system (where the total number of professorial positions in a faculty were pre-determined) to a performance-based one.
He even offered an attractive retirement package to faculty members who were not producing much research.
However, in order for universities to be able to do that, Prof Smith said they need autonomy.
“The university has to be free to offer different contracts (to academics and scientists).
“And within the university, a lot of power needs to be devolved to the young people.
“It's all about having decisions taken at the lowest level practicable.
“That’s a major change,” he said.
What's Happened to Malaysia?
It's that time of year again. A few days ago I received an invitation to take part in the THES-QS "peer review" Last year it came via World Scientific, the Singapore-based publishing company whose subscription list has been used by QS to construct their list of "smart people " who pass judgement on the world's universities. This year it came directly from QS. I do not know whether this means that QS is now using the THES subscription list as its database. If so, we can expect to see some wild fluctuations in the "peer review" and hence the overall rankings in October.
Anyway, here is message from QS.
It's that time of year again. Each year the response to the academic peer review for the Times Higher - QS World University Rankings goes from strength to strength.
QS and Times Higher Education are committed to making these rankings as strong and robust as they can be. Many enhancements have been made to the system in the last 12 months (you can read about them on http://www.topuniversities.com/) but amongst the most important has been your help in increasing the response to our academic peer review questionnaire.
Put simply... your opinion counts. Please share it with us.
You'll notice some slight differences to the survey this year - mainly in that we have included a lot more universities - as a result there are two questions about universities - one asking about those around the world, and a second asking about your own country specifically.
Please be as accurate and honest as possible. Help us make sure that your university contributes a representative response to the survey this year.
http://research.qsnetwork.com/academic
The deadline for response is July 15th.
At the end of the survey, from a selection of offers, you will have the chance to either...
Opt for a $100 discount on delegate's fee for QS APPLE (Asia Pacific Professional Leaders in Education Conference & Exhibition) 2009 in Kuala Lumpur
Enter a draw to win your university a free exhibition table at the QS World Grad School Tour (the world's leading circuit of Masters & PhD recruitement fairs) in a city of your choice
Receive a month month trial subscription to Times Higher Education
Thank you for taking the time to share your opinions with us, and please look out for the results of the Times Higher - QS World University Rankings 2008 - due to be published on October 9th.
Many thanks,
Ben SowterHead of ResearchQS
I was disappointed that I would not have a chance of getting a BlackBerry this time around. I started to fill out the form but stopped when I noticed something very odd. There are no Malaysian universities listed this year. Possible explanations are:
A. For some reason, QS have decided that no Malaysian university is of sufficient quality to even be included in the survey. This is unlikely considering some of the others that are included.
B. QS state at the start of the survey that the respondent's own university will be excluded from consideration. Since I work at a Malaysian university it is to be expected that that particular university would not show up. Perhaps some sort of error has meant that all Malaysian universities have been excluded from the list presented to me.
C. QS have made a principled decision that respondents are not allowed to choose any university in the country in which they work. This would be a good idea and therefore can probably be ruled out straightaway.
D. QS just forgot about Malaysia.
E. A computer error that affected me and nobody else.
Based on past experience, D seems the most likely, followed by B. If D, then in October Malaysian universities are going to get zero on this year's "peer review" and therefore will fall even further in the rankings. There will no doubt be an mass outbreak of soul searching in Malaysian universities and jeering by the opposition. If B, The fall will not be so great but Malaysian universities would still suffer if they are the only ones who cannot receive votes from within the country.
I appeal to any reader of this blog who has completed or is about to complete the THES-QS survey to let me know whether they have also noticed the omission of Malaysian universities or of any other country.
It's that time of year again. A few days ago I received an invitation to take part in the THES-QS "peer review" Last year it came via World Scientific, the Singapore-based publishing company whose subscription list has been used by QS to construct their list of "smart people " who pass judgement on the world's universities. This year it came directly from QS. I do not know whether this means that QS is now using the THES subscription list as its database. If so, we can expect to see some wild fluctuations in the "peer review" and hence the overall rankings in October.
Anyway, here is message from QS.
It's that time of year again. Each year the response to the academic peer review for the Times Higher - QS World University Rankings goes from strength to strength.
QS and Times Higher Education are committed to making these rankings as strong and robust as they can be. Many enhancements have been made to the system in the last 12 months (you can read about them on http://www.topuniversities.com/) but amongst the most important has been your help in increasing the response to our academic peer review questionnaire.
Put simply... your opinion counts. Please share it with us.
You'll notice some slight differences to the survey this year - mainly in that we have included a lot more universities - as a result there are two questions about universities - one asking about those around the world, and a second asking about your own country specifically.
Please be as accurate and honest as possible. Help us make sure that your university contributes a representative response to the survey this year.
http://research.qsnetwork.com/academic
The deadline for response is July 15th.
At the end of the survey, from a selection of offers, you will have the chance to either...
Opt for a $100 discount on delegate's fee for QS APPLE (Asia Pacific Professional Leaders in Education Conference & Exhibition) 2009 in Kuala Lumpur
Enter a draw to win your university a free exhibition table at the QS World Grad School Tour (the world's leading circuit of Masters & PhD recruitement fairs) in a city of your choice
Receive a month month trial subscription to Times Higher Education
Thank you for taking the time to share your opinions with us, and please look out for the results of the Times Higher - QS World University Rankings 2008 - due to be published on October 9th.
Many thanks,
Ben SowterHead of ResearchQS
I was disappointed that I would not have a chance of getting a BlackBerry this time around. I started to fill out the form but stopped when I noticed something very odd. There are no Malaysian universities listed this year. Possible explanations are:
A. For some reason, QS have decided that no Malaysian university is of sufficient quality to even be included in the survey. This is unlikely considering some of the others that are included.
B. QS state at the start of the survey that the respondent's own university will be excluded from consideration. Since I work at a Malaysian university it is to be expected that that particular university would not show up. Perhaps some sort of error has meant that all Malaysian universities have been excluded from the list presented to me.
C. QS have made a principled decision that respondents are not allowed to choose any university in the country in which they work. This would be a good idea and therefore can probably be ruled out straightaway.
D. QS just forgot about Malaysia.
E. A computer error that affected me and nobody else.
Based on past experience, D seems the most likely, followed by B. If D, then in October Malaysian universities are going to get zero on this year's "peer review" and therefore will fall even further in the rankings. There will no doubt be an mass outbreak of soul searching in Malaysian universities and jeering by the opposition. If B, The fall will not be so great but Malaysian universities would still suffer if they are the only ones who cannot receive votes from within the country.
I appeal to any reader of this blog who has completed or is about to complete the THES-QS survey to let me know whether they have also noticed the omission of Malaysian universities or of any other country.
Friday, June 27, 2008
Why are German universities so bad?
A recent post by Steve Sailer compares the Shanghai Jiao Tong and the QS-Times Higher Education Supplement rankings (although he refers to the London Times, a completely different publication).
He asserts
"This Chinese list seems less chauvinistically biased than the London Times rankings I cited in tonight's VDARE article (Harvard #1 in both, but Stanford is #2 on the Chinese list vs. #19 on the English list, behind a number of obscure provincial colleges in England). Because it's a better list, it supports the point I made in VDARE even more strongly than the previous list did: that America's exclusive universities are now enormously prestigious relative to Germany's and the rest of the world's.
German colleges that would have dominated the list 100 years ago have been hit hard by sincere, leftist anti-elitism"
That the Chinese rankings are less chauvinistic than QS-THES is absolutely correct. And if he is being deliberately offensive by describing Oxford, Cambridge and Imperial and University Colleges, London as "obscure provincial colleges in England", I suppose that I couldn't really argue.
But surely the decline of German universities began well before leftist anti-elitism appeared on the scene? Didn't it begin with the mass expulsion of Jewish students and academics after 1933?
A recent post by Steve Sailer compares the Shanghai Jiao Tong and the QS-Times Higher Education Supplement rankings (although he refers to the London Times, a completely different publication).
He asserts
"This Chinese list seems less chauvinistically biased than the London Times rankings I cited in tonight's VDARE article (Harvard #1 in both, but Stanford is #2 on the Chinese list vs. #19 on the English list, behind a number of obscure provincial colleges in England). Because it's a better list, it supports the point I made in VDARE even more strongly than the previous list did: that America's exclusive universities are now enormously prestigious relative to Germany's and the rest of the world's.
German colleges that would have dominated the list 100 years ago have been hit hard by sincere, leftist anti-elitism"
That the Chinese rankings are less chauvinistic than QS-THES is absolutely correct. And if he is being deliberately offensive by describing Oxford, Cambridge and Imperial and University Colleges, London as "obscure provincial colleges in England", I suppose that I couldn't really argue.
But surely the decline of German universities began well before leftist anti-elitism appeared on the scene? Didn't it begin with the mass expulsion of Jewish students and academics after 1933?
Tuesday, June 24, 2008
Resumption of Posting
Teaching and family affairs have kept me away from this blog for a few months. I hope to start posting regularly again soon.
QS’s Greatest Hits: Part One
For the moment, it might be interesting to review some of the more spectacular errors of QS Quacquarelli Symonds Ltd (QS), the consultants who collect the data for the Times Higher Education Supplement’s (THES) World University Rankings.
During its short venture into the ranking business QS has shown a remarkable flair for error. In terms of quantity and variety they have no peers. All rankers make mistakes now and then but so far there has been nobody quite like QS.
Here is a list of what I think are QS’s ten best errors, mainly from the rankings and their book, Guide to the World’s Top Universities (2007). Most of them have been discussed in earlier posts on this blog. The date of these posts is given in brackets. There is one error, relating to Washington University in St. Louis, from last year’s rankings,
It has to be admitted that QS seem to be doing better recently. Or perhaps I have not been looking as hard as I used to. I hope that another ten errors will follow shortly.
One: Faculty Student Ratio in Guide to the World’s Top Universities (2007). (July 27, 2007; May 11, 2007)
This is a beautiful example of the butterfly effect, with a single slip of the mouse leading to literally hundreds of mistakes.
QS’s book, Guide to the World’s Top Universities, was produced at the end of 2006 after the publication of the rankings for that year and contained data about student faculty ratios of over 500 ranked universities. It should have been obvious immediately that there was something wrong with this data. Yale is given a ratio of 34.1, Harvard 18, Cambridge 18.9 and Pretoria 590 .3. On the other hand, there are some ridiculously low figures such as 3.5 for Dublin Institute of Technology and 6.1 for the University of Santo Tomas (Philippines).
Sometimes the ratios given flatly contradict information given on the same page. So, on page 127 in the FACTFILE, we are told that Yale has a student faculty ratio of 34.3. Over on the left we are informed that Yale has around 10,000 students and 3,333 faculty.
There is also no relationship between the ratios and the scores out of 100 in the THES QS rankings for student faculty ratio, something that Matt Rayner asked about, without ever receiving a reply, on QS’s topuniversities web site.
So what happened? It’s very simple. Someone slipped three rows when copying and pasting data and every single student faculty ratio in the book, over 500 of them, is wrong. Dublin Institute of Technology was given Duke’s ratio (more about that later), Pretoria got Pune’s, Aachen RWT got Aberystwyth’s (Wales). And so on. Altogether over 500 errors.
Two: International Students and Faculty in Malaysian Universities.
In 2004 there was great jubilation at Universiti Malaya (UM) in Malaysia. The university had reached 89th place in the THES-QS world rankings. Universiti Sains Malaysia (USM) also did very well. Then in 2005 came disaster. UM crashed 100 places, seriously damaging the Vice-Chancellor’s career, and USM disappeared from the top 200 altogether. The Malaysian political opposition had a field day blasting away at the supposed incompetence of the university leadership.
The dramatic decline should have been no surprise at all. A Malaysian blogger had already noticed that the figures for international students and faculty in 2004 were unrelated to reality. What happened was that in 2004 QS were under the impression that larger numbers of foreigners were studying and teaching at the two Malaysian universities. Actually, there were just a lot of Malaysian citizens of Indian and Chinese descent. In 2005 the error was corrected causing the scores for international faculty and students to fall precipitously.
Later, THES referred to this as “a clarification of data”, a piece of elegant British establishment obfuscation that is almost as good as “being economical with the truth”
Three: Duke’s student faculty ratio 2005 ( October 30, 2006 )
Between 2004 and 2005 Duke rose dramatically in the rankings. It did so mainly because it had been given a very low and incredible student faculty ratio in the latter year, less than two students per faculty. This was not the best ratio in the rankings. That supposedly belonged to Ecole Polytechnique in Paris (more of that later). But it was favourable enough to give Duke a powerful boost in the rankings.
The ratio was the result of a laughable error. QS listed Duke as having 6,244 faculty, well in excess of anything claimed on the university’s web site. Oddly enough, this was exactly the number of undergraduate students enrolled at Duke in the fall of 2005. Somebody evidently had copied down the figure for undergraduate students and counted them as faculty, giving Duke four times the number of faculty it should have.
Four: Duke’s student faculty ratio 2006 (December 16, 2006)
Having made a mess of Duke’s student faculty ratio in 2005, QS pulled off a truly spectacular feat in 2006 by making an even bigger mess. The problem, I suspect, was that Duke’s public relations office had its hands full with the Lacrosse rape hoax and that the web site had not been fully updated since the fall of 2005. For students, QS apparently took undergraduate student enrollment in the fall of 2005, subtracted the number of undergraduate degrees awarded and added the 2005 intake. This is a bit crude because some students would leave without taking a degree, Reade Seligmann and Colin Finnerty for example, but probably not too inaccurate. Then, there was a bit of a problem because while the number of postgraduate degrees awarded was indicated on the site there was no reference to postgraduate admissions. So, QS seem to have deducted the degrees awarded and added what they thought was number of postgraduate students admitted, 300 of them, to the Pratt School of Engineering, which is an undergraduate, not a graduate school. Then, in a final flourish they calculated the number of faculty by doubling the figure on the Duke site, apparently because Duke listed the same number classified first by department and then by status.
The result was that the number of students was undercounted and the number of faculty seriously overcounted, giving Duke the best student faculty ratio for the year. Although the ratio was higher than in 2005 Duke was now in first place for this section because QS had calculated more realistic ratios for the Ecole Polytechnique and the Ecole Normale Superieure.
Five: Omission of Kenan Flagler from the Fortune business school rankings. (March 05, 2007)
On the surface this was a trivial error compared to some that QS has committed. They got the business school at the University of North Carolina mixed up with that of North Carolina State University. The grossness of this error is that while most American universities seem unconcerned about the things that QS writes or does not write about them, business schools evidently feel that more is at stake and also have considerable influence over the magazines and newspaper that publish rankings. Kenan-Flagler protested vociferously over its omission, Fortune pulled the ranking off its site, Nunzio Quacquarelli, director of QS, explained that it was the result of a lapse by a junior employee and stated that this sort of thing had never happened before and would never happen again.
Six: "Beijing University"
China’s best or second best university is Peking University. The name has not been changed to Beijing University apparently to avoid confusion with Beijing Normal University. There are also over twenty specialist universities in Beijing: Traditional Chinese Medicine, Foreign Languages, Aeronautics and so on.
In 2004 and 2005 THES and QS referred to Beijing University finally correcting it to Peking University in 2006.
This was perhaps not too serious an error except that it revealed something about QS’s knowledge of its own sources and procedures.
In November 2005. Nunzio Quacquarelli went to a meeting in Kuala Lumpur, Malaysia. Much of the meeting was about the international students and faculty at UM and USM. There was apparently also a question about how Beijing University could have got such a magnificent score on the peer review while apparently producing almost no research. The correct answer would have been that QS was trying to find research written by scholars at Beijing University, which does not exist. Quacquarelli, however, answered that “we just couldn’t find the research” because Beijing University academics published in Mandarin (Kuala Lumpur New Straits Times 20/11/05).
This is revealing because QS’s “peer review” is actually nothing more than a survey of the subscribers to World Scientific, a Singapore-based company that publishes academic books and journals, many of them Asia-orientated and mostly written in English. World Scientific has very close ties with Peking University. If Quacquarelli knew very much about the company that produces his company’s survey he would surely have known that it had a cozy relationship with Peking University and that Chinese researchers, in the physical sciences at least, do quite a lot of publishing in English.
Seven: Student faculty ratios at Yonsei and Korea universities (November 08, 2006)
Another distinguished university administrator whose career suffered because of a QS error was of Yonsei University. This university is a rival of Korea University and was on most measures its equal or superior. But on the THES – QS rankings it was way behind, largely because of a poor student faculty ratio. As it happened, the figure given for Korea University was far too favourable and much better even than the ratio admitted by the university itself. This did not, however, help Jung Chang-Young who had to resign.
Eight: Omission of SUNY – Binghamton, Buffalo and Albany
THES and QS have apologized for omitting the British universities of Lancaster, Essex and Royal Holloway. A more serious omission is the omission of the State University of New York’s (SUNY) University Centres at Buffalo, Albany and Binghamton. SUNY has four autonomous university centres which are normally treated as independent and are now often referred to as the Universities of Buffalo and Albany and Binghamton University. THES-QS does refer to one university centre as Stony Brook University, probably being under the impression that this is the entirety of the SUNY system. Binghamton is ranked 82nd according to the USNWR and 37th among public national universities (2008). It can boast several internationally known scholars such as Melvin Dubofsky in labour history and Immanuel Wallerstein in sociology. To exclude it from the rankings while including the likes of Dublin Institute of Technology and the University of Pune is ridiculous.
Nine: Student faculty ratio at Ecole Polytechnique (September 08, 2006)
In 2005 the Ecole Polytechnique went zooming up the rankings to become the best university in continental Europe. Then in 2006 it went zooming down again. All this was s because of extraordinary fluctuations in the student faculty ratio. What happened could be determined by looking at the data on QS’s topgraduate site. Clicking on the rankings for 2005 led to the data that was used for that year (it is no longer available). There were two sets of data for students and faculty for that year, evidently one containing part-time faculty and another with only full time faculty. It seems that in 2005 part-time faculty were counted but not in 2006.
Ten: Washington University in St Louis (November 11, 2007)
This is a leading university in every respect. Yet in 2007, QS gave it a score of precisely one for citations per faculty, behind Universitas Gadjah Mada, the Dublin Institute of Technology and Politecnico di Milano and sent it falling from 48th to 161st in the overall rankings. What happened was that QS got mixed up with the University of Washington (in Seattle) and gave all WUSL’s citations to the latter school.
Teaching and family affairs have kept me away from this blog for a few months. I hope to start posting regularly again soon.
QS’s Greatest Hits: Part One
For the moment, it might be interesting to review some of the more spectacular errors of QS Quacquarelli Symonds Ltd (QS), the consultants who collect the data for the Times Higher Education Supplement’s (THES) World University Rankings.
During its short venture into the ranking business QS has shown a remarkable flair for error. In terms of quantity and variety they have no peers. All rankers make mistakes now and then but so far there has been nobody quite like QS.
Here is a list of what I think are QS’s ten best errors, mainly from the rankings and their book, Guide to the World’s Top Universities (2007). Most of them have been discussed in earlier posts on this blog. The date of these posts is given in brackets. There is one error, relating to Washington University in St. Louis, from last year’s rankings,
It has to be admitted that QS seem to be doing better recently. Or perhaps I have not been looking as hard as I used to. I hope that another ten errors will follow shortly.
One: Faculty Student Ratio in Guide to the World’s Top Universities (2007). (July 27, 2007; May 11, 2007)
This is a beautiful example of the butterfly effect, with a single slip of the mouse leading to literally hundreds of mistakes.
QS’s book, Guide to the World’s Top Universities, was produced at the end of 2006 after the publication of the rankings for that year and contained data about student faculty ratios of over 500 ranked universities. It should have been obvious immediately that there was something wrong with this data. Yale is given a ratio of 34.1, Harvard 18, Cambridge 18.9 and Pretoria 590 .3. On the other hand, there are some ridiculously low figures such as 3.5 for Dublin Institute of Technology and 6.1 for the University of Santo Tomas (Philippines).
Sometimes the ratios given flatly contradict information given on the same page. So, on page 127 in the FACTFILE, we are told that Yale has a student faculty ratio of 34.3. Over on the left we are informed that Yale has around 10,000 students and 3,333 faculty.
There is also no relationship between the ratios and the scores out of 100 in the THES QS rankings for student faculty ratio, something that Matt Rayner asked about, without ever receiving a reply, on QS’s topuniversities web site.
So what happened? It’s very simple. Someone slipped three rows when copying and pasting data and every single student faculty ratio in the book, over 500 of them, is wrong. Dublin Institute of Technology was given Duke’s ratio (more about that later), Pretoria got Pune’s, Aachen RWT got Aberystwyth’s (Wales). And so on. Altogether over 500 errors.
Two: International Students and Faculty in Malaysian Universities.
In 2004 there was great jubilation at Universiti Malaya (UM) in Malaysia. The university had reached 89th place in the THES-QS world rankings. Universiti Sains Malaysia (USM) also did very well. Then in 2005 came disaster. UM crashed 100 places, seriously damaging the Vice-Chancellor’s career, and USM disappeared from the top 200 altogether. The Malaysian political opposition had a field day blasting away at the supposed incompetence of the university leadership.
The dramatic decline should have been no surprise at all. A Malaysian blogger had already noticed that the figures for international students and faculty in 2004 were unrelated to reality. What happened was that in 2004 QS were under the impression that larger numbers of foreigners were studying and teaching at the two Malaysian universities. Actually, there were just a lot of Malaysian citizens of Indian and Chinese descent. In 2005 the error was corrected causing the scores for international faculty and students to fall precipitously.
Later, THES referred to this as “a clarification of data”, a piece of elegant British establishment obfuscation that is almost as good as “being economical with the truth”
Three: Duke’s student faculty ratio 2005 ( October 30, 2006 )
Between 2004 and 2005 Duke rose dramatically in the rankings. It did so mainly because it had been given a very low and incredible student faculty ratio in the latter year, less than two students per faculty. This was not the best ratio in the rankings. That supposedly belonged to Ecole Polytechnique in Paris (more of that later). But it was favourable enough to give Duke a powerful boost in the rankings.
The ratio was the result of a laughable error. QS listed Duke as having 6,244 faculty, well in excess of anything claimed on the university’s web site. Oddly enough, this was exactly the number of undergraduate students enrolled at Duke in the fall of 2005. Somebody evidently had copied down the figure for undergraduate students and counted them as faculty, giving Duke four times the number of faculty it should have.
Four: Duke’s student faculty ratio 2006 (December 16, 2006)
Having made a mess of Duke’s student faculty ratio in 2005, QS pulled off a truly spectacular feat in 2006 by making an even bigger mess. The problem, I suspect, was that Duke’s public relations office had its hands full with the Lacrosse rape hoax and that the web site had not been fully updated since the fall of 2005. For students, QS apparently took undergraduate student enrollment in the fall of 2005, subtracted the number of undergraduate degrees awarded and added the 2005 intake. This is a bit crude because some students would leave without taking a degree, Reade Seligmann and Colin Finnerty for example, but probably not too inaccurate. Then, there was a bit of a problem because while the number of postgraduate degrees awarded was indicated on the site there was no reference to postgraduate admissions. So, QS seem to have deducted the degrees awarded and added what they thought was number of postgraduate students admitted, 300 of them, to the Pratt School of Engineering, which is an undergraduate, not a graduate school. Then, in a final flourish they calculated the number of faculty by doubling the figure on the Duke site, apparently because Duke listed the same number classified first by department and then by status.
The result was that the number of students was undercounted and the number of faculty seriously overcounted, giving Duke the best student faculty ratio for the year. Although the ratio was higher than in 2005 Duke was now in first place for this section because QS had calculated more realistic ratios for the Ecole Polytechnique and the Ecole Normale Superieure.
Five: Omission of Kenan Flagler from the Fortune business school rankings. (March 05, 2007)
On the surface this was a trivial error compared to some that QS has committed. They got the business school at the University of North Carolina mixed up with that of North Carolina State University. The grossness of this error is that while most American universities seem unconcerned about the things that QS writes or does not write about them, business schools evidently feel that more is at stake and also have considerable influence over the magazines and newspaper that publish rankings. Kenan-Flagler protested vociferously over its omission, Fortune pulled the ranking off its site, Nunzio Quacquarelli, director of QS, explained that it was the result of a lapse by a junior employee and stated that this sort of thing had never happened before and would never happen again.
Six: "Beijing University"
China’s best or second best university is Peking University. The name has not been changed to Beijing University apparently to avoid confusion with Beijing Normal University. There are also over twenty specialist universities in Beijing: Traditional Chinese Medicine, Foreign Languages, Aeronautics and so on.
In 2004 and 2005 THES and QS referred to Beijing University finally correcting it to Peking University in 2006.
This was perhaps not too serious an error except that it revealed something about QS’s knowledge of its own sources and procedures.
In November 2005. Nunzio Quacquarelli went to a meeting in Kuala Lumpur, Malaysia. Much of the meeting was about the international students and faculty at UM and USM. There was apparently also a question about how Beijing University could have got such a magnificent score on the peer review while apparently producing almost no research. The correct answer would have been that QS was trying to find research written by scholars at Beijing University, which does not exist. Quacquarelli, however, answered that “we just couldn’t find the research” because Beijing University academics published in Mandarin (Kuala Lumpur New Straits Times 20/11/05).
This is revealing because QS’s “peer review” is actually nothing more than a survey of the subscribers to World Scientific, a Singapore-based company that publishes academic books and journals, many of them Asia-orientated and mostly written in English. World Scientific has very close ties with Peking University. If Quacquarelli knew very much about the company that produces his company’s survey he would surely have known that it had a cozy relationship with Peking University and that Chinese researchers, in the physical sciences at least, do quite a lot of publishing in English.
Seven: Student faculty ratios at Yonsei and Korea universities (November 08, 2006)
Another distinguished university administrator whose career suffered because of a QS error was of Yonsei University. This university is a rival of Korea University and was on most measures its equal or superior. But on the THES – QS rankings it was way behind, largely because of a poor student faculty ratio. As it happened, the figure given for Korea University was far too favourable and much better even than the ratio admitted by the university itself. This did not, however, help Jung Chang-Young who had to resign.
Eight: Omission of SUNY – Binghamton, Buffalo and Albany
THES and QS have apologized for omitting the British universities of Lancaster, Essex and Royal Holloway. A more serious omission is the omission of the State University of New York’s (SUNY) University Centres at Buffalo, Albany and Binghamton. SUNY has four autonomous university centres which are normally treated as independent and are now often referred to as the Universities of Buffalo and Albany and Binghamton University. THES-QS does refer to one university centre as Stony Brook University, probably being under the impression that this is the entirety of the SUNY system. Binghamton is ranked 82nd according to the USNWR and 37th among public national universities (2008). It can boast several internationally known scholars such as Melvin Dubofsky in labour history and Immanuel Wallerstein in sociology. To exclude it from the rankings while including the likes of Dublin Institute of Technology and the University of Pune is ridiculous.
Nine: Student faculty ratio at Ecole Polytechnique (September 08, 2006)
In 2005 the Ecole Polytechnique went zooming up the rankings to become the best university in continental Europe. Then in 2006 it went zooming down again. All this was s because of extraordinary fluctuations in the student faculty ratio. What happened could be determined by looking at the data on QS’s topgraduate site. Clicking on the rankings for 2005 led to the data that was used for that year (it is no longer available). There were two sets of data for students and faculty for that year, evidently one containing part-time faculty and another with only full time faculty. It seems that in 2005 part-time faculty were counted but not in 2006.
Ten: Washington University in St Louis (November 11, 2007)
This is a leading university in every respect. Yet in 2007, QS gave it a score of precisely one for citations per faculty, behind Universitas Gadjah Mada, the Dublin Institute of Technology and Politecnico di Milano and sent it falling from 48th to 161st in the overall rankings. What happened was that QS got mixed up with the University of Washington (in Seattle) and gave all WUSL’s citations to the latter school.
Friday, February 22, 2008
More on Cambridge and Harvard
Alejandro Pisanty has an interesting comment on the previous post. I will reproduce a large part of here.
“In particular for Harvard it's darn tricky. "Harvard University" will
yield only a fraction of the production and the citations from there.
There's also Harvard Medical School, Harvard Business School, Harvard
Law School, etc., and nifty arrangements like Harvard-Smisthsonian
Astronomy Project (also in Cambridge MA; surely a half-floor of a physics or
astronomy unit in the best of cases) and so on.
QS and THES admit quite cynically that they don't really know too well
how to treat "children institutions". One can be sure that officials from
Harvard and Cambridge, and all British universities, have been well on
top of this by constant contact with QS and their staff. And, it all
happens in English.
One would reasonably excpect that QS does not apply the same care to
Malaysian or Mexican universities...”
The number of papers produced by the Harvard Business and Law Schools is relatively small although still a lot more than the Judge School of Business at Cambridge or Addenbrookes Hospital. Harvard Medical School, however, does produce a massive number of papers, over 35,000 according to Scopus between 2002 and 2006. Compare this with 12, 736 for "Harvard University" over the same period.
If QS did indeed count the papers produced by authors with a Harvard Medical School affiliation this would be an adequate -- probably more than adequate -- explanation for Harvard’s superiority over Cambridge in terms of citations. But another problem now arises. The number of citations per faculty would now be much larger than Caltech which does a bit better than Harvard in the THES-QS citations per faculty section.
It is possible that QS included the papers produced by HMS and then also counted " about [sic}10,674 medical school faculty". Not to do so would be absurd since any other procedure would mean that linguists, sociologists and engineers were getting credit for producing medical research.
But if QS counted papers with a Harvard Medical School affiliation and also counted all the medical faculty then we would be back where we started.
It still seems to me that the most plausible reconstruction of Harvard’s citations per faculty score is that QS did not count papers produced by the various schools, or least not by the Harvard Medical School, and that for the faculty figure they used the number given on the Harvard website or in QS’s school profile.
All this speculation would be unnecessary if QS told us exactly what they did but I wouldn’t bother waiting for that to happen.
Alejandro Pisanty has an interesting comment on the previous post. I will reproduce a large part of here.
“In particular for Harvard it's darn tricky. "Harvard University" will
yield only a fraction of the production and the citations from there.
There's also Harvard Medical School, Harvard Business School, Harvard
Law School, etc., and nifty arrangements like Harvard-Smisthsonian
Astronomy Project (also in Cambridge MA; surely a half-floor of a physics or
astronomy unit in the best of cases) and so on.
QS and THES admit quite cynically that they don't really know too well
how to treat "children institutions". One can be sure that officials from
Harvard and Cambridge, and all British universities, have been well on
top of this by constant contact with QS and their staff. And, it all
happens in English.
One would reasonably excpect that QS does not apply the same care to
Malaysian or Mexican universities...”
The number of papers produced by the Harvard Business and Law Schools is relatively small although still a lot more than the Judge School of Business at Cambridge or Addenbrookes Hospital. Harvard Medical School, however, does produce a massive number of papers, over 35,000 according to Scopus between 2002 and 2006. Compare this with 12, 736 for "Harvard University" over the same period.
If QS did indeed count the papers produced by authors with a Harvard Medical School affiliation this would be an adequate -- probably more than adequate -- explanation for Harvard’s superiority over Cambridge in terms of citations. But another problem now arises. The number of citations per faculty would now be much larger than Caltech which does a bit better than Harvard in the THES-QS citations per faculty section.
It is possible that QS included the papers produced by HMS and then also counted " about [sic}10,674 medical school faculty". Not to do so would be absurd since any other procedure would mean that linguists, sociologists and engineers were getting credit for producing medical research.
But if QS counted papers with a Harvard Medical School affiliation and also counted all the medical faculty then we would be back where we started.
It still seems to me that the most plausible reconstruction of Harvard’s citations per faculty score is that QS did not count papers produced by the various schools, or least not by the Harvard Medical School, and that for the faculty figure they used the number given on the Harvard website or in QS’s school profile.
All this speculation would be unnecessary if QS told us exactly what they did but I wouldn’t bother waiting for that to happen.
Subscribe to:
Posts (Atom)