Top Universities Guide

I have finally forced myself to buy and read the second edition of QS's book, retitled Top Universities Guide.

The first edition, which was called Guide to the World's Top Universities, was an unqualified disaster, full of errors the worst of which was getting every single student faculty ratio wrong as a result of somebody moving every university three rows down while copying data. Others included putting the Technical University of Munich in the profiles twice in positions 82 and 98 and listing an "Official University of California, Riverside". It also started to fall apart within a few days.

The new edition, however, tuned out to be a pleasant surprise. The previous errors seem to have been corrected and there do not appear to be any new ones. There is a new dignified purple cover and so far not a single page has fallen out.

This does not mean that the THE-QS rankings are faultless. Far from it. The peer review remains extremely biased, the international components are largely meaningless and the student faculty ratio is misleading and easily manipulable. Still, the book does show that in a technical sense, QS has improved quite a bit recently.

The change in title should be noted. There may be another reason, but this would appear to be a clever attempt to distance QS and the authors from the first edition without admitting that there was anything wrong with it.

And I wonder why Blackwell is no longer distributing the book.

A Must for University Ranking Fans

University rankings have been sprouting up all over the place recently. Laura Milligan at LearningXL has an annotated list of "100 Free College Rankings". It is very Americocentric -- neither the Shanghai nor the THE-QS rankings are there -- but there is something for nearly everybody. The strangest one is probably the Campus Squirrel Listings. The best, for those who are interested, include Berkeley, Mary Baldwin College and Kansas State University.

Comment on the THE-QS Academic Survey

QS have published some information on the academic survey that makes up 40 % of their World University Rankings. The table listing repondents by country is especially interesting. Here are some highlights

United States 638
United Kingdom 563
Australia 286
Inonesia 228
Malaysia 180
Philippones 201
France 125
Belgium 124
Germany 182
China 116
Kong Kong 100
Japan 96
Ireland 78
Thailand 43
Russia 41

So there are almost as many respondents from the UK as from the USA. There are more from Indonesia than from Germany. There are more from Thailand than from Russia. There are more from Hong Kong than from Japan.

Overall, there is pronounced bias in favour of the UK and Ireland, Southeast Asia and Australasia and against the USA, Japan and Southwest Asia.

What does 100 on the academic survey mean?

This year 21 universities got a score of 100 on the academic survey section of the THE-QS rankings. A look at the subject rankings, which are combined with equal weighting to form the total score for this indicator, shows that such a perfect score can mean many different things.

Harvard scored 1oo in Arts and Humanities, 100 in Life Sciences and Biomedicine, 96.1 in Natural Sciences, 100 in Social Sciences and 59.6 in Engineering and IT.

The Australian National University scored 74 in Arts and Humanities, 46 .9 in Life Sciences and Biomedicine, 66.1 in Natural Sciences, 71.4 in Social sciences and 49.9 in Engineering and IT.

Peking University scored 56.4 for Arts and humanities, 56.9 for Life sciences and Biomedicine, 73 for Natural Sciences, 57.8 for Social Sciences, and 39.2 for Engineering and IT.

Farewell, Alabama

In 2007 the University of Alabama was listed by THE-QS as the fifth best research university in the world, as measured by citations per faculty. This year it is not even even in the top100. What happened?

Did all those researchers go on strike?

Or is just that that last year the number of faculty was underestimated and this year the mistake was corrected?

Ratio Inflation or Genuine Improvement?

QS have helpfully provided the means and standard deviations for their ranking indicators. For student faculty the mean is 0.09, which works out as a mean of 11.11 students per faculty.

But looking at the data for 2006, the mean number of students per faculty (using the scores provided by QS and cross-checking with the data on their website) was 16.44 students per faculty (N = 531). There has, it would seem, been a very substantial improvement on this indicator in only two years.

Is this a real improvement?

Comment on a THE Editorial and Comments

Times Higher Education has published an editorial on the 2008 rankings that deserves comment. It says:

This is the fifth year we have published the rankings and the methodology
has remained unchanged for the past two. Along with our partners in the venture,
Quacquarelli Symonds, we make enormous efforts to ensure that our
quality-control processes and anti-cheating mechanisms are as robust as
possible.

I accept that so far this year the rankings have not been disfigured by the sort of spectacular errors that have occurred in the past but I would feel more confident about those enormous efforts if THE and QS owned up to their past errors, such as putting Duke in top place for student faculty ratio by counting undergraduate students as faculty, and indicated exactly what they are doing to stop similar mistakes from occurring again.

The editorial continues:

We try to ensure that the results are produced with a large amount of data. For example, there were more than 6,000 participants in the academic survey alone, producing an average of 20 responses per head. That is a staggering 120,000 data points, making it the largest known survey of university quality.

This is fallacious. The validity of any survey depends on how representative the respondents are of the larger population, not how many they are. If QS continued to report a response rate of around 3 percent they must expect continued criticism.

After proclaiming its lack of bias THE concludes:

For 2008, we congratulate Harvard University for its success
in topping the rankings yet again. However, it is worth remembering that its
endowment now totals more than $35 billion (£19 billion), roughly equivalent to
the total income received by the entire UK sector last year. By that measure,
the UK, with its 29 institutions in the Top 200 (and four in the Top 10), can
stand proud on the world stage.

According to THE's citations per faculty indicator the research impact of Cambridge academics is less than that of 35 other institutions including Resselaer Polytechnic Institute, Tufts, McMaster, Tel Aviv, UC Davis, Minnesota, Leiden, Emory, Toronto, Kyoto, Brown and ETH Zurch. Cambridge probably is not quite that bad -- there may be problems with the faculty side of the equation that are causing distortions -- but it does look as though the academic survey is in part an attempt to cover up the steady decline of British higher education and research.

There is a comment on the editorial by bgc:

The Times uses a non-transparent, undefined opinion
survey for most of their weightings - presumably this is what leads to such
nonsensical results.
Anyone who knows anything about international HE would
realize that the Times ranking lack basic validity.
For goodness sake,
sort-out the ranking methodology before next year. Or please stop inflicting
this annual embarrassment on those of us who practice scientometrics and try to
use objective methods of educational evaluation.

This is perhaps over-dramatic. The THE- QS rankings do seem to be improving in some respects. However, a reply by Martin Ince, editor of the rankings is rather unfortunate:

By contrast, we have measures relating to teaching,
globalisation and employability, and our research indicators cover the full
range of subjects. We set out exactly who and where our respondents are - there
is a nice pie chart in today's paper. These expert academics provide us with
about 126,000 data points (20 per person for 6,300 people) and make up the
biggest and best survey of university quality.

Whether the ability to sign on to a mailing list makes one an academic expert is debatable. And telling experts in scientometrics about your nice pie charts does you no good at all.

Not What It used To Be

The THE-QS rankings seem to be getting better. So far this year, no outrageous errors like ranking a non existent university (Beijing University), turning Malaysian ethnic minorities into foreigners or giving Washington University in St Louis a near zero for research, have surfaced.

But comparing the scores for the academic survey with those for citations for research, both of which are supposed to measure research quality, suggests that the former has is a large and systematic bias.

Here is a list of universities whose score on the academic survey exceeds their score on the citations per faculty by forty points or more.

Mcgill
Sydney
Melbourne
New York University
Monash
Trinity College Dublin
Peking
Seoul National University
Tsinghua
Auckland
London School of Economics
Warwick
Nanyang Technological University Singapore
University College Dublin
Fudan
Humboldt University Berlin
Nanjing
Shanghai Jiao Tong
Aotonomous National University of Mexico
Chulalongkorn University Thailand
Waseda
Lomonosov State University Moscow
Bologna
Sao Paulo
Beunos Aires

Maybe LSE and NYU can be explained by excellence in subjects that produce few publications or citations. But is it not possible that there is a pronounced geographical bias in the survey?

THE-QS 2008 Update

This year there has been only one methodological change, namely the separation of the lists in the academic survey section into international and domestic sections and then their recombination. This would probably work against universities that receive a lot of votes from their own countries and might explain why Hong Kong, Peking and several Australian universities have fallen quite a bit.

Also, it is likely that the geographical spread of the academic and employer surveys has expanded and that this has benefitted universities in Latin America, Africa and India.

The biggest change in the top 100 is that Washington University in St Louis has risen to 60th place from 161st in 2007. This, presumably, is because it is now getting a realistic score for citations per faculty instead of the 1 it got in 2007, when QS seem to have confused it with the University of Washington. I am a little bit suspicious though about these two places being next to each other in this year's ranking.

The University of Hong Kong has fallen from 18th to 26th, Peking from 36th to 50th, Nanyang from 69th to 77th, Melbourne from 21st to 38th and Macquarie from 168th to 183rd (I wonder what Dr. Schwartz will say about that.)

On the other hand, the National Autonomous University of Mexico has risen from 192nd to 150th, the Indian Institute of Technology Delhi from 307th to 154th and Chulalongkorn from 223rd to 166th.

One oddity that I've noticed is that Stony Brook University, which is an autonomous university centre of the State University of new York, has risen dramatically to 127th place from 224th, while the other three centres at Binghamton, Buffalo and Albany which are of equal or better quality do not even get into QS's initial list.

The THE-QS Top 200: 2008

The QS topuniversities site has listed the top 200 universities in their 2008 rankings. I hope to make a few comments in a little while.

Sarah Palin and the University of Idaho

One of the more distasteful aspects of the US presidential election is the obsession in some quarters with Sara Palin's IQ. See here for example. Some have suggested that she is not particularly bright because she graduated from the University of Idaho and not from Columbia or Harvard.

May I point out that the University of Idaho, although it is not Harvard or Columbia, is ranked in the 400s in the Shanghai Jiao Tong rankings, which puts it way above most institutions in the world and in the United States.

May I also point out that Palin, unlike Joe Biden, has not been accused of plagiarism. Also, if we are going to be snobby about universities, one also wonders why Biden, if he were so clever, would plagiarise from Neil Kinnock who had to take his final exams twice at a university that also wasn't Harvard or Columbia

Sour Grapes Watch

There is a definite smell of very sour grapes coming out of Imperial College London with the news that University College London has moved ahead in the Shanghai Jiao Tong University rankings, rising from 26th to 22th. Imperial has fallen from 23rd to 27th, a trivial distance by THES-QS standards. This account is from London Student, an online newspaper.

Jovan Nedic, Editor-in-Chief of Imperial’s student paper Felix, said: “I mean
just look at the [ARWU] website, that alone tells you that this ranking is a
joke. None of the scoring categories are explained and the ones that are obvious
are not the best indication on what university is all about. It calls itself an
academic ranking, so why is a score on Alumni important?”This ranking is nothing
more than a joke and I’m surprised that London Student are even bothering to
look at it.”As to whether the new rankings meant UCL was ‘better‘, he continued:
“Academics and employers alike all recognise the Times one as the standard, only
those results will mean anything to Imperial students, until then, UCL and the
other 21 above Imperial can (in case they don’t get that, I mean all the ones
that are between 6th and 27th) can only dream to be better!”The difference
between criteria used in university rankings is a long-standing bone of
contention. While ARWU gives higher scores to universities whose staff and
students win Nobel Prizes, The Times bases 40 per cent of its scoring on
researchers peer-review - where experts are asked to list institutions they
think are top in their area.

There are a few errors here. The ARWU categories are explained in the methodology section. Not all academics and employers recognise the Times (actually Times Higher Education, not the same thing) as the standard, especially in the USA. It is stretching things a bit to refer to the experts in the THE-QS "peer review". To be a reviewer requires no greater expertise than the ability to sign on to the mailing list of World Scientific, an academic publishing company that has -- this is no doubt completely irrelevant -- an interest in Imperial College Press.

Another University Ranking

This one is from the Global Language Monitor. which "has ranked the nation’s [USA] colleges and universities according their appearance in the global print and electronic media, as well as on the Internet and throughout the Blogosphere. "

The top ten universities are:

1. Harvard University

2. Columbia University

3. University of Michigan, Ann Arbor

4. University of California, Berkeley

5. Stanford University

6. University of Chicago

7. University of Wisconsin, Madison

8. Yale University

9. Princeton University

10. Cornell University


The top ten colleges are:

1. Colorado College

2. Williams College

3. University of Richmond

4. Middlebury College

5. Wellesley College

6. Bucknell University

7 . Amherst College

8. Oberlin College

9. Vassar College

10. Pomona College

University Research Expenditure Rankings

There is an interesting ranking of US universities by the National Science Foundation according to total research and development expenditure.

Here are the top ten:

1. Johns Hopkins
2. Wisconsin (Madison)
3. UCLA
4. Minnesota
5. UC San Francisco
6. Washington
7. UC San Diego
8. Stanford
9. University of Pennsylvania
10. Duke

But does spending a lot of money necessarily lead to a high level of research productivity?

Plagiarism Multiple Standards Watch


A couple of recent news stories illustrate the multiple standards regarding plagiarism. Allison Routman, an undergraduate student, was kicked off a university of Virginia educational cruise ship for copying precisely three phrases from Wikipedia into a film review. The exact phrases were "when the Germans attacked the Soviet Union during Operation Barbarossa", "German speaking minority outside of Germany" and "who had been released from a concentration camp".


Assuming that the student has given a full and accurate account -- we have not heard from her instructors -- this would seem far too draconian. Would we consider a researcher who wrote "recent research has clearly shown that" to be plagiarising?. That phase gets 748 results from a Google Search.


Meanwhile , a new ranking by Forbes has put the University of Southern Illinois at Carbondale in 489th place among US colleges. The president of the university, Glenn Poshard, had plagiarised a substantial part of his doctoral dissertation written for the university's Department of Higher Education but a committee found that the plagiarism was inadvertent and he just had to revise the offending sections. Neither Poshard nor the university seem to have suffered much from the affair.



Next, Joe Biden has been picked as Barack Obama's running mate. Biden is well known for his plagiarism of a speech by Neil Kinnock during his presidential campaign and that was not his first offense. Still it has not hurt very much in the long run.


Incidentally, one wonders why Biden bothered to copy such a ridiculous passage . Kinnock said

Why am I the first Kinnock in a thousand generations to be able to get
to university? Why is Glenys the first woman in her family in a thousand
generations to be able to get to university? Was it because our predecessors
were thick? Does anybody really think that they didn't get what we had because
they didn't have the talent or the strength or the endurance or the commitment?
Of course not. It was because there was no platform upon which they could
stand.

Well, for 977 of those thousand generations it was because there were no universities in Scotland for anyone to go to.

The Shanghai Rankings 2008



Shanghai Jiao Tong University (SJTU) has just released their rankings for 2008. Compared to the THE-QS rankings, public response, especially in Asia and Australia, has been slight. This is largely because ascent and descent within the Shanghai index is minimal, a tribute to their reliability. In contrast, the THE-QS rankings, with their changes in methodology and frequent errors, arouse almost as much interest as a country's performance in the Olympics.



Still, it is instructive to check how well various universities do on the different components of the Shanghai rankings.



The current top ten are as follows:

1. Harvard
2. Stanford
3. Berkeley
4. Cambridge
5. MIT
6. Caltech
7. Columbia
8. Princeton
9. Chicago
10. Oxford

The Shanghai index includes two categories based on Nobel prizes and Fields medals. These measure the quality of research that might have been produced decades ago. Looking at the other criteria gives a rather different picture of current research.

It is interesting to see what happens to these ten if we rank them according to SJTU's PUB category, the total number of articles indexed in the Science Citation Index-Expanded (SCIE) and Social Science Citation Index (SSCI) in 2007. The SSCI gets a double weighting.

Harvard remains at number 1

Stanford goes down to number 8

Berkeley goes down to 11

Cambridge goes down to 23

MIT is down at 34

Caltech tumbles to 86

Columbia is down just a bit at 10

Princeton crashes to 120

Chicago falls to 72

Oxford goes down to 18

If this category represents current research output then it looks as though some American universities and Oxbridge have entered a period of decline. Of course, Caltech and MIT may suffer from the PUB category including social science research but would that explain why Princeton and Chicago are now apparently producing a relatively small amount of research?

The top ten for PUB is

1. Harvard

2. Tokyo

3. Toronto

4. University of Michigan

5. UCLA

6. University of Washington

7. Stanford

8. Kyoto

9. Columbia

10. Berkeley

Are Things Improving at QS?

I think it worth quoting from a recent comment.

Just wanted to add a note about submitting data to THES based on my experience
here at a large Australian university. We forwarded our submission earlier this
year and now we have received a query on some of our numbers - a check just to
confirm if they are correct. And I can understand why they would need to be
checked. The numbers we submitted for staff (in the thousands) seem to have
changed to a number in the very low hundreds. Also, comments (made by us) that
were attached to specific sections have, in the past come back with typos... I
think this indicates that there is greater scope for human error in the
compilation of the data, even at such an early and relatively uncomplicated
phase of the data gathering process...

There are signs that QS is making a commendable effort to avoid the errors that have been so prevalent in previous rankings. Still it is rather disconcerting that thousands of faculty have turned into hundreds, especially since it is not altogether impossible that some universities might conveniently forget to correct an error that might be to their advantage.

So, I was wondering how common simple basic errors are in the QS rankings. I have been looking at QS's topuniversities site and checking the number of students listed in the descriptions for each university, comparing the number of undergraduates and postgraduates with the total of all students. Here are the results just for the Universities beginning with A.

For these twelve universities no problems were noticed: Aarhus, Aberdeen, Aberystwyth, Antwerp, Arizona State University, Athens, Aston, Amsterdam, Adelaide, Australian National University, Austral, Vrije Universiteit Amsterdam. In some cases there were minor discrepancies but not enough to cause concern.

About two weeks ago there were, in three cases, discrepancies between the number of total students and the combined numbers of undergraduates and post graduates: University of Arizona (more combined undergraduates and post graduates than total students), University of Auckland (number of postgraduates and total students the same) and Athens University of Economics and Business (more undergraduate international students than total international students) .

In the above three cases, at the time of writing the errors have been corrected with new entries.
There were however three cases where the errors at the time of writing had not been corrected. These are:

University of Arkansas
Full Time Equivalent (FTE) undergraduates. 20,416.
FTE graduate/postgraduate students 4,163.
Total FTE students 15,182.
Over 9,000 students "missing" from the total.

(correction: not the University of Arizona as was indicated in an earlier version of this post)


University of Alabama
FTE undergraduates 33, 986
FTE graduate/postgraduate students 8,291.
FTE total of 19,651 students.
Over 22,000 students “missing” from the total.

University of Alberta
FTE undergraduates 29,178.
FTE graduate/postgraduate students 5,419.
Total FTE total students 32,341.
About 2,500 students “missing” from the total.

I suspect that the problem with these three schools is that the totals of students were complied and entered separately and that the data for undergraduates and postgraduates included students in branch campuses, professional schools and/or research institutes and the data for total students did not. It will be interesting to see whether these errors will be corrected and whether new ones will emerge.

Ranking Fever Sweeps America

The US college scene is buzzing about the latest ranking news. No, its not the impending THE-QS rankings but Princeton Review's list of top party schools. See here, here, here, here, here, here, here, here, here and here.

It seems that US colleges are more concerned about being in the top 20 party schools or, in a few cases, in the top 20 stone-cold sober schools than their position in the THE-Qs rankings. And who is to say that they they are wrong.

The Princeton Review Rankings

The Princeton Review's Annual College Rankings, based on a survey of 120,000 US students, is now out. There are 62 lists and there seems to be something for everyone. Here are some of the top schools.

Middlebury College -- best professors
Wheaton College -- tastiest campus food
Loyola College -- best dorms
Yale -- most beautiful campus
Northeastern University -- best career and job placement service
Stanford -- best classroom experience
Texas A and M (College Station) -- most conservative students
Occidental College -- most liberal students
University of Florida (Gainesville) -- top party school
Brigham Young University -- top stone-cold sober school
City University of new York -- most diverse student body
University of Maryland at College Part -- best athletic facilities

Another University Ranking

The Ecole des Mines de Paris has produced a new ranking based on the number of corporate leaders trained by universities. Its website reports:

The École des Mines de Paris ranking is based on a quite different
criterion: the number of alumni holding a Chief Executive Officer position (CEO)
in one of the 500 leading worldwide companies as of the date of the Shanghai
ranking 2006. This criterion is aimed at being the equivalent among companies of
the criterion for alumni who have been awarded the Nobel Prize or the Fields
Medal, as the numbers involved are similar.
We have chosen to identify the
500 leading enterprises on the basis of the "Global Fortune 500", based on the
criterion of the published annual turnover and conducted by Fortune
magazine.

It is good to be reminded that universities do other things and not just research. Still this is a very limited measure of excellence.

Here are the top ten

1. Harvard Univ

2. Tokyo Univ

3 Stanford Univ

4 Ecole Polytechnique Paris

5 École des Hautes Études Commerciales Paris

6 Univ Pennsylvania

7 Massachusetts Inst Tech (MIT)

8 Sciences Po - Paris

9 ENA Paris

10 Ecole des Mines de Paris


There are five French schools in the top ten which may say something about the excellence of these institutions or perhaps something about French cultural introversion. Only four foreign institutions have contributed to the training of 37 French CEOs. For the 38 British CEOs the corresponding figure is 22, including Gettysburg College, the US Naval Academy, University College Cork, Utrecht, Witwatersrand and, of course, the Ecole des Mines.

I noticed that in at least two cases, Strode's College in Surrey and Malay College in Malaysia, secondary schools were listed as diploma-granting institutions.

Another Use of Rankings

BBC News reports that graduates of lower ranking universities are more dishonest than those from institutions further up the ranking ladder.

"Analysis of 3,876 financial service job applications found embellishments on the forms of 43% of applicants from the UK's lowest ranking universities.

Only 14% of applicants from the top 20 UK universities were found to have fibbed in their applications.

The survey was commissioned by a pre-employment screening firm, Powerchex.
Its managing director Alexandra Kelly said: "What this survey says is that graduates from lesser-known universities may feel the need to alter their background to compete."

There is no definitive ranking of universities. For the survey the researchers at the Shell technology and enterprise programme used the Times Online 2009 ranking. "


Given the amount of soft marking and numbers massaging that British universities do to get a good place in the rankings, is it posible that graduates of the good ones do not need to embellish their CVs because their universities do all the necessary embellishment for them?

France Wants a New Ranking System

One consequence of the ranking craze is the proliferation of "minor leagues" as universities and countries that do badly on prominent rankings encourage the creation of more and more indexes and league tables on which they hope to better.

The French Senate is now proposing a new ranking system for European universities. It is not happy with the attention given to the Shanghai Jiao Tong index where French institutions do not perform very well.

The report on the euractiv site continues:

"France's key bone of contention with the Shanghai index is that the number of citations of a institution's scientific research is used as a ranking factor. Paris says this works against countries that do not publish in English.

Senator Joël Bourdin, the rapporteur on the report, argues that the intrinsic value of the Shanghai ranking is highly questionable ["très discutable"] and owes its interest only to its "mobilising effect".

The highest-ranking French university in the 2006 Shanghai ranking was Paris VI in 39th place, while American universities occupy more than half of the top 100. The only European universities in the top 10 are the UK's Universities of Cambridge and Oxford.

While French Higher Education Minister Valerie Pécresse has said she wishes to use the French EU Presidency to "lay down the foundations of common European criteria" for university classification, Bourdin argues that France should already go ahead and develop its own national classification system, as an EU-level process would take years. "


One wonders why the French do not simply use the citations per faculty section of the THE-QS rankings which shows the Ecole Normale Superieure in 4th place just ahead of the University of Alabama (5th), Pohang University of Science and Technology (11th), University of Helsinki (17th), Hong Kong University of Science and Technology (20th). On this measure the ecole does better than Princeton, Harvard, Cornell and Columbia.

The report can viewed here.

Descending into Farce

A report in the Independent (UK) provides further evidence of the destructive effects of the obsession with rankings.

"The university degree classification system is "descending into farce", the chairman of the Commons Select Committee on Universities has said.

Phil Willis was speaking as MPs questioned Peter Williams, the chief executive of the Quality Assurance Agency (QAA), the higher education watchdog, on degree standards. "An individual institution can award as many firsts as it wants, provided it satisfies its own criteria on what is a first," Mr Willis said.

It followed comments from Professor Geoffrey Alderman, the former head of quality at the University of London, which were reported in The Independent, that lecturers had been told to "mark softly" to ensure enough first-class degree passes were awarded to win a high ranking in league tables. He also alleged universities were turning a "blind eye" to plagiarism by international students because they were dependent on income from their fees. "

Comments on Ben Sowter's Comments

This is in reply to comments by Ben Sowter, head of research at QS, on my post of July 10th.

First of all, I was mistaken about the BlackBerry. At the end of the survey there is a box to tick if respondents want a chance to win one.

Next, I am glad to learn that Ben is a frequent reader of this blog. Thank you also for the comment that "Much of your analysis is detailed and shrewd. In fact, whilst I have declined to comment in the past we have kept an eye on your blog and, in some areas, chosen to make improvements based on your observations".

The clarification that the current survey is of prior respondents only and that later e-mails will also be sent to World Scientific subscribers is also welcome.

As for wondering why Malaysian universities were not on the first list, the introductory messages says that "You'll notice some slight differences to the survey this year - mainly in that we have included a lot more universities - as a result there are two questions about universities - one asking about those around the world, and a second asking about your own country specifically". I assumed that "around the world" meant that every country in the world would be included. The information should have read "around the world excluding your your own country". Still, I accept that I jumped to an unwarranted conclusion.

I think that there are serious questions remaining about the implications of this change. If QS are going to use only the responses to the first list when calculating the score for the academic opinion criterion then this would surely mean a noticeable reduction in the number of votes for universities that got a large number of responses from other places in that country. I suspect that this would have a disproportionate effect on American and Japanese universities. Wouldn't it also have a relatively beneficial impact on British universities with lots of graduates in the Commonwealth and Australia with graduates in South East Asia and China, countries with a lot of names in the World Scientific database?

Ben's comment that the purpose of having two lists is so that "we can do some more analysis on response dynamics and, potentially, more effectively counteract country biases" suggests that this might be the case since the implication is that the second list will not be used to construct the scores for the "peer review".

Is it possible that such a change would give Cambridge, Oxford and Imperial College London a further advantage over Harvard and push one of the former three into the number one spot?

Comments on "What's Happened to Malaysia?"

I recently posted on what I thought was the removal of Malaysian universities from the THES-QS survey of academic opinion.

It appears, as comments on the post and my own fiddling about with the survey indicate, that what has happened is that the survey is divided into two parts: the first in which all universities in the country where the respondent works are excluded and a second in which only those universities -- minus the one where the respondent works -- are presented.

It seems that I have been unfair to QS on this occasion. There are, however, some issues about the survey that I will discuss in a little while.

Two comments refer to the absence on the State University of New York from the list. In fact, Stony Brook is there but not the other three campuses at Binghamton, Albany and Buffalo.