FAQs at QS

QS have answered some questions over at QS TOPUNIVERSITIES. Here are some of the questions and ATFAQS (Answers to ...) or extracts and some comments.


"1) How do you plan to address the perceived bias towards English-speaking (and particularly UK) universities?

...The reality is, however, that in many areas of university competitiveness, operating in English is an advantage. English language journals are more widely read and cited, the top four destinations for international students (and I suspect also faculty) are the US, Canada, UK and Australia – all English speaking. Many universities in non-English speaking Asia, recognising this are operating more programs in English and all global rankings currently carry this bias, not just ours. Our objective is to minimise the bias, but it is far from clear whether eliminating it entirely would be appropriate."

Fair enough. But the bias within the English speaking world (the high scores for Oxbridge, the London schools and colleges, Edinburgh and Australian universities compared to the US and Canada) in the THE-QS rankings was probably more significant.

"3) Following the launch of the government-funded Assessment of Higher Education Learning Objectives (AHELO) pilot scheme, how do you respond to the suggestion that an insufficient emphasis is given to teaching standards and student skills within the more research-oriented established methodologies?


QS absolutely concurs that teaching and learning is inadequately embraced in any of the existing global rankings, including our own and is watching the AHELO exercise with great interest to see if lessons can be drawn and applied to the much broader geographical scope of our rankings. QS is also assessing whether student and alumni inputs can help draw a clearer picture of comparative performance in teaching and learning. On the student skills side of things, QS is currently the only global ranking taking this aspect seriously – via the Employer Review indicator."

Assessing the quality of teaching has so many pitfalls that it may never be possible to do it objectively on an international scale. A global version of RateMyProfessor might be feasible but there is obvious potential for rigging. It also has to be said that for more proficient students -- and that would include many or most of those in universities that will be in the top 200 0r 300 in any sort of ranking -- teaching is largely irrelevant. I doubt if any high fliers from the Ivy League or the grandes ecoles were ever quizzed by interviewers about the staff-student ratio in their classes or whether their instructors explained desired learning outcomes or whether they felt safe in their lecture halls. If teaching is to be assessed an opinion survey is probably no worse than anything else that might be proposed.

"4) Do you think that the low ranking of LSE in the 2009 rankings (67th) is reflective of an inherent bias toward scientific subjects within citations-based methodologies, and if so how do you plan to address this in 2010?

The QS World University Rankings™ are designed to assess the all-round quality of universities across all disciplines and levels, in teaching, research, employability and internationalisation. LSE is a fantastic institution, as is reflected by their persistent high position in the Social Sciences – the faculty area in which they are focused. In fact, it is so strong with its narrower focus that it manages to compete with world leading institutions with a much broader range. Even if we only take the proportion of world universities recognised by UNESCO a Top 100 placing represents the top 1% - a prolific achievement for an institution that focuses on only a small part of the academic spectrum. To put things in perspective, LSE fails to break the top 200 in the Shanghai Ranking."

It seems that the position of LSE in the forthcoming rankings will be closely watched. Yes, there has been a bias against institutions with strengths in the social sciences and this may be corrected in the THE rankings but anything that benefits LSE will also benefit general universities as much or more.

"5) How can the shift in position of some universities in the THE -QS World University Rankings 2004-2009 be explained?"

QS essentially answers this questions, or rather avoids answering it, by pointing out that later editions of the THE-QS rankings showed more stability and that national rankings of British universities were even more volatile.

One reason why the THE-QS rankings were so unstable is simply the large number of errors that were made. These include counting ethnic minorities in Malaysia as international faculty and students, giving 1 out of 100 for citations to Washington University in St. Louis, the Indian Institutes of technology and Technion Israel and then boosting their scores in the following year, overcounting the number of faculty at Duke University and overcounting the number of citations or undercounting the number of faculty at the University of Alabama.

Such errors do, however, appear to have been eliminated from the most recent rankings.

Another problem arose from the the frequent changes in methods and sources of data. Here there is a real and serious dilemma . Methodological improvements are necessary to maintain validity but at the same time they can undermine credibility by causing noticeable fluctuations.

One solution to this might simply be to publish two sets of rankings every year, one with an unchanged methodology called the QS Classic or the Shanghai Classic ranking and another incorporating the latest methodological changes called the New, Alpha.. Mega.. or whatever ranking.