Comment on the QS Subject Rankings

An enjoyable although perhaps a little intemperate comment on the QS philosophy rankings from the Leiter Reports.

Several readers sent this item, the latest worthless misinformation from the "world universities" ranking industry, in which "QS" (which, contrary to rumor, does not actually stand for 'Quirky Silliness") is a main player. As a commenter at The Guardian site notes, five of the universities ranked tops in Geography do not even have geography departments! And which are the "top five" US universities in philosophy?

1. Harvard University
2. University of California, Berkeley
3. Princeton University
4. Stanford University
5. Yale University

That corresponds decently to the top five American research universities, to be sure, but it has nothing to do with the top five U.S. philosophy departments, at least not in the 21st-century. But it should hardly be surprising that if you ask academics teaching in philosophy departments in Japan or Italy to rank the best philosophy departments, many of them will use general university reputation as a proxy. Indeed, every department that is pretty obviously "overrated" in philosophy in this list is at a top research university, and every department obviously underrated is not: so, e.g., Rutgers comes in at a mere 13th, Pittsburgh at 18th (behind Brown and Penn), and North Carolina at 20th.
One may hope that no student thinking about post-graduate work will base any decisions on this nonsense.

Important Dates

September 11th. From QS Intelligence Unit

QS Intelligence Unit is pleased to invite you to attend this afternoon event featuring the global exclusive release of the full QS World University Rankings® 2012-2013 Tuesday 11th September 2012 in Trinity College Dublin. Be among 300 university delegates present for a focused ninety minute session and networking reception on the eve of the EAIE conference.

September 12th. From Morse Code

The 2013 edition of U.S. News's Best Colleges rankings will go live on usnews.com on Wednesday, September 12. National Universities, National Liberal Arts Colleges, Regional Universities, and Regional Colleges are included in these rankings.

Our website will have the most complete version of the rankings, tables, and lists. It will have extensive statistical profiles for each school as well as wide-ranging interactivity and a college search to enable students and parents to find the school that best fits their needs. These exclusive rankings will also be published in our Best Colleges 2013 edition guidebook, which will go on sale September 18 on newsstands and at usnews.com.


October 3rd.  Times Higher Education
 
The annual THE rankings, which UK universities and science minister David Willetts said are "fast becoming something of a fixture in the academic calendar", will be published live online at 21.00 BST on 3 October.
A special rankings print supplement will also be published with the 4 October edition of THE, and the results will be available on a free interactive iPhone application.

The Shanghai Rankings 3

Two of the indicators in the Shanghai rankings measure research achievement at the highest level. The highly cited researchers indicator is based on a list of those scientists who have been cited most frequently by other researchers. Since ARWU counts current but not past affiliations of researchers, it is possible for a university to boost its score by recruiting researchers. This indicator might then  be seen as signalling a willingness to invest in and to retain international talent and hence a sign of future excellence. 

The top five for this indicator are

1,  Harvard
2.  Stanford
3.  UC Berkeley
4.  MIT
5.  Princeton

This indicator shows that there are a lot of US state universities and non-Ivy League schools that are doing well on this indicator. There is the University of Michigan (6th), University of Washington (13th), University of Minnesota (19th), Penn State (23rd), and Rutgers  (42nd).

Before this year, the methodology for this indicator was simple. If a highly cited researcher had two affiliations then there was a straightforward fifty-fifty division. Things were complicated when King Abdulaziz University (KAU) in Jeddah signed up scores of researchers on part time contracts, a story recounted in Science. ARWU has responded deftly by asking researchers to indicate how their time was divided if they had joint affiliations and this seems to have deflated KAU's score considerably but has had no or minimal effect for anyone else.

The top five universities for papers in Nature and Science are:

1.  Harvard
2.  Stanford
3.  MIT
4.  UC Berkeley
5.  Cambridge

High fliers on this indicator include several specialised science and medical institutions such as Imperial College London, Rockefeller University, Karolinka Institutet and the University of Texas Southwestern Medical Center.

Self Citation

In 2010 Mohamed El Naschie, former editor of the journal Chaos, Solitons and Fractals, embarrassed a lot of people by launching the University of Alexandria into the world's top five universities for research impact in the new Times Higher Education (THE) World University Rankings. He did this partly by  diligent self citation and partly by lot of mutual citation with a few friends and another journal. He was also helped by a ranking indicator that gave  the university disproportionate credit for citations in a little cited field, for citations in a short period of time and for being in a country were there are few citations.

Clearly self citation was only part of he story of Alexandria's brief and undeserved success but it was not an insignificant one.

It now seems that Thomson Reuters (TR), who collect and process the data for THE beginning to get a bit worried about  "anomalous citation patterns" . According to an article by Paul Jump in THE.

When Thomson Reuters announced at the end of June that a record 26 journals had been consigned to its naughty corner this year for "anomalous citation patterns", defenders of research ethics were quick to raise an eyebrow.

"Anomalous citation patterns" is a euphemism for excessive citation of other articles published in the same journal. It is generally assumed to be a ruse to boost a journal's impact factor, which is a measure of the average number of citations garnered by articles in the journal over the previous two years.

Impact factors are often used, controversially, as a proxy for journal quality and, even more contentiously, for the quality of individual papers published in the journal and even of the people who write them.

When Thomson Reuters discovers that anomalous citation has had a significant effect on a journal's impact factor, it bans the journal for two years from its annual Journal Citation Reports (JCR), which publishes up-to-date impact factors.

"Impact factor is hugely important for academics in choosing where to publish because [it is] often used to measure [their] research productivity," according to Liz Wager, former chair of the Committee on Publication Ethics.

"So a journal with a falsely inflated impact factor will get more submissions, which could lead to the true impact factor rising, so it's a positive spiral."

One trick employed by editors is to require submitting authors to include superfluous references to other papers in the same journal.

A large-scale survey by researchers at the University of Alabama in Huntsville's College of Business Administration published in the 3 February edition of Science found that such demands had been made of one in five authors in various social science and business fields.

That TR are beginning to crack down on self citation is good news. But will they follow their rivals QS and stop counting self citation in the citation indicator in their rankings? When I spoke to Simon Pratt of TR at the Shanghai World Class Universities conference in Shanghai at the end of last year he seemed adamant that they would go on counting self citations.

Even if TR and THE start excluding self citations, it would probably not be enough.. It may soon become necessary to exclude intra-journal citations as well.