Powered by Thomson Reuters

Thanks to Kris Olds in Global higher Ed for noticing the above in the THE World UniversityRankings banner.

A quotation from his article.

Thomson Reuters is a private global information services
firm, and a highly respected one at that. Apart from ‘deep pockets’, they have
knowledgeable staff, and a not insignificant number of them. For example, on 14
September Phil Baty, of Times Higher Education sent out this fact via their
Twitter feed:

2 days to #THEWUR. Fact: Thomson Reuters involved more
than 100 staff members in its global profiles project, which fuels the rankings

The incorporation of Thomson Reuters into the rankings games by Times
Higher Education was a strategically smart move for this media company for it
arguably (a) enhances their capacity (in principle) to improve ranking
methodology and implementation, and (b) improves the respect the ranking
exercise is likely to get in many quarters. Thomson Reuters is, thus, an
analytical-cum-legitimacy vehicle of sorts.

Comment on the THE rankings

From The Age (Australia)

Les Field, the deputy vice-chancellor (research) at the University of NSW, said the new Times methodology had produced some curious results, such as Hong Kong Baptist University ranking close behind Harvard on citations.

''There are some anomalies which to my mind don't pass the reasonableness test,'' he said.

And Alexandria University, UC Santa Cruz, UC Santa Barbara, Pohang University of Science and Technology, Bilkent University, William & Mary, Royal Holloway, University of Barcelona, University of Adelaide.

Alexandria University

According to the THE rankings Alexandria University in Egypt (no. 147 overall) is the fourth university in the world for research impact, surpassed only by Caltech, MIT and Princeton.

Alexandria is not ranked by Shanghai Jiao Tong University or HEEACT. It is way down the SCImago rankings. Webometrics puts it in 5,882nd place and 7,253rd for the "Scholar" indicator.

That is not the only strange result for this indicator, which looks as though it will spoil the rankings as a whole.

More on Alexandria and some other universities in a few hours.

The Good News

There are some worthwhile improvements in the new THE World University Rankings.

First the weighting given to the subjective opinion survey has been reduced although probably not by enough. Very sensibly, the survey asked respondents to evaluate teaching as well as research.

The task ahead for THE now is to refine the sample of respondents and the questions they are invited to answer. It would make sense to exclude those with a non-university affiliation from answering questions about teaching. Similarly, there ought to be some way of eliciting the views of university teachers who do not do research, perhaps by some sort of rigorously validated sign up system. Something like this might also be developed to discover the views of students, at least graduate students.

The weighting given to international students has been reduced from five to two per cent.

There is a substantial weighting for a mixed bag of teaching indicators, including the survey. Some of these are questionable though such as the ratio of doctoral to undergraduate students.

For most indicators, the present rankings represent a degree of progress.

The problem with these rankings is the Citations Indicator, which has produced results that, to say the least, are bizarre.

First the Bad News about the THE Rankings

There is something seriously wrong with the citations indicator data. I am doing some checking right now.

Highlights of the THE Rankings

The top ten are:
1. Harvard
2. Caltech
3. MIT
4. Stanford
5. Princeton
6. Cambridge
6. Oxford
8. UC Berkeley
9. Imperial College
10. Yale

The best Asian university is the University of Hong Kong. Sao Paulo is best in South America and Melbourne in Australia. Cape Town is top in Africa followed by the University of Alexandria which is ranked 149th, a rather surprising result.

The new THE World University Rankings are out. Discussion follows in a few hours.

HEEACT Rankings Out

The Taiwan Rankings are out. They are based on articles and citations over the last eleven years and the last year, highly cited articles, articles in high impact journals and the H-index. Essentially they measure research productivity, research excellence and research impact.

The top 10 are:

1. Harvard
2. Stanford
3. Johns Hopkins
4. University of Washington -- Seattle
5. UCLA
6. UC Berkeley
7. MIT
8. University of Michigan -- Ann Arbor
9. Tornto
10. Oxford

Tokyo is 14th, Cambridge 16th, University College London 17th, Yale 18th, Imperial 21st, Caltech 31st, Melbourne 43rd, Seoul National University 67th.

Academic Fraud in China

An article by Sam Geall in The New Humanist shows something of the other side of China's rapid scientific development in recent years.

Access to Rankings Data

Global Higher Ed has an excellent article by Kris Olds and Susan Robertson about the need for transparency in the collection and distribution of ranking data.

Yet More Reactions to the QS

As the world holds its breath waiting for the THE World University Rankings here are a few more reactions to the QS World University Rankings.

AUSTRALIAN universities have responded with a deafening silence to their contentious downgrading in last week's Quacquarelli Symonds World University Rankings.

The Australian


World's top universities: Four IITs slip in rankings

Sify Finance


Ranking is not everything

The Nation (Thailand)


University climbs fourteen places in world rankings

leedsstudent


A slow but steady climb

Malaysia Star Online

Ranking Research Impact

A team at the University of Western Australia has ranked the world' top 500 universities by research impact. The table is based on citations data derived from Scopus and covers the period 2000 to 2009. The ranking seems technically to be very competent.

The top five are:

1. Harvard

2. Stanford

3. MIT

4. UCLA

5. UC Berkeley

Cambridge is 13th and University College London 36th.

.

More Reactions to the QS Rankings

Australian Higher Education Sector Down in Rankings and Nervous on International Enrolments
AIEC QUEST Australian International Education

4 Chinese universities rank among world's top 50
Peoples Daily Online

Some Reactions to the 2010 QS Rankings

Cambridge Knocks Harvard Off Top in University League

San Francisco Chronicle

Nine Taiwan universities listed among the world's top 500 Radio Taiwan International Israeli universities drop in international rankings Haaretz Trinity and UCD slip down rankings of top universities Independent.ie

Cambridge Beats Harvard -- Sort of

The big news from the QS World University Rankings today is that Cambridge is finally top after trailing Harvard for six years.

This seems a little odd since Cambridge is way behind Harvard, and a few other places, on all the indicators in the Shanghai rankings. So what happened? Looking at the indicator scores we find that on the "Academic Peer Review" -- more accurately called an Academic Reputation Index elsewhere on the site -- Cambridge is first and Harvard second. For the Employer Review Cambridge is third and Harvard first, reversing their places last year. For citations per faculty Harvard was third and Cambridge 36th, behind Tufts, Emory and UC Santa Cruz among others. For student faculty ratio, Cambridge was 18th and Harvard 40th. At the time of writing data was not available for International Faculty and Students.

It seems that the main factor in Cambridge's success was the academic survey. QS indicates the sources of the survey.

• 1,648 previous respondents who returned. If QS have continued the practice of previous years , they also counted respondents from 2009 and 2008 even if they did not submit a form.
• 180,00 out of 300,000 persons on the mailing list of World Scientific, a Singapore-based publishing company with links to Imperial College London. World Scientific, by the way, claim to have 400,000 subscribers.
• 48,125 records from Mardev-DM2
• 2,000 academics who signed up at the QS site
• Lists provided by institutions. In 2010 160 universities provided more than 40,000 names.

I will let readers decide how representative or accurate such a survey can be.

Incidentally, QS should be given credit for the detailed description of the methodology of this criterion.

QS announces Date

Times Higher Education have already announced that their World University Rankings will be published on September 16th.

This morning QS indicated on their topuniversities site that theirs will be out on September 8th.

World Class Universities as a Measure of System Quality

This a list of the percentage of each country's universities that are included in the top 500 of the Academic Ranking of World Universities produced by Shanghai Jiao Tong University. It might be considered a limited indicator of the overall quality of a country's higher education system.

The number of universities in each country included in the ARWU Top 500 is from ARWU . The total number of universities in each country is from Webometrics. A university is simply defined by the possession of a distinct URL.

It is of course easier to start a university in the US than in Israel where the country's first Arabic speaking university has only just been approved. However, this table does put the large number of American universities in global rankings in a different perspective.


1. Israel 21.21
2. Sweden 20.37
3. Australia 19.77
4. UK 16.17
5= Finland 11.76
5= Singapore 11.76
7. South Africa 11.54
8. Canada 11.33
9. New Zealand 11.11
10. Italy 10.89
11. Austria 10.61
12. Germany 9.75
13. Netherlands 8.21
14. Belgium 7.14
15. Switzerland 6.67
16. Ireland 6.00
17. Norway 5.89
18. USA 4.70
19. Spain 4.59
20. Saudi Arabia 4.44
21. Hungary 3.85
22. France 3.77
23. Denmark 3.57
24. Japan 3.50
25= Greece 3.125
25= Slovenia 3.125
27. South Korea 2.55
28. China 2.52
29. Chile 2.47
30. Portugal 1.79
31. Czech Republic 1.75
32. Argentina 0.95
33. Turkey 0.67
34. Poland 0.46
35. Brazil 0.40
36. Russia 0.30
37. Iran 0.19
38. India 0.13
39. Mexico 0.11

New Rankings on the Way

Times Higher Education have announced that their new rankings will be published on September 16th and have revealed the outline of their methodology.

The rankings will include five groups of indicators as follows:

A new broad category, called "Teaching - the learning environment", will be
given a weighting of 30 per cent.

Using five separate indicators, this category will use data on an institution's income, staff-student ratios and undergraduate-postgraduate mix, as well as the results of the first-ever global academic reputation survey examining the quality of teaching.

A further 30 per cent of the final rankings score will be based on another new indicator, "Research - volume, income and reputation".

This category will use four separate indicators, including data on research income, research output (measured by publications in leading peer-reviewed journals) and the results of the academic reputation survey relating to research.

The highest-weighted category is "Citations - research influence".

This category will examine a university's research influence, measured by the number of times its published work is cited in other academics' papers.

Based on the 12,000 journals indexed by Thomson Reuters' Web of Science, and taken over a five-year period, the citations data will be normalised to take account
of different volumes of citations between disciplines.

Reflecting the high levels of correlation between citations data and research excellence, this category will be given a weighting of 32.5 per cent.

A fourth category, "International mix - staff and students", will use data on the proportion of international staff and students on campus. This indicator will be given a 5 per cent weighting.

Knowledge transfer activities will be reflected in "Industry income - innovation", a new category worth 2.5 per cent of the total rankings score. This will be based on just one measure in 2010 - research income from industry.

There is still a lot apparently left undecided such as the distribution of indicators within the groups and exactly what faculty will count for scaling. In general, though, the broad outlines of the new ranking look promising with the exception of the large weighting -- nearly one third -- assigned to a single indication, citations. Certainly citations are a good measure of research impact and more difficult to manipulate than some others but putting so much emphasis on just one indicator will be a problem for face validity and will also amplify any data entry errors should they occur.

Finally, I wonder if it is a good idea to refer to the "seventh annual survey". Wouldn't it better to start all over again with the First THE Rankings?

From THE

I am reproducing Phil Baty's column from Times Higher Education in its entirety

One of the things that I have been keen to do as editor of the Times Higher
Education World University Rankings is to engage as much as possible with our
harshest critics.

Our editorial board was trenchant in its criticism of our old rankings. In particular, Ian Diamond, principal of the University of Aberdeen and former chief executive of the Economic and Social Research Council, was scathing about our use of research citations.

The old system failed to normalise data to take account of the dramatically different citation volumes between different disciplines, he said - unfairly hitting strong work in fields with lower average figures. We listened, learned and have corrected this
weakness for the 2010 rankings.

Another strong critic is blogger Richard Holmes, an academic at the Universiti Teknologi MARA in Malaysia. Through his University Ranking Watch blog, he has perhaps done more than anyone to highlight the weaknesses in existing systems: indeed, he highlighted many of the problems that helped convince us to develop a new methodology with a new data provider, Thomson Reuters.

He has given us many helpful suggestions as we develop our improved methodology. For example, he advised that we should reduce the weighting given to the proportion of international students on campus, and we agreed. He added that we should increase the weighting given to our new teaching indicators, and again we concurred.

Of course, there are many elements that he and others will continue to disagree with us on, and we welcome that. We are not seeking anyone's endorsement. We simply ask for open engagement - including criticism - and we expect that process will continue long after the new tables are published.

There are still issues to be resolved but it does appear that the new THE rankings are making progress on several fronts. There is a group of indicators that attempts to measure teaching effectiveness. The weighting given to international students, an indicator that is easily manipulable and that has had very negative backwash effects, has been reduced. The inclusion of funding as a criterion, while obviously favouring wealthy regions, does measure an important input. The weighting assigned to the subjective academic survey has been reduced and it is now drawn from a clearly defined and at least moderately qualified set of respondents.



There are still areas where questions remain. I am not sure that citations per paper is the only way to measure impact. At the very least, the h-index could be added, which would add another ingredient to the mix.



Also, there are details that need to be sorted out. Exactly what sort of faculty will be counted in the various scalings? Is self-citation be counted? I also suspect that not everybody will be enthusiastic about using statistics from UNESCO for weighting the results of the reputational survey. That is not exactly the most efficient organization in the world. There is also a need for a lot more information about the workings of the reputational survey. What was the response rate and exactly how many responses were there from individual countries?

Something that may well cause problems in the future is the proposed indicator of the ratio of doctoral degrees to undergraduate degrees. if this is retained it is easy to predict that universities everywhere will be encouraging or coercing applicants to master's programs to switch to doctoral programs.

Still, it does seem that THE is being more open and honest about the creation of the new rankings than other ranking organizations and that the final result will be a significant improvement.

America' Best Colleges 2011

US News and World Report's Ameica's Best Colleges 2011 is now out.

The top ten National Universities are:

1. Harvard
2. Princeton
3. Yale
4. Columbia
5. Stanford
6. University of Pennsylvania
7 = Caltech
7 = MIT
9 = Dartmouth
9 = Duke
9 = Chicago

Shanghai Rankings: Shifting Research Landscape

My article in University World News on the 2010 Academic Ranking of World Universisities can be viewed here.

More from THE

Phil Baty in Times Higher Education gives us some clues about what the forthcoming THE World University Rankings will contain.

"While all the self-reported material bears the imprimatur of the supplying
institutions (and our tables include only those that have cooperated with our exercise) and it has been vetted for quality, the consultation had some concerns
about its consistency and robustness - especially in this inaugural year. For example, not all institutions could provide a clear or internationally comparable figure for their research income from industry.

For maximum robustness, we plan to give extra weighting to data that have been sourced independently of the institutions themselves and are globally consistent.

Citations data, for example, which are widely accepted as a strong proxy for research quality, will have a high weighting - perhaps about 30 per cent of the total ranking score.

We also have high confidence in the validity and independence of the results of our reputation survey. Although we may yet adjust its weighting, this subjective measure will not be weighted as highly as it was in our old methodology (2004-09), where reputation was worth 40 per cent."

It looks as though citations per paper, a measure of its influence throughout a research community, will count for a lot in the forthcoming rankings. It is questionable whether such a high weighting for a single component is justified. At the very least it could be combined with other measures of quality such as the h-index which is, in effect, a measure of both productivity and impact.

The reluctance to place too much emphasis on research income and perhaps other types of income, is understandable but perhaps unfortunate. This indicator would give the new rankings a distinctive feature and might also allow us to see whether institutions are giving value for money.

It is inevitable that the reputational survey would never be given the same weight that its predecessors received in the THE-QS rankings. Whether its results are really valid -- we still do not know the response rate -- remains to be seen.

Why international students are not a good indicator of quality

Times Higher Education describes a dispute between Coventry University and a recruiting agent in Chennai. According to the article, Ram Beegala was hired as a recruiting consultant and would only be paid if he succeeded in getting the number of Indian students above 450.

There is a comment by "To John" which might be slightly exaggerated:

"It is no secret that the Indian students who cannot get into any of their universities and colleges are the ones that are willing to come to the UK. Their intention is the 20 hour/week work allowed and assume rightly once they use the university route to get into UK they can stay in the country to work. In my university which recruits these students, the drop out rates for such students is high as they work more than 20 weeks to meet their expenses. Their attendance drops down after a few months. I have yet to come across a single non-EU student who comes with enough funds to complete a 3 UG degree. They are told by agents that they can work in the UK to meet part of their fees and all the living expenses. The students coming in to do MSc are poorly equipped and struggle to pass their modules and write project proposals."

Big Names and Unsung Heroes

In Times Higher Education, Phil Baty hints that the reduction in the weighting for subjective indicators in the forthcoming THE rankings will mean that those dominant in the past will suffer a decline and that there may be some new schools at the top.

"We can expect some big-name institutions to take a hit in the new World
University Rankings.

Why? Because the rankings we will publish this autumn will be based less on subjective opinion and more on objective evidence".

..........................................................................................................

"Under the initial proposals for our methodology, currently being refined in line with responses from the global academy, reputational measures are worth no more than 20 per cent of overall scores.

I have also set a cap to ensure that subjective elements are never again anywhere near the 50 per cent used in our previous methodology. This means that big names with big reputations that lack world-class research output and influence to match will suffer in comparison with previous exercises. Conversely, unsung heroes have a better chance of recognition".

Another Ranking

The ic4u ranking of 200 top universities is based on web popularity.

The top five are:

1. Stanford
2. MIT
3. National Autonomous University of Mexico
4. Berkeley
5. Peking