Some More on the THE World University Rankings 2


I have calculated the mean scores for the indicator groups in the 2012-13 Times Higher education. world university rankings. The mean scores are for the 400 universities included in the published 2012-13 rankings are:

Teaching   41.67
International Outlook 52.35
Industry Income 50.74
Research 40.84
Citations 65.25

For Industry Income, N is 363 since 37 universities, mainly in the US, did not submit data. This might be a smart move if the universities realized that they were likely to receive a low score. N is 400 for the others.

There are considerable differences between the indicators which are probably due to Thomson Reuters' methodology. Although THE publishes data for 200 universities on its website and another 200 on an iPad/iPhone app there are in fact several hundred more universities that are not included in the published rankings but whose scores are used to calculate the overall mean from which scores for the ranked universities are derived.

A higher score on an indicator means a greater distance from all the institutions in the Thomson Reuters database.

The high scores for citations mean that there is a large gap between the top 400 and the lesser places outside the top 400.

I suspect that the low scores for teaching and research are due to the influence of the academic survey which contributes to both indicator clusters. We have already seen that after the top six, the curve for the survey is relatively flat.

The citations indicator already has a disproportionate influence contributing 30 % to the overall weighing. That 30 % is of course a maximum. Since universities on average are getting more for citations than the the  indicators, it has in practice a correspondingly greater weighting.

Some More on the THE World University Rankings 2012-13

Here are some observations based on a simple analysis of the Times Higher Education World University Rankings of 2012-13.

First, calculating the Pearson correlation between the indicator groups produces some interesting points. If a ranking is valid we would expect the correlations between indicators to  be fairly high but not too high. If the correlations between indicators are above .800 this suggests that they are basically measuring the same thing and that there is no point in having more than one indicator. On the other hand it is safe to assume that if an indicator does measure quality or desired characteristics in some way it will have a positive relationship with other valid indicators.

One thing about the 2012-13 rankings is that the relationship between international outlook (international faculty, students and research collaboration)  and the other indicators is negative or very slight. With teaching it is .025 (not significant), industry income .003 (not significant), research .156 and citations 158. This adds to  my suspicion that internationalisation, at least among those universities that get into the world rankings, does not per se  say very much about quality.

Industry income correlates modestly with teaching (.350) and research (.396), insignificantly with international outlook (.003) and negatively and insignificantly with citations (-.008).

The correlation between research  and teaching is very high at .905. This may well be because  the survey of academic opinion contributes to the teaching and the research indicators. There are different questions -- one about research and one about postgraduate supervision -- but the difference between the responses is probably quite small.

It is also very interesting that the correlation between scores for research and citations is rather modest at .410. Since volume of publications, funding and reputation should contribute to research influence, which is what citations are supposed to measure, this suggests that the citations indicator needs a careful review.

Teaching, research and international outlook are composites of several indicators. It would be very helpful if THE or Thomson Reuters released the scores for the separate indicators.

The California Paradox

Looking at the Times Higher Education reputation rankings, I noticed that there were two Californian universities in the  superbrand six and seven in the top 50. This is not an anomaly. A slightly different seven can be found in the THE World University Rankings. California does even better in the Shanghai ARWU with three in the top six and 11 in the top 50. This is a slight improvement on 2003 when there were ten. According to ARWU, California would be the second best country in the world for higher education if it became independent.

California’s performance is not so spectacular according to QS who have just four Californian institutions in their top fifty, a fall from 2004 when they had five (I am not counting the University of California at San Francisco which, being a single subject medical school, should not have been there). Even so it is still a creditable performance.

But, if we are to believe many commentators, higher education in California, at least public higher education, is dying if not already dead.

According to Andy Kroll in Salon:

"California’s public higher education system is, in other words, dying a slow death. The promise of a cheap, quality education is slipping away for the working and middle classes, for immigrants, for the very people whom the University of California’s creators held in mind when they began their grand experiment 144 years ago. And don’t think the slow rot of public education is unique to California: that state’s woes are the nation’s".

The villains according to Kroll are Californian taxpayers who refuse to accept adding to a tax burden that is the among the highest in the world. 

It is surprising that the death throes of higher education in California have gone unnoticed by the well known international rankers.

It is also surprising that public and private universities that are still highly productive and by international standards still lavishly funded exist in the same state as secondary and elementary schools that are close to being the worse in the nation in terms of student performance. The relative and absolute decline in educational achievement is matched by a similar decline in the overall economic performance of the state.

It may be just a matter of time and in the coming decades Californian universities will follow primary and secondary education into irreversible decline.

 

 

 

Preserving data

Times Higher and QS have both renovated their ranking pages recently and both seem to have removed access to some data from previous years. THE used to provide links to the Times Higher Education (Supplement) - Quacquarelli Symonds rankings of 2004-2010 but apparently not any more. QS do not seem to give access to these rankings before 2007. In both cases, I will update if it turns out that there is a way to get to these rankings.


There is, however,  a site which has the rankings for the top 200 of the THES - QS Rankings of 2004-2007.

The THE Reputation Rankings

Times Higher Education have published their reputation rankings based on data collected from the World University Rankings of 2012.

They are not very interesting. Which is exactly what they should be. When rankings show massive changes from one year to another a certain amount of scepticism is required.

The same six, Harvard, MIT, Stanford, Berkeley, Oxford and Cambridge are well ahead of everybody else as they were in 2012 and in 2011.

Taking a quick look at the top fifty, there is little movement between 2011 and 2013. Four universities for the US, Japan, Netherlands and Germany have dropped out. In their place there is one more from Korea and from the the UK and two more from Australia.

I was under the impression that Australian universities were facing savage cuts in research funding and were going to be deserted by international students and researchers..

Maybe it is the other universities that are being cut. or maybe a bit of blood letting is good for the health.

I also noticed that the number of respondents went down a bit in 2012. It could be that the academic world is beginning to suffer from ranking fatigue.

GRE Country Ranking: Verbal Reasoning

Arranging the mean scores for the 2011-12 GRE Verbal Reasoning test, we can see that the bottom looks rather similar to  the Quantitative Reasoning test. It comprises African and Arab countries. The top is very different, with five out of six places held by countries where English is currently the native language of a majority of the population.

1	Australia	158.40
2	New Zealand	157.30
3=	Singapore	157.10
3=	Ireland	157.10
3=	UK	157.10
6	Canada	156.00
7	Netherlands	155.50
8	Belgium	155.00
9	US White	154.10
10	Switzerland	153.70
11	Romania	153.50
12=	Sweden	153.30
12=	South Africa	153.30
14	Bulgaria	153.20
15	Norway	153.10
16	USA	152.90
17	Argentina	152.80
18	France	152.70
19	US Asian	152.60
20	Austria	152.50
21	Germany	152.30
22	Denmark	152.30
23	Italy	152.20
24	Croatia	151.70
25	Finland	151.70
26	Uruguay	151.60
27	US American Indian	151.50
28=	Czech Republic	151.40
28=	Israel	151.40
28=	Trinidad	151.40
31	Hungary	151.20
32=	Portugal	150.90
32=	Spain	150.90
34	Poland	150.40
35	Lithuania	150.30
36	US Hispanic	150.20
37	Iceland	149.80
38	US Mexican	149.70
39=	Malaysia	149.50
39=	Barbados	149.50
41	Greece	149.40
42=	Costa Rica	149.10
42=	Philippines	149.10
44	Guatemala	149.00
45	Brazil	148.90
46=	Zimbabwe	148.80
46=	Jamaica	148.80
48	US Porto Rican	148.70
49=	Georgia	148.60
49=	Bosnia - Herzogovina	148.60
51	Guyana	148.60
52	Moldova	148.40
53	Macedonia	148.30
54=	Peru	148.20
54=	Mexico	148.20
54=	Bahamas	148.20
57	Chile	148.00
58	Belarus	147.90
59	Russia	147.80
60=	Latvia	147.70
60=	Colombia	147.70
62	Albania	147.60
63=	Estonia	147.60
63=	Venezuela	147.60
63=	El Salvador	147.60
63=	Cuba	147.60
63=	St Lucia	147.60
68=	Hong Kong	147.50
68=	South Korea	147.50
70=	Ukraine	147.40
70=	Bahrain	147.40
72=	Serbia	147.30
72=	Bolivia	147.30
74=	Eritrea	147.20
74=	Honduras	147.20
76=	Zambia	147.10
76=	Afghanistan	147.10
78	Pakistan	147.00
79	Panama	146.80
80	US Black	146.70
81=	Ecuador	146.50
81=	Kenya	146.50
83	Nigeria	146.40
82=	Morocco	146.30
82=	Senegal	146.30
82=	Nicaragua	146.30
87=	Cyprus	146.10
87=	Dominican Republic	146.10
87=	Sierra leone	146.10
90	Uzbekistan	146.00
91	China	145.90
92	Mongolia	145.80
93=	Vietnam	145.70
93=	Myanmar	145.70
95	Kazakhstan	145.60
96=	Togo	145.50
96=	Ghana	145.50
98=	Tunisia	145.20
98=	Uganda	145.20
100	Niger	145.10
101=	Burkina Faso	145.00
101=	Malawi	145.00
103	Kyrgyzstan	144.90
104=	India	144.70
104=	Indonesia	144.70
106	Cote d'Ivoire	144.60
107=	Japan	144.50
107=	Nepal	144.50
107=	Haiti	144.40
110=	Taiwan	144.20
110=	Bangladesh	144.20
110=	Ethiopia	144.20
110=	Benin	144.20
110=	Congo DR	144.20
115	Turkey	144.10
116=	Egypt	143.80
116=	Azerbaijan	143.80
118	Armenia	143.70
119=	Turkmenistan	143.50
119=	Cameroon	143.50
121=	Macao	143.40
121=	Sri Lanka	143.40
121=	Tanzania	143.40
124	Thailand	142.80
125=	Syria	142.70
125=	Rwanda	142.70
127	Qatar	142.50
128	Algeria	141.60
129	Jordan	141.40
130	Oman	141.30
131	Yemen	141.00
132	Congo Republic	140.90
133	Kuwait	140.80
134	Sudan	140.60
135=	UAE	140.30
135=	Mali	140.30
137	Namibia	140.20
138	Saudi Arabia	137.40