Monday, January 20, 2020

Boycotting the Shanghai Rankings?

John Fitzgerald of Swinburne University of Technology has written an article in the Journal of Political Risk that argues that Western universities should boycott, that is not participate in or refer to, the Shanghai rankings in order to show opposition to the current authoritarian trend in Chinese higher education.

There seems a bit of selective indignation at work here. China is hardly the only country in the world with authoritarian governments, ideologically docile universities, or crackdowns on dissidents. Nearly everywhere in Africa, most of the Middle East, Russia, much of Eastern Europe, and perhaps India would seem as guilty as China, if not more so.

American and other Western universities themselves are in danger of becoming one party institutions based on an obsessive hatred of Trump or Brexit, a pervasive cult of "diversity", political tests for admission, appointment and promotion, and periodic media or physical attacks on dissenters or those who associate with dissenters. 

Perhaps academics should boycott the THE or other rankings to protest the treatment by Cambridge University of Noah Carl or Jordan Peterson? 

One way of resisting the wave of repression, according to Professor Fitzgerald, is to "no longer reference the ARWU rankings or participate in the Shanghai Jiaotong rankings process which risks spreading the Chinese Communist Party's university model globally. Universities that continue to participate or reference the Shanghai rankings should be tasked by their faculty and alumni to explain why they are failing to uphold the principles of free inquiry and institutional autonomy as fiercely as Xi Jinping is undermining them."

It is hard to see what Fitzgerald means by not participating in the Shanghai rankings. The Academic Ranking of World Universities (ARWU) uses publicly available data from western sources, the Web of Science, Nature, Science, the Clarivate Analytics list of Highly Cited Researchers, and Nobel and Fields awards. Universities cannot avoid participating in them. They can denounce and condemn the rankings until their faces turn bright purple but they cannot opt out. They are ranked by ARWU whether they like it or not. 

As for referencing, presumably citing the Shanghai rankings or celebrating university achievements there, Fitzgerald's proposals would seem self defeating. The rankings actually understate the achievements of leading Chinese universities. In the latest ARWU Tsinghua University and Peking University are ranked 43rd and 53rd. The QS World University Rankings puts them 16th and 22nd and the THE world rankings 23rd and 24th. 

If anyone wanted to protest the rise of Chinese universities they should turn to the QS and THE rankings where they do well because of reputation, income (THE), and publications in high status journals. It is also possible to opt out of the THE rankings simply by not submitting data.

If oppressive policies did affect the quality of research produced by Chinese universities this would be more likely to show up in the Shanghai rankings through the departure of highly cited researchers or declining submissions to Nature or Science than in the THE or QS rankings where a decline would be obscured if reputation scores continued to hold steady.

Fitzgerald's proposals are pointless and self defeating and ascribe a greater influence to rankings than they actually have.






Thursday, January 16, 2020

The decline of standardised testing

Over the last few years there has been a trend in American higher education to reduce the significance of standardised tests -- SAT, ACT, GRE, GMAT, LSAT -- in university admissions. A large number of institutions have gone test optional, meaning that it is up to students whether or not they submit their scores. Those who do not submit will not be rejected but will be assessed by other criteria such as high school grades and ranks, recommendations, social awareness, grit, coping with adversity, leadership, sports, membership of protected groups, and so on.

Most test optional schools are small liberal arts colleges but recently they were joined by the University of Chicago, an elite university by any standards.

Going test optional has a number of advantages. Students with low test scores will be more likely to apply and that will lower the percentage of applicants admitted which will make the universities look more selective. It could also help with the rankings which may at first sight seem a bit of a paradox. US News has declared that for the America's Best Colleges rankings up to 25% of applicants can withhold their SAT or ACT scores without the college being penalised.

It seems that if universities can arrange to admit 75% of applicants wholly or partly on the basis of their test scores and allow another quarter to be admitted because of a "holistic" assessment then they may suffer a measurable fall in the average academic ability of their students but not enough to get in trouble with the rankers or to undermine their reputation for academic excellence.

It seems likely that US News will continue to adjust its rankings to accommodate the test optional trend, Recently, for example, the ranking of online graduate education courses lowered the threshold for full credit for quantitative and verbal GRE scores from 75% to 25%  of admitted students. The justification for this is that "although many ranked programs with selective admissions made use of GRE scores in limited circumstances very often submitting these scores was optional or waived for applicants."


This will lead to the problem of a substantial number of students being admitted with significantly lower test scores or without taking the tests and so the gap between the most and least able students is likely to widen. Many of those admitted without submitting scores will suffer a serious blow to their self respect as they go from being the academic super stars of their high school or undergraduate program to ranking at the bottom of any assessed test or assignment.

There will accordingly be pressure on colleges and graduates schools to relax grading standards, give credit for group work, allow students to repeat courses, mandate contextualised assessment policies, hold instructors responsible for the performance of students. Faculty who talk about the decline in standards or disparities in achievement will be disciplined and ostracised.

The significant thing about standardised tests is that they correlate quite highly with general intelligence or cognitive ability and also with each other. They played a significant role in the growth of American higher education and research in the twentieth century and the transformation of the Ivy league from a place for producing literate and well behaved young gentlemen into intellectual powerhouses that contributed to the economic and scientific dominance of the US in the second half of the twentieth century.

They are also a good predictor of academic performance although perhaps not quite as good by themselves as high school grades which are influenced by conscientiousness and social conformity

It now seems likely that there will be increasing pressure to get rid of standardised tests altogether. In California there is a court case in process to make it illegal to used tests for university admissions and Carol Christ, Chancellor of the University of California (UC) Berkeley, has declared in favour of abolition.

Getting rid of tests will mean getting rid of an objective measure of students' intelligence and academic ability. Grades are, as noted above, a slightly better predictor overall of academic performance but there are contexts where tests can add vital information to the admission process. Grade inflation throughout US high schools is creating a large number of students with perfect or near perfect grades but with huge differences in cognitive skills. Without tests there will be no way of distinguishing the truly capable from the diligently mediocre or the aggressively conformist.

If  UC does stop using the SAT or ACT for admissions it is unlikely that it will institute a policy of open admissions, at least not yet. It is more probable they it will shift the criteria for selection to high school grades, teachers' recommendations, group membership, and unsupervised personal essays. The consequences of selection though inflatable high school grades and other subjective measures will almost certainly be a significant decline in the average cognitive skills of students at currently selective universities.

American universities will probably become more representative of the ethnic, gender and racial structure of America or the world, more conscientious, more extroverted, more socially aware. Perhaps this will be compensation for the decline in cognitive ability.

It is unlikely that the levelling process will end there. In the years to come there will very probably be demands that universities stop using high school grades or admission essays or anything else that shows a social or racial gap. Studies will be cited showing that wealthy white parents help their children with homework or drive them to volunteering activities or pay for sports equipment or get professional advice about their diversity essays. 

Ultimately there will be a situation where American universities  see a noticeable decline in the academic and cognitive ability of students and graduates in comparison with China and the Chinese diaspora, Japan, Korea, Russia and Eastern Europe and maybe India. Almost certainly this will be attributed by educational experts to the stinginess of federal and state authorities.

Perhaps there will come another Sputnik moment when America  realises that it has fallen behind its competitors. If so it will probably be too late. 


Sunday, December 15, 2019

Sometimes Nice Guys Finish First for Something: the Case of Dartmouth College

There is a crisis approaching for the universities of the global North. A fundamental problem is that declining or stagnant birth rates are reducing the number of potential students, especially in North America, and that will eventually undermine their economic viability. See this article in Inside Higher Ed for the situation in the US.

The options seem to be limited. Universities could downsize and reduce the numbers of staff and students and, at elite US institutions, spending on country club facilities and an ever expanding army of administrators. They could revise their missions by offering fewer graduate courses, especially in the humanities and social sciences, and more vocational programs.

There seems, however, to be little appetite at the moment for such measures. Many universities are trying to maintain income and size by recruiting from abroad. For a while it appeared that western universities would be saved by thousands of international, mainly Chinese, students. But now it looks like fewer Chinese will be coming and there seems to be no substitute in sight. European universities got excited about Middle Eastern refugees filling the empty seats in lecture halls but then it turned out that most lacked the linguistic and cognitive skills for higher education.

The problem is exacerbated by the general decline or flatlining of cognitive skills of potential students, measured by PISA scores or standardized tests. There have been various hypotheses about the cause: smart phones, too much screen time, immigration, dysgenic fertility, inadequate teaching methods, lack of funding, institutional racism and sexism, toxic Trumpism. But, whatever the cause there seems little hope of a recovery any time soon.

Business schools appear to be part of this trend. MBA students tend to be highly mobile and they are not limited to choosing, as many US undergraduates are,  between a community college, the local state university and a struggling private college. Faced with competition from European and Asian schools and online courses, soaring costs and declining applications, many US business schools are at best treading water and at worst in serious danger of drowning.

Dartmouth College, a venerable Ivy League school, is no exception. Back in 2014 it reported the biggest drop in applications in 21 years. Although the college continues to hold its place in the US News Best Colleges rankings it has fallen in the Shanghai rankings, suggesting that it is failing to attract leading researchers as well as talented students.

Dartmouth's Tuck School of Business has suffered as much or more than the rest of the institution. In 2011 it was first in the Economist's Full Time MBA ranking and second in 2012, starting a steady decline until 2019 when it was twelfth.

In 2018 Tuck tried to reverse the steady decline by adopting a new approach to admissions. It was not enough for Tuck students  to be smart, accomplished and aware. They have to be nice

Back in my days in grammar school my English teacher would be outraged by the use of that word. But standards have changed. 

How to measure niceness? By an essay and a referee's report. One does not have to be excessively cynical to see that there is obvious room for gaming and bias here. Their is a  large amount of writing and talking about coaching for standardized tests, none about whether essays like these have the any real authenticity or validity.

But perhaps I am being too cynical. Maybe Dartmouth's business school has done something right. The latest THE business and economics subject rankings puts Dartmouth 44th in the world for business and economics, which is very creditable, ahead of Boston University,  Zhejiang, Edinburgh and Johns Hopkins..

With THE whenever there is a surprisingly high overall score it is a good idea to check the citations indicator which is supposedly a measure of research impact or influence. Sure enough, Dartmouth is second in the world for citations in business and economics just behind Central South University in China and just ahead of Peter the Great St Petersburg polytechnic University.

Could it be that all that niceness is somehow radiating out from the Tuck and causing researchers around the world to cite Dartmouth articles? 

Sunday, November 10, 2019

When will Tsinghua Overtake Harvard?

One of the most interesting trends in higher education over the last few years is the rise of China and the relative decline of the USA.

Winston Churchill said the empires of the future will be empires of the mind. If that is so then this century will very likely be the age of Chinese hegemony. Chinese science is advancing faster than that of the USA on all or nearly all fronts. Unless we count things like critical race theory or queer studies.

This is something that should show up in the global rankings if we track them over at least a few years. So, here is a comparison of the top two universities in the two countries according to indicators of research output and research quality over a decade.

Unfortunately, most international rankings are not very helpful in this respect. Few of the current ones provide data for a decade or more. QS and THE have seen frequent changes in methodology and THE's citation indicator although charmingly amusing is not useful unless you think that Aswan University, Anglia Ruskin University and the University of Peradeniya are world beaters for research impact. Two helpful rankings here are Shanghai Academic Ranking of World Universities (ARWU), and Leiden Ranking.

Let's compare the comparative performance of Tsinghua University and Harvard in the Shanghai Ranking's indicator of research output, papers over a one year period, excluding arts and humanities. The published scores are derived from the square roots of the raw data with the top scorer getting a score of 100.

In 2009 Harvard's score  was 100 while that for Tsinghua was 55.8. In 2019 it was 100 for Harvard and 79.5 for Tsinghua. So the gap is closing 2.37 points every year. At that rate it would take about nine years for Tsinghua to catch up so look out for 2028.

Of course, this is quantity not quality so take a look at another indicator, Highly Cited Researchers. This is a moderately gamable metric and I suspect that Shanghai might have to abandon it one day but it captures the willingness and ability of universities to sponsor research of a high quality. In 2009 Tsinghua's score was zero compared to Harvard's 100. In 2019 it is 37.4. If everything continues at the same rate Tsinghua will overtake Harvard in another 17 years.

Looking at the default indicator in Leiden Ranking, total publications, in 2007-10 Tsinghua was 35% of Harvard and in 2014-17 56%. Working from that Tsinghua would achieve parity in 2029-33, in the rankings published in 2035.

Looking at a measure of research quality, publications in the top 10% of journals, Tsinghua was 15% of Harvard in 2007-10 and 34% in 2014-17.  From that, Tsinghua should reach parity  in 2038-42. in the rankings published in 2044, assuming Leiden is still following its current methodology.

So it looks like Tsinghua will reach parity in  research output in a decade or a decade or a decade and a half and high quality research in a decade and a half or two decades and a half.






Saturday, October 26, 2019

Using Webometrics to Rank University Systems

Recently there has been some interest in ranking higher education systems in addition to institutions or departments. See here and here.  But both of these efforts, from Universitas 21 and QS, rank only 50 countries.

The Webometrics rankings attempt to cover every university in the world or anything that might conceivably claim to be a university, institute or college. The indicators comprise web activity and research output. So, there is data here to create a simple and comprehensive  ranking of countries. Below is the list of countries and territories ranked according to the world rank of the highest ranked university. If the Webometrics methodology remains unchanged it will be updated twice a year.

The table is not very surprising overall but it is worth noting that the leading Asian countries are already in the top ten and that Brazil and Mexico  are not too far behind. The performance of Arab countries is not too impressive even if they are rich in oil.

It's a safe bet that the highest ranked Chinese university will rise steadily over the next few years followed by South Korea and Singapore, but probably not Hong Kong and Australia.






Rank
Country
Rank of highest ranked university
1
USA           
       1
2
UK            
       7
3
Canada        
      19
4
Switzerland   
      32
5
China         
      33
6
Hong Kong     
      45
7
Australia     
      46
8
Singapore     
      50
9
Netherlands   
      63
10
Japan         
      69
11
Brazil        
      74
12
Denmark       
      76
13
Belgium       
      78
14
Finland       
      87
15
Norway        
      93
16
Germany       
      97
17
Sweden        
     106
18
Taiwan        
     111
19
South Korea       
     116
20
Italy         
     120
21
Spain         
     133
22
Mexico        
     141
23
Austria       
     150
24
New Zealand   
     153
25
Israel        
     157
26
Czech Republic         
     204
27
Portugal      
     208
28
Greece        
     224
29
Russia        
     226
30
Argentina     
     228
31
Ireland       
     230
32
South Africa  
     274
33
France        
     292
34
Chile          
     323
35
Malaysia      
     352
36
Argentina     
     372
37
Poland        
     388
38
Saudi Arabia  
     415
39
Iran          
     417
40
Estonia       
     440
41
Serbia        
     464
42
India         
     471
43
Turkey        
     475
44
Thailand      
     513
45
Iceland       
     533
46
Hungary       
     563
47
Egypt         
     602
48
Colombia      
     614
49
Croatia       
     619
50
Luxembourg    
     631
51
Puerto Rico   
     649
52
Belarus       
     684
53
Cyprus        
     700
54
Macau         
     720
55
Slovakia      
     732
56
Lithuania     
     750
57
Indonesia     
     771
58
Costa Rica    
     844
59
Malta         
     866
60
Romania       
     881
61
Bulgaria      
     934
62
Jamaica       
     953
63
Qatar         
     958
64
Peru          
     971
65
Kenya         
     987
66
Vietnam       
    1013
67
Slovenia      
    1103
68
Latvia        
    1106
69
Uganda        
    1129
70
Jordan        
    1149
71
UAE           
    1158
72
Philippines   
    1199
73
Ghana         
    1209
74
Nigeria       
    1233
75
Pakistan      
    1269
76
Ethiopia      
    1314
77
Oman           
    1346
78
Georgia       
    1423
79
Morocco       
    1515
80
North Macedonia   
    1569
81
Venezuela     
    1593
82
Ecuador       
    1638
83
Palestine     
    1646
84
Bosnia        
    1669
85
Kazakhstan    
    1793
86
Trinidad      
    1794
87
Iraq          
    1804
88
Brunei        
    1829
89
Fiji          
    1831
90
Bangladesh    
    1895
91
Tanzania      
    1913
92
Ukraine       
    1977
93
Sri Lanka     
    1981
94
Zimbabwe      
    2014
95
Algeria       
    2061
96
Cuba          
    2134
97
Bahrain       
    2161
98
Kuwait        
    2200
99
Mozambique    
    2280
100
Paraguay      
    2297
101
Mauritius     
    2422
102
Guatemala     
    2458
103
Uruguay       
    2499
104
Botswana      
    2583
105
Grenada       
    2583
106
Armenia       
    2643
107
Liechtenstein 
    2761
108
Montenegro    
    2878
109
Guam          
    2900
110
Sudan         
    2936
111
Bolivia       
    2960
112
Mongolia      
    2962
113
Benin         
    2980
114
Malawi        
    3001
115
Zambia        
    3001
116
Senegal       
    3008
117
Moldova       
    3151
118
Tunisia       
    3198
119
Rwanda        
    3220
120
Nepal         
    3243
121
Namibia       
    3316
122
Panama        
    3391
123
Cameroon      
    3527
124
Barbados      
    3538
125
Azerbaijan    
    3573
126
US Virgin Islands
    3579
127
Syria         
    3593
128
Burkina Faso  
    3634
129
Dominica      
    3679
130
Honduras      
    3892
131
Uzbekistan    
    4017
132
Libya         
    4040
133
Yemen         
    4126
134
Faroe Islands      
    4368
135
Madagascar    
    4372
136
Togo          
    4392
137
Eswatini      
    4428
138
Laos          
    4431
139
Nicaragua     
    4458
140
El Salvador   
    4542
141
Kyrgyzstan     
    4554
142
French Polynesia
    4640
143
Albania       
    4735
144
Monaco        
    4842
145
Dominican Republic
    4903
146
Cambodia      
    5060
147
San Marino    
    5107
148
Papua New Guinee
    5205
149
Greenland     
    5378
150
Afghanistan   
    5676
151
Lesotho       
    5872
152
Antigua       
    6040
153
Guyana        
    6149
154
Ivory Coast   
    6306
155
Anguilla      
    6374
156
Suriname       
    6641
157
Democratic Republic of the Congo     
    7033
158
American Samoa      
    7213
159
Myanmar       
    7221
160
Belize        
    7497
161
Micronesia    
    7962
162
Haiti         
    8082
163
Angola        
    8091
164
Bhutan        
    8159
165
Niger         
    8384
166
Sierra Leone  
    8560
167
Somalia       
   10154
168
St Kitts & Nevis
   10527
169
Cape Verde    
   10685
170
Andorra       
   10772
171
Gambia        
   11020
172
Seychelles    
   11235
173
South Sudan   
   12329
174
Cayman Islands
   13011
175
Samoa         
   13132
176
Bermuda       
   13431
177
British Virgin Islands
   13694
178
Maldives      
   13864
179
Palau         
   13864
180
St Lucia      
   13981
181
Tajikistan    
   14180
182
Djibouti      
   14186
183
Central African Republic           
   14433
184
Northern Marianas
   14444
185
Marshall Islands   
   15827
186
Gabon         
   16002
187
Aruba         
   16347
188
Solomon Islands    
   17867
189
Montserrat    
   18103
190
East Timor       
   18433
191
Guinea        
   18588
192
French Guiana 
   18703
193
Liberia       
   19463
194
Isle of Man   
   20029
195
Mali          
   20172
196
Mauretania    
   22144
197
Equatorial Guinea     
   23382
198
Niue          
   23892
199
Eritrea       
   24481
200
Turks & Caicos Islands
   27918