Friday, October 25, 2019

Using Webometrics to Rank University Systems

Recently there has been some interest in ranking higher education systems in addition to institutions or departments. See here and here.  But both of these efforts, from Universitas 21 and QS, rank only 50 countries.

The Webometrics rankings attempt to cover every university in the world or anything that might conceivably claim to be a university, institute or college. The indicators comprise web activity and research output. So, there is data here to create a simple and comprehensive  ranking of countries. Below is the list of countries and territories ranked according to the world rank of the highest ranked university. If the Webometrics methodology remains unchanged it will be updated twice a year.

The table is not very surprising overall but it is worth noting that the leading Asian countries are already in the top ten and that Brazil and Mexico  are not too far behind. The performance of Arab countries is not too impressive even if they are rich in oil.

It's a safe bet that the highest ranked Chinese university will rise steadily over the next few years followed by South Korea and Singapore, but probably not Hong Kong and Australia.






Rank
Country
Rank of highest ranked university
1
USA           
       1
2
UK            
       7
3
Canada        
      19
4
Switzerland   
      32
5
China         
      33
6
Hong Kong     
      45
7
Australia     
      46
8
Singapore     
      50
9
Netherlands   
      63
10
Japan         
      69
11
Brazil        
      74
12
Denmark       
      76
13
Belgium       
      78
14
Finland       
      87
15
Norway        
      93
16
Germany       
      97
17
Sweden        
     106
18
Taiwan        
     111
19
South Korea       
     116
20
Italy         
     120
21
Spain         
     133
22
Mexico        
     141
23
Austria       
     150
24
New Zealand   
     153
25
Israel        
     157
26
Czech Republic         
     204
27
Portugal      
     208
28
Greece        
     224
29
Russia        
     226
30
Argentina     
     228
31
Ireland       
     230
32
South Africa  
     274
33
France        
     292
34
Chile          
     323
35
Malaysia      
     352
36
Argentina     
     372
37
Poland        
     388
38
Saudi Arabia  
     415
39
Iran          
     417
40
Estonia       
     440
41
Serbia        
     464
42
India         
     471
43
Turkey        
     475
44
Thailand      
     513
45
Iceland       
     533
46
Hungary       
     563
47
Egypt         
     602
48
Colombia      
     614
49
Croatia       
     619
50
Luxembourg    
     631
51
Puerto Rico   
     649
52
Belarus       
     684
53
Cyprus        
     700
54
Macau         
     720
55
Slovakia      
     732
56
Lithuania     
     750
57
Indonesia     
     771
58
Costa Rica    
     844
59
Malta         
     866
60
Romania       
     881
61
Bulgaria      
     934
62
Jamaica       
     953
63
Qatar         
     958
64
Peru          
     971
65
Kenya         
     987
66
Vietnam       
    1013
67
Slovenia      
    1103
68
Latvia        
    1106
69
Uganda        
    1129
70
Jordan        
    1149
71
UAE           
    1158
72
Philippines   
    1199
73
Ghana         
    1209
74
Nigeria       
    1233
75
Pakistan      
    1269
76
Ethiopia      
    1314
77
Oman           
    1346
78
Georgia       
    1423
79
Morocco       
    1515
80
North Macedonia   
    1569
81
Venezuela     
    1593
82
Ecuador       
    1638
83
Palestine     
    1646
84
Bosnia        
    1669
85
Kazakhstan    
    1793
86
Trinidad      
    1794
87
Iraq          
    1804
88
Brunei        
    1829
89
Fiji          
    1831
90
Bangladesh    
    1895
91
Tanzania      
    1913
92
Ukraine       
    1977
93
Sri Lanka     
    1981
94
Zimbabwe      
    2014
95
Algeria       
    2061
96
Cuba          
    2134
97
Bahrain       
    2161
98
Kuwait        
    2200
99
Mozambique    
    2280
100
Paraguay      
    2297
101
Mauritius     
    2422
102
Guatemala     
    2458
103
Uruguay       
    2499
104
Botswana      
    2583
105
Grenada       
    2583
106
Armenia       
    2643
107
Liechtenstein 
    2761
108
Montenegro    
    2878
109
Guam          
    2900
110
Sudan         
    2936
111
Bolivia       
    2960
112
Mongolia      
    2962
113
Benin         
    2980
114
Malawi        
    3001
115
Zambia        
    3001
116
Senegal       
    3008
117
Moldova       
    3151
118
Tunisia       
    3198
119
Rwanda        
    3220
120
Nepal         
    3243
121
Namibia       
    3316
122
Panama        
    3391
123
Cameroon      
    3527
124
Barbados      
    3538
125
Azerbaijan    
    3573
126
US Virgin Islands
    3579
127
Syria         
    3593
128
Burkina Faso  
    3634
129
Dominica      
    3679
130
Honduras      
    3892
131
Uzbekistan    
    4017
132
Libya         
    4040
133
Yemen         
    4126
134
Faroe Islands      
    4368
135
Madagascar    
    4372
136
Togo          
    4392
137
Eswatini      
    4428
138
Laos          
    4431
139
Nicaragua     
    4458
140
El Salvador   
    4542
141
Kyrgyzstan     
    4554
142
French Polynesia
    4640
143
Albania       
    4735
144
Monaco        
    4842
145
Dominican Republic
    4903
146
Cambodia      
    5060
147
San Marino    
    5107
148
Papua New Guinee
    5205
149
Greenland     
    5378
150
Afghanistan   
    5676
151
Lesotho       
    5872
152
Antigua       
    6040
153
Guyana        
    6149
154
Ivory Coast   
    6306
155
Anguilla      
    6374
156
Suriname       
    6641
157
Democratic Republic of the Congo     
    7033
158
American Samoa      
    7213
159
Myanmar       
    7221
160
Belize        
    7497
161
Micronesia    
    7962
162
Haiti         
    8082
163
Angola        
    8091
164
Bhutan        
    8159
165
Niger         
    8384
166
Sierra Leone  
    8560
167
Somalia       
   10154
168
St Kitts & Nevis
   10527
169
Cape Verde    
   10685
170
Andorra       
   10772
171
Gambia        
   11020
172
Seychelles    
   11235
173
South Sudan   
   12329
174
Cayman Islands
   13011
175
Samoa         
   13132
176
Bermuda       
   13431
177
British Virgin Islands
   13694
178
Maldives      
   13864
179
Palau         
   13864
180
St Lucia      
   13981
181
Tajikistan    
   14180
182
Djibouti      
   14186
183
Central African Republic           
   14433
184
Northern Marianas
   14444
185
Marshall Islands   
   15827
186
Gabon         
   16002
187
Aruba         
   16347
188
Solomon Islands    
   17867
189
Montserrat    
   18103
190
East Timor       
   18433
191
Guinea        
   18588
192
French Guiana 
   18703
193
Liberia       
   19463
194
Isle of Man   
   20029
195
Mali          
   20172
196
Mauretania    
   22144
197
Equatorial Guinea     
   23382
198
Niue          
   23892
199
Eritrea       
   24481
200
Turks & Caicos Islands
   27918





Saturday, September 28, 2019

Rankings case shows need to reform citations indicator

My previous post has just been republished by University World News.

Comments here are welcome.

Thursday, September 26, 2019

Trinity College Dublin: Time to forget about THE?:

Global rankings, especially THE's, have been very useful to British universities, at least to those sitting at the apex of the system. If a Russell Group university falls in the rankings then it is the fault of impending Brexit and/or the terrible austerity inflicted on the nation's research prowess. If it rises then this is cause for congratulation but with a hint of foreboding. How can they keep advancing with Ebenezer Scrooge controlling  the treasury and the bitter winds of Brexit howling at the door? The universities have reciprocated by not inquiring about how THE constructs its rankings, particularly the citations and industry income indicators. 

Across the Irish Sea, universities have for the most part also been loyal to THE. Trinity College Dublin (TCD) has continued to submit data to THE and also to QS and has passively accepted the results of the rankings even when they show the institution going down and down. The steady decline is usually blamed on the meanness of the Irish government and its failure to provide sufficient funds.

I have dealt with Trinity's misfortunes here,  herehere, and here, In 2015 TCD fell seven places in the QS world rankings and 22 in THE's. In contrast it had been rising in the Shanghai ARWU rankings since 2004 and in the Round University Rankings (RUR) since 2010, although everybody pretended not to notice this.

This year history repeats itself all over again. TCD has fallen in THE world rankings from 120th place to 164th. Again this supposedly is the fault of the Irish state to provide enough money.

But we get a very different picture when we look at the Shanghai Rankings. TCD has risen from 167th place to 154th, getting close to the 101-150 band. Leaving aside the Nobel and Fields Awards, Trinity has gained 6.6 points for highly cited researchers, 1.4 for publications, and 1.4 for productivity per capita. It has, however, fallen 0.6 for papers in Nature and Science.

Looking at RUR, TCD has risen from 75th to 57th for Research and 35th to 29th for international diversity. It has fallen slightly for financial sustainability from 191st to 197th and for Teaching from 275th to 335th, mainly because of a fall in the number of academic staff.

It seems perverse for TCD to keep on about its decline in the THE rankings when it can point to a steady rise in the Shanghai rankings which are not perfect but are certainly more stable, consistent and realistic than THE. 

Does THE really want to be judged by rankings that apparently think that Anadolu University is best for Innovation, Luxembourg for International Orientation and Aswan for research impact measured by Citations?

But if TCD really insist on sticking with THE then I suggest that they recruit a few researchers taking part in the Global Burden of Disease Study, funded by the Bill and Melinda Gates Foundation.*

They should also think about amalgamating with the Royal College of Surgeons.

*assuming no methodological change




Tuesday, September 17, 2019

Seven Modest Suggestions for Times Higher Education


My post of the 24th of August has been republished in Arabia Higher Education







Going Up and Up Down Under: the Case of the University of Canberra

It is a fact almost universally ignored that when a university suddenly rises or falls many places in the global rankings the cause is not transformative leadership, inclusive excellence, team work, or strategic planning but nearly always a defect or a change in the rankers' methodology.

Let's take a look at the fortunes of the University of Canberra (UC) which THE world rankings now have in the world's top 200 universities and Australia's top ten. This is a remarkable achievement since the university did not appear in these rankings until 2015-16 when it was placed in the 500-600 band with very modest scores of 18.4 for teaching, 19.3 for research, 29.8 for citations, which is supposed to measure research impact, 36.2 for industry income, and 54.6 for international outlook.

Just four years later the indicator scores are 25.2 for teaching, 31.1 for research, 99.2 for citations, 38.6 for industry income, and 86.9 for international orientation. 

The increase in the overall score over four years, calculated with different weightings for the indicators, was composed of 20.8 points for citations and 6.3 for the other four indicators combined. Without those 20.8 points Canberra would be in the 601-800 band.

I will look at where that massive citation score came from in a moment. 

It seems that the Australian media is reporting on this superficially impressive performance with little or no scepticism and without noting how different it is from the other global rankings. 

The university has issued a statement quoting vice-chancellor Professor Deep Saini as saying that the "result confirms the steady strengthening of the quality at the University of Canberra, thanks to the outstanding work of our research, teaching and professional staff" and that the "increase in citation impact is indicative of the quality of research undertaken at the university, coupled with a rapid growth in influence and reach, and has positioned the university as amongst the best in the world."

The Canberra Times reports that the vice-chancellor has said  that part of the improvement was the result of a talent acquisition campaign while noting that many faculty were complaining about pressure and excessive workloads.

Leigh Sullivan, DVC for research and innovation, has a piece in the Campus Morning Mail that hints at reservations about UC's apparent success, which is " a direct result of its Research Foundation Plan (2013-2017) and "a strong emphasis on providing strategic support for research excellence in a few select research areas where UC has strong capability." He notes that when the citation scores of research stars are excluded there has still been a significant increase in citations and warns that what goes up can go down and that performance can be affected by changes in the ranking methodology.

The website riotact quotes the vice-chancellor on the improvement in research quality as evidence by the citation score and as calling for more funding for universities: the "government has to really think and look hard at how well we support our universities. That's not to say it badly supports us, it's that the university sector deserves to be on the radar of our government as a major national asset."

The impressive ascent of UC is unique to THE. No serious ranking puts it in the top 200 or anywhere near. In the current Shanghai Rankings it is in the 601-700 band and has been falling for the last two years. In Webometrics it is 730th in the world and 947th for Excellence, that is publications in the 10% most cited in 25 disciplines.  In  University Ranking by Academic Performance it is 899th and in the CWUR Rankings it doesn't even make the top 1,000.

Round University Ranking and Leiden Ranking do not rank UC at all.

Apart from THE UC does best in the QS rankings where it is 484th in the world and 26th in Australia.

So how could UC perform so brilliantly in THE rankings when nobody else has recognised that brilliance? What does THE know that nobody else does? Actually, it does not perform brilliantly in the THE rankings, just in the citations indicator which is supposed to measure research influence or research impact.

This year UC has a score of 99.2 which puts it in the top twenty for citations just behind Nova Southeastern University in Florida and Cankaya University in Turkey and ahead of Harvard, Princeton and Oxford. The top university this year is Aswan University in Egypt replacing Babol Noshirvani University of Technology in Iran. 

No, THE is not copying the interesting methodology of the Fortunate 500. This is the result of an absurd methodology that THE is unable or unwilling for some reason to change.

THE has a self-inflicted  problem with  a small number of papers that have hundreds or thousands of "authors" and collect thousands of citations. Some of these are from the CERN project and THE has dealt with them  by using a modified form of fractional counting for papers with more than a thousand authors. That has removed the privilege of institutions that contribut to CERN projects but has replaced it with the privilege of those that contribute to the Global Burden of Disease Study (GBDS) whose papers tend to have hundreds but not thousands of contributors and sometimes receive over a thousand citations. As a result, places like Tokyo Metropolitan University, National Research University MEPhI and Royal Holloway London have been replaced as citation super stars by St Georges' London, Brighton and Sussex Medical School, and Oregon Health and Science University.

It would be a simple matter to apply fractional counting to all papers, dividing the number of citations by the number of authors. After all Leiden Ranking and Nature Index manage to do it but THE for some reason has chosen not to follow.

The problem is compounded by counting self-citations, by hyper-normalisation so that the chances of hitting the jackpot with an unusually highly cited paper are increased, and by the country bonus that boosts the scores for universities by virtue of their location in low scoring countries. 

And so to UC's apparent success this year. This is entirely the result of it's citation score which is entirely dependent on THE's methodology. 

Between 2014 and 2018 UC had 3,825 articles in the Scopus database of which 27 were linked to the GBDS which is funded by the Bill and Melinda Gates Foundation. Those 27 articles, each with hundreds of contributors, have received 18,431 citations all of which are credited to UC and its contributor. The total number of citations is 53,929 so those 27 articles accounted for over a third of UC's citations. Their impact might be even greater if they were cited disproportionately soon after publication.

UC has of course improved its citation performance even without those articles but it is clear that they have made an outsize contribution. UC is not alone here. Many universities in the top 100 for citations in the THE world rankings owe their status to the GBDS: Anglia Ruskin, Reykjavik, Aswan, Indian Institute of Technology Ropar, the University of Peradeniya, Desarrollo, Pontifical Javeriana and so on.

There is absolutely nothing wrong with the GBDS nor with UC encouraging researchers to take part. The problem lies with THE and its reluctance to repair an indicator that produces serious distortions and is an embarrassment to those universities who apparently look to the THE rankings to validate their status.

Monday, September 16, 2019

What should universities do about organised cheating?

Every so often the world of higher education is swept by a big panic about systemic and widespread cheating. The latest instance is concern about contract cheating or essay mills that provide bespoke essays or papers for students.

It seems that the Australian government will introduce legislation to penalise the supply or advertising of cheating services to students. There are already laws in several American states and there have been calls for the UK to follow suit.

There is perhaps a bit of hypocrisy here. If universities in Europe Australia and North America admit more and more students who lack the cognitive or language skills to do the required work and if they chose to use assessment methods that are vulnerable to deception and dishonesty such as unsupervised essays and group projects then cheating is close to inevitable. 

On the supply side there appear to be large numbers of people around the world without decent academic jobs or jobs of any sort who are capable of producing academic of a high standard, sometimes worth an A grade or a first. The Internet has made it possible for lazy or incompetent students to link up with competent writers.

The Daily Mail has reported that Kenya hosts a medium sized industry with  students and academics slaving away to churn out essays for British and American students. This is no doubt a hugely exploitative business but consider the consequences of shutting down the essay mills. Many educated Kenyans are going to suffer financially. Many students will drop out, resort to  other forms of cheating, or will demand more support and counselling and transitional or foundation programmes.

If universities are serious about the scourge of essay mills they need to work on both the supply and the demand side. They might start by offering the essay writers in Kenya to apply scholarships for undergraduate or postgraduate courses or posts in EAP departments. 

On the demand side the solution seems to be simple. Stop admitting students because they show leadership ability, have overcome adversity, will make the department look like Britain, America or the world, will help craft an interesting class, and admit them because they have demonstrated an ability to do the necessary work.












https://www.studyinternational.com/news/australia-essay-mills-contract-cheating-penalty-law/