Polish Universities in International Rankings

My short article on Polish universities and international rankings has just been published by Forum Akademikie. The article in Polish can be accessed here. Translation and editing by Piotr Kieracinski. The full journal issue is here.

Here is the English version.

Richard Holmes

Polish Universities and International Rankings

A Brief History of International University Rankings

After a false start with the Asiaweek rankings of 1999-2000, international university rankings took off in 2003 with the Academic Ranking of World Universities (ARWU), published by Shanghai Jiao Tong University and then by the Shanghai Ranking Consultancy.

In 2004 two new rankings appeared: the Ranking Web of Universities, better known as Webometrics, which originally measured only web activity, and the Times Higher Education Supplement (THES) – Quacquarelli Symonds (QS) World University Rankings, which emphasised research and also included faculty resources and internationalisation indicators.

Since then the number of rankings, metrics and data points has increased prompting ranking consultant Daniel Guhr to talk about “vast data lakes”. Rankings have become more complex and sophisticated and often use statistical techniques such as standardisation and field normalisation.

In addition to global rankings, specialist rankings of regions, subjects, and business schools have appeared. International rankings continue to have a bias towards research but some try to find a way of capturing data that might be relevant to teaching and learning or to university third missions such as sustainability, gender equity and open access. They have also become significant in shaping national higher education policy and institutional strategy.

Although the media usually talk about the big four rankings or sometimes the big three or big two, there are now many more. The IREG Inventory of International Rankings includes 17 global rankings in addition to regional and specialised rankings and various spin offs. Since the publication of the inventory more global rankings have appeared and no doubt there are more to come.

Media Perceptions of Rankings

It is unfortunate that the media and public perception of global rankings has little relation to reality. The Times Higher Education (THE) World University Rankings are by far the most prestigious but they have serious defects. They lack transparency with eleven indicators bundled into three broad groups. They rely on subjective surveys and questionable data submitted by institutions. They are unbalanced with a 30% weighting going towards a citation indicator that can be influenced by a handful of papers in a multi-author international project and that has elevated a succession of little-known places to world research leadership. These include the University of Reykjavik, Babol Noshirvani University of Technology, Aswan University, and Anglia Ruskin University.

The problems of the THE world rankings are illustrated by looking at the fate of a leading Polish university in recent editions. In the 2014-15 rankings the University of Warsaw was ranked 301-350 but in 2015-16 it fell to 501-600. This was entirely the result of a dramatic fall in the score for citations and that was entirely the result of a methodological change. In 2015 THE stopped counting citations to papers with over a thousand “authors”. This badly affected the University of Warsaw which, along with Warsaw University of Technology, had been contributing to the Large Hadron Collider project a producer of many such papers. The University of Warsaw’s decline in the rankings had nothing to do with any defect. It was simply the result of THE’s tweaking.

Although they receive little attention from the media there are now several global rankings published by universities and research councils that include more universities, cover a broader range of indicators and are technically as good as or better than the Big Four. These include the National Taiwan University Rankings, University Ranking by Academic Performance published by Middle East Technical University, the Scimago Institution Rankings and CWTS Leiden Ranking.

Polish Universities in Global Rankings

Turning to the current position of Polish universities in international rankings there is a great deal of variation. There are 14 in the THE rankings with 4 in the top 1000, but 410 in the Webometrics rankings of which 10 are in the top 1000. The ranking with the best representation of Polish universities is Scimago with 54 in the top 1000.

Of the “big four” rankings -- THE, QS, Shanghai, US News -- the best for analysing the current standing of the International Visibility Project (IntVP) universities is the US News Best Global Universities (BGU). THE and QS are unbalanced with too much emphasis on a single indicator, citations and academic survey respectively. The Shanghai rankings include Nobel and Fields awards some of which are several decades old. It should be noted that BGU is an entirely research based ranking.

The list below indicates the world rank of Polish universities in the latest US News BGU:

University of Warsaw 286

Jagiellonian University 343

Warsaw University of Technology 611

AGH University of Science and Technology 635

Adam Mickiewicz University 799

University of Wroclaw 833

Medical University of Wroclaw 926

Wroclaw University of Science and Technology 961

Nicholas Copernicus University 984

Medical University of Gdansk 995

Medical University of Warsaw 1033

University of Silesia 1082

University of Gdansk 1096

University of Lodz 1119

Gdansk University of Technology 1148

Poznan University of Technology 1148

Lodz University of Technology 1194

Lodz Medical University 1203

Warsaw University of Life Sciences 1221

Maria Curie-Sklodowska University 1303

Pomeranian Medical University 1303  

Poznan University of Medical Sciences 1312

Silesian University of Technology 1351

Krakow University of Technology   1363

University of Warmia 1363

Medical University Silesia    1399

Medical University of Lublin 1414

University of Rseszov 1430

Technical University Czestochowa 1445

Poznan University of Life Sciences 1457

Wroclaw University of Life and Environmental Sciences 1465

Agricultural University of Lublin. No overall rank. 214 for agriculture

Medical University of Bialystok. No overall rank. 680 for clinical medicine.

One thing that emerges from this list is that the Polish university system suffers from a serious handicap in this ranking and in others due to the existence of independent specialist universities of technology, business and medicine. Consolidation of small specialist institutions could bring about significant improvements as has recently happened in France.

There is also some variation in the rank of the best performing Polish universities. In most rankings the top scoring Polish university is in the 300s or 490s. There are, however, some exceptions. The default indicator in Leiden Ranking, total publications, has Jagiellonian University at 247, GreenMetric has Adam Mickiewicz University at 160 and the Moscow Three Missions University Rankings puts the University of Warsaw at 113. On the other hand, no Polish university gets higher than the 600-800 band in the THE world rankings.

The various rankings have very different methodologies and indicators. THE for example includes income in three indicators. The QS rankings gave a combined weighting of 50 % to reputation surveys. Scimago counts patents and the Center for World University Rankings (CWUR), now based in the Arab Gulf, the achievements of alumni. It would be a good idea to look carefully at the content and format of all the rankings before using them for evaluation or benchmarking.

Polish Universities: Strengths and Weaknesses

Poland has certain advantages with regard to international rankings. It has an excellent secondary school system as shown by above average performance in PISA and other international standardised tests. Current data indicates that it has adequate teaching resources, shown by statistics for staff student ratio. It has cultural and economic links to the East, with the Anglosphere and within the EU that are likely in the future to produce fruitful research partnerships and networks.

On the other hand, the evidence of current rankings is that Polish universities relatively underfunded and that doctoral education is still relatively limited. The reputation for research in the world is not very high although is better for regional reputation.

One exception to the limited international visibility of Polish universities is a recent British film, The Last Passenger, in which a hijacked train is saved by a few heroes one of whom has an engineering degree from Gdansk University of Technology.

Poland and the Rankings

It would be unwise for Poland, or indeed any country, to focus on a single ranking. Some rankings have changed their methodology and will probably continue to do so and this might lead to unexpected rises or falls. THE has announced that there will be a new 3.0 version of the world rankings towards the end of this year.

Any university or university system wishing to engage with the rankings should be aware that they often favour certain types of institutions. The Shanghai rankings, for example, privilege medical research and totally ignore the arts and humanities. Scimago includes data about patents which would give technological universities an advantage. THE’s current methodology gives a massive privilege to participants in multi-contributor projects. QS uses several channels to obtain respondents for its academic and employer surveys. One of these is a list of potential respondents provided by the universities. Universities are very likely to nominate those who are likely to support them in the surveys. 

Guidelines for Polish universities as they seek to establish and extend their international presence.

First, improving international visibility will take time. Quick fixes such as recruiting highly cited adjunct faculty or taking part in high profile projects may be counterproductive, especially if there is an unannounced change in methodology.

Second, before launching a campaign to rise in the global rankings some universities might consider regional or specialist rankings first, such as the THE Europe Teaching Rankings, the QS graduate employability ranking, the Indonesian GreenMetric rankings or business school rankings.

Third, Universities should also consider the cost of taking part in the rankings. US News, QS and THE require universities to submit data and this can be time consuming especially for THE which asks for data in ten subjects. Many universities seem to need three or four staff dedicated to rankings.

Fourth, it would be wise to monitor all international rankings for data that can be used for internal evaluation or publicity.

Fifth, universities should match their profiles, strengths and weaknesses with the methodology of specific rankings. Universities with strengths in medical research might perform well in the Shanghai rankings.

Sixth, it is not a good idea to focus exclusively on any single ranking or to make any one ranking the standard of excellence.

1.