Rankings and Higher Education Policy

Two examples of how the need to perform well in the rankings is shaping national research and higher education policy.

From the Irish Examiner

"Ireland must apply for membership of the world-renowned European Organisation for Nuclear Research (Cern) in order to combat the effect of Brexit and boost university rankings.

That is according to Cork senator Colm Burke as the campaign to join Cern gains momentum, after Ireland recently became a member of the European Space Observatory."

From Times Higher Education

"France’s programme of university mergers is paying off, improving the research performance and international visibility of its top providers, according to the Times Higher Education World University Rankings 2019.

Paris Sciences et Lettres – PSL Research University Paris, a 2010 merger of numerous institutions, climbed 31 places to 41st this year, becoming the first French university to feature in the top 50 best universities since 2011. PSL made its debut in the global table last year.

Its teaching and research scores improved, driven by increased global visibility and votes in the academic teaching and research reputation surveys.

Meanwhile, Sorbonne University, which was founded in January this year following the merger of Pierre and Marie Curie University and Paris-Sorbonne University, has joined the list at 73rd place – making it the highest-ranked newcomer in the table."

https://www.irishexaminer.com/breakingnews/business/cern-membership-vital-for-irish-universities-872312.html

Philosophy Department Will Ignore GRE Scores

The philosophy department at the University of Pennsylvania has taken a step away from fairness and objectivity in university admissions. It will no longer look at the GRE scores of applicants to its graduate programme. 

The department is good but not great. It is ranked 27th in the Leiter Report rankings and in the 101-150 band in the QS world subject rankings.

So how will students be selected without GRE scores? It seems it will be by letters of recommendation, undergraduate GPA, writing samples, admission statements.

Letters of recommendation have very little validity. The value of undergraduate grades has eroded in recent years and very likely will continue to do so. Admission essays and diversity statements  say little about academic ability and a lot about political conformism.

The reasons for the move are not convincing. Paying for the GRE is supposed to be a burden on low income students. But the cost is much less than Penn's exorbitant tuition fees. It is also claimed that the GRE and other standardised tests do not predict performance in graduate school. In fact they are a reasonably good predictor of academic success although they should not be used by themselves. 

Then there is the claim that the GRE "sometimes" underpredicts the performance of minorities and women. No doubt it sometimes does but then presumably sometimes it does not. Unless there is evidence that the underprediction is significant and that it is greater than that of other indicators this claim is meaningless.

What will be the result of this? The department will be able to admit students who "do not test well" but who can get good grades, something that is becoming less difficult at US colleges, or  persuade letter writers at reputable schools that they will do well.

It is likely that more departments across the US will follow Penn's lead. American graduate programmes will slowly become less rigorous and less able to compete with the rising universities of Asia.

Ranking Global Rankings: Information

Another indicator for ranking global rankings might be the amount of information that they contain. Here are 17 global rankings in the IREG Inventory ranked according to the number of indicators or groups of indicators for which scores or ranks are given. The median and the mode are both six.

The number for U-Multirank is perhaps misleading since data is not provided for all universities. 

 Number of indicators or indicator groups with scores or ranks

Rank	Ranking	Address of publisher	Number of indicators
1	U-Multirank	Germany	112
2	RUR Round University Rankings	Russia	20
3	CWTS Leiden Ranking	Netherlands	19
4	US News Best Global Universities	USA	13
5	NTU Ranking	Taiwan	8
6	CWUR University Rankings	UAE	7
7=	QS World University Rankings	UK	6
7=	Shanghai Ranking ARWU	China	6
7=	UI GreenMetric Ranking	Indonesia	6
7=	URAP University Ranking by Academic Performance	Turkey	6
11	THE World University Rankings	UK	5
12	Ranking Web of Universities (Webometrics)	Spain	4
13	SCImago Institutions Ranking	Spain	3
14	Nature Index	UK	2
15=	Emerging/Trendence Global University Employability	France	1
15=	Reuters Top 100 Innovative Universities	USA	1
15=	uniRank University Ranking	Australia	1

Ranking Global Rankings: Inclusion

The number of international global universities continues to grow and it is becoming harder to keep track of them. Earlier this year IREG published an inventory of international rankings that included 17 global rankings. Here are those rankings in order of the number of institutions that they rank in the most recent edition.

Webometrics is the clear winner, followed by uniRank and SCImago. There are, of course, other indicators to think about and some of these will be covered later.

Number of Institutions ranked

Rank	Ranking	Address of publisher	Number ranked
1	Ranking Web of Universities (Webometrics)	Spain	28,077
2	uniRank University Ranking	Australia	13,146
3	SCImago Institutions Ranking	Spain	5,637
4	URAP University Ranking by Academic Performance	Turkey	2,500
5	U-Multirank	Germany	1,500
6	US News Best Global Universities	USA	1,250
7	THE World University Rankings	UK	1,000+
8=	Shanghai Ranking ARWU	China	1,000
8=	CWUR University Rankings	UAE	1,000
10	QS World University Rankings	UK	916
11	CWTS Leiden Ranking	Netherlands	903
12	NTU Ranking	Taiwan	800
13	RUR Round University Rankings	Russia	783
14	UI GreenMetric Ranking	Indonesia	619
15	Nature Index	UK	500
16	Emerging/Trendence Global University Employability	France	150
17	Reuters Top 100 Innovative Universities	USA	100

Ranking US Rankings

Forbes Magazine has an article by Willard Dix that ranks US  ranking sites. The ranking is informal without specifying indicators but the author does give us an idea of what he thinks a good ranking should do.

Here are the top five of thirteen:
1.  US News: America's Best Colleges
2.  Money magazine: Best Colleges Ranking
3.  Forbes: America's Top Colleges
4.  Kiplinger's Best College Values
5.  Washington Monthly: College Guide and Rankings.

Reading through the comments it is possible to get an idea of the criteria of a good ranking. Rankings should contain a lot of information, they should be comprehensive and include a large number of institutions, they should provide data that helps prospective students and stakeholders, they should be published for several years, if they use surveys they should have a lot of respondents, they should have face validity (a list with a "revolutionary algorithm" that puts non-Ivy places at the top is in 13th place).