Top universities for research impact in the emerging economies

It is always interesting to read the reactions of the good and the great of the academic world to the latest rankings from Times Higher Education (THE). Rectors, vice chancellors and spokespersons of various kinds gloat over their institutions' success. Opposition politicians and journalists demand rolling heads if a university falls.

What nobody seems to do is take a look at the ranking methodology or the indicator scores which can often say as much about a university's success or failure than any amount of government funding, working as a team or dynamic leadership.

The citations indicator in the latest THE Emerging Economies Rankings is supposed to measure research impact and officially accounts for twenty per cent of the total weighting. In practice its weighting is effectively larger because universities in every country except the one with the most impact benefit from a country adjustment.

Here are the top ten universities in the emerging world for research impact according to THE.

1.  Jordan University of Science and Technology
2.  Cyprus University of Technology
3.  University of Desarrolio
4.  Diego Portales University
5.  Southern University of Science and Technology China
6.  University of Crete
7.  University of Cape Town
8.  Indian Institute of Technology Indore
9.  Pontifical Javeriana University
10. University of Tartu

Where does prestige come from? : Age, IQ, research or money?

Prestige is a wonderful thing. Universities use it to attract capable students, prolific researchers, grants and awards and, of course, to rise in the global rankings.

This post was inspired by a series of tweets that started with a claim that the prestige of  universities was dependent on student IQ. 

That is a fairly plausible idea. There is good evidence that employers expect universities to guarantee that graduates have a certain level of cognitive ability, are reasonably conscientious and, especially in recent years, conform to prevailing social and political orthodoxy. At the moment, general cognitive ability appears to be what contributes most to graduate employability although it may be less important than it used to be.

Then there was a suggestion that when it came to prestige it was actually age and research that mattered. Someone also said that it might be money.

So I have compared these metrics or proxies with universities' scores on various reputation surveys, which could be indicative, perhaps not perfectly, of their prestige

I have taken the median ACT or SAT scores of admitted students at the top fifty selective colleges in the USA as a substitute for  IQ, with which they have a high correlation. The data is from the supplement to a paper in the Journal of Intelligence by Wai, Brown and Chabris. 

The  endowments of those colleges and the financial sustainability scores in the Round University Rankings are used to measure money. The  number of research publications listed in the latest CWTS Leiden Ranking represents research. 

I have looked at the correlations of these with the reputation scores in the rankings by QS (academic reputation and employer reputation), Times Higher Education (THE) (research reputation and teaching reputation), RUR  (research reputation, teaching reputation and reputation outside region), and the Emergence/Trendence survey of graduate employability.

Since we are looking at a small fraction of the world's institutions in just one country the generalisability of this exercise is limited.

So what do we come up with? First, there are several highly selective liberal arts colleges in the US that are overlooked by international rankings. About half of the top 50 schools by SAT/ACT scores in the US do not show in the global rankings. An international undergraduate student wanting to study in the USA would do well to look beyond these rankings and think about places that are still highly selective such as Harvey Mudd, Pomona and Amherst Colleges.

Let's take a look at the four attributes. Age doesn't matter. There is no significance correlation between an institution's age and any of the reputation indicators. The lowest correlation, -.15, is with the RUR world research reputation indicator and the highest, but still not significant, .36, is with the THE teaching reputation indicator.

Research, however, is important. The correlation between total publications in the most recent Leiden Ranking varies from .48, RUR reputation outside region, to .63, THE teaching reputation.

So are standardised test scores. There is a significant correlation between SAT/ACT scores and the reputation indicators in the QS, RUR and Emerging/Trendence survey, ranging from .48 for the RUR world research reputation and reputation outside region to .72 for the Emerging/Trendence ranking. But the correlation with the THE teaching and research reputation indicators is not significant.

The RUR composite financial sustainability indicator correlates highly with the QS, RUR and Emerging/Trendence rankings, ranging from .47 for the QS employers' survey to .71 for the RUR world teaching reputation score but not for the THE indicators with which it is .15 for research and .16 for  teaching.

Endowment value appears to be the biggest influence on reputation. it correlates significantly with all reputation indicators, ranging from .42 for the RUR world research reputation indicator to .72 for Emerging/Trendence.

Of the four inputs the one that has the highest correlation with the three RUR reputation indicators, .71, .63, and .64 and the QS academic survey, .59, is financial sustainability.

Endowment value has the highest correlation with the QS employer survey, .57, and the two THE indicators, .66 and .71. Endowment and SAT are joint top for the Emerging/Trendence employability survey, .72. 

So it's seems that the best way to a good reputation, at least for selective American colleges, would be money. Test scores and research output can also help. But age doesn't matter.