Saturday, January 12, 2019

Where does prestige come from? : Age, IQ, research or money?

Prestige is a wonderful thing. Universities use it to attract capable students, prolific researchers, grants and awards and, of course, to rise in the global rankings.

This post was inspired by a series of tweets that started with a claim that the prestige of  universities was dependent on student IQ. 

That is a fairly plausible idea. There is good evidence that employers expect universities to guarantee that graduates have a certain level of cognitive ability, are reasonably conscientious and, especially in recent years, conform to prevailing social and political orthodoxy. At the moment, general cognitive ability appears to be what contributes most to graduate employability although it may be less important than it used to be.

Then there was a suggestion that when it came to prestige it was actually age and research that mattered. Someone also said that it might be money.

So I have compared these metrics or proxies with universities' scores on various reputation surveys, which could be indicative, perhaps not perfectly, of their prestige

I have taken the median ACT or SAT scores of admitted students at the top fifty selective colleges in the USA as a substitute for  IQ, with which they have a high correlation. The data is from the supplement to a paper in the Journal of Intelligence by Wai, Brown and Chabris. 

The  endowments of those colleges and the financial sustainability scores in the Round University Rankings are used to measure money. The  number of research publications listed in the latest CWTS Leiden Ranking represents research. 

I have looked at the correlations of these with the reputation scores in the rankings by QS (academic reputation and employer reputation), Times Higher Education (THE) (research reputation and teaching reputation), RUR  (research reputation, teaching reputation and reputation outside region), and the Emergence/Trendence survey of graduate employability.

Since we are looking at a small fraction of the world's institutions in just one country the generalisability of this exercise is limited.

So what do we come up with? First, there are several highly selective liberal arts colleges in the US that are overlooked by international rankings. About half of the top 50 schools by SAT/ACT scores in the US do not show in the global rankings. An international undergraduate student wanting to study in the USA would do well to look beyond these rankings and think about places that are still highly selective such as Harvey Mudd, Pomona and Amherst Colleges.

Let's take a look at the four attributes. Age doesn't matter. There is no significance correlation between an institution's age and any of the reputation indicators. The lowest correlation, -.15, is with the RUR world research reputation indicator and the highest, but still not significant, .36, is with the THE teaching reputation indicator.

Research, however, is important. The correlation between total publications in the most recent Leiden Ranking varies from .48, RUR reputation outside region, to .63, THE teaching reputation.

So are standardised test scores. There is a significant correlation between SAT/ACT scores and the reputation indicators in the QS, RUR and Emerging/Trendence survey, ranging from .48 for the RUR world research reputation and reputation outside region to .72 for the Emerging/Trendence ranking. But the correlation with the THE teaching and research reputation indicators is not significant.

The RUR composite financial sustainability indicator correlates highly with the QS, RUR and Emerging/Trendence rankings, ranging from .47 for the QS employers' survey to .71 for the RUR world teaching reputation score but not for the THE indicators with which it is .15 for research and .16 for  teaching.

Endowment value appears to be the biggest influence on reputation. it correlates significantly with all reputation indicators, ranging from .42 for the RUR world research reputation indicator to .72 for Emerging/Trendence.

Of the four inputs the one that has the highest correlation with the three RUR reputation indicators, .71, .63, and .64 and the QS academic survey, .59, is financial sustainability.

Endowment value has the highest correlation with the QS employer survey, .57, and the two THE indicators, .66 and .71. Endowment and SAT are joint top for the Emerging/Trendence employability survey, .72. 

So it's seems that the best way to a good reputation, at least for selective American colleges, would be money. Test scores and research output can also help. But age doesn't matter.


Gavin Moodie said...

Thanx for this informative analysis, except that correlation does not imply causation.

Alex Usher said...

Re: Age. If you're looking internationally, no it doesn't matter because universities evolved in Europe before anywhere else. But run the same regression *within* each country and I suspect you will find a different answer. In most countries, the older the university is, the likelier it will be ranked more highly internationally.

(also, remember if you're restricting to top 500 you're looking at a biased may be that within top 500 age does not matter but it does matter in order to get into top 500 in first place.).

Anonymous said...

Faculty IQ, not student IQ.