Can Grit Save Higher Education?

American, Australian and British universities are facing a serious input crisis. The number of students capable of anything resembling a conventional university education is drying up and the spectre of extinction is haunting British and American campuses. London Metropolitan University is closing two campuses and cutting 400 jobs.  Twelve of those jobs will be managerial ones so the situation must be really desperate. Hull University has closed a campus at Scarborough. The Open University is closing several regional centres with 500 plus jobs at risk.

Meanwhile small colleges in the US are shutting down: Dowling College, New York, Burlington College, Vermont, Tennessee Temple University and no doubt more to come.

Many English speaking universities  have tried to cover the deficit by aggressively recruiting international students. That is helping a little and science and technology departments in the Russell Group, the Group of Eight and the Ivy League are becoming increasingly dependent on graduate students and faculty of East Asian origin or descent. But the number of students who can perform adequately at degree level is not infinite. There are signs that the Flynn Effect has run its course even in China and there seems to be an increasing large amount of test and credential fraud, plagiarism and ghost-writing associated with the influx of international students.

The problem is compounded by the pressure to admit increasing numbers of historically underrepresented  groups who may come with substantial loans and grants but are often inadequately prepared for higher education. Such students frequently find that attending classes with classmates who perform much better is a deeply painful and humiliating experience, all the more so since they have from childhood been steeped in a warm bath of self esteem and excused almost any anti-social behaviour.

The admission of increasing numbers of unprepared students can also have serious consequences down the road towards and after graduation. More students with poor ACT or SAT scores or failing to graduate on time, if ever, means that ranking scores will suffer, with serious consequences for applications and admissions, and that employers and graduate schools will be less welcoming.

So we have increasingly desperate efforts to find something, anything, that will predict academic success but where less able students can do just as well or better. The problem is that so far nothing has been found that matches the predictive validity of standardised tests that are highly correlated with general intelligence.

There has even been a serious proposal by a group of elite admissions officers to reward applicants for doing ordinary things like baby sitting or punish them for a lack of authenticity in their resume compliant extra curricular activities. Taken seriously this would effectively randomise university admissions and turn US higher education into a flat swamp of mediocrity.

The latest in a succession of attempts to find the really effective non-cognitive factor that will transform American higher education and achieve the holy grail of true diversity is something called "grit", supposedly discovered by Angela Duckworth a psychologist at the University of Pennsylvania and a certified genius and author of an instant New York Times best seller. Grit is supposed to be a combination of passion and perseverance and is claimed to be important in determining job and academic success. It is allegedly a better predictor of success than IQ, health, good looks, or even social intelligence. It is, it would seem, "going to change the world".

Unfortunately, a thorough meta-analysis suggests that grit is almost the same thing as conscientiousness, a long established personality trait, and that its impact on academic success is modest and much less than that of cognitive ability.

And so the search for the Really Significant Non-Cognitive Factor continues.

David Cameron, Donald Trump, Leicester City, University Rankings and the end of deference

Recently there have several setbacks for the experts on both sides of the Atlantic. In May 2015 the pollsters, armed with all the techniques of scientific social science, got it very wrong about the UK general elections, drastically underestimating the Conservative margin of victory.

Now they seem to be doing even worse with Donald Trump. Pundits have queued  up to denounce him as a racist, misogynist, transphobic, xenophobic, anti-semitic, Ku Klux Klan appeasing liar. Successive ceilings above which he could not possibly rise have been declared only to evaporate and replaced by another. And yet he has won the Republican nomination.

Pundits, critics and mainstream journalists are now predicting that his campaign will implode. that he will never have enough money, needs the support of the party grandees, does not have enough support among women, Hispanics, gays or trans people, is a poor organiser, does not read from a teleprompter, has disgusting hair, talks in slow monosyllables, fails to grasp the nuances of various things and so on. Perhaps this time the army of the qualified and credentialed will be right and we will have another wonderful four or eight years of a Clinton presidency with Bill as the First Gentleman. Or perhaps not.

Then there is  Leicester City winning the premier league title, against 5000 to one odds.  At the beginning of the season was there anyone who predicted anything other than relegation?

The press has had a multitude of stories about the experts of various kinds who have been humbled by Leicester's rise. John Micklethwait of the Economist has recounted how every year but the last he bet on the team winning the premiership title (later on the story was betting that they would come top of their division). Had he not forgotten to do so so this year then he would have have won 100,000 pounds.

I too have a story about the perils of underestimating Leicester City. A few decades ago I was the owner of a complete set of LCFC autographs some of which I got from from my father who went around the town as an inspector for the Ministry of Pensions and National Insurance and occasionally got souvenirs from the organisations whose books he helped tidy up.

I got Gordon Banks's by queueing up at Lewis's in Humberston Gate and Jimmy Walsh's, then Leicester's captain just arrived from Glasgow Celtic, because he lived down the road in a semi detached now worth 130,000 pounds (Jamie Vardy's car is more than that) although I had to suffer the public embarrassment of being called a wee laddie by Mrs Walsh.

But at some point in the early or late seventies after Leicester left Division One, I gave the autographs away to somebody I can't remember. Today four signatures from that era plus Norman Wisdom are worth 275 pounds on ebay. That was an error even worse than selling an authentic vinyl copy of the Dylan Royal Albert Hall bootleg, now worth thirty, for ten dollars.

And of course we have the Brexit vote. It is unlikely that that there has ever been such unanimity about any matter of public concern from the various components of the dominant elite. Nearly every vice chancellor, all the managers of the premier league plus an imposing array of pop stars, rock stars and film stars have admonished the great unread (some of whom probably don't have passports!) that no decent person could possibly even dream of voting Leave.

The universities further elaborated on the perils of Brexit. Think about all the money that we get from the EU for research. The merit of this argument may have been blunted by the revelation that the field that got the most from the EU was Education.

The polls conducted with the latest markers of rigour such as sample sizes and margins of error, appeared to confirm that the British electorate was fully aware of the wisdom of their intellectual betters.

But clearly the academic elite had absolutely no idea about what was going on in the minds of over half of the population just as they had no idea of what was going on in the minds of Republican voters.

One wonders whether the economic catastrophe supposed to follow Brexit will actually be so catastrophic. A fall in the value of the pound or the FTSE index is not really a problem if you don't have any shares or pounds to start with.

It is possible that even the university rankers may also be suffering a loss of credibility. Last year the "revered" QS reported that the National University of Singapore and Nanyang Technological University were overtaking the Ivy League, a claim that met with some scepticism even, or especially, in Singapore, and the universally "trusted" and prestige dispensing THE rankers do not seem to have received very much support for their pilot projects in Africa and the Middle East.

Perhaps the age of deference to expertise is coming to an end.

Article in the Japan Times and an Unsuccessful Comment

Here is an article published in the Japan Times. followed by a comment which I attempted to post.

Todai tumbles from top of Asia university rankings to seventh place

LONDON – The University of Tokyo, locally known as Todai, lost its crown in this year’s Times Higher Education Asia University rankings released Monday.

After occupying the number one spot for the past three years, the University of Tokyo came seventh in a list of Asia’s top 200 universities. Times Higher Education described this year’s results as “challenging” for Japan and blamed a lack of funding and poor international outlook for the country’s position.

Singapore achieved unprecedented success in this year’s rankings by taking the top two places, with the National University of Singapore at the top and Nanyang Technological University joint ranked second with Peking University, the highest-ranked Chinese institution.

A total of 39 Japanese universities made it into the top 200, with 14 listed in the top 100.

Kyoto University (11th), Tohoku University (23rd), Tokyo Institute of Technology (24th) and Osaka University (30th) all made the top 30, with four more ranked in the top 50.

Phil Baty, a Times Higher Education rankings editor, said, “Japan claims almost a fifth — 39 — of Asia’s top 200 universities in this year’s table, making it the most-represented nation, in joint place with China.

“However, while the list proves Japan has strength in depth, the majority of its universities appear in the bottom half of the table; just 14 Japanese institutions make the top 100, compared with 22 in China.

“Furthermore, Japan’s top-ranked institution — the University of Tokyo in seventh place — has been knocked off the number one position after three years at the helm. It is Japan’s only top 10 representative. Meanwhile, China, Singapore and Hong Kong have two, while South Korea has three.”

Baty notes that the past few years have seen a shift in the balance of power from West to East in terms of higher education funding and performance.

Comment

Todai fell from 1st place in last year's THE Asian rankings to 7th this year, while Nanyang Technological University rose from 10th to 2nd. Changes in International outlook and funding levels could not have had such a large effect in the space of just 12 months.

It should be noted that this year THE did a recalibration of the weighting of its indicators. That for research and teaching reputation, in which Todai does much better than Singapore and Hong Kong universities, was reduced from 33% to 25%.The weighting for industry income, in which Todai has an average score and Nanyang Technological University an almost perfect one, was increased from 2.5% to 7.5%.

In addition, THE has changed the process of collecting and analysing citations data, including not counting large-scale multi-author projects, in a way that has worked to the detriment of the University of Tokyo and to the advantage of the Singaporean universities.

The recommendations of THE should be taken with a big bucket of salt.

See University Ranking Watch

THE's bespoke Asian rankings: the strange decline of the University of Tokyo and the rise of Singapore

Times Higher Education (THE), in conjunction with their prestigious summit in Hong Kong, have revealed this year's Asian University Rankings which use essentially the same methodology as the world rankings but with some recalibration.

The most noticeable aspect of the new rankings is that the University of Tokyo (UT), which was first in 2013, 2014 and 2015, has now suddenly dropped to seventh place, behind the National University of Singapore (NUS) in first place, Nanyang Technological University (NTU) in Singapore up from tenth to second, Peking University, the University of Hong Kong, Tsinghua University and Hong Kong University of Science and Technology.

Tokyo is not the only Japanese university to suffer in these rankings. Tokyo Institute of Technology has gone from 15th last year to 24th, Osaka University from 18th to 30th and Tokyo Metropolitan University from 33rd to 52nd. 

The rise of NTU and the fall of Tokyo need some explanation. When we are talking about institutions with thousands of students and faculty that produce thousands of papers, citations and patents, it is not good enough to say that one has been investing and networking and the other has not. The time from the publication of budgets via research proposals to publication and citation is usually closer to a decade than to a year.

Let's take a look at the details. Between 2015 and this year UT suffered a modest fall for teaching (a cluster of five indicators) international outlook and industry income, a substantial fall of 5.6 points for research (a cluster of three indicators) and a large fall from 76.1 to 67.8 points for field-normalised citations.

Evidently the methodological changes introduced last year by THE and Elsevier, their new data partners, have had an effect on the citations indicator score of UT. The changes were excluding papers, mostly in physics, with a large number of authors, switching from the Web of Science to Scopus as a source of data about papers and citations and reducing the impact of the "regional modification" that awards a bonus to universities in countries with a low citation impact.

Meanwhile NUS rose 4.7 points for citations and NTU 9.7 points. It would seem then that these changes contributed significantly to Tokyo's decline and to the ascent of NUS and even more so that of NTU.

There is another factor at work. THE have told told us that they did some recalibration, that is changing the weighting of the indicators. They reduced the weighting of the teaching reputation survey from 15% to 10% and that of the research reputation survey from 18% to 15%. The weighting for research productivity and research income was increased from 6% to 7.5% each  and for income from industry from 2.5% to 7.5%.

So why did THE do this?  It seems that it was done after consulting with Asian universities because "many Asian institutions have only relatively recently arrived on the world stage, with investment focused on recent decades, so have had less time to accumulate reputation around the world."

But one could say something similar about all the indicators: Asian universities have only recently arrived on the world stage and so have had less time to accumulate research funds or research expertise, build up their faculty, develop international networks and so on.

And why give the large extra weighting to industry income because "many Asian nations have put their universities at the forefront of economic growth plans, where industry links are crucial?" Perhaps some countries have plans where industry links are not crucial or perhaps other criteria are equally or more crucial. In any case, industry income is  a very questionable indicator. Alex Usher of Higher Education Strategy Associates has already pointed out some of its flaws.

Anyway, whatever THE 's ostensible reasons for this recalibration, the consequences are quite clear. Taking points from the reputation survey has worked to the disadvantage of UT, which in THE's 2015 reputation ranking had scores of 18.0 for teaching reputation and 19.8 for research reputation, and in favor of NUS which had scores of 9.2 and 10.9. The scores for NTU, the University of Hong Kong and Hong Kong University of Science and Technology are much lower and are withheld. It is not clear what the exact effect is since this year the reputation scores are subject to an "exponential component" which has presumably reduced the spread of scores and therefore UT's advantage.

It is not possible to determine the effect of giving extra weighting to research productivity and research income since these are bundled with other indicators.

Giving a greater weight to industry income has hurt UT, which has a score of only 50.8, and helped NTU with a score of 99.9, the University of Hong Kong with a perfect score of 100 and Kong Kong University of Science and Technology with a score of 68.1.

It appears that Japanese universities do relatively badly in these rankings and those in Singapore and Hong Kong do so well largely because of the changes last year in the collection and processing of citations data and the recalibration this year of the indicator weightings.

The co-host of the Asian summit was Hong Kong University of Science and Technology and the list of "prestigious university leaders from around the world"  includes those from Hong Kong, Singapore and China but  not from Japan.

Worth reading 6: The Berlin principles

Just heard about this from Gary Barron.

Barron, Gary R.S. 2016. "The Berlin Principles on Ranking Higher Education Institutions: limitations, legitimacy, and value conflict." Higher Education, Online First, pp.1-17.

Abstract

University rankings have been widely criticized and examined in terms of the environment they create for universities. In this paper I reverse the question by examining how ranking organizations have responded to criticisms. I contrast ranking values and evaluation with those practiced by academic communities. I argue that the business of ranking higher education institutions is not one that lends itself to isomorphism with scholarly values and evaluation and that this dissonance creates reputational risk for ranking organizations. I argue that such risk caused global ranking organizations to create the Berlin Principles on Ranking Higher Education Institutions, which I also demonstrate are decoupled from actual ranking practices. I argue that the Berlin Principles can be best regarded as a legitimizing practice to institutionalize rankings and symbolically align them with academic values and systems of evaluation in the face of criticism. Finally, I argue that despite dissonance between ranking and academic evaluation there is still enough similarity that choosing to adopt rankings as a strategy to distinguish one's institution can be regarded as a legitimate option for universities.

Dot Connection Time

Singapore-based World Scientific Publishing, whose subscription lists were used to collect names for the QS academic opinion survey, are advertising a new book, Top the IELTS: Opening the Gates to Top QS-Ranked Universities.  by Kaiwen Leong of Nanyang Technological University and Elaine Leong.

Nanyang Technological University is ranked 13th in the QS world rankings, ahead of Yale, Johns Hopkins and King's College London, and third in the Asian rankings.

World Scientific owns Imperial College Press.

Imperial College is eighth in the QS world rankings, ahead of Chicago and Princeton.

Dumbing Down at Oxbridge

The relentless levelling of British universities continues. The latest sign is a report from Oxford where the university is getting ready to crack down on colleges that make their students work too hard. Some of them apparently have to write as many as three essays a week and most work at least 40 hours a week, some longer, which is apparently twice as much as places like Northumbria University.

Many commentators have mocked the poor fragile students who cannot cope with with a fifty hour week. After all, that is nothing to what they can expect if they start legal, medical or research careers.

Something else that is a bit disturbing is that Oxford students apparently need so much time to do that amount of work. One would expect the admissions system at Oxford to select academically capable students who can do as little work as those at Northumbria and still perform much better. If Oxford students can only stay ahead by working so hard doesn't this mean that Oxford is failing to find the most intelligent students and has to make do with diligent mediocrities instead?

The villain of the piece is probably the abolition of the essay based Oxford entrance exam in 1995 (Cambridge abolished theirs in 1986) which threw the burden of selection onto A level grades and interviews. The subsequent wholesale inflation of A level grades has meant that an undue importance is now given to interviews which have been shown repeatedly to be of limited value as a selection tool, particularly at places like Oxbridge where the interviewers have sometimes been biased and eccentric.

So Oxford and  Cambridge are now planning to reintroduce written admission tests. They had better do it quickly if they want their graduates to compete with the Gaokao-hardened students from the East.

THE is coming to America

Times Higher Education (THE) has just announced that American university rankings are not fit for purpose.

We have heard that before. In 2009 THE said the same thing about the world rankings that they had published in partnership with the consulting firm Quacquarelli Symonds (QS) since 2004.

The subsequent history of THE's international rankings provides little evidence that the magazine is qualified to make such a claim.

The announcement of 2009 was followed by months of consultation with all sorts of experts and organisations. In the end the world rankings of 2010, powered by data from Thomson Reuters (TR), were not quite what anyone had expected. There was an increased dependence on self-submitted data, a reduced but still large emphasis on subjective surveys, and four different measures of income, reduced to three in 2011. Altogether there were 14 indicators, reduced to 13 in 2011, all but two of which were bundled into three super-indicators, making it difficult for anyone to figure exactly why any institution was falling or rising.

There were also some extraordinary elements in the 2010 rankings the most obvious of which was  placing Alexandria University in 4th place in the world for  research impact
.

The rankings received a chorus of criticism mixed with some faint praise for trying hard. Philip Altbach of Boston College summed up the whole affair pretty well.

“Some of the rankings are clearly inaccurate. Why do Bilkent University in Turkey and the Hong Kong Baptist University rank ahead of Michigan State University, the University of Stockholm, or Leiden University in Holland? Why is Alexandria University ranked at all in the top 200? These anomalies, and others, simply do not pass the smell test."  

THE and TR returned to the drawing board. They did some tweaking here and there and in 2011 got Alexandria University out of the top 200 although more oddities would follow over the next few years, usually associated with the citations indicator. Tokyo Metropolitan University, Cadi Ayyad University of Marrakech, Santa Maria Federico Technical University, Middle East Technical University and the University of the Andes were at one point or another declared world class for research impact across the full range of the disciplines.

Eventually the anomalies got too much and after breaking with TR in 2015 THE decided to have a bit of a spring cleaning and tidied things up a bit.

For many universities and countries  the results of the 2015 methodological changes were catastrophic. There was a massive churning with universities going up and down the tables. Universite Paris-Sud, the Korean Advanced Institute of Science and Technology, Bogazici University and The Middle East Technical university fell scores of places.

THE claimed that this was an improvement. If it was then the previous editions must have been hopelessly inadequate. But if the previous rankings were the gold standard of rankings then those methodological changes were surely nothing but gratuitous vandalism.

THE has also ventured into far away regions with snapshot or pilot rankings. The Middle East was treated to a ranking with a single indicator that put Texas A and M University Qatar, a branch campus housing a single  faculty, in first place. For Africa there was a ranking consisting of data extracted from the world rankings without any modification of the indicators, which did not seem to impress anyone.

So one wonders where THE got the chutzpah to tell the Americans that their rankings are not fit for purpose. After all, US News was doing rankings for two decades before THE and their America's Best Colleges include metrics about retention and reputation as well as resources and selectivity. Also, there are now several rankings that already deal directly with  the concerns raised by THE.

The Forbes/CCAP rankings include measures of student satisfaction , degree of student indebtedness, graduation on time, and career success.

The Brookings Institution has a value added ranking that includes data from the college scorecard

The Economist has produced a very interesting ranking that compares expected and actual value added.

So exactly what is THE proposing to do? 

It seems that there will be a student engagement survey which apparently will be launched this week and will cover 1,000 institutions. They will also use data on cost, graduation rates and salaries from the Integrated Postsecondary Data System (IPEDS) and the College Scorecard. Presumably they are looking for some way of monetising all of this so probably large chunks of the data will only be revealed as part of benchmarking or consultancy packages.

I suspect that  the new rankings will like something like the Guardian university league tables just published in the UK but much bigger.

The Guardian rankings include measures of student satisfaction, selectivity, spending, staff student ratio and value added. The latter compares entry qualifications with the number of students getting good degrees (a first or upper second).

It seems that THE are planning something different from the research centred industry orientated university rankings that they have been doing so far and are venturing out into new territory, institutions that are two or three tiers below the elite and do little or no research.

There could be a market for this kind of ranking but is very far from certain that THE are capable of doing it and whether it is financially feasible.