What's the Problem with U-Multirank and AHELO?

In a recent post, I discussed the contrast between the poor skills of young people in the UK (strictly speaking Northern Ireland and England) and the high regard in which  British universities are held by the brand name rankers.

There is a piece of data in the skills report from the OECD that is interesting in this respect. Figure 2.2 shows the average numeracy skills of new graduates (age 16-29, 2012). It is depressing reading. The data for tertiary graduates shows that only Italy does worse than the UK and Ireland is either the same or almost the same.The US is very slightly ahead. The top scorers are Austria, Flanders and the Czech Republic.

Something that should have everybody running around doing research and forming  committees is that British tertiary graduates are only very slightly better than most European secondary graduates and slightly better than South Koreans with less than an upper secondary qualification.

It is possible, indeed quite probable, that British tertiary graduates do better on verbal skills and likely that they could conduct themselves well in interviews. Perhaps also, it is places like the University of East London and Bolton University that are dragging down the British average. But this dramatically poor performance is such a glaring contrast to the preening self satisfaction of the higher education establishment that some discussion at least is called for.

We may be seeing an explanation for the reluctance of the Russell Group and its orbiters and the Ivy League to cooperate with U-Multirank and their disdain for the AHELO project that is in marked contrast with their support for the trusted and prestigious THE rankings. They are quite happy to be assessed on reputation, resources, income and citations but comparison with the cognitive skills of graduates from the upstarts of East Asia and perhaps Eastern and Central Europe  is something to be avoided.

Why Russia Might Rise Fairly Quickly in the Rankings After Falling a Bit

An article  by Alex Usher in Higher Education in Russia and Beyond, reprinted in University World News, suggests five structural reasons why Russian universities will not rise very quickly in the global rankings. These are:

• the concentration of resources in academies rather than universities

• excessive specialisation among existing universities

• a shortage of researchers  caused by the economic crisis of the nineties

• excessive bureaucratic control over research projects

• limited fluency in English.

Over the next couple of years things might even get a bit worse. QS are considering introducing a sensible form of field normalisation, just for the five main subject groups. This might not happen since they are well aware of the further advantages this will give to English speaking universities, especially Oxbridge and places like Yale and Princeton, that are strong in the humanities and social sciences. But if it did it would not be good for Russian universities. Meanwhile, THE has spoken about doing something about hugely cited multi-authored physics papers and that could drastically affect institutions like MEPhI.

But after that, there are special features in the QS and THE world rankings that could be exploited by Russian universities. 

Russia is surrounded by former Soviet countries where Russian is widely used and which could provide large numbers of international research collaborators, an indicator in the THE rankings, and could be a source of international students and faculty, indicators in the THE and QS rankings and a source of respondents to the THE and QS academic surveys.

Russia might also consider tapping the Chinese supply of bright students for STEM subjects. It is likely that the red bourgeoisie will start wondering about the wisdom of sending their heirs to universities that give academic credit for things like walking around with a mattress or  not shaving armpit hair and think about a degree in engineering from Moscow State or MEPhI.

Russian universities also appear to have a strong bias towards applied sciences and vocational training that should, if marketed properly, produce high scores in the QS employer survey and the THE Industry Income: Innovation indicator.

Italy and France Accept Gaokao Scores

What will happen when universities find gaokao is a better predictor of academic ability than A levels or SAT?

This is from YIBADA. 

"Up to 1,000 universities in France, Italy, and other 14 popular overseas destinations for Chinese applicants are now accepting national college entrance test scores or "gaokao" scores as admission criteria, according to a report published on Monday by MyOffer, a London-based online student placement portal.
The findings reflect the growing international recognition for China's national college entrance tests despite lagging behind other exams.
MyOffer, which helps international students with university placements, overseas internships and career development, released the study as this year's "gaokao" scores were announced in several parts of China.
Earlier reports claimed that "gaokao" test results were accepted in 20 countries and regions, but MyOffer's study has by far the most detailed findings available."

Today, Cuba and China, Tomorrow North Korea?

Another sign of the growing desperation of American colleges to find international students to take the courses American students just won't take is the four Cuban students who will take the TOEFL in Havana a week from now. There are plans for the GRE to be offered in Cuba in October.

The Implications of the University of San Francisco Accepting Gaokao scores.

The University of San Francisco has announced that it will admit a limited number of students on the basis of their scores on the Gaokao, the rigorous Chinese national university entrance exam, plus an interview and English language test in Beijing. The candidates will be spared the necessity of  taking TOEFL prep courses and flying to Hong Kong or Singapore for the SAT test.
 
American and British universities are running out of students capable of taking tertiary education courses. Average cognitive skills of local students are stagnant or declining, which explains the obsession of universities with finding students from overseas to bring in revenue and balance the books. China appears to have a large number of students capable of high achievement in numeracy-based fields.
 
What would happen if American universities found that Gaokao scores were more predictive of academic success than a dumbed down SAT? What if the English language component turned out to be just as good a measure of language proficiency as IELTS or TOEFL? The consequence might be that the Gaokao could become the normal route for admission to universities outside China.

And looking ahead several decades, what would happen if the Gaokao was offered in languages other than Chinese with test centres being set up outside China?

Which is the Real Fraud?

The Australian via Inside higher Ed has an article by Kylar Loussikian about a shadowy organisation apparently based in Colchester, England, that supplies ghostwritten academic essays. Australian universities, and maybe others, are getting very concerned about the racket. 

'The most common issue, ghostwritten essays, represents a “wicked problem,” said John Shields, deputy dean of the University of Sydney’s business school. “It’s deep and embedded and it’s hard to catch and kill,” he said. “In one sense, ghostwriting has emerged as an area of key concern in academic honesty because many universities are using a first-line defense in terms of [text matching software], and the simple plagiarism approach being detectable has forced those who, for whatever reason, choose to engage in dishonest conduct, to go one level deeper.” '

No doubt there will be a lot of finger pointing and tongue wagging. But are companies like these the real frauds? When millions of students are unable to do the work in courses for which they have been selected shouldn't we conclude that the entire admission process is flawed?

Why are there people capable of turning out essays and papers at a few hours or days notice not employed in universities? Doesn't this suggest that that there is a problem with the recruitment process?

Meanwhile the ghost writing virus seems to be spreading to graduate and faculty research. In the last few weeks I have received messages from Gulf Dissertation Online, which has "expertly helped and consulted PhD Professors, Lecturers and Scholars with their Thesis, Dissertations and Research Papers for over 12 Years" and Publish Pedia, which "is now offering a unique opportunity to Scholars and Professors who are pursuing their first publication ISI indexed journal or due to insufficient time not able to follow up on their new papers for publication to high impact factor top tier journals keeping the mandatory guidelines for ISI journal approved by the University"

The British Paradox Again

We have been told many times before that British universities are punching above their weight and are outperforming their international counterparts. Year after year they do extremely well in the QS and THE world rankings although perhaps not as well in the Shanghai ARWU.

This excellent performance is in glaring contrast to the well documented decline in the cognitive skills of young people in the United Kingdom. A recent publication from the OECD on youth, skills and employability  show that the proportion of 16-39 olds in the UK (actually England and Northern Ireland) with low literacy skills was well above the OECD average and slightly above the United States. Only Spain and Italy did worse. Not unexpectedly, the top performers here were Japan, Korea and Finland.

What is even more frightening is that the UK is very distinctive in that the proportion of 16-29 olds with poor literacy skills is lower than that of 30-54 years. In every other country except Japan, where literacy is very high among both groups,  and Norway, literacy has risen among the younger generation.

For numeracy skills of 16-29 year olds, the UK is again well below the OECD average. The share of young people with limited numeracy is higher than any other country except Italy and the US. Again there is a decline from the 30-54 year olds.

The OECD has also published data on problem solving abilities in technology-rich environments. This time the UK, like every country assessed, has improved over time but still is behind everyone else except  the US, Ireland and Poland.

So how can British students be so bad at literacy, numeracy and problem solving when the universities are, according to international rankers, so brilliant?

Some suggestions.

Perhaps, the rankings are biased towards British universities.

Perhaps, British higher education is highly differentiated with a few outstanding institutions that get high scores in the global league tables and a mass of others that cannot even squeeze into the 400s or 500s or do not even try.

Perhaps, it is just a question of time and in the next few years British universities will collapse under the weight of thousands of students with low cognitive skills who must be admitted to keep revenues flowing.

Why is Bogazici University considered so great in Turkey although it actually is at 400th position in the QS world rankings?

Another question from Quora.

The answer is that the QS rankings favour universities with an established reputation in those countries that are interested in rankings, those that have extensive international linkages, those with a lot of faculty and those with strengths in medical research.

In contrast, the Times Higher Education  (THE) rankings favour those powered by hadron driven citations and with the good fortune to be located in countries where most universities produce few citations.

What will happen if THE does reform its citations indicator?

Is The QS Computer Scence Ranking Accurate?

Ben Zhao, Professor at UC Santa Barbara, doesn't think so.

"There's a bunch of rankings, US News, Shanghai, US National Research Council, QS.  Of all of these, I would probably say that QS is one of the least useful. Why do I say that? I get SPAMMED on multiple email addresses to respond to a survey on QS university rankings.  I don't respond, and they just send more mail.  This is NOT the behavior of a reputable organization trying to gather a legitimate view of universities and their research quality.   ... "

I wouldn't disagree with him about the QS subject rankings, which outside the ranks of the world elite are based on very small samples of employers and academics and small numbers of citations. But it might be unfair to complain about being spammed all the time. This is probably happening because many universities are submitting his name to QS for the academic opinion survey.

As Oscar Wilde probably would have said the only thing worse than being spammed is not being spammed.

What do Indian Scientists do on Their Holidays?

The Indian Express has an interesting interview with the Vice-Chancellor of Panjab University, which Times Higher Education (THE), but nobody else, considers to be the best or second best university in India,  a feat achieved by an outstanding score for citations.

Here is an extract:

"Did the four-year period, 2010-2014, counted for the Times ranking include old research papers as well?Yes. It is not about papers that came out in this period but also the papers in which PU figures and which have a high citation. It is a mix of so many things. God particle came up in 2012. So, all those papers are being cited multiple times. Every theorist is cited. So, PU was already doing well, and discovery of God particle made it even better. When there was a lull and Fermilab was closed down for a while, and they were re-building CERN, PU and TIFR went on and joined the groups in B-factory in Japan.
The thing is that you have a job in the university, you have a job for life, you can decide to sleep, still you will get the salary. These professors at PU, or those at IIT-Guwahati, TIFR people, they are conscious that their productivity should not suffer. They should continuously be valued as a member of these collaborations. So, they keep working. So, when there is a holiday, when [other] people spend time here and there,what do High energy physicists do? Class khatam hoti hai. The next day they take a flight, and go to CERN or Chicago, and there they work hard. You are actually trying to make up for the time you could not do anything because you were doing teaching. That is how international faculty values them also, and they are continuously being included."

So, the Vice-Chancellor is aware that it is the CERN project that is cause of PU's ranking success. It will be interesting to see what happens if THE does bite the unpleasant tasting bullet and introduce fractionated counting of citations.

But if PU and other Indian institutions continue to improve, even if there is a (temporary?) dip in the THE rankings, then the key to that success may be here. Indian scientists can draw  a salary while sleeping if they want but they can also go to Switzerland and discover the fundamental particles of the universe if so inclined. Increasingly, western scientists are apparently expected to spend their days and nights filling out forms, applying for grants, writing teaching philosophies, attending sexual harassment seminars, making safe spaces all over the place, undergoing diversity sensitivity training and so on and so on.