The October 8th post has been republished in University World news
NORTH KOREA
Richard Holmes21 October 2016 Issue No:433
Discussion and analysis of international university rankings and topics related to the quality of higher education. Anyone wishing to contact Richard Holmes without worrying about ending up in comments can go to rjholmes2000@yahoo.com
Sunday, October 23, 2016
Monday, October 17, 2016
More lamentation from Dublin
Rankings have become a major weapon in the struggle of universities around the world to get their fair share or what they think is their fair share of public money. The Times Higher Education (THE) world and regional rankings are especially useful in this regard. They have a well known brand name, occasionally confused with the "Times of London", and sponsor prestigious summits at which rankers, political leaders and university heads wallow together in a warm bath of mutual flattery.
In addition, the THE rankings are highly volatile with significant methodological changes in 2011, 2015 and 2016. Another source of instability is the growing number of ranked universities. The scores used for calculating the various indicators in these rankings are not raw but standardised scores derived from means and standard deviations. So if there is an influx of new universities then mean scores are likely to change and consequently the processed scores of those above or below the mean.
The THE rankings can be interpreted to provide useful arguments whatever happens. If Western universities rise that is a sign of authentic excellence but one that is threatened by reduced funding, restrictions on foreign students and researchers, and reputations sullied by xenophobic electorates. If they fall that means of course that those threats have materialised.
The QS rankings are also sometimes unstable, having made significant methodological changes in 2015 and giving a 50% weighting to very subjective reputation indicators.
Irish universities seem to be especially fond of using these rankings as a ploy to gain public favour and largess. In 2015 Ireland's top university, Trinity College Dublin (TCD), fell seven places in the QS world rankings and 22 places in THE's.
TCD announced of course that government cuts had a lot do with it. The Dean of Research said:
University College Dublin has also continued to do well in the Shanghai rankings with the publications score rising this year from 34.1 (27.3 in 2004) to 34.2 and PCP from 18.0 (8.1 in 2014) to 18.1.
The Shanghai rankings are famous for not counting the arts and humanities or trying to measure anything related to teaching. The RUR rankings from Russia are based on Thomson Reuters data, also used by THE until two years ago and they do include publications in the humanities and teaching-related metrics. They have 12 out of the 13 indicators in the THE world rankings, plus eight others, but with a sensible weighting, for example 8% instead of 30% for field normalised citations.
The RUR rankings show that TCD rose from 174th overall in 2010 to 102nd in 2016. (193rd to 67th for research).
University College Dublin (UCD) rose from 213th overall to 195th (157th to 69th for research) although some Irish universities, NUI Galway, NUI Maynooth, University College Cork, and Dublin City University, have fallen.
Nonetheless TCD decided in March of this year to develop a rankings strategy aimed at QS and THE with a Rankings Steering Group chaired by the Provost. The competence and knowledge displayed by such groups and committees often have little relationship to the status and salaries of its members and that appears to be the case for TCD.
It seems that there was a misplaced decimal point in the financial data submitted to THE for the 2016 rankings and that would have left TCD with a lower rating than it deserved and so it has withdrawn from the rankings until the error is corrected.
If TCD cannot find an administrator or a statistician to check things like that it really has no business asking for taxpayers' money. I suspect that decimal points are not misplaced -- or if they are it is to the right rather than the left -- in submissions for grants or subsidies.
This raises the question of whether the THE checking procedures are adequate. I was under the impression that if there was a change of 20% then red flags would start waving. For THE to allow a large change in reported income and therefore at least one, maybe two or three, income indicators sounds rather odd. What about that unique game changing audit?
Meanwhile UCD, 176th in the THE rankings last year, has dropped out of the top 200 altogether.
The QS rankings were also bad news for Ireland. Every university fell except for NUI Galway and there were none in the top 100.
But has there in fact been any real decline in the quality of TCD and UCD?
The evidence of RUR and the Shanghai rankings is that the two main universities are steadily improving or at least holding their own, especially with regard to research. Possibly less highly regarded places like NUI Galway and NUI Maynooth are struggling but that could be fairly easy to remedy.
The Irish Universities Association issued a statement:
It is a shame that researchers and scholars should collude with those rankings that show them in a bad light while ignoring more stable and less biased ones that show a continuing and genuine improvement especially in research.
In addition, the THE rankings are highly volatile with significant methodological changes in 2011, 2015 and 2016. Another source of instability is the growing number of ranked universities. The scores used for calculating the various indicators in these rankings are not raw but standardised scores derived from means and standard deviations. So if there is an influx of new universities then mean scores are likely to change and consequently the processed scores of those above or below the mean.
The THE rankings can be interpreted to provide useful arguments whatever happens. If Western universities rise that is a sign of authentic excellence but one that is threatened by reduced funding, restrictions on foreign students and researchers, and reputations sullied by xenophobic electorates. If they fall that means of course that those threats have materialised.
The QS rankings are also sometimes unstable, having made significant methodological changes in 2015 and giving a 50% weighting to very subjective reputation indicators.
Irish universities seem to be especially fond of using these rankings as a ploy to gain public favour and largess. In 2015 Ireland's top university, Trinity College Dublin (TCD), fell seven places in the QS world rankings and 22 places in THE's.
TCD announced of course that government cuts had a lot do with it. The Dean of Research said:
“Notwithstanding these combined achievements the cuts in funding and increased investments made by our global competition, continue to have a direct impact on the rankings. Trinity is battling against intense international competition, particularly from Asian universities and from certain European countries where governments are investing heavily in higher education. The continued reduction in government investment in Irish universities has impacted negatively on the international standing of our universities and our ability to compete in a global arena.”“Trinity’s top 100 position globally and top 30 in Europe is remarkable in the context of its reduced income. Trinity’s annual budget per academic is 45% lower than that of the average university in the world top 200. It is to the credit of Trinity’s dedicated teaching and research staff that the University continues to maintain its global position against such challenges.”
“As a knowledge economy we need an excellent competitive education system. Trinity remains a world leading research-intensive university and the knowledge and innovation created are critical for the economic development of Ireland.”I pointed out in 2015 that TCD had been steadily rising in the Shanghai ARWU rankings since 2004, especially in the Publications indicator (papers in the Science Citation Index and the Social Science Citation Index) and PCP (productivity per capita, that is the combined indicator scores divided by the number of faculty). This year, TCD's publication score again went up very slightly from 31 to 31.1 and the PCP quite significantly from 19 to 20.8, compared to top scores of 100 for Harvard and Caltech respectively.
University College Dublin has also continued to do well in the Shanghai rankings with the publications score rising this year from 34.1 (27.3 in 2004) to 34.2 and PCP from 18.0 (8.1 in 2014) to 18.1.
The Shanghai rankings are famous for not counting the arts and humanities or trying to measure anything related to teaching. The RUR rankings from Russia are based on Thomson Reuters data, also used by THE until two years ago and they do include publications in the humanities and teaching-related metrics. They have 12 out of the 13 indicators in the THE world rankings, plus eight others, but with a sensible weighting, for example 8% instead of 30% for field normalised citations.
The RUR rankings show that TCD rose from 174th overall in 2010 to 102nd in 2016. (193rd to 67th for research).
University College Dublin (UCD) rose from 213th overall to 195th (157th to 69th for research) although some Irish universities, NUI Galway, NUI Maynooth, University College Cork, and Dublin City University, have fallen.
Nonetheless TCD decided in March of this year to develop a rankings strategy aimed at QS and THE with a Rankings Steering Group chaired by the Provost. The competence and knowledge displayed by such groups and committees often have little relationship to the status and salaries of its members and that appears to be the case for TCD.
It seems that there was a misplaced decimal point in the financial data submitted to THE for the 2016 rankings and that would have left TCD with a lower rating than it deserved and so it has withdrawn from the rankings until the error is corrected.
If TCD cannot find an administrator or a statistician to check things like that it really has no business asking for taxpayers' money. I suspect that decimal points are not misplaced -- or if they are it is to the right rather than the left -- in submissions for grants or subsidies.
This raises the question of whether the THE checking procedures are adequate. I was under the impression that if there was a change of 20% then red flags would start waving. For THE to allow a large change in reported income and therefore at least one, maybe two or three, income indicators sounds rather odd. What about that unique game changing audit?
Meanwhile UCD, 176th in the THE rankings last year, has dropped out of the top 200 altogether.
The QS rankings were also bad news for Ireland. Every university fell except for NUI Galway and there were none in the top 100.
But has there in fact been any real decline in the quality of TCD and UCD?
The evidence of RUR and the Shanghai rankings is that the two main universities are steadily improving or at least holding their own, especially with regard to research. Possibly less highly regarded places like NUI Galway and NUI Maynooth are struggling but that could be fairly easy to remedy.
The Irish Universities Association issued a statement:
'The continued slide of the Irish Universities in the QS World University Rankings should be greeted with alarm. Strenuous efforts on the part of the universities has resulted in strong performance on some measures in the rankings such as those relating to research citations and internationalisation of the staff and student cohort. Unfortunately, this good work is being undermined by the negative impact of underfunding on key indicators such as the student:faculty ratio. The latter is highly influential in scoring in the QS rankings.
It would also appear likely that almost a decade of austerity is spilling over into the reputational component of the rankings, with consequent negative repercussions. IUA Chief Executive, Ned Costello said: “we can no longer hide from the corrosive effect which years of cutbacks are having on our higher education system. At a time when we are more dependent than ever on the talent of our people for our economic future, we simply must invest in our universities. An immediate injection of funding is required in the upcoming Budget and Estimates to fund more lecturers, deliver smaller group teaching and restore quality in our system.” 'The decline of TCD and and UCD in the QS and THE rankings cannot reasonably be attributed to any real deficiencies on the part of those universities. A decline in the number of lecturers would have a negative effect on the faculty student metric but would help indicators scaled for faculty size. The alleged decline is largely a consequence of methodological changes and adjustments, the instability resulting from the influx of new universities and growing ranking sophistication in other places.
It is a shame that researchers and scholars should collude with those rankings that show them in a bad light while ignoring more stable and less biased ones that show a continuing and genuine improvement especially in research.
Saturday, October 08, 2016
Will North Korea Engage with the Rankings?
Kim Jong-un has declared that Kim Il-sung University must become a world-class institution. No doubt there will be chuckles at Oxford, Anglia Ruskin University, the University of Iceland and the Free University of Bozen - Bolzano but it could be surprisingly easy if being world class means getting a high place in the rankings. After all, there are now quite a few places appearing in the various global and regional tables that would have been just as surprising just a few years ago.
First, I should mention that there already is a ranking in which Kim Il-sung University is listed: a ranking of international influence as measured by Google's ranking of search results where the institution is 254th.
Here is my plan for North Korea to become world class in just a few years.
1. Offer adjunct professorships to 150 researchers and ask them to put the university as a secondary affiliation. Maybe they can come and visit Pyongyang sometimes but that is not really necessary. In a little while they will be producing 150 papers or more a year with the university name on, eventually one thousand over a five year period, which will meet the threshold for inclusion in the THE world rankings.
2. Make sure that one or two of those adjunct professors are involved in multi-author, multi-cited projects (but make sure below 1,000 authors) with multiple citations. Medicine is probably a better bet than physics at the moment. This will get a good score in the THE citations indicator.
3. Make sure that research funds to the university go through something with the word industry in it. That way the university will go to the top of the THE Industry Income: Innovation indicator.
4. Don't forget the other rankings. Give the university a boost in the QS world rankings by drafting lots of research assistants who will count in the the student faculty ratio indicator.
5. Start a branch campus somewhere and get a high score in the international indicators that nearly everybody has nowadays. If the branch is in the USA go for Princeton Review's top party school.
6. Send a few hundred closely supervised graduate students abroad and tell them they know what to do for the QS reputation survey. When they come back as faculty with a co-authored article or two tell them they know what to do for the THE survey.
7. When Kim Il-sung University is a rising star of the university world, try hosting a summit to rise even higher. Better make sure that hotel is finished though.
First, I should mention that there already is a ranking in which Kim Il-sung University is listed: a ranking of international influence as measured by Google's ranking of search results where the institution is 254th.
Here is my plan for North Korea to become world class in just a few years.
1. Offer adjunct professorships to 150 researchers and ask them to put the university as a secondary affiliation. Maybe they can come and visit Pyongyang sometimes but that is not really necessary. In a little while they will be producing 150 papers or more a year with the university name on, eventually one thousand over a five year period, which will meet the threshold for inclusion in the THE world rankings.
2. Make sure that one or two of those adjunct professors are involved in multi-author, multi-cited projects (but make sure below 1,000 authors) with multiple citations. Medicine is probably a better bet than physics at the moment. This will get a good score in the THE citations indicator.
3. Make sure that research funds to the university go through something with the word industry in it. That way the university will go to the top of the THE Industry Income: Innovation indicator.
4. Don't forget the other rankings. Give the university a boost in the QS world rankings by drafting lots of research assistants who will count in the the student faculty ratio indicator.
5. Start a branch campus somewhere and get a high score in the international indicators that nearly everybody has nowadays. If the branch is in the USA go for Princeton Review's top party school.
6. Send a few hundred closely supervised graduate students abroad and tell them they know what to do for the QS reputation survey. When they come back as faculty with a co-authored article or two tell them they know what to do for the THE survey.
7. When Kim Il-sung University is a rising star of the university world, try hosting a summit to rise even higher. Better make sure that hotel is finished though.
Tuesday, October 04, 2016
About those predictions
On September 16th I made some predictions about the latest Times Higher Education (THE) world rankings and summit at Berkeley. My record is not perfect but probably a bit better than the professional pollsters who predicted a hung parliament at the last UK elections, a crushing defeat for Brexit and humiliation for Donald Trump in the Republican primaries.
I predicted that Trump would not be invited to give a keynote speech. I was right but it was a pity. He would certainly have added a bit of diversity to a rather bland affair and he does seem to have a talent for helping unpromising beginners into successful careers, something that the current fad for value added ranking is supposed to measure.
I also said that UC Berkeley as the summit host would get into the top ten again after falling to thirteenth last year. This has now become a tradition at THE summits. I suspect though that even THE will find it hard to get King's College London, the 2017 world summit host, into the top ten. Maybe they will have to settle for top twenty.
The prediction that adding books to the indicator mix would help British universities seems to have been fulfilled. Oxford was number one for the first time. I was also right about the renewed rise of Asia, some of it anyway. The Korean favourites, Seoul National University, POSTECH, KAIST, Sungkyunkwan University, Korea University, have all risen significantly this year.
The decline of US public universities blamed on lack of funding? Yes, although I never thought Robert Reich would say that public higher education is dying.
Danger of Brexit and immigration controls for UK universities? I did not see anything specific but I did not look very hard and probably everybody thinks it's self evident.
I have to confess that I have not counted the number of times that the words prestige and prestigious were used at the summit or in the Christopher Priest novel. In the latter it is a contraction of prestidigitation and refers to the effect or the third segment of a stage illusion following the setup and the performance, the moment when the rabbit is pulled out of the hat or Anglia Ruskin revealed to have a greater world research impact than Cambridge or Imperial.
Phil Baty gave a masterclass and so did did Duncan Ross. I am pretty certain that no feminists complained about this outrageous sexism so I am prepared to admit that I was wrong there.
Incidentally, according to wikipedia a master class is "a class given to students of a particular discipline by an expert of that discipline -- usually music, but also painting, drama, any of the arts, or on any other occasion where skills are being developed."
I predicted that Trump would not be invited to give a keynote speech. I was right but it was a pity. He would certainly have added a bit of diversity to a rather bland affair and he does seem to have a talent for helping unpromising beginners into successful careers, something that the current fad for value added ranking is supposed to measure.
I also said that UC Berkeley as the summit host would get into the top ten again after falling to thirteenth last year. This has now become a tradition at THE summits. I suspect though that even THE will find it hard to get King's College London, the 2017 world summit host, into the top ten. Maybe they will have to settle for top twenty.
The prediction that adding books to the indicator mix would help British universities seems to have been fulfilled. Oxford was number one for the first time. I was also right about the renewed rise of Asia, some of it anyway. The Korean favourites, Seoul National University, POSTECH, KAIST, Sungkyunkwan University, Korea University, have all risen significantly this year.
The decline of US public universities blamed on lack of funding? Yes, although I never thought Robert Reich would say that public higher education is dying.
Danger of Brexit and immigration controls for UK universities? I did not see anything specific but I did not look very hard and probably everybody thinks it's self evident.
I have to confess that I have not counted the number of times that the words prestige and prestigious were used at the summit or in the Christopher Priest novel. In the latter it is a contraction of prestidigitation and refers to the effect or the third segment of a stage illusion following the setup and the performance, the moment when the rabbit is pulled out of the hat or Anglia Ruskin revealed to have a greater world research impact than Cambridge or Imperial.
Phil Baty gave a masterclass and so did did Duncan Ross. I am pretty certain that no feminists complained about this outrageous sexism so I am prepared to admit that I was wrong there.
Incidentally, according to wikipedia a master class is "a class given to students of a particular discipline by an expert of that discipline -- usually music, but also painting, drama, any of the arts, or on any other occasion where skills are being developed."
Saturday, October 01, 2016
Who says rankings are of no significance?
From Mansion Global
Stanford is, in case you haven't noticed, top of the Wall Street Journal/Times Higher Education US college ranking [subscription required for full results] and, more significantly, the world's 100 most innovative universities.
Six High-End Homes Near America’s Top-Ranked University
Who needs dorms at Stanford when you can live in one of these?
Stanford is, in case you haven't noticed, top of the Wall Street Journal/Times Higher Education US college ranking [subscription required for full results] and, more significantly, the world's 100 most innovative universities.