Wednesday, April 25, 2012

LSE overtakes Oxford in British Ranking

The Complete University Guide has just published its rankings of British universities. According to Brendan O'Malley in University World News.




It is the first time since 2000 that Oxford and Cambridge have not shared the top two spots – in that year Imperial College London knocked Oxford into third place.


In separate listings for the leading universities and higher education institutions covering 62 subjects, Cambridge is in the top 10 for all 46 subjects it offers, and top in 30. Oxford is in the top 10 for all 32 of its subjects, and is placed first in 12. The LSE is in the top 10 for all 12 subjects offered, and top for three.


There are two new entrants to the Top 20 – the University of Glasgow (17th) and Leicester (19th). They have replaced Sussex, which just missed out in 21st place, and the School of Oriental and Asian Studies, which fell from 15th to 30th position.



O'Malley continues:


The interactive guide ranks universities on nine factors: student satisfaction; research assessment; entry standards; student-to-staff ratio; spending on academic services; spending on student facilities; good honours degrees achieved; graduate prospects; and completion.

The subject tables are based on four factors: student satisfaction; research assessment; entry standards; and graduate prospects.


When LSE offers 12 subjects and still beats Oxford with 32 subjects, these results need to be approached with caution. Bernard Kingston is honest enough to issue a health warning.

The good degrees and completion rates indicator are obvious incentives to game the system.
London Met Update

It seems that the proposal to ban alchohol from one campus was not quite what was reported. Apparently the vice-chancellor was looking for an excuse to save money. The latest report is that Muslim students are disassociating themselves from his plan.

Tuesday, April 24, 2012

Another Quality Initiative

Every so often education bureaucrats come up with bright ideas to boost quality and raise standards and proclaim their faith that students and teachers at all levels can perform as well as anybody anywhere.

A recent example is the decree by the the Director General of Higher Education at the Indonesian Education and Culture Ministry, that in order to graduate undergraduate students must publish a paper in a scientific journal, master's students in a national scientific journal and doctoral students in an international scientific journal.

It is easy to predict what will happen if this idea survives. Every university will set up a website and call it an  online journal to which  an essay, term paper or thought paper by every undergraduate or group of undergraduates will be uploaded, after running it through turnitin.

It would be almost as easy to set up national websites and call them national online scientific journals and publish term papers by masters students.

But getting papers into international journals will be another matter if the term international has any meaning and if the papers are to be single-author papers. We can expect a lot of creative workarounds here.
QS Paper 

Here are the links to a paper and presentation slides by Ben Sowter of QS that discusses the spin-offs: subject, regional and best student city rankings.
What about the Kindergarten Rankings?

US News will be publishing the Best High Schools rankings on May 1st.

Sunday, April 22, 2012

Does Not Compute

London Metropolitan University seems to be getting rather desperate in its attempts to attract students. On the one hand:


"A London university is considering establishing alcohol-free zones on its campuses because so many of its students consider drinking to be immoral.
Professor Malcolm Gillies, vice-chancellor of London Metropolitan University, said the selling of alcohol was an issue of "cultural sensitivity" at his institution where a fifth of students are Muslim.
Speaking to a conference of university administrators in Manchester, he said that for many students, drinking alcohol was "an immoral experience".
"Because there is no majority ethnic group [at London Metropolitan], I think [selling alcohol] is playing to particular parts of our society much more [than to others]," he was reported as saying in the Times Higher Education magazine."
On the other:

"As part of a master's course in events experience management, London Metropolitan University will offer a module in partnership with Chillisauce, known for organising custom stag dos across Europe.
The firm's website lists options including mud-wrestling with scantily clad women in Budapest, a "spa with strippers" in Riga, or the option to be "punished" at a Tallinn "lap dancing dreamland".
The link has drawn criticism from unions, although students will be involved only in the company's more straightforward commercial activities - including corporate dinners and conferences.
Participants on the course will be asked to create a Guinness World Record attempt that doubles as a PR event for a consumer brand. They will devise a "creative concept" and pitch it to Chillisauce executives, who will attend seminars and lectures during the module.
But a University and College Union spokesman questioned "how employing a company that specialises in stag weekends offering wrestling with scantily clad women in jelly is likely to do much for a university's reputation"."
Opening the Black Box

I have just seen an article by Mu-Hsuang Huan of National Taiwan University published by Research Evaluation, an Oxford University Press journal. Here is the abstract:

"In the era of globalization, the trend of university rankings gradually shifts from country-wide analyses to world-wide analyses. Relatively high analytical weightings on reputational surveys have led Quacquarelli Symonds (QS) World University Rankings to criticisms over the years. This study provides a comprehensive discussion of the indicators and weightings adopted in the QS survey. The article discusses several debates stirred in the academia on QS. Debates on this ranking system are presented in the study. Firstly, problems of return rate, as well as unequal distribution of returned questionnaires, have incurred regional bias. Secondly, some universities are listed in both domestic and international reputation questionnaires, but some others are listed only in the domestic part. Some universities were evaluated only by domestic respondents, limiting their performance of the ranking results. Thirdly, quite a few universities exhibit the same indicator scores or even full scores, rendering the assessment questionable. Lastly, enormous changes of single indicator scores suggest that the statistic data adopted by QS Rankings should be further questioned."

The article is useful and interesting, especially the table of rates of return for different countries. But it does not seem to go beyond what this blog and others have been saying for a long time.

This seems to be another case of mainstream academia lagging behind the mass media which in turn  is way behind the blogosphere.

Wednesday, April 18, 2012

The Euro-Rankings

The U-Multirank project has slowly crept past the feasibility study stage. While any attempt to undermine the triopoly of the big three global rankers, ARWU (Shanghai), QS, Times Higher Education, is welcome, it does seem to be taking a long time. One wonders whether it will get going before the Eurozone disintegrates. Still the project does have its defenders.


"Jordi Curell, director of lifelong learning, higher education and international affairs at the directorate general for education and culture, conceded that there was opposition to its development.

"When we started working on the project of U-Multirank, many people from the higher education community were opposed to it,” he told an international symposium on university rankings and quality assurance in Brussels on 12 April.

But the system had intrinsic value, he said, because it would provide an evidence-based measure of the performance of European universities that would help them improve.

According to Curell, if higher education is to help Europe emerge from its current financial and economic crisis, the EU needs to know how its universities are performing and universities need to know how they are doing.
"Rankings which are carefully thought out are the only transparency tools which can give a comparative picture of higher education institutions at a national, European and global level," he told the symposium."



There are critics of course. One of them is a committee of the British House of Lords which has argued that U-Multirank is a waste of money. Four million Euros sounds rather a lot but considering what the EU has spent money on, it is trivial.

And as for wasting the taxpayer's money, well, the committee could look at the other house and think about a  floating duck island, pantyliners, nappies, soundproofed bedrooms and so on and so on.

Tuesday, April 17, 2012

The Ranking Business

In INSIDE HIGHER ED, Kris Olds comments on the growing commercialisation of university rankings.

Saturday, April 14, 2012

US Faculty Salaries

The Chronicle of Higher Education features a report from the AAUP on faculty salaries, along with data on student faculty ratio. Predictably, Harvard, where the average fill professors gets $198,000 per annum, is at the top.

If anyone ever has the time, it ought to be  possible to work out the productivity of faculty with regard to publications, citations , students taught and graduated.

Thursday, April 12, 2012

News from Australia

There is now a website MyUniversity that allows anyone to compare Australian public universities for  attributes such as staff qualifications, student to staff ratio, graduate employment and so on.

Of course, the value of the site will be little better than the quality of the information uploaded. Perhaps someone could take a look at the site and see how accurate the data is.

According to University World News , the Australian faculty staff union is not happy.

'Rea said that if students were looking to base their choice of institution on whether a campus had an automatic teller machine, the site might be useful. But if they wanted an indication of the quality of teaching and research at any given institution, the information provided relied on a set of indicators that had been under question for many years.

The union had been critical for some time of the misuse of statistical data, such as graduate employment outcomes and student satisfaction results, in determining the quality of learning and teaching. Yet these were included as measurable indicators of quality by the website.

“The use of student satisfaction scores in particular is prone to manipulation and does not reflect quality in teaching. Indeed, if institutions based their courses on whether students liked their subjects, which is essentially what these metrics capture, they would risk driving down the quality of degrees from Australian universities.

“There is always a danger of teaching to the test – or the survey, in this case,” Rea said.

She said the diversity of Australian universities made it difficult to attempt any comparisons. Although the union believed students should be able to make an informed choice of where best to study, it should be just that – an informed choice based on accurate, clear and transparent information.

“This can only happen if the indicators or measures used to create this information are specific, widely understood and agreed, and incapable of institutional manipulation.” '

Tuesday, April 10, 2012

Ranking News: QS World University  Rankings

Like many others, I have just received an invitation to take in the QS academic survey which contributes 40 per cent to their World University Rankings. If I understand it correctly, QS now get the majority of their respondents from the Mardev mailing lists. That means that subscribing to an academic journal is sufficient qualification to rank the research capabilities of universities around the world.

The Times Higher Education World University Rankings also have a reputation survey. It is a little more difficult to join this club. To be a respondent it is necessary to have written (or co-written or perhaps "co-written" (these days the concept of authorship is getting quite blurred) an article in an ISI-indexed journal. On balance, the THE survey is likely to more reliable although I think QS could argue that those who read academic journals but do little research are a constituency whose views should be considered.
Ranking News: US News

Robert Morse has announced that US News has begun to collect data for the 2013 US college rankings.

"We recently started collecting the statistical data that will be used for the 2013 edition of our college rankings, which will be published later this year. Data collection for the three U.S. News statistical surveys—the main one, financial aid, and finance—began on March 27.

These surveys gather information on such factors as enrollment, faculty, tuition, room and board, SAT and ACT scores, admissions criteria, graduation and retention rates, college majors, faculty, school finances, activities, sports, and financial aid. This data is used in the Best Colleges rankings that will be published on usnews.com and in the print guidebook that will be available on newsstands."

Monday, March 26, 2012

The University Challenge Rankings

The  quiz show, University Challenge, provides a plausible supplement to the established British league tables. If we award 2 points for winning and one for being runner up (I confess this is from Wikipedia) then we get this ranking:

1.      Oxford                                                             39
2.      Cambridge                                                       21
3.      Manchester                                                        8
4=.    Imperial College London                                  5
4=.    Open University                                                5
6=.    Durham                                                             4
6=.    Sussex                                                               4
8=.    St. Andrews                                                      3
8=.    Birkbeck College, University of London         3
10=.  Bradford                                                            2
10=.  Dundee                                                              2
10=.  Keele                                                                 2
10=.  Leicester                                                            2
10=.  Belfast                                                               2
10=.  Warwick                                                            2   
16 =  Lancaster                                                           2
16=.  LSE                                                                   1
16=.  Cranfield                                                           1
16=.  Sheffield                                                            1
16=.  York                                                                   1

Bristol, Edinburgh and University College are not there at all and LSE does not perform very well.

The show inspired an Indian version and in one memorable "cup winners' cup" final Sardar Patel College of Engineering beat Gonville and Caius, Cambridge.

Apologies for the weird spacing. I am looking up how to do tables in blogspot.

Tuesday, March 20, 2012

Comments on the THE Reputation Rankings

 From Kris Olds at GlobalHigherEd

The 2012 Times Higher Education (THE) World Reputation Rankings were released at 00.01 on 15 March by Times Higher Education via its website. It was intensely promoted via Twitter by the ‘Energizer Bunny’ of rankings, Phil Baty, and will be circulated in hard copy format to the magazine’s subscribers. As someone who thinks there are more cons than pros related to the rankings phenomenon, I could not resist examining the outcome, of course! See below and to the right for a screen grab of the Top 20, with Harvard demolishing the others in the reputation standings.

I do have to give Phil Baty and his colleagues at Times Higher Education and Thomson Reuters credit for enhancing the reputation rankings methodology. Each year their methodology gets better and better.

From Alex Usher at Higher Education Strategy Associates

There actually is a respectable argument to be made for polling academics about “best” universities. Gero Federkeil of the Centrum für Hochschulentwicklung in Gütersloh noted a few years ago that if you ask professors which institution in their country is “the best” in their field of study, you get a .8 correlation with scholarly publication output. Why bother with tedious mucking around with bibliometrics when a survey can get you the same thing?

Two reasons, actually. One is that there’s no evidence this effect carries over to the international arena (could you name the best Chinese university in your discipline?) and second is that there’s no evidence it carries over beyond an academic’s field of study (could you name the best Canadian university for mechanical engineering?).

So, while the Times makes a big deal about having a globally-balanced sample frame of academics (and of having translated the instrument into nine languages), the fact that it doesn’t bother to tell us who actually answered the questionnaire is a problem. Does the fact that McGill and UBC do better on this survey than on more quantitatively-oriented research measures have to do with abnormally high participation rates among Canadian academics? Does the fact that Waterloo fell out of the top 100 have to do with the fact that fewer computer scientists, engineers and mathematicians responded this year? In neither case can we know for sure.

Sunday, March 18, 2012

The THE Reputation Rankings

Times Higher Education has produced its third reputation ranking based on a survey of researchers published in ISI indexed journals. The top ten are:

1.  Harvard
2.  MIT
3.  Cambridge
4.  Stanford
5.  UC Berkeley
6.  Oxford
7.  Princeton
8.  Tokyo
9.  UCLA
10. Yale

This does not look all that dissimilar to the academic survey indicator in the  2011 QS World University Rankings. The top ten there is as follows:

1.  Harvard
2.  Cambridge
3.  Oxford
4.  UC Berkeley
5.  Stanford
6.  MIT
7.  Tokyo
8.  UCLA
9. Princeton
10. Yale

Once we step outside the top ten there are some differences. The National University of Singapore is 23rd in these rankings but 11th in the QS academic survey, possibly because QS still has respondents on its list from the time when it used World scientific, a Singapore based publishing company.

The Middle East Technical University in Ankara is in the top 100 (In the QS academic survey it is not even in the top 300), sharing the 90-100 band with Ecole Polytechnique, Bristol and Rutgers. At first glance this seems surprising since its research output is exceeded by other universities in the Middle East. But the technical excellence of its University Ranking by Academic Performance suggests that its research might be of a high quality.

Sunday, March 11, 2012

Law School Rankings

Anyone interested in the current arguments about American law school rankings might visit the blog of Lawyers against the Law School Scam.
University Ranking in Pakistan

University World News has an article on the ranking of Pakistani universities by the country's Higher Education Commission (HEC). According to Ameen Amjad Khan:

According to the Higher Education Commission (HEC) ranking, Islamabad’s Quaid-e-Azam University tops 136 public and private sector institutions, followed by the Pakistan Institute of Engineering and Applied Sciences, with Karachi’s Agha Khan University in third place.

Academics from the University of Karachi and the University of Peshawar have rejected the ranking, which does not place either institution in the top 10.

They have accused the HEC of tampering with the standard formula to favour some institutions and have demanded that their vice-chancellors formally convey their disapproval to HEC bosses.

Faculty members of Hyderabad’s Liaquat University of Medical and Health Sciences have even warned that they will take the matter to the court if the ranking is not revoked. They said in a statement on 29 February: “The HEC announced the rankings in haste and caused chaos in both public and private higher education institutions.”

It seems that the rankings were based on a modified of the QS World University Rankings. Among the modifications were the introduction of indicators based on the number of journals published and the number of grants from the HEC. 
International Patent Filing

The World International Property Organization has issued a report on the filing of patents. The top university is the University of California followed by MIT, University of Texas, Johns Hopkins and the Korea Institute of Advanced Technology.

Tuesday, March 06, 2012

O, My Felony is Rank

The Social Science  Research Network has just produced a paper by Morgan Cloud and George Shepherd, 'Law Deans in Jail'. Notice the absence of a question mark. Here is the abstract:

A most unlikely collection of suspects - law schools, their deans, U.S. News
& World Report and its employees - may have committed felonies by publishing
false information as part of U.S. News' ranking of law schools. The possible
federal felonies include mail and wire fraud, conspiracy, racketeering, and
making false statements. Employees of law schools and U.S. News who
committed these crimes can be punished as individuals, and under federal law
the schools and U.S. News would likely be criminally liable for their agents'
crimes. Some law schools and their deans submitted false information about the
schools' expenditures and their students' undergraduate grades and LSAT
scores. Others submitted information that may have been literally true but was
misleading. Examples include misleading statistics about recent graduates'
employment rates and students' undergraduate grades and LSAT scores. U.S.
News itself may have committed mail and wire fraud. It has republished, and
sold for profit, data submitted by law schools without verifying the data's
accuracy, despite being aware that at least some schools were submitting false
and misleading data. U.S. News refused to correct incorrect data and rankings
errors and continued to sell that information even after individual schools
confessed that they had submitted false information. In addition, U.S. News
marketed its surveys and rankings as valid although they were riddled with
fundamental methodological errors.

It is unlikely that we will ever see law school deans in jail. It seems that it would be necessary to show that there is a connection between the submission or publication (or failure to retract publication) of false or misleading information and monetary gain and that might be rather difficult to prove.

Also, the authors of the paper are from Emory University, a middling institution. If it ever did come to criminal proceedings you can bet that the big law schools would be lined up behind the rankings.

Monday, March 05, 2012

Best Student Cities

QS has just announced its Best Student Cities ranking. The criteria are performance in the QS World university rankings, student mix, quality of living, employer activity and affordability.

To be included a city must have a population of at least 250,000 and at least two institutions in the QS rankings That explains why Oxford and Cambridge are not on the list but Cairo and Santiago are. 
The top five are:

1.  Paris
2.  London
3.  Boston
4.  Melbourne
5.  Vienna

Saturday, February 18, 2012

Light Posting Ahead

For the next few weeks posting will be light as I am attending to family matters.

Wednesday, February 08, 2012

Eight years of ranking: What have we learned?

My post at the University World News World Blog can be viewed here.

Wednesday, February 01, 2012

University Ranking by Academic Performance

Perhaps too much attention is given to the big three international rankings -- ARWU (the Shanghai Rankings), the QS World University Rankings and the Times Higher Education World University Rankings.

So, here is what I hope will be the first of  a number of posts about the other rankings that have appeared recently.

The University Ranking of Academic Performance is produced by the Informatics Institute of the Middle East Technical University in Ankara.

The indicators are as follows:

Number of articles -- measures current scientific productivity
Google scholar results -- measures long term overall productivity
Citations -- measures research impact
Cumulative Journal Impact -- measures scientific impact
H- index --- measures research quality
International collaboration -- measures international acceptance


These are the top ten:

1.  Harvard
2.  Toronto
3.  Johns Hopkins
4.  Stanford

5.  UC Berkeley
6.  Tokyo
7.  Michigan, Ann Arbor
8.  Washington, Seattle
9.  UCLA
10. Oxford
Think Tank Ranking

There seems to be little that cannot be ranked these days. In the UK there are primary school league tables although nowhere in the world is there a kindergarten ranking. Yet.

The University of Pennsylvania has produced a report on think tanks. These are apparently flourishing everywhere: even Andorra has one.

Here are the top five world wide:

1.  Brookings Institution, USA

2.  Chatham House Royal Institute of International Affairs, UK

3.  Carnegie Endowment for International Peace, USA

4.  Council on Foreign Relations, USA

5.  Center for Strategic and International Studies, USA

Thursday, January 26, 2012

Guest Post

Today's post is by Masturah Alatas. It is in response to a comment in the Kuala Lumpur New Straits Times by James Campbell that begins:

"Any discussion of Malaysian tertiary educational policy needs to take into account the needs of national development in a specific and historical context. Recent debates in regard to the competitive position of Malaysian higher education globally is one area where the pressures of competition and liberalisation must be balanced by the interests of inclusion and social sustainability."


Over the last few years there has been an ongoing debate between those Malaysian academics who accept the challenge of globalisation  and those who more concerned with, as Campbell puts it, "ensuring national values, addressing horizontal social inequality, rural disadvantage and looking into the needs of sustainable and inclusive economic and social development".

The comment continued by invoking the name of Syed Hussein Alatas, a Malaysian social and political theorist who had considerable  international influence, being cited by, among others, Edward Said.



"The discourse of neo-liberal globalisation is itself still arguably beholden to what Syed Hussein Alatas critiqued as the discourse of “The Lazy Native”. Higher educational institutions’ commitment to inclusion and social justice is central to their merit in society."


The following is a response to this comment by the daughter and biographer of Alatas.

Finding the clarity

by Masturah Alatas

As biographer of late Malaysian sociologist Syed Hussein Alatas (1928 – 2007), I do not consider his life story to end with the end of his life. Any continuing narrative about Alatas also has to take into account how, for example, he is talked about in the media today.

As a case in point, I refer to the article Finding the Balance (Learning Curve, New Straits Times, 20 November 2011) by Dr James Campbell, a lecturer in Education at Deakin University, Australia, and researcher with Universiti Sains Malaysia.

Even though the article carries a moribund photo of Alatas, it contains only one sentence in direct reference to him. And the sentence is difficult to understand. “The discourse of neo-liberal globalization is itself still arguably beholden to what Syed Hussein Alatas critiques as the discourse of The Lazy Native”.

The article does not explain what, if at all, neo-liberal globalization has to do with “the discourse of The Lazy Native”. And nowhere in the article is it clearly stated that many of Malaysia’s current higher education policies are neo-liberal to begin with. So Campbell is vague. He seems to be criticizing neo-liberalism in general, but not what is specifically neo-liberal about Malaysia’s higher education policies.

Campbell argues that comparisons between the National University of Singapore and Universiti Malaya may not always be valid “given important distinctions and differences in national policies and political culture between the two nations.”

But how does this reasoning square with the fact that a Malaysian sociologist like Syed Hussein Alatas taught at the National University of Singapore for over twenty years, and was appointed Head of Department of Malay studies there? What criteria of merit did NUS apply when they tenured Syed Hussein Alatas? Is Campbell suggesting that the same criteria cannot be applied in a Malaysian university? And if so, why not?
Malaysians may still remember Syed Hussein Alatas’ short-lived, controversial term (1988 – 1991) as Vice-Chancellor of Universiti Malaya when he tried to promote academic staff based on merit. For Alatas, one way to establish merit was to look at the publications of academics. The New Straits Times itself carried reports of the controversy, so it is no secret. Some of them contained statements like “five members of the students association have come out in support of Vice-Chancellor Syed Hussein Alatas’s stand on the appointment of non-Malay deans to faculties in the university” (NST, 12 March 1990).

When Campbell writes that any discussion of Malaysia’s higher education policies “needs to be placed in perspective against the needs of national development in a specific historical context”, and that notions of merit must take into account ideas of inclusion and social justice, what exactly does he mean? Inclusion, of course, necessarily entails exclusion. But who is being excluded and on what grounds are they being excluded? And whose sense of social justice are we talking about here?

Syed Hussein Alatas’ works from The Myth of the Lazy Native to his writings on corruption precisely warn against the dangers of relativising notions of social justice. So it is quite odd that Campbell would refer to Alatas in his article.

All written legacies can be appropriated, rightly and wrongly, to support a particular persuasion or agenda, and it depends on critics to call attention to what is right or wrong appropriation. One of the best writers I know, for example, who has creatively applied Syed Hussein Alatas’ ideas on mental captivity and the inability to raise original problems to the role of education and the Arts, is former New Straits Times columnist U-En Ng. “You can use entertainment to pull wool over people’s eyes and divert attention away from whatever it is you don’t want them to see or think about. Or, more positively, you can use artistic expression to build civic participation and the capacity to raise original problems,” he writes in the article Governing through the Arts (NST, 09 January 2011). “Rebellious performance art by university students tells you both what you can expect from the current education system as well as how the public might react to new ideas.”

At the same time, a young Italian poet from Osimo by the name of Andrea Palazzo reminds us that an excess of “the so-called need to express oneself can be the mortal enemy of Beauty and Truth. Overshadowed by popular opinion, great art dies, or rather it disappears, or languishes in museums.” And for Palazzo, the prospects for Philosophy in some Italian universities, which he feels are “conservative rather than selective”, are not much brighter either.

Syed Hussein Alatas believed that any process of change must necessarily be accompanied by a philosophical set of criteria for selection of what to reject, retain and strive for (see Erring Modernization, 1975). Quoting from German philosopher Friedrich Nietzsche’s The Use and Abuse of History, Alatas stressed the importance of having a horizon of thinking, “a line that divides the visible and the clear from the vague and shadowy” in order for an individual, a community and a nation to thrive. “We must know the right time to forget as well as the right time to remember, and instinctively see when it is necessary to feel historically and when unhistorically.”

It is, of course, extremely difficult to know when and what to remember and forget, especially when the spectre of Malaysia’s 1969 May 13 Racial Riots still haunts us, and when Malaysia is making itself dizzy by rushing to become a High Income Nation before eight years are up.


But one way for a line, a horizon of thinking, to become clearer is through good writing. And this may just be what will rescue Alatas’ work from languishing in the shadows.



Masturah Alatas


Sunday, January 22, 2012

Worth Reading

Ranking in Higher Education: Its Place and Impact by Jan Sadlak. Originally appeared in the Europa World of Learning 2010.

Wednesday, January 18, 2012

The Power of Small Numbers

My attention has just been drawn to the citations per paper indicator in the 2011 QS Asian University Rankings. In first place is University of Santo Tomas in the Philippines, a very good school in some ways but not usually considered as a research rival to Hong Kong or Tokyo. It seems that UST's success was the result of just one much cited medical paper of which just one UST researcher was a co-author.

 Another highly cited many-authored medical paper seems to explain Universitas Padjadjaran's appearance in sixth place in this indicator even though the total number of papers is extremely small.

Leiden University have started offering fractional counting of publications in their rankings:

The fractional counting method gives less weight to collaborative publications than to non-collaborative ones. For instance, if the address list of a publication contains five addresses and two of these addresses belong to a particular university, then the publication has a weight of 0.4 in the calculation of the bibliometric indicators for this university. The fractional counting method leads to a more proper normalization of indicators and to fairer comparisons between universities active in different scientific fields. Fractional counting is therefore regarded as the preferred counting method in the Leiden Ranking.

This would be one way of avoiding giving a high position to universities that produce little but manage to get researchers included as co-authors a few papers, usually in medical journals,
FindThe Best

This is a site that has just come to my attention. There is a great variety of rankings of things like antivirus, snowboards and fertility clinics and also of colleges and universities, including business, law and medical schools.

The colleges and universities ranking is US only and includes a "smart ranking" combining  statistical information with the Forbes, US News and ARWU (Shanghai) rankings. This sounds like a good idea but there does not seem to be any information about the methodology.

Tuesday, January 17, 2012

The Happiest University in Britain?

According to the Daily Telegraph it's St Mary's University College Belfast. I thought Belfast was in Ireland but then again I did not do geography in secondary school.
Update: The Journal Bubble

Jeffrey Bealle has a list of predatory journals.

 Note  that some of the comments dispute the inclusion of some journals.
The Journal Bubble

Inside Higher Ed has a piece by Philip Altbach about the proliferation of dubious academic journals.

Clever people have figured out that there is a growing demand for outlets for scholarly work, that there are too few journals or other channels to accommodate all the articles written, that new technology has created confusion as well as opportunities, and (finally) and somewhat concerning is that there is money to be made in the knowledge communication business. As a result, there has been a proliferation of new publishers offering new journals in every imaginable field. The established for-profit publishers have also been purchasing journals and creating new ones so that they “bundle” them and offer them at high prices to libraries through electronic subscriptions.

I suspect that this is causing increasing problems for struggling but respectable journals in the academic periphery. I know of of least one journal that has had several submitters "disappear" after being asked  to make modest revisions or even deal with problems identified by a turnitin report. One wonders where the papers will reappear.

Sunday, January 15, 2012

Research Fraud in the UK

AN article in Times Higher Education by Jack Grove indicates that there is a large amount of research fraud going on in UK universities, although the comments raise questions about the validity of the study.

I am wondering if there is any comparative international data available.
Primary School League Tables

The craze for rankings continues unabated. There is now a League Table for English primary schools.


The league tables show the percentage of 11-year-olds in each school reaching Level 4 – the standard expected for their age group – in both English and maths at primary school.
Officially, this means they can spell properly, start to use grammatically complex sentences and employ joined up handwriting in English. In maths, they should be able to multiply and divide whole numbers by 10 or 100 and be able to use simple fractions and percentages.

Pupils exceeding this standard are awarded a higher Level 5.Data for individual schools also shows three other measures: average points score, value-added and pupil progress.

Saturday, January 14, 2012

Who says college isn't worth it?

There is a web site called SeekingArrangement which puts sugar daddies and sugar babies in touch with another. Apparently, large numbers of college graduates are signing up in the latter category, perhaps because of increasing difficulties in paying off student loans.

There is now a ranking of the top 20 colleges among sugar baby sign ups. I have doubts about the validity of the ranking. All that is necessary to be a certified college sugar baby, and get three times as many enquiries from sugar daddies (it's good to know that American society still values education), is an edu email address.

New York University might be number 1 because its tuition fees are so high or job prospects for graduates so meagre or maybe because the sugar daddies are in New York.

Here are the top twenty:

1. New York University (NYU) -- 185
2. University of Georgia -- 155
3. University of Phoenix -- 144
4. Tulane University -- 129
5. Temple University -- 113
6. Virginia Community College -- 108
7. University of Southern Florida -- 93
8. Arizona State University -- 85
9. Michigan State University -- 81
10. Ivy Tech Community College -- 78
11. Georgia State University -- 74
12. University of Wisconsin -- 73
13. Penn State University -- 72
14. University of Central Florida -- 67
15. Kent University -- 65
16. Maricopa Community College -- 63
17. Indiana University -- 62
18. University of California, Berkeley -- 61
19. The Art Institutes -- 60
20. Florida International University -- 59

Please note that I found this story via the Chronicle of Higher Education.
Online Education Rankings

US News has announced its rankings of American online education programs. There are six categories: Bachelor's, Business, Education, Engineering, Info tech and Nursing. Within each category there are rankings for faculty credentials and training, student services and technology, student engagement and assessment and, except for bachelor's, admissions selectivity. For bachelor programs the top universities are:

Faculty Credentials and Training
Westfield State University, Massachusetts

Student Engagement and Assessment  
Bellevue University, Nebraska

Student Services and Technology
Arizona State University, Tempe

Wednesday, January 11, 2012

The end of the university as we know it?


MIT has already been putting its course materials online for anyone to access free of charge. Now they are going a step further.

"MIT today announced the launch of an online learning initiative internally called “MITx.” MITx will offer a portfolio of MIT courses through an online interactive learning platform that will:
  • organize and present course material to enable students to learn at their own pace
  • feature interactivity, online laboratories and student-to-student communication
  • allow for the individual assessment of any student’s work and allow students who demonstrate their mastery of subjects to earn a certificate of completion awarded by MITx
  • operate on an open-source, scalable software infrastructure in order to make it continuously improving and readily available to other educational institutions.
MIT expects that this learning platform will enhance the educational experience of its on-campus students, offering them online tools that supplement and enrich their classroom and laboratory experiences. MIT also expects that MITx will eventually host a virtual community of millions of learners around the world.

There are a lot of questions that come to mind. Will students be assessed according to the same standards as conventional MIT students? If someone accumulates sufficient certificates of completion will they be entitled to an MITx degree? What will happen if employers and graduate school  start accepting MITx certificates as equivalent to standard academic credentials? If so, will MIT be able to resist the temptation to start charging hefty fees for a certificate.

MIT may, perhaps unwittingly,  have started a process that will end with universities becoming something very different.

Monday, January 02, 2012

How Did I Miss This?

The blog Registrarism has discovered a fascinating article, published in 2002 in Higher Education Quarterly, that compares university league tables (that is British university rankings) with the football (soccer to Americans) league tables.

Sunday, December 18, 2011

Leiden Ranking: Many Ways to Rate Research

My article on the Leiden Rankings in University World News can be found here.

 It looks as though a two-tier international university ranking system is emerging.

At the top we have the 'big three', Shanghai's Academic Ranking of World Universities, the QS World University Rankings and, since 2010, the Times Higher Education World University Rankings.

These receive massive attention from the media, are avidly followed by academics, students and other stakeholders and are often quoted in promotional literature. Graduation from a university included in these has even been proposed as a requirement for immigration.

Then we have the rankings by SCImago and Webometrics, both from Spain, the Performance Ranking of Scientific Papers for World Universities produced by the Higher education Evaluation and Accreditation Council of Taiwan, and the Leiden Ranking, published by the Centre for Science and Technology Studies at Leiden University.

These rankings get less publicity but are technically very competent and in some ways more reliable than the better-known rankings.

Wednesday, December 07, 2011

Update 6 on El Naschie vs Nature

There have been no reports for several days and the trial is now over. There will be a judgement in January.
What to do about the research bust

Mark Bauerlein has an article in the Chronicle of Higher Education on the disparity between the extraordinary effort and intelligence poured into scholarly writing in the humanities and the meager attention such writing receives.

"I devised a study of literary research in four English departments at public universities—the University of Georgia, the University at Buffalo, the University of Vermont, and the University of Illinois at Urbana-Champaign—collecting data on salaries, books and articles published, and the reception of those works. The findings:
  • Those universities pay regular English faculty, on average, around $25,000 a year to produce research. According to the faculty handbooks, although universities don't like to set explicit proportions, research counts as at least one-third of professors' duties, and we may calculate one-third of their salaries as research pay. This figure does not include sabbaticals, travel funds, and internal grants, not to mention benefits, making the one-third formula a conservative estimate.
  • Professors in those departments respond diligently, producing ample numbers of books and articles in recent years. At Georgia, from 2004 to 2009, current faculty members produced 22 authored books, 15 edited books, and 200 research essays. The award of tenure didn't produce any drop-off in publication, either. Senior professors continue their inquiries, making their departments consistently relevant and industrious research centers.
  • Finally, I calculated the impact of those publications by using Google Scholar and my own review of books published in specific areas to count citations. Here the impressive investment and productivity appear in sobering context. Of 13 research articles published by current SUNY-Buffalo professors in 2004, 11 of them received zero to two citations, one had five, one 12. Of 23 articles by Georgia professors in 2004, 16 received zero to two citations, four of them three to six, one eight, one 11, and one 16. "
Bauerlain suggests that these limited citation counts are telling us something, that talented scholars might find better things to do and that society might direct resources elsewhere.

The QS World University Rankings would apparently agree. Their citations indicator simply counts the total number of citations and divides it by the total number of faculty. This is a very crude measure, especially since it counts the current number of faculty but then counts the citations to articles written over a five year period. Any university seeking a boost in the QS rankings could simply axe a few English, history and philosophy specialists and replace them with oncologists and engineers. True, the world would lose studies about Emily Dickinson's Reluctant Ecology of Place, cited once according to Google Scholar, or Negotiations of Homoerotic Tradition in Paradise Regained, but if this was accompanied by even a small advance in cancer treatment, who would really care? There would be an even better effect on the Shanghai rankings which do not not count publications or citations in the humanities but still include the faculty in their productivity indicator.

But there are those who would argue that while disciplines go about citing differently they must be regarded as being on the same level in all other respects. Thomson Reuters, who collect the data for the Times Higher Education rankings, now normalise their data so that citations in a specific discipline in a specific country in a specific year are benchmarked against the average for that discipline in that country in that year. That would mean that the article by the Buffalo professors with five citations might look quite good.

I have a suggestion for those professors of English and other disciplines which hardly anyone seems to read anymore. Go to some Central Asian or East African republic where the papers in your field get only a few citations: the next article you write with its miserable handful of citations will be well above average for that country and your new university will suddenly perform well in the Times Higher rankings. Just make sure that your employer produces two hundred papers a year altogether.

Friday, December 02, 2011

European Universities and Rankings

Research Trends, the newsletter from Scopus,  reports on  a conference of European universities that discussed international rankings. The participants found positive aspects to rankings but also had criticisms:

Going through the comparison of the various methodologies, the report details what is actually measured, how the scores for indicators are measured, and how the final scores are calculated — and therefore what the results actually mean.
The first criticism of university rankings is that they tend to principally measure research activities and not teaching. Moreover, the ‘unintended consequences’ of the rankings are clear, with more and more institutions tending to modify their strategy in order to improve their position in the rankings instead of focusing on their main missions.
For some ranking systems, lack of transparency is a major concern, and the QS World University Ranking in particular was criticized for not being sufficiently transparent.
The report also reveals the subjectivity in the proxies chosen and in the weight attached to each, which leads to composite scores that reflect the ranking provider’s concept of quality (for example, it may be decided that a given indicator may count for 25% or 50% of overall assessment score, yet this choice reflects a subjective assessment of what is important for a high-quality institute). In addition, indicator scores are not absolute but relative measures, which can complicate comparisons of indicator scores. For example, if the indicator is number of students per faculty, what does a score of, say, 23 mean? That there are 23 students per faculty member? Or does it mean that this institute has 23% of the students per faculty compared with institutes with the highest number of students/faculty? Moreover, considering simple counts or relative values is not neutral. As an example, the Academic Ranking of World Universities ranking does not take into consideration the size of the institutions.

I am not sure these criticisms are entirely fair. It seems that the weighting of the various indicators in the Times Higher Education rankings emerged from a lot of to and fro-ing between various stakeholders and advisers. In the end, far too much weighting was given to citations but that is not quite the same as assigning arbitrary or subjective values.

The Shanghai rankings do have an indicator, productivity per capita , that takes  faculty size into account although it is only ten per cent of the total ranking. The problem here is that faculty in the humanities are counted but not their publications.

I am not sure why QS is being singled out with regard to transparency. The THE rankings are also, perhaps in a different way, quite opaque. Aggregate scores are given for teaching environment, research and international orientation without indicating the scores that make up these criteria.

So what is to be done?


The EUA report makes several recommendations for ranking-makers, including the need to mention what the ranking is for, and for whom it is intended. Among the suggestions to improve the rankings, the following received the greatest attention from the audience:
  1. Include non-journal publications properly, including books, which are especially important for social sciences and the arts and humanities;
  2. Address language issues (is an abstract available in English, as local language versions are often less visible?);
  3. Include more universities: currently the rankings assess only 1–3% of the 17,000 existing universities worldwide;
  4. Take into consideration the teaching mission with relevant indicators.

The first of these may become feasible now that Thomson Reuters has a book citation index. The second and third are uncontroversial. The fourth is very problematical in many ways.

The missing indicator here is student quality. To be very blunt, universities can educate and instruct students but they can do very little to make them brighter.  A big contribution to any university ranking would be a comparison of the relative cognitive ability of its students. That, however, is a goal that requires passing through many minefields.

Thursday, December 01, 2011

Diversity and Rankings

Robert Morse, director of data research at the US News and World Report discusses the question of whether "diversity" should be included in the ranking of American law schools.

"I was one of speakers on the "Closing Plenary: Reforming U.S. News Rankings to Include Diversity" panel, which discussed many of the issues pertaining to whether U.S. News should add a measure of diversity directly into the Best Law Schools rankings. I pointed out that U.S. News believes diversity is important and that is why we all ready publish a separate law school diversity index.

Our current index identifies law schools where law students are most and least likely to encounter classmates from a different racial or ethnic group. However, the current index does not measure how successful each law school is at meeting a diversity goal or benchmark at the school, state, local, or national level. It also gives schools enrolling one ethnic group a low score, though that school's enrolment may match its state's ethnic population or the school may be a Historically Black College or University. It's for these reasons the current index would not be appropriate to add into the rankings".

Diversity here does not mean diversity of ideology, religion, class, politics or nationality. It simply means the numbers of recognised minorities, mainly African-Americans, Hispanics and Asian Americans.

It is interesting to look at the diversity index and to see the likely effect of including diversity in the law school rankings. The most diverse law school is the University of Hawaii. The University of the District of Columbia and Florida International University also get high scores. Low scorers include Harvard, Yale and UCLA.

Somehow, I do not think that an indicator that benefited Florida International University at the expense of Harvard would add to the credibility of these rankings.

Unless it can be demonstrated that there is something magically transforming about the statistical profile of a law school reflecting that of its city, state, or nation or future nation, this proposal does not sound like a very good idea.
The Utility of Rankings

Another advantage of a good performance in international university rankings is that graduates will be able to get into Russian postgraduate programs (if your university is in a G8 country).


Russia’s education ministry is currently drawing up a list of foreign universities whose qualifications will be recognized.

The list will include only universities located within the G8 countries that enter the top 300 in the Academic Ranking of World Universities or the QS World University Rankings. Officials say there will be around 300 institutions meeting the criteria.

The reform is intended to attract more students to take part in Russian MA and PhD programs.

Saturday, November 26, 2011

Update 5 on El Naschie vs Nature

The BBC has another piece on the Nature case by Pallab Ghosh. It seems that El Naschie is now admitting that his papers were not peer reviewed but argues that this was because he had no peers who could review them:


He said that he would discuss his papers with fellow scientists, and only when he thought that they were of a sufficiently high standard would he publish them. "I am too arrogant and have too much self-respect to allow a bad paper to pass through," he said.
Prof El Naschie called one witness, Prof Otto Rossler - an honorary editor of Chaos, Solitons and Fractals.
He told the court that there was no-one who could peer review him, referring to Prof El Naschie, because "if you have something new to offer, peer review is dangerous", adding that in such cases "peer review delays progress in science".
Prof El-Naschie asked his witness whether he thought that his (Prof El Naschie) papers were of "poor quality".
Prof Rossler replied: "On the contrary, they were very important and will become more important in the future."
And he added: "You are the most hard-working and diligent scientists I have ever met."

It is a useful to compare Rossler with Neil Turok, Nature's expert witness. Rossler is best known lately for warning that the Large Hadron Collider risked creating black holes that would destroy the world. See El Naschie Watch for this and other information. He is also a self-proclaimed simultaneous submitter, something that for journal editors is almost as bad as plagiarism. A comment on Rossler's claims concludes "To conclude: this text would not pass the referee process in a serious journal".

It seems that it is increasingly difficult to argue that Alexandria University's remarkable scores for research impact in the Times Higher Education World University Rankings were the result of outstanding, excellent or even controversial research.

I am not sure what an honorary editor is but at the moment Rossler is not listed as any sort of editor at the official Chaos Solitons and Fractals home page.


Incidentally, since the trial will presumably turn to the question of El Naschie's affiliations at some point, this page lists him as Founding Editor but does not give any affiliation.
The Politics of Ranking

One of the more interesting aspects of the university ranking business is the way it is used by local politicians to advance their agenda. This is especially obvious in Malaysia where errors and methodological changes have sent local universities bouncing up down the QS rankings. Every rise is proclaimed to be a vindication of government policy while every fall is accompanied by head shaking from the opposition.

This year Universiti Malaya moved into the QS top 200. There is nothing surprising about that: it has been there before. More significant was getting into the Shanghai Academic ranking of World Universities top 500. That is a lot harder but also more likely to reflect real underlying changes. It seems that UM has finally realised that a little bit of encouragement and financial support can produce quite significant results in a short period time.

Patrick Lee, in the blog of opposition leader Lee Kit Siang, comments that:  


PETALING JAYA: Malaysia has little to show for its universities despite spending more money on tertiary education than do many other countries.
Malaysian universities lag behind many counterparts in Asia, including those located in neighbouring countries like Thailand and Singapore, according to a World Bank report released today.
“While Malaysia spends slightly more than most countries on its university students, leading Malaysian universities perform relatively poorly in global rankings,” said the report, entitled Malaysia Economic Monitor: Smart Cities.
Citing the Quacquarelli Symonds (QS) World University Rankings 2010, it noted that Universiti Malaya (UM) was ranked 207th worldwide and 29th in Asia.
It also quoted a US News and World 2011 report on the World’s Best Universities, which put UM, Universiti Kebangsaan Malaysia, Universiti Sains Malaysia and Universiti Putra Malaysia at 167th, 279th, 335th and 358th place respectively.
Even more worrying, the World Bank report observed, was the “increasing gap” between Malaysia’s and Singapore’s universities.
It compared UM with the National University of Singapore (NUS), which QS cited as the leading university in Southeast Asia.
“The gap between UM and NUS has been high and generally increasing, especially in the sciences,” the report said.
According to the report, UM and NUS were on par when it came to science and technology in 2005. However, UM has lost out to NUS over the past six years.
The report also said many of Malaysia’s university graduates did not seem to have the skills that would help them get employment.

Firstly, the QS  and the US News and World Report rankings are the same. Secondly, it is a lot easier to start a university in Malaysia than in Singapore.

Even so, moving into the Shanghai rankings is a real advance and should be recognised as such. 
Update 4 on El Naschie vs Nature

Gervase de Wilde has an article on the case in Inform's Blog. He refers to the Guardian's account and then adds the following comment:


Comment
The case seems to offer ammunition to libel reformers. Even in the absence of the ill-advised and incoherent aspects of his case which were excluded before trial, and of the implicit comparisons of his work to Einstein’s made during the first five days at the High Court, his claim against a venerable and highly respected scientific journal seems a poor substitute for meeting their allegations head on in some form of correspondence or public debate. Moreover, the journal had published the Claimant’s own defence of his methods in running CSF, that he sought to emphasise scientific content above impressive affiliations, in the original article.

A spokesperson for the Libel Reform campaign, speaking to the Guardian, commented that reform can’t come soon enough, since
“Scientists expect publications like Nature to investigate and write about controversies within the scientific community. The threat of libel action is preventing scientific journals from discussing what is good and bad science.”

However, the public interest defence argued for by campaigners is one which is already being employed. The BBC reports that Andrew Caldecott QC’s opening statement for the Defendants described their defence as relying on the article being “true, honest opinion and responsible journalism on an issue of public interest”.

As the choice of witnesses indicates, the case does touch on the seemingly incomprehensible branch of physics in which the Claimant has made his academic career. In this respect there is a threat of a libel action stifling academic debate, and a similarity to BCA v Singh 2010 EWCA Civ 350, where opinions expressed in a controversy on what was essentially a scientific matter were at issue. But it is also about the methods he employed in running a publication in the context of a widely recognised system of accreditation and review, and about allegations regarding the professional affiliations which feature on his website. These are the kind of criticisms that might be made about any professional person, and would not necessarily come under scope of a scientific exception for “rigorous debate” on good and bad science urged by campaigners.

It is questionable whether the phrase "academic career" is appropriate since El Naschie has not apparently held any formal permanent academic posts recently unless one counts the award of an Emeritus Professorship by Alexandria University, a strange distinction since there is no sign of the professorship from which he retired.

Friday, November 18, 2011

Update 3 on El Naschie vs Nature

The Guardian has a substantial report on the case by Alo Jha. It seems that El Naschie believes that expert witness Neil Turok is unqualified to understand his work.It is difficult to see how this argument, even if valid, is relevant to the point of whether or not peer review took place.

Should the court decide in favour of El Naschie, it would provide some sort of justification for the methods used in the citations indicator in the Times Higher Education rankings which gave high scores to Alexandria University mainly or partly because of the many citations of papers by El Naschie.

El Naschie is suing Nature as a result of a news article published in 2008, after the scientist's retirement as editor-in-chief of the journal Chaos, Solitons and Fractals. The article alleged that El Naschie had self-published several research papers, some of which did not seem to have been peer reviewed to an expected standard and also said that El Naschie claimed affiliations and honorary professorships with international institutions that could not be confirmed by Nature. El Naschie claims the allegations in the article were false and had damaged his reputation.
On Friday, Nature called Professor Neil Turok, a cosmologist and director of the Perimeter Institute in Canada, as an expert witness to assess some of the work published by El Naschie.
Turok described his expertise as being in cosmology. "I work at the theoretical end of cosmology … my work consists of applying unified theories, such as string theory, to the most difficult questions in cosmology, namely the beginning of the universe or the initial singularity, the moment where everything was at a single point in the conventional description."
In his evidence, Turok said he found it difficult to understand the logic in some of El Naschie's papers. The clear presentation of scientific ideas was an important step in getting an idea accepted, he said. "There are two questions – one is whether the work is clearly presented and readers would be able to understand it. It would be difficult for a trained theoretical physicist to understand [some of El Naschie's papers]. I couldn't understand it and I made a serious attempt to understand it. The second question is about the correctness of the theory and that will be decided by whether it agrees with experiments. Most theories in theoretical physics are speculative – we form a logical set of rules and deductions and we try, ultimately, to test the deductions in experiments. For me, clear presentation is the first thing in the presentation of a theory."
In response, El Naschie pointed out that even Albert Einstein had made mistakes in his publications. "Einstein is the most sloppy scientist ever. He never defined his quantities, he doesn't put in references and he made so many mistakes of mathematics and concepts. He was a very natural man when he explained something to lay people. But Einstein, whom I admire very much because he had imagination and the courage to stand up to the bloody Nazis, Einstein was an extremely sloppy man."
Later in the session, El Naschie accused Turok of having "no idea" about mathematics and being unqualified to assess his work. "If somebody doesn't understand things, it's his own limitation," El Naschie said.

Thursday, November 17, 2011

Times Higher Social Science Rankings


1.  Stanford
2.  Harvard
3.  Oxford
The Influence of Rankings

Varsity, the student newspaper at Cambridge, suggests that British universities are recruiting staff in order to improve their position in the QS rankings:


Matthew Knight, chairman of Universities HR and the University of Leeds HR director, said: “Within the context of £9,000 fees, many universities have a strategic drive to improve the quality of the student experience.
“Therefore, many are taking the opportunity to improve student staff ratios regardless of the numbers of applicants. So there’s a lot of recruitment going on at some universities, although there’s no specific pattern to this.”
As the QS World University Rankings use student-faculty ratios as the only globally comparable indicator to determine their tables, an increase in employment can be used to promote a university’s image and attract students.