RateMyProfessors and Amy Bishop

Richard Vedder of the Center for College Affordability and Productivity produces an interesting US ranking based on value for money for students. One key element is data provided from the famous or notorious site, RateMyProfessors. I used to think that it would a step forward in international university rankings to try to do something like this on a global scale. Now I am not so sure.

I assume that everbody has heard of the tragic shooting at the University of Alabama at Huntsville. My first suspicion was that the alleged murderer, Amy Bishop, was a talented but socially awkward academic who had snapped after being denied tenure on flimsy grounds of collegiality or for being politically incorrect.

That does not look like being the case. A blog, Shepherds and Black Sheep has analysed her research output and found that it seems inflated, with one article having her children as c0-authors and another published in an "online vanity press' and several more being co-authored with her husband

Bishop's pages at RateMyProfessors are also interesting. At first sight they look quite impressive with a total score of 3.6 out of 5, fifth best in her faculty, and 3.4 for clarity and 3.7 for helpfulness. But there are some oddities.

The user comments start with three excellent reviews, one on 26 May, 2009 and two on 19 May. A little odd. Back in June 2004 there were also two rave reviews again posted on the same day.

It is also noticeable that the good reviews tend to cluster together with three consecutve good reviews in January and February 2006 and another three, one after the other, in November, 2004 and January, 2005.

Another odd thing is that for the helpfulness and clarity indicators there is a very distinctive distribution curve. For helpfulness, Bishop had 6 ones (the worse), 2 twos , 5 threes, 3 fours, and then 17 fives (the best) For clarity, it was 7 ones, 5 twos, 4 threes, 3 fours and 15 fives. Note the dramatic jump from four to five in both categories.

Compare this with a low scoring teacher who gets 4 fours and 4 fives for helpfulness and 5 fours and 6 fives for clarity.

Compare also a high scoring faculty member with 13 fours and 25 fives for helpfulness and 15 fours and 28 fives for clarity. A big jump from four to five but proportionately much less than Bishop's.

Is it possible to rig RateMyProfessors?

According to Tenured Radical it is very easy.

"To test my theory that ratings could be posted by people who had never been my students, I went to the dreaded site, and registered myself, under my own name, as a Zenith student. Easy-peasy. The only false information I provided was a birth date that made me 19 years old (I wish!) and the box I checked that affirmed my status as a Zenith sophomore. I then successfully added a rating about myself. You can see it here: it's the anxious looking green emoticon that has the comment "interesting." I thought it only fair to add something right down the middle, neither good nor bad. Inflammatory perhaps, but arrogant never, that's my motto. "

So, I have a strong suspicion that someone had been going to RateMyProfessors and posting effusive commuents about Bishop (and nasty ones about other faculty members.?)

Perhaps RateMyProfessors is not such as good indicator after all.