Are All Disciplines Equal?

Rankers, evaluators and assessors of all sorts have to face the problem that academic disciplines do not go about writing, publishing and citing in exactly the same way. Researchers in some disciplines write shorter papers that  have more authors, are more easily divided into smaller units and get more citations sooner after publication than others. Medical research papers tend to be short, frequent,  co-authored and cited very soon after publication compared to history or philosophy.

Different ranking organisations follow different approaches when dealing with this diversity of practice. Scimago just counts the total number of publications. ARWU of Shanghai counts publications but gives a double weighting to social sciences and none to the humanities. Thomson Reuters, who power the Times Higher Education world rankings, normalize by field and by country.

Here are some fairly simple things that rankers might try to overcome disparities between disciplines. They could  count total pages rather than the number of papers. They could look into counting citations of conference papers or books. Something worth doing might be giving a  reduced weighting to co-authored papers, which would shift the balance a little bit towards the arts and humanities and might also help to discourage dubious practices like supervisors adding their names to papers written by their graduate students.

We should also ask whether there are limits to how far field and country normalization should go. Do we really believe that someone who has received an average number of citations for political science in Belarus deserves the same score as someone with an average number of citations for chemical engineering in Germany?

It does seem that there are substantial and significant variations in the cognitive skills required to compete effectively in the academic arena. Here are combined verbal and quantitative GRE scores for selected disciplines by intended majors for 2011-2012.

Physics and Astronomy                          317
Economics                                              313
Biology                                                   308
Philosophy                                              310
English Language and Literature           305
Education: Secondary                             305
History                                                    304
Education: Higher                                  304
Psychology                                             300
Sociology                                               300
Education: Administration                     299
Education: Curriculum                           299
Education: Early Childhood                   294


The scores look as they are very close together but this is largely a a consequence, perhaps intended, of the revision (dumbing down?) of the GRE.

Is it possible that one reason why physicists and economists publish more papers which are read more than those by education specialists is simply that the former have more ability and interest than the latter?