The Best University in India

No, it is not one of the IITs. Or even Delhi or Calcutta.

It is Panjab University, which has come from nowhere to seize the 245th place.

NDTV thinks that it is something to do with a meeting last May.

"The increased representation for India in the rankings has been attributed to a two-day National Policy Dialogue in May on international rankings, when representatives of Times Higher Education were invited to meet with senior university leaders by India's Ministry of Human Resources Development and Planning Commission."

Someone at the MHRD thinks it is all about presenting the data to Thomson Reuters.

'“It is not that Panjab university has done significantly very well this year. The way we see it is that, they may have been able to present their data and information in a better and convincing way. The vice-chancellor himself is a researcher and he may have taken the effort to logically present the data," said a senior MHRD official.'

Phil Baty of THE is more helpful.

"Panjab University has done well in citation impact with good scores indicating that its academics are producing research which is widely used and valued by the academic community around the world as per Phil Baty, editor, Times Higher Education World University Rankings. He said that Panjab University though has not fared well on the research front. According to him, Indian Institutes have done well this year"

This is what actually happened. Panjab University performed poorly for Teaching, International Outlook and Industry Income - Innovation. It performed very poorly for Research with a score of 14 but "brilliantly" for Citations with a score of 84.7.

Panjab University's citations score is very largely due to its participation in the Large Hadron Collider project, which involves more than 2000 physicists in more than 150 research centers and 37 countries. Below is the list of physicists. Somewhere in there is someone from Panjab University. The publications related to the project are widely cited. For most universities this brings little benefit since the citations are spread out over many publications. PU, however, has a relatively limited research output so those citations lead to a very high mean citations per paper (normalised for field and year) and thus to a very high score, after adding in the regional modification.

So if Indian universities want to do well in the Rankings this is one way. Don't do too much research and get on the LHC project or something similar.

Author(s): Bayatian, GL (Bayatian, G. L.); Chatrchyan, S (Chatrchyan, S.); Hmayakyan, G (Hmayakyan, G.); Sirunyan, AM (Sirunyan, A. M.); Adam, W (Adam, W.); Bergauer, T (Bergauer, T.); Dragicevic, M (Dragicevic, M.); Ero, J (Eroe, J.); Friedl, M (Friedl, M.); Fruehwirth, R (Fruehwirth, R.); Ghete, V (Ghete, V.); Glaser, P (Glaser, P.); Hrubec, J (Hrubec, J.); Jeitler, M (Jeitler, M.); Krammer, M (Krammer, M.); Magrans, I (Magrans, I.); Mikulec, I (Mikulec, I.); Mitaroff, W (Mitaroff, W.); Noebauer, T (Noebauer, T.); Pernicka, M (Pernicka, M.); Porth, P (Porth, P.); Rohringer, H (Rohringer, H.); Strauss, J (Strauss, J.); Taurok, A (Taurok, A.); Waltenberger, W (Waltenberger, W.); Walzel, G (Walzel, G.); Widl, E (Widl, E.); Wulz, CE (Wulz, C-E); Fedorov, A (Fedorov, A.); Korzhik, M (Korzhik, M.); Missevitch, O (Missevitch, O.); Zuyeuski, R (Zuyeuski, R.); Chekhovsky, V (Chekhovsky, V.); Dvornikov, O (Dvornikov, O.); Emeliantchik, I (Emeliantchik, I.); Litomin, A (Litomin, A.); Mossolov, V (Mossolov, V.); Shumeiko, N (Shumeiko, N.); Solin, A (Solin, A.); Stefanovitch, R (Stefanovitch, R.); Gonzalez, JS (Gonzalez, J. Suarez); Tikhonov, A (Tikhonov, A.); Petrov, V (Petrov, V.); D'Hondt, J (D'Hondt, J.); De Weirdt, S (De Weirdt, S.); Goorens, R (Goorens, R.); Heyninck, J (Heyninck, J.); Lowette, S (Lowette, S.); Tavernier, S (Tavernier, S.); Van Doninck, W (Van Doninck, W.); Van Lancker, L (Van Lancker, L.); Bouhali, O (Bouhali, O.); Clerbaux, B (Clerbaux, B.); De Lentdecker, G (De Lentdecker, G.); Dewulf, JP (Dewulf, J. P.); Mahmoud, T (Mahmoud, T.); Marage, PE (Marage, P. E.); Neukermans, L (Neukermans, L.); Sundararajan, V (Sundararajan, V.); Vander Velde, C (Vander Velde, C.); Vanlaer, P (Vanlaer, P.); Wickens, J (Wickens, J.); Assouak, S (Assouak, S.); Bonnet, JL (Bonnet, J. L.); Bruno, G (Bruno, G.); Caudron, J (Caudron, J.); De Callatay, B (De Callatay, B.); De Jeneret, JD (De Jeneret, J. De Favereau); De Visscher, S (De Visscher, S.); Delaere, C (Delaere, C.); Demin, P (Demin, P.); Favart, D (Favart, D.); Feltrin, E (Feltrin, E.); Forton, E (Forton, E.); Gregoire, G (Gregoire, G.); Kalinin, S (Kalinin, S.); Kcira, D (Kcira, D.); Keutgen, T (Keutgen, T.); Leibenguth, G (Leibenguth, G.); Lemaitre, V (Lemaitre, V.); Liu, Y (Liu, Y.); Michotte, D (Michotte, D.); Militaru, O (Militaru, O.); Ninane, A (Ninane, A.); Ovyn, S (Ovyn, S.); Pierzchala, T (Pierzchala, T.); Piotrzkowski, K (Piotrzkowski, K.); Roberfroid, V (Roberfroid, V.); Rouby, X (Rouby, X.); Teyssier, D (Teyssier, D.); Van der Aa, O (Van der Aa, O.); Vander Donckt, M (Vander Donckt, M.); Daubie, E (Daubie, E.); Herquet, P (Herquet, P.); Mollet, A (Mollet, A.); Romeyer, A (Romeyer, A.); Beaumont, W (Beaumont, W.); Cardaci, M (Cardaci, M.); De Langhe, E (De Langhe, E.); De Wolf, EA (De Wolf, E. A.); Rurua, L (Rurua, L.); Souza, MHG (Souza, M. H. G.); Oguri, V (Oguri, V.); Santoro, A (Santoro, A.); Sznajder, A (Sznajder, A.); Vaz, M (Vaz, M.); Gregores, EM (Gregores, E. M.); Novaes, SF (Novaes, S. F.); Anguelov, T (Anguelov, T.); Antchev, G (Antchev, G.); Atanasov, I (Atanasov, I.); Damgov, J (Damgov, J.); Darmenov, N (Darmenov, N.); Dimitrov, L (Dimitrov, L.); Genchev, V (Genchev, V.); Iaydjiev, P (Iaydjiev, P.); Panev, B (Panev, B.); Piperov, S (Piperov, S.); Stoykova, S (Stoykova, S.); Sultanov, G (Sultanov, G.); Vankov, I (Vankov, I.); Dimitrov, A (Dimitrov, A.); Kozhuharov, V (Kozhuharov, V.); Litov, L (Litov, L.); Makariev, M (Makariev, M.); Marinov, A (Marinov, A.); Marinova, E (Marinova, E.); Markov, S (Markov, S.); Mateev, M (Mateev, M.); Pavlov, B (Pavlov, B.); Petkov, P (Petkov, P.); Sabev, C (Sabev, C.); Stoynev, S (Stoynev, S.); Toteva, Z (Toteva, Z.); Verguilov, V (Verguilov, V.); Chen, GM (Chen, G. M.); Chen, HS (Chen, H. S.); He, KL (He, K. L.); Jiang, CH (Jiang, C. H.); Li, WG (Li, W. G.); Liu, HM (Liu, H. M.); Meng, X (Meng, X.); Shen, XY (Shen, X. Y.); Sun, HS (Sun, H. S.); Yang, M (Yang, M.); Zhao, WR (Zhao, W. R.); Zhuang, HL (Zhuang, H. L.); Ban, Y (Ban, Y.); Cai, J (Cai, J.); Liu, S (Liu, S.); Qian, SJ (Qian, S. J.); Yang, ZC (Yang, Z. C.); Ye, YL (Ye, Y. L.); Ying, J (Ying, J.); Wu, J (Wu, J.); Zhang, ZP (Zhang, Z. P.); Godinovic, N (Godinovic, N.); Puljak, I (Puljak, I.); Soric, I (Soric, I.); Antunovic, Z (Antunovic, Z.); Dzelalija, M (Dzelalija, M.); Marasovic, K (Marasovic, K.); Brigljevic, V (Brigljevic, V.); Ferencek, D (Ferencek, D.); Kadija, K (Kadija, K.); Morovic, S (Morovic, S.); Planinic, M (Planinic, M.); Nicolaou, C (Nicolaou, C.); Papadakis, A (Papadakis, A.); Razis, PA (Razis, P. A.); Tsiakkouri, D (Tsiakkouri, D.); Hektor, A (Hektor, A.); Kadastik, M (Kadastik, M.); Kannike, K (Kannike, K.); Lippmaa, E (Lippmaa, E.); Muntel, M (Muntel, M.); Raidal, M (Raidal, M.); Aarnio, PA (Aarnio, P. A.); Czellar, S (Czellar, S.); Haeggstroem, E (Haeggstroem, E.); Heikkinen, A (Heikkinen, A.); Harkonen, J (Harkonen, J.); Karimaki, V (Karimaki, V.); Kinnunen, R (Kinnunen, R.); Lampen, T (Lampen, T.); Lassila-Perini, K (Lassila-Perini, K.); Lehti, S (Lehti, S.); Linden, T (Linden, T.); Luukka, PR (Luukka, P. R.); Michal, S (Michal, S.); Maenpaa, T (Maenpaa, T.); Nysten, J (Nysten, J.); Stettler, M (Stettler, M.); Tuominen, E (Tuominen, E.); Tuominiemi, J (Tuominiemi, J.); Wendland, L (Wendland, L.); Tuuva, T (Tuuva, T.); Guillaud, JP (Guillaud, J. P.); Nedelec, P (Nedelec, P.); Sillou, D (Sillou, D.); Anfreville, M (Anfreville, M.); Beauceron, S (Beauceron, S.); Bougamont, E (Bougamont, E.); Bredy, P (Bredy, P.); Chipaux, R (Chipaux, R.); Dejardin, M (Dejardin, M.); Denegri, D (Denegri, D.); Descamps, J (Descamps, J.); Fabbro, B (Fabbro, B.); Faure, JL (Faure, J. L.); Ganjour, S (Ganjour, S.); Gentit, FX (Gentit, F. X.); Givernaud, A (Givernaud, A.); Gras, P (Gras, P.); de Monchenault, GH (de Monchenault, G. Hamel); Jarry, P (Jarry, P.); Kircher, F (Kircher, F.); Lemaire, MC (Lemaire, M. C.); Levesy, B (Levesy, B.); Locci, E (Locci, E.); Lottin, JP (Lottin, J. P.); Mandjavidze, I (Mandjavidze, I.); Mur, M (Mur, M.); Pasquetto, E (Pasquetto, E.); Payn, A (Payn, A.); Rander, J (Rander, J.); Reymond, JM (Reymond, J. M.); Rondeaux, F (Rondeaux, F.); Rosowsky, A (Rosowsky, A.); Sun, ZH (Sun, Z. H.); Verrecchia, P (Verrecchia, P.); Baffioni, S (Baffioni, S.); Beaudette, F (Beaudette, F.); Bercher, M (Bercher, M.); Berthon, U (Berthon, U.); Bimbot, S (Bimbot, S.); Bourotte, J (Bourotte, J.); Busson, P (Busson, P.); Cerutti, M (Cerutti, M.); Chamont, D (Chamont, D.); Charlot, C (Charlot, C.); Collard, C (Collard, C.); Decotigny, D (Decotigny, D.); Delmeire, E (Delmeire, E.); Dobrzynski, L (Dobrzynski, L.); Gaillac, AM (Gaillac, A. M.); Geerebaert, Y (Geerebaert, Y.); Gilly, J (Gilly, J.); Haguenauer, M (Haguenauer, M.); Karar, A (Karar, A.); Mathieu, A (Mathieu, A.); Milleret, G (Milleret, G.); Mine, P (Mine, P.); Paganini, P (Paganini, P.); Romanteau, T (Romanteau, T.); Semeniouk, I (Semeniouk, I.); Sirois, Y (Sirois, Y.); Berst, JD (Berst, J. D.); Brom, JM (Brom, J. M.); Didierjean, F (Didierjean, F.); Drouhin, F (Drouhin, F.); Fontaine, JC (Fontaine, J. C.); Goerlach, U (Goerlach, U.); Graehling, P (Graehling, P.); Gross, L (Gross, L.); Houchu, L (Houchu, L.); Juillot, P (Juillot, P.); Lounis, A (Lounis, A.); Maazouzi, C (Maazouzi, C.); Mangeol, D (Mangeol, D.); Olivetto, C (Olivetto, C.); Todorov, T (Todorov, T.); Van Hove, P (Van Hove, P.); Vintache, D (Vintache, D.); Ageron, M (Ageron, M.); Agram, JL (Agram, J. L.); Baulieu, G (Baulieu, G.); Bedjidian, M (Bedjidian, M.); Blaha, J (Blaha, J.); Bonnevaux, A (Bonnevaux, A.); Boudoul, G (Boudoul, G.); Chabanat, E (Chabanat, E.); Combaret, C (Combaret, C.); Contardo, D (Contardo, D.); Della Negra, R (Della Negra, R.); Depasse, P (Depasse, P.); Dupasquier, T (Dupasquier, T.); El Mamouni, H (El Mamouni, H.); Estre, N (Estre, N.); Fay, J (Fay, J.); Gascon, S (Gascon, S.); Giraud, N (Giraud, N.); Girerd, C (Girerd, C.); Haroutunian, R (Haroutunian, R.); Ianigro, JC (Ianigro, J. C.); Ille, B (Ille, B.); Lethuillier, M (Lethuillier, M.); Lumb, N (Lumb, N.); Mathez, H (Mathez, H.); Maurelli, G (Maurelli, G.); Mirabito, L (Mirabito, L.); Perries, S (Perries, S.); Ravat, O (Ravat, O.); Kvatadze, R (Kvatadze, R.); Roinishvili, V (Roinishvili, V.); Adolphi, R (Adolphi, R.); Brauer, R (Brauer, R.); Braunschweig, W (Braunschweig, W.); Esser, H (Esser, H.); Feld, L (Feld, L.); Heister, A (Heister, A.); Karpinski, W (Karpinski, W.); Klein, K (Klein, K.); Kukulies, C (Kukulies, C.); Olzem, J (Olzem, J.); Ostapchuk, A (Ostapchuk, A.); Pandoulas, D (Pandoulas, D.); Pierschel, G (Pierschel, G.); Raupach, F (Raupach, F.); Schael, S (Schael, S.); Schwering, G (Schwering, G.); Thomas, M (Thomas, M.); Weber, M (Weber, M.); Wittmer, B (Wittmer, B.); Wlochal, M (Wlochal, M.); Adolf, A (Adolf, A.); Biallass, P (Biallass, P.); Bontenackels, M (Bontenackels, M.); Erdmann, M (Erdmann, M.); Fesefeldt, H (Fesefeldt, H.); Hebbeker, T (Hebbeker, T.); Hermann, S (Hermann, S.); Hilgers, G (Hilgers, G.); Hoepfner, K (Hoepfner, K.); Hof, C (Hof, C.); Kappler, S (Kappler, S.); Kirsch, M (Kirsch, M.); Lanske, D (Lanske, D.); Philipps, B (Philipps, B.); Reithler, H (Reithler, H.); Rommerskirchen, T (Rommerskirchen, T.); Sowa, M (Sowa, M.); Szczesny, H (Szczesny, H.); Tonutti, M (Tonutti, M.); Tsigenov, O (Tsigenov, O.); Beissel, F (Beissel, F.); Davids, M (Davids, M.); Duda, M (Duda, M.); Flugge, G (Fluegge, G.); Franke, T (Franke, T.); Giffels, M (Giffels, M.); Hermanns, T (Hermanns, T.); Heydhausen, D (Heydhausen, D.); Kasselmann, S (Kasselmann, S.); Kaussen, G (Kaussen, G.); Kress, T (Kress, T.); Linn, A (Linn, A.); Nowack, A (Nowack, A.); Poettgens, M (Poettgens, M.); Pooth, O (Pooth, O.); Stahl, A (Stahl, A.); Tornier, D (Tornier, D.); Weber, M (Weber, M.); Flossdorf, A (Flossdorf, A.); Hegner, B (Hegner, B.); Mnich, J (Mnich, J.); Rosemann, C (Rosemann, C.); Flucke, G (Flucke, G.); Holm, U (Holm, U.); Klanner, R (Klanner, R.); Pein, U (Pein, U.); Schirm, N (Schirm, N.); Schleper, P (Schleper, P.); Steinbruck, G (Steinbrueck, G.); Stoye, M (Stoye, M.); Van Staa, R (Van Staa, R.); Wick, K (Wick, K.); Blum, P (Bluem, P.); Buege, V (Buege, V.); De Boer, W (De Boer, W.); Dirkes, G (Dirkes, G.); Fahrer, M (Fahrer, M.); Feindt, M (Feindt, M.); Felzmann, U (Felzmann, U.); Menendez, JF (Menendez, J. Fernandez); Frey, M (Frey, M.); Furgeri, A (Furgeri, A.); Hartmann, F (Hartmann, F.); Heier, S (Heier, S.); Jung, C (Jung, C.); Ledermann, B (Ledermann, B.); Muller, T (Mueller, Th.); Niegel, M (Niegel, M.); Oehler, A (Oehler, A.); Gomez, TO (Gomez, T. Ortega); Piasecki, C (Piasecki, C.); Quast, G (Quast, G.); Rabbertz, K (Rabbertz, K.); Saout, C (Saout, C.); Scheurer, A (Scheurer, A.); Schieferdecker, D (Schieferdecker, D.); Schmidt, A (Schmidt, A.); Simonis, HJ (Simonis, H. J.); Theel, A (Theel, A.); Vest, A (Vest, A.); Weiler, T (Weiler, T.); Weiser, C (Weiser, C.); Weng, J (Weng, J.); Zhukov, V (Zhukov, V.); Karapostoli, G (Karapostoli, G.); Katsas, P (Katsas, P.); Kreuzer, P (Kreuzer, P.); Panagiotou, A (Panagiotou, A.); Papadimitropoulos, C (Papadimitropoulos, C.); Anagnostou, G (Anagnostou, G.); Barone, M (Barone, M.); Geralis, T (Geralis, T.); Kalfas, C (Kalfas, C.); Koimas, A (Koimas, A.); Kyriakis, A (Kyriakis, A.); Kyriazopoulou, S (Kyriazopoulou, S.); Loukas, D (Loukas, D.); Markou, A (Markou, A.); Markou, C (Markou, C.); Mavrommatis, C (Mavrommatis, C.); Theofilatos, K (Theofilatos, K.); Vermisoglou, G (Vermisoglou, G.); Zachariadou, A (Zachariadou, A.); Aslanoglou, X (Aslanoglou, X.); Evangelou, I (Evangelou, I.); Kokkas, P (Kokkas, P.); Manthos, N (Manthos, N.); Papadopoulos, I (Papadopoulos, I.); Sidiropoulos, G (Sidiropoulos, G.); Triantis, FA (Triantis, F. A.); Bencze, G (Bencze, G.); Boldizsar, L (Boldizsar, L.); Hajdu, C (Hajdu, C.); Horvath, D (Horvath, D.); Laszlo, A (Laszlo, A.); Odor, G (Odor, G.); Sikler, F (Sikler, F.); Toth, N (Toth, N.); Vesztergombi, G (Vesztergombi, G.); Zalan, P (Zalan, P.); Molnar, J (Molnar, J.); Beni, N (Beni, N.); Kapusi, A (Kapusi, A.); Marian, G (Marian, G.); Raics, P (Raics, P.); Szabo, Z (Szabo, Z.); Szillasi, Z (Szillasi, Z.); Zilizi, G (Zilizi, G.); Bawa, HS (Bawa, H. S.); Beri, SB (Beri, S. B.); Bhandari, V (Bhandari, V.); Bhatnagar, V (Bhatnagar, V.); Kaur, M (Kaur, M.); Kaur, R (Kaur, R.); Kohli, JM (Kohli, J. M.); Kumar, A (Kumar, A.); Singh, JB (Singh, J. B.); Bhardwaj, A (Bhardwaj, A.); Bhattacharya, S (Bhattacharya, S.); Chatterji, S (Chatterji, S.); Chauhan, S (Chauhan, S.); Choudhary, BC (Choudhary, B. C.); Gupta, P (Gupta, P.); Jha, M (Jha, M.); Ranjan, K (Ranjan, K.); Shivpuri, RK (Shivpuri, R. K.); Srivastava, AK (Srivastava, A. K.); Borkar, S (Borkar, S.); Dixit, M (Dixit, M.); Ghodgaonkar, M (Ghodgaonkar, M.); Kataria, SK (Kataria, S. K.); Lalwani, SK (Lalwani, S. K.); Mishra, V (Mishra, V.); Mohanty, AK (Mohanty, A. K.); Topkar, A (Topkar, A.); Aziz, T (Aziz, T.); Banerjee, S (Banerjee, S.); Bose, S (Bose, S.); Cheere, N (Cheere, N.); Chendvankar, S (Chendvankar, S.); Deshpande, PV (Deshpande, P. V.); Guchait, M (Guchait, M.); Gurtu, A (Gurtu, A.); Maity, M (Maity, M.); Majumder, G (Majumder, G.); Mazumdar, K (Mazumdar, K.); Nayak, A (Nayak, A.); Patil, MR (Patil, M. R.); Sharma, S (Sharma, S.); Sudhakar, K (Sudhakar, K.); Tonwar, SC (Tonwar, S. C.); Acharya, BS (Acharya, B. S.); Banerjee, S (Banerjee, S.); Bheesette, S (Bheesette, S.); Dugad, S (Dugad, S.); Kalmani, SD (Kalmani, S. D.); Lakkireddi, VR (Lakkireddi, V. R.); Mondal, NK (Mondal, N. K.); Panyam, N (Panyam, N.); Verma, P (Verma, P.); Arabgol, M (Arabgol, M.); Arfaei, H (Arfaei, H.); Hashemi, M (Hashemi, M.); Mohammadi, M (Mohammadi, M.); Najafabadi, MM (Najafabadi, M. Mohammadi); Moshaii, A (Moshaii, A.); Mehdiabadi, SP (Mehdiabadi, S. Paktinat); Grunewald, M (Grunewald, M.); Abbrescia, M (Abbrescia, M.); Barbone, L (Barbone, L.); Colaleo, A (Colaleo, A.); Creanza, D (Creanza, D.); De Filippis, N (De Filippis, N.); De Palma, M (De Palma, M.); Donvito, G (Donvito, G.); Fiore, L (Fiore, L.); Giordano, D (Giordano, D.); Iaselli, G (Iaselli, G.); Loddo, F (Loddo, F.); Maggi, G (Maggi, G.); Maggi, M (Maggi, M.); Manna, N (Manna, N.); Marangelli, B (Marangelli, B.); Mennea, MS (Mennea, M. S.); My, S (My, S.); Natali, S (Natali, S.); Nuzzo, S (Nuzzo, S.); Pugliese, G (Pugliese, G.); Radicci, V (Radicci, V.); Ranieri, A (Ranieri, A.); Romano, F (Romano, F.); Selvaggi, G (Selvaggi, G.); Silvestris, L (Silvestris, L.); Tempesta, P (Tempesta, P.); Trentadue, R (Trentadue, R.); Zito, G (Zito, G.); Abbiendi, G (Abbiendi, G.); Bacchi, W (Bacchi, W.); Benvenuti, A (Benvenuti, A.); Bonacorsi, D (Bonacorsi, D.); Braibant-Giacomelli, S (Braibant-Giacomelli, S.); Capiluppi, P (Capiluppi, P.); Cavallo, FR (Cavallo, F. R.); Ciocca, C (Ciocca, C.); Codispoti, G (Codispoti, G.); D'Antone, I (D'Antone, I.); Dallavalle, GM (Dallavalle, G. M.); Fabbri, F (Fabbri, F.); Fanfani, A (Fanfani, A.); Giacomelli, P (Giacomelli, P.); Grandi, C (Grandi, C.); Guerzoni, M (Guerzoni, M.); Guiducci, L (Guiducci, L.); Marcellini, S (Marcellini, S.); Masetti, G (Masetti, G.); Montanari, A (Montanari, A.); Navarria, F (Navarria, F.); Odorici, F (Odorici, F.); Perrotta, A (Perrotta, A.); Rossi, A (Rossi, A.); Rovelli, T (Rovelli, T.); Siroli, G (Siroli, G.); Travaglini, R (Travaglini, R.); Albergo, S (Albergo, S.); Chiorboli, M (Chiorboli, M.); Costa, S (Costa, S.); Galanti, M (Galanti, M.); Rotondo, GG (Rotondo, G. Gatto); Noto, F (Noto, F.); Potenza, R (Potenza, R.); Russo, G (Russo, G.); Tricomi, A (Tricomi, A.); Tuve, C (Tuve, C.); Bocci, A (Bocci, A.); Ciraolo, G (Ciraolo, G.); Ciulli, V (Ciulli, V.); Civinini, C (Civinini, C.); D'Alessandro, R (D'Alessandro, R.); Focardi, E (Focardi, E.); Genta, C (Genta, C.); Lenzi, P (Lenzi, P.); Macchiolo, A (Macchiolo, A.); Magini, N (Magini, N.); Manolescu, F (Manolescu, F.); Marchettini, C (Marchettini, C.); Masetti, L (Masetti, L.); Mersi, S (Mersi, S.); Meschini, M (Meschini, M.); Paoletti, S (Paoletti, S.); Parrini, G (Parrini, G.); Ranieri, R (Ranieri, R.); Sani, M (Sani, M.); Fabbricatore, P (Fabbricatore, P.); Farinon, S (Farinon, S.); Greco, M (Greco, M.); Cattaneo, G (Cattaneo, G.); De Min, A (De Min, A.); Dominoni, M (Dominoni, M.); Farina, FM (Farina, F. M.); Ferri, F (Ferri, F.); Ghezzi, A (Ghezzi, A.); Govoni, P (Govoni, P.); Leporini, R (Leporini, R.); Magni, S (Magni, S.); Malberti, M (Malberti, M.); Malvezzi, S (Malvezzi, S.); Marelli, S (Marelli, S.); Menasce, D (Menasce, D.); Moroni, L (Moroni, L.); Negri, P (Negri, P.); Paganoni, M (Paganoni, M.); Pedrini, D (Pedrini, D.); Pullia, A (Pullia, A.); Ragazzi, S (Ragazzi, S.); Redaelli, N (Redaelli, N.); Rovelli, C (Rovelli, C.); Rovere, M (Rovere, M.); Sala, L (Sala, L.); Sala, S (Sala, S.); Salerno, R (Salerno, R.); Tabarelli de Fatis, T (Tabarelli de Fatis, T.); Vigano, S (Vigano, S.); Comunale, G (Comunale, G.); Fabozzi, F (Fabozzi, F.); Lomidze, D (Lomidze, D.); Mele, S (Mele, S.); Paolucci, P (Paolucci, P.); Piccolo, D (Piccolo, D.); Polese, G (Polese, G.); Sciacca, C (Sciacca, C.); Azzi, P (Azzi, P.); Bacchetta, N (Bacchetta, N.); Bellato, M (Bellato, M.); Benettoni, M (Benettoni, M.); Bisello, D (Bisello, D.); Borsato, E (Borsato, E.); Candelori, A (Candelori, A.); Checchia, P (Checchia, P.); Conti, E (Conti, E.); De Mattia, M (De Mattia, M.); Dorigo, T (Dorigo, T.); Drollinger, V (Drollinger, V.); Fanzago, F (Fanzago, F.); Gasparini, F (Gasparini, F.); Gasparini, U (Gasparini, U.); Giarin, M (Giarin, M.); Giubilato, P (Giubilato, P.); Gonella, F (Gonella, F.); Kaminskiy, A (Kaminskiy, A.); Karaevskii, S (Karaevskii, S.); Khomenkov, V (Khomenkov, V.); Lacaprara, S (Lacaprara, S.); Lippi, I (Lippi, I.); Loreti, M (Loreti, M.); Lytovchenko, O (Lytovchenko, O.); Mazzucato, M (Mazzucato, M.); Meneguzzo, AT (Meneguzzo, A. T.); Michelotto, M (Michelotto, M.); Montecassiano, F (Montecassiano, F.); Nigro, M (Nigro, M.); Passaseo, M (Passaseo, M.); Pegoraro, M (Pegoraro, M.); Rampazzo, G (Rampazzo, G.); Ronchese, P (Ronchese, P.); Torassa, E (Torassa, E.); Ventura, S (Ventura, S.); Zanetti, M (Zanetti, M.); Zotto, P (Zotto, P.); Zumerle, G (Zumerle, G.); Belli, G (Belli, G.); Berzano, U (Berzano, U.); De Vecchi, C (De Vecchi, C.); Guida, R (Guida, R.); Necchi, MM (Necchi, M. M.); Ratti, SP (Ratti, S. P.); Riccardi, C (Riccardi, C.); Sani, G (Sani, G.); Torre, P (Torre, P.); Vitulo, P (Vitulo, P.); Ambroglini, F (Ambroglini, F.); Babucci, E (Babucci, E.); Benedetti, D (Benedetti, D.); Biasini, M (Biasini, M.); Bilei, GM (Bilei, G. M.); Caponeri, B (Caponeri, B.); Checcucci, B (Checcucci, B.); Fano, L (Fano, L.); Lariccia, P (Lariccia, P.); Mantovani, G (Mantovani, G.); Passeri, D (Passeri, D.); Pioppi, M (Pioppi, M.); Placidi, P (Placidi, P.); Postolache, V (Postolache, V.); Ricci, D (Ricci, D.); Santocchia, A (Santocchia, A.); Servoli, L (Servoli, L.); Spiga, D (Spiga, D.); Azzurri, P (Azzurri, P.); Bagliesi, G (Bagliesi, G.); Basti, A (Basti, A.); Benucci, L (Benucci, L.); Bernardini, J (Bernardini, J.); Boccali, T (Boccali, T.); Borrello, L (Borrello, L.); Bosi, F (Bosi, F.); Calzolari, F (Calzolari, F.); Castaldi, R (Castaldi, R.); Cerri, C (Cerri, C.); Cucoanes, AS (Cucoanes, A. S.); D'Alfonso, M (D'Alfonso, M.); Dell'Orso, R (Dell'Orso, R.); Dutta, S (Dutta, S.); Foa, L (Foa, L.); Gennai, S (Gennai, S.); Giammanco, A (Giammanco, A.); Giassi, A (Giassi, A.); Kartashov, D (Kartashov, D.); Ligabue, F (Ligabue, F.); Linari, S (Linari, S.); Lomtadze, T (Lomtadze, T.); Lungu, GA (Lungu, G. A.); Mangano, B (Mangano, B.); Martinelli, G (Martinelli, G.); Massa, M (Massa, M.); Messineo, A (Messineo, A.); Moggi, A (Moggi, A.); Palla, F (Palla, F.); Palmonari, F (Palmonari, F.); Petrucciani, G (Petrucciani, G.); Raffaelli, F (Raffaelli, F.); Rizzi, A (Rizzi, A.); Sanguinetti, G (Sanguinetti, G.); Segneri, G (Segneri, G.); Sentenac, D (Sentenac, D.); Serban, AT (Serban, A. T.); Sguazzoni, G (Sguazzoni, G.); Slav, A (Slav, A.); Spagnolo, P (Spagnolo, P.); Tenchini, R (Tenchini, R.); Tonelli, G (Tonelli, G.); Venturi, A (Venturi, A.); Verdini, PG (Verdini, P. G.); Vos, M (Vos, M.); Baccaro, S (Baccaro, S.); Barone, L (Barone, L.); Bartoloni, A (Bartoloni, A.); Cavallari, F (Cavallari, F.); Costantini, S (Costantini, S.); Dafinei, I (Dafinei, I.); Del Re, D (Del Re, D.); Diemoz, M (Diemoz, M.); Gargiulo, C (Gargiulo, C.); Longo, E (Longo, E.); Meridiani, P (Meridiani, P.); Organtini, G (Organtini, G.); Rahatlou, S (Rahatlou, S.); Accomando, E (Accomando, E.); Arneodo, M (Arneodo, M.); Ballestrero, A (Ballestrero, A.); Bellan, R (Bellan, R.); Biino, C (Biino, C.); Bolognesi, S (Bolognesi, S.); Cartiglia, N (Cartiglia, N.); Cerminara, G (Cerminara, G.); Cordero, M (Cordero, M.); Costa, M (Costa, M.); Dellacasa, G (Dellacasa, G.); Demaria, N (Demaria, N.); Maina, E (Maina, E.); Mariotti, C (Mariotti, C.); Maselli, S (Maselli, S.); Mereu, P (Mereu, P.); Migliore, E (Migliore, E.); Monaco, V (Monaco, V.); Nervo, M (Nervo, M.); Obertino, MM (Obertino, M. M.); Pastrone, N (Pastrone, N.); Petrillo, G (Petrillo, G.); Romero, A (Romero, A.); Ruspa, M (Ruspa, M.); Sacchi, R (Sacchi, R.); Staiano, A (Staiano, A.); Trapani, PP (Trapani, P. P.); Belforte, S (Belforte, S.); Cossutti, F (Cossutti, F.); Della Ricca, G (Della Ricca, G.); Penzo, A (Penzo, A.); Cho, K (Cho, K.); Ham, SW (Ham, S. W.); Han, D (Han, D.); Kim, DH (Kim, D. H.); Kim, GN (Kim, G. N.); Kim, JC (Kim, J. C.); Kim, WY (Kim, W. Y.); Lee, MW (Lee, M. W.); Oh, SK (Oh, S. K.); Park, WH (Park, W. H.); Ro, SR (Ro, S. R.); Son, DC (Son, D. C.); Suh, JS (Suh, J. S.); Kim, JY (Kim, J. Y.); Jung, SY (Jung, S. Y.); Rhee, JT (Rhee, J. T.); Hong, BS (Hong, B. S.); Hong, SJ (Hong, S. J.); Lee, KS (Lee, K. S.); Park, I (Park, I.); Park, SK (Park, S. K.); Sim, KS (Sim, K. S.); Won, E (Won, E.); Kim, SB (Kim, S. B.); Moreno, SC (Carrillo Moreno, S.); Valdez, HC (Castilla Valdez, H.); Hernandez, AS (Sanchez Hernandez, A.); Ibarguen, HAS (Salazar Ibarguen, H. A.); Pineda, AM (Morelos Pineda, A.); Gray, RNC (Gray, R. N. C.); Krofcheck, D (Krofcheck, D.); Rodrigues, NB (Rodrigues, N. Bernardino); Butler, PH (Butler, P. H.); Williams, JC (Williams, J. C.); Aftab, Z (Aftab, Z.); Ahmad, M (Ahmad, M.); Ahmad, U (Ahmad, U.); Ahmed, I (Ahmed, I.); Jan, JA (Jan, J. Alam); Asghar, MI (Asghar, M. I.); Asghar, S (Asghar, S.); Hafeez, M (Hafeez, M.); Hoorani, HR (Hoorani, H. R.); Ibrahim, M (Ibrahim, M.); Iftikhar, M (Iftikhar, M.); Khan, MS (Khan, M. S.); Qaiser, N (Qaiser, N.); Rehman, I (Rehman, I.); Solaija, T (Solaija, T.); Toor, S (Toor, S.); Blocki, J (Blocki, J.); Cyz, A (Cyz, A.); Gladysz-Dziadus, E (Gladysz-Dziadus, E.); Mikocki, S (Mikocki, S.); Turnau, J (Turnau, J.); Wlodarczyk, Z (Wlodarczyk, Z.); Zychowski, P (Zychowski, P.); Bunkowski, K (Bunkowski, K.); Czyrkowski, H (Czyrkowski, H.); Dabrowski, R (Dabrowski, R.); Dominik, W (Dominik, W.); Doroba, K (Doroba, K.); Kalinowski, A (Kalinowski, A.); Konecki, M (Konecki, M.); Krolikowski, J (Krolikowski, J.); Kudla, IM (Kudla, I. M.); Pietrusinski, M (Pietrusinski, M.); Pozniak, K (Pozniak, K.); Zabolotny, W (Zabolotny, W.); Zych, P (Zych, P.); Bluj, M (Bluj, M.); Gokieli, R (Gokieli, R.); Goscilo, L (Goscilo, L.); Gorski, M (Gorski, M.); Nawrocki, K (Nawrocki, K.); Traczyk, P (Traczyk, P.); Wrochna, G (Wrochna, G.); Zalewski, P (Zalewski, P.); Alemany-Fernandez, R (Alemany-Fernandez, R.); Almeida, C (Almeida, C.); Almeida, N (Almeida, N.); Trindade, AA (Trindade, A. Araujo); Bordalo, P (Bordalo, P.); Rodrigues, PD (Rodrigues, P. Da Silva); Husejko, M (Husejko, M.); Jain, A (Jain, A.); Kazana, M (Kazana, M.); Musella, P (Musella, P.); Ramos, S (Ramos, S.); Da Silva, JR (Da Silva, J. Rasteiro); Ribeiro, PQ (Ribeiro, P. Q.); Santos, M (Santos, M.); Semiao, J (Semiao, J.); Silva, P (Silva, P.); Teixeira, I (Teixeira, I.); Teixeira, JP (Teixeira, J. P.); Varela, J (Varela, J.); Afanasiev, S (Afanasiev, S.); Babich, K (Babich, K.); Belotelov, I (Belotelov, I.); Elsha, V (Elsha, V.); Ershov, Y (Ershov, Y.); Filozova, I (Filozova, I.); Golunov, A (Golunov, A.); Golutvin, I (Golutvin, I.); Gorbounov, N (Gorbounov, N.); Gramenitski, I (Gramenitski, I.); Kalagin, V (Kalagin, V.); Kamenev, A (Kamenev, A.); Karjavin, V (Karjavin, V.); Khabarov, S (Khabarov, S.); Khabarov, V (Khabarov, V.); Kiryushin, Y (Kiryushin, Y.); Konoplyanikov, V (Konoplyanikov, V.); Korenkov, V (Korenkov, V.); Kozlov, G (Kozlov, G.); Kurenkov, A (Kurenkov, A.); Lanev, A (Lanev, A.); Lysiakov, V (Lysiakov, V.); Malakhov, A (Malakhov, A.); Melnitchenko, I (Melnitchenko, I.); Mitsyn, VV (Mitsyn, V. V.); Moisenz, K (Moisenz, K.); Moisenz, P (Moisenz, P.); Movchan, S (Movchan, S.); Nikonov, E (Nikonov, E.); Oleynik, D (Oleynik, D.); Palichik, V (Palichik, V.); Perelygin, V (Perelygin, V.); Petrosyan, A (Petrosyan, A.); Rogalev, E (Rogalev, E.); Samsonov, V (Samsonov, V.); Savina, M (Savina, M.); Semenov, R (Semenov, R.); Shmatov, S (Shmatov, S.); Shulha, S (Shulha, S.); Smirnov, V (Smirnov, V.); Smolin, D (Smolin, D.); Tcheremoukhine, A (Tcheremoukhine, A.); Teryaev, O (Teryaev, O.); Tikhonenko, E (Tikhonenko, E.); Vassiliev, S (Vassiliev, S.); Vishnevskiy, A (Vishnevskiy, A.); Volodko, A (Volodko, A.); Zamiatin, N (Zamiatin, N.); Zarubin, A (Zarubin, A.); Zarubin, P (Zarubin, P.); Zubarev, E (Zubarev, E.); Bondar, N (Bondar, N.); Golovtsov, V (Golovtsov, V.); Golyash, A (Golyash, A.); Ivanov, Y (Ivanov, Y.); Kim, V (Kim, V.); Kozlov, V (Kozlov, V.); Lebedev, V (Lebedev, V.); Makarenkov, G (Makarenkov, G.); Orishchin, E (Orishchin, E.); Shevel, A (Shevel, A.); Sknar, V (Sknar, V.); Smirnov, I (Smirnov, I.); Sulimov, V (Sulimov, V.); Tarakanov, V (Tarakanov, V.); Uvarov, L (Uvarov, L.); Velichko, G (Velichko, G.); Volkov, S (Volkov, S.); Vorobyev, A (Vorobyev, A.); Andreev, Y (Andreev, Yu); Anisimov, A (Anisimov, A.); Gninenko, S (Gninenko, S.); Golubev, N (Golubev, N.); Gorbunov, D (Gorbunov, D.); Kirsanov, M (Kirsanov, M.); Kovzelev, A (Kovzelev, A.); Krasnikov, N (Krasnikov, N.); Matveev, V (Matveev, V.); Pashenkov, A (Pashenkov, A.); Postoev, VE (Postoev, V. E.); Sadovski, A (Sadovski, A.); Solovey, A (Solovey, A.); Solovey, A (Solovey, A.); Soloviev, D (Soloviev, D.); Stepanova, L (Stepanova, L.); Toropin, A (Toropin, A.); Gavrilov, V (Gavrilov, V.); Ilina, N (Ilina, N.); Kaftanov, V (Kaftanov, V.); Kiselevich, I (Kiselevich, I.); Kolosov, V (Kolosov, V.); Kossov, M (Kossov, M.); Krokhotin, A (Krokhotin, A.); Kuleshov, S (Kuleshov, S.); Oulianov, A (Oulianov, A.); Safronov, G (Safronov, G.); Semenov, S (Semenov, S.); Stolin, V (Stolin, V.); Vlasov, E (Vlasov, E.); Zaytsev, V (Zaytsev, V.); Fomenko, AM (Fomenko, A. M.); Konovalova, N (Konovalova, N.); Kozlov, V (Kozlov, V.); Lebedev, AI (Lebedev, A. I.); Lvova, N (Lvova, N.); Rusakov, SV (Rusakov, S. V.); Terkulov, A (Terkulov, A.); Boos, E (Boos, E.); Dubinin, M (Dubinin, M.); Dudko, L (Dudko, L.); Ershov, A (Ershov, A.); Gribushin, A (Gribushin, A.); Ilyin, V (Ilyin, V.); Klyukhin, V (Klyukhin, V.); Kodolova, O (Kodolova, O.); Lokhtin, I (Lokhtin, I.); Petrushanko, S (Petrushanko, S.); Sarycheva, L (Sarycheva, L.); Savrin, V (Savrin, V.); Sherstnev, A (Sherstnev, A.); Snigirev, A (Snigirev, A.); Teplov, K (Teplov, K.); Vardanyan, I (Vardanyan, I.); Abramov, V (Abramov, V.); Azhguirei, I (Azhguirei, I.); Bitioukov, S (Bitioukov, S.); Datsko, K (Datsko, K.); Filine, A (Filine, A.); Goncharov, P (Goncharov, P.); Grishin, V (Grishin, V.); Inyakin, A (Inyakin, A.); Kachanov, V (Kachanov, V.); Khmelnikov, A (Khmelnikov, A.); Konstantinov, D (Konstantinov, D.); Korablev, A (Korablev, A.); Krychkine, V (Krychkine, V.); Levine, A (Levine, A.); Lobov, I (Lobov, I.); Petrov, V (Petrov, V.); Pikalov, V (Pikalov, V.); Ryutin, R (Ryutin, R.); Slabospitsky, S (Slabospitsky, S.); Sourkov, A (Sourkov, A.); Sytine, A (Sytine, A.); Tourtchanovitch, L (Tourtchanovitch, L.); Troshin, S (Troshin, S.); Tyurin, N (Tyurin, N.); Uzunian, A (Uzunian, A.); Volkov, A (Volkov, A.); Zelepoukine, S (Zelepoukine, S.); Adzic, P (Adzic, P.); Krpic, D (Krpic, D.); Maletic, D (Maletic, D.); Milenovic, P (Milenovic, P.); Puzovic, J (Puzovic, J.); Smiljkovic, N (Smiljkovic, N.); Zupan, M (Zupan, M.); Aguilar-Benitez, M (Aguilar-Benitez, M.); Alberdi, J (Alberdi, J.); Maestre, JA (Alcaraz Maestre, J.); Martin, MA (Aldaya Martin, M.); Arce, P (Arce, P.); Barcala, JM (Barcala, J. M.); Lazaro, CB (Burgos Lazaro, C.); Bejar, JC (Caballero Bejar, J.); Calvo, E (Calvo, E.); Montes, MC (Cardenas Montes, M.); Cerrada, M (Cerrada, M.); Llatas, MC (Chamizo Llatas, M.); Colino, N (Colino, N.); Daniel, M (Daniel, M.); De la Cruz, B (De la Cruz, B.); Bedoya, CF (Fernandez Bedoya, C.); Ferrando, A (Ferrando, A.); Fouz, MC (Fouz, M. C.); Garcia-Abia, P (Garcia-Abia, P.); Hernandez, JM (Hernandez, J. M.); Josa, MI (Josa, M. I.); Luque, JM (Luque, J. M.); Marin, J (Marin, J.); Merino, G (Merino, G.); Molinero, A (Molinero, A.); Navarrete, JJ (Navarrete, J. J.); Oller, JC (Oller, J. C.); Calle, EP (Perez Calle, E.); Romero, L (Romero, L.); Salicio, J (Salicio, J.); Munoz, CV (Villanueva Munoz, C.); Willmott, C (Willmott, C.); Yuste, C (Yuste, C.); Albajar, C (Albajar, C.); de Troconiz, JF (de Troconiz, J. F.); Fernandez, M (Fernandez, M.); Jimenez, I (Jimenez, I.); Teixeira, RF (Teixeira, R. F.); Cuevas, J (Cuevas, J.); Lopez, JM (Lopez, J. M.); Sordo, HN (Naves Sordo, H.); Garcia, JMV (Vizan Garcia, J. M.); Calderon, A (Calderon, A.); Fernandez, DC (Cano Fernandez, D.); Merino, ID (Diaz Merino, I.); Moral, LAG (Garcia Moral, L. A.); Gomezo, G (Gomezo, G.); Cabellero, IG (Gonzalez Cabellero, I.); Sanchez, JG (Gonzalez Sanchez, J.); Virto, AL (Lopez Virto, A.); Marco, J (Marco, J.); Marco, R (Marco, R.); Rivero, CM (Martinez Rivero, C.); del Arbol, PMR (Ruiz del Arbol, P. Martinez); Matorras, F (Matorras, F.); Revuelta, AP (Patino Revuelta, A.); Rodrigo, T (Rodrigo, T.); Gonzalez, DR (Rodriguez Gonzalez, D.); Jimeno, AR (Ruiz Jimeno, A.); Sanudo, MS (Sobron Sanudo, M.); Vila, I (Vila, I.); Cortabitarte, RV (Vilar Cortabitarte, R.); Abbaneo, D (Abbaneo, D.); Abbas, SM (Abbas, S. M.); Agostino, L (Agostino, L.); Ahmed, I (Ahmed, I.); Akhtar, S (Akhtar, S.); Amapane, N (Amapane, N.); Meleiro, BA (Meleiro, B. Araujo); Argiro, S (Argiro, S.); Ashby, S (Ashby, S.); Aspell, P (Aspell, P.); Auffray, E (Auffray, E.); Axer, M (Axer, M.); Ball, A (Ball, A.); Bangert, N (Bangert, N.); Barney, D (Barney, D.); Bernet, C (Bernet, C.); Bialas, W (Bialas, W.); Bloch, C (Bloch, C.); Bloch, P (Bloch, P.); Bonacini, S (Bonacini, S.); Bosteels, M (Bosteels, M.); Boyer, V (Boyer, V.); Branson, A (Branson, A.); Brett, AM (Brett, A. M.); Breuker, H (Breuker, H.); Bruneliere, R (Bruneliere, R.); Buchmuller, O (Buchmuller, O.); Campi, D (Campi, D.); Camporesi, T (Camporesi, T.); Cano, E (Cano, E.); Carrone, E (Carrone, E.); Cattai, A (Cattai, A.); Chierici, R (Chierici, R.); Christiansen, T (Christiansen, T.); Cittolin, S (Cittolin, S.); Corrin, E (Corrin, E.); Corvo, M (Corvo, M.); Cucciarelli, S (Cucciarelli, S.); Cure, B (Cure, B.); De Roeck, A (De Roeck, A.); Delikaris, D (Delikaris, D.); Della Negra, M (Della Negra, M.); D'Enterria, D (D'Enterria, D.); Dierlamm, A (Dierlamm, A.); Elliott-Peisert, A (Elliott-Peisert, A.); Eppard, M (Eppard, M.); Foeth, H (Foeth, H.); Folch, R (Folch, R.); Fratianni, S (Fratianni, S.); Wfunk, O (Wfunk, O.); Gaddi, A (Gaddi, A.); Gastal, M (Gastal, M.); Gayde, JC (Gayde, J. C.); Gerwig, H (Gerwig, H.); Gill, K (Gill, K.); Giolo-Nicollerat, AS (Giolo-Nicollerat, A. S.); Glege, F (Glege, F.); Garrido, RGR (Garrido, R. Gomez-Reino); Goudard, R (Goudard, R.); Gutleber, J (Gutleber, J.); Hansen, M (Hansen, M.); Hartert, J (Hartert, J.); Herve, A (Herve, A.); Hoffmann, HF (Hoffmann, H. F.); Honma, A (Honma, A.); Huhtinen, M (Huhtinen, M.); Iles, G (Iles, G.); Innocente, V (Innocente, V.); Jank, W (Jank, W.); Janot, P (Janot, P.); Kloukinas, K (Kloukinas, K.); Lasseur, C (Lasseur, C.); Lebeau, M (Lebeau, M.); Lecoq, P (Lecoq, P.); Leonidopoulos, C (Leonidopoulos, C.); Letheren, M (Letheren, M.); Ljuslin, C (Ljuslin, C.); Loos, R (Loos, R.); Magazzu, G (Magazzu, G.); Malgeri, L (Malgeri, L.); Mannelli, M (Mannelli, M.); Marchioro, A (Marchioro, A.); Meijers, F (Meijers, F.); Meschi, E (Meschi, E.); Moser, R (Moser, R.); Mulders, M (Mulders, M.); Nash, J (Nash, J.); Ofierzynski, RA (Ofierzynski, R. A.); Oh, A (Oh, A.); Olbrechts, P (Olbrechts, P.); Onnela, A (Onnela, A.); Orsini, L (Orsini, L.); Pal, I (Pal, I.); Papotti, G (Papotti, G.); Paramatti, R (Paramatti, R.); Passardi, G (Passardi, G.); Solano, BP (Solano, B. Perea); Perinic, G (Perinic, G.); Petagna, P (Petagna, P.); Petrilli, A (Petrilli, A.); Pfeiffer, A (Pfeiffer, A.); Pimia, M (Pimiae, M.); Pintus, R (Pintus, R.); Postema, H (Postema, H.); Principe, R (Principe, R.); Pelayo, JP (Pelayo, J. Puerta); Racz, A (Racz, A.); Rehn, J (Rehn, J.); Reynaud, S (Reynaud, S.); Risoldi, M (Risoldi, M.); Moreira, PRS (Moreira, P. Rodrigues Simoes); Rolandi, G (Rolandi, G.); Rosinsky, P (Rosinsky, P.); Rumerio, P (Rumerio, P.); Sakulin, H (Sakulin, H.); Samyn, D (Samyn, D.); Schilling, FP (Schilling, F. P.); Schwick, C (Schwick, C.); Schafer, C (Schaefer, C.); Segoni, I (Segoni, I.); Sharma, A (Sharma, A.); Siegrist, P (Siegrist, P.); Sinanis, N (Sinanis, N.); Sphicas, P (Sphicas, P.); Spiropulu, M (Spiropulu, M.); Szoncso, F (Szoncso, F.); Teller, O (Teller, O.); Treille, D (Treille, D.); Troska, J (Troska, J.); Tsesmelis, E (Tsesmelis, E.); Tsirigkas, D (Tsirigkas, D.); Tsirou, A (Tsirou, A.); Ungaro, D (Ungaro, D.); Vasey, F (Vasey, F.); Acosta, MV (Acosta, M. Vazquez); Veillet, L (Veillet, L.); Vichoudis, P (Vichoudis, P.); Wertelaers, P (Wertelaers, P.); Wijnant, A (Wijnant, A.); Wilhelmsson, M (Wilhelmsson, M.); Willers, IM (Willers, I. M.); Bertl, W (Bertl, W.); Deiters, K (Deiters, K.); Erdmann, W (Erdmann, W.); Gabathuler, K (Gabathuler, K.); Heising, S (Heising, S.); Horisberger, R (Horisberger, R.); Ingram, Q (Ingram, Q.); Kaestli, HC (Kaestli, H. C.); Kotlinski, D (Kotlinski, D.); Konig, S (Koenig, S.); Renker, D (Renker, D.); Rohe, T (Rohe, T.); Spira, M (Spira, M.); Betev, B (Betev, B.); Davatz, G (Davatz, G.); Dissertori, G (Dissertori, G.); Dittmar, M (Dittmar, M.); Djambazov, L (Djambazov, L.); Ehlers, J (Ehlers, J.); Eichler, R (Eichler, R.); Faber, G (Faber, G.); Freudenreich, K (Freudenreich, K.); Fuchs, JF (Fuchs, J. F.); Grab, C (Grab, C.); Holzner, A (Holzner, A.); Ingenito, P (Ingenito, P.); Langenegger, U (Langenegger, U.); Lecomte, P (Lecomte, P.); Leshev, G (Leshev, G.); Lister, A (Lister, A.); Luckey, PD (Luckey, P. D.); Lustermann, W (Lustermann, W.); Maillefaud, JD (Maillefaud, J. D.); Moortgat, F (Moortgat, F.); Nardulli, A (Nardulli, A.); Nessi-Tedaldi, F (Nessi-Tedaldi, F.); Pape, L (Pape, L.); Pauss, F (Pauss, F.); Rykaczewski, H (Rykaczewski, H.); Roser, U (Roeser, U.); Schinzel, D (Schinzel, D.); Starodumov, A (Starodumov, A.); Stockli, F (Stoeckli, F.); Suter, H (Suter, H.); Tauscher, L (Tauscher, L.); Trub, P (Trueb, P.); von Gunten, HP (von Gunten, H. P.); Wensveen, M (Wensveen, M.); Alagoz, E (Alagoz, E.); Amsler, C (Amsler, C.); Chiochia, V (Chiochia, V.); Hoermann, C (Hoermann, C.); Prokofiev, K (Prokofiev, K.); Regenfus, C (Regenfus, C.); Robmann, P (Robmann, P.); Speer, T (Speer, T.); Steiner, S (Steiner, S.); Wilke, L (Wilke, L.); Blyth, S (Blyth, S.); Chang, YH (Chang, Y. H.); Chen, EA (Chen, E. A.); Go, A (Go, A.); Hung, CC (Hung, C. C.); Kuo, CM (Kuo, C. M.); Lin, W (Lin, W.); Chang, P (Chang, P.); Chao, Y (Chao, Y.); Chen, KF (Chen, K. F.); Gao, Z (Gao, Z.); Hsiung, Y (Hsiung, Y.); Lei, YJ (Lei, Y. J.); Schumann, J (Schumann, J.); Shiu, JG (Shiu, J. G.); Ueno, K (Ueno, K.); Velikzhanin, Y (Velikzhanin, Y.); Yeh, P (Yeh, P.); Aydin, S (Aydin, S.); Bakirci, MN (Bakirci, M. N.); Cerci, S (Cerci, S.); Dumanoglu, I (Dumanoglu, I.); Erturk, S (Erturk, S.); Esen, S (Esen, S.); Eskut, E (Eskut, E.); Topaksu, AK (Topaksu, A. Kayis); Kurt, P (Kurt, P.); Ozkurt, H (Ozkurt, H.); Polatoz, A (Polatoz, A.); Sogut, K (Sogut, K.); Topakli, H (Topakli, H.); Vergili, M (Vergili, M.); Yetkin, T (Yetkin, T.); Onengut, G (Onengut, G.); Gamsizkan, H (Gamsizkan, H.); Ozkan, C (Ozkan, C.); Sekmen, S (Sekmen, S.); Serin-Zeyrek, M (Serin-Zeyrek, M.); Sever, R (Sever, R.); Yazgan, E (Yazgan, E.); Zeyrek, M (Zeyrek, M.); Cakir, A (Cakir, A.); Cankocak, K (Cankocak, K.); Deliomeroglu, M (Deliomeroglu, M.); Demir, D (Demir, D.); Dindar, K (Dindar, K.); Gulmez, E (Gulmez, E.); Isiksal, E (Isiksal, E.); Kaya, M (Kaya, M.); Kaya, O (Kaya, O.); Ozkorucuklu, S (Ozkorucuklu, S.); Sonmez, N (Sonmez, N.); Grinev, B (Grinev, B.); Lyubynskiy, V (Lyubynskiy, V.); Senchyshyn, V (Senchyshyn, V.); Levchuk, L (Levchuk, L.); Sorokin, P (Sorokin, P.); Bailey, DS (Bailey, D. S.); Barrass, T (Barrass, T.); Brooke, JJ (Brooke, J. J.); Croft, R (Croft, R.); Cussans, D (Cussans, D.); Evans, D (Evans, D.); Frazier, R (Frazier, R.); Grant, N (Grant, N.); Hansen, M (Hansen, M.); Heath, GP (Heath, G. P.); Heath, HF (Heath, H. F.); Huckvale, B (Huckvale, B.); Lynch, C (Lynch, C.); Mackay, CK (Mackay, C. K.); Metson, S (Metson, S.); Newbold, DM (Newbold, D. M.); Smith, VJ (Smith, V. J.); Tapper, RJ (Tapper, R. J.); Baird, SA (Baird, S. A.); Bell, KW (Bell, K. W.); Brown, RM (Brown, R. M.); Cockerill, DJA (Cockerill, D. J. A.); Coughlan, JA (Coughlan, J. A.); Flower, PS (Flower, P. S.); Francis, VB (Francis, V. B.); French, M (French, M.); Greenhalgh, J (Greenhalgh, J.); Halsall, R (Halsall, R.); Hill, J (Hill, J.); Jones, L (Jones, L.); Kennedy, BW (Kennedy, B. W.); Lintern, L (Lintern, L.); Lodge, AB (Lodge, A. B.); Maddox, J (Maddox, J.); Morrissey, Q (Morrissey, Q.); Murray, P (Murray, P.); Pearson, M (Pearson, M.); Quinton, S (Quinton, S.); Salisbury, J (Salisbury, J.); Shah, A (Shah, A.); Shepherd-Themistocleous, C (Shepherd-Themistocleous, C.); Smith, B (Smith, B.); Sproston, M (Sproston, M.); Stephenson, R (Stephenson, R.); Taghavirad, S (Taghavirad, S.); Tomalin, IR (Tomalin, I. R.); Williams, JH (Williams, J. H.); Arteche, F (Arteche, F.); Bainbridge, R (Bainbridge, R.); Barber, G (Barber, G.); Barrillon, P (Barrillon, P.); Beuselinck, R (Beuselinck, R.); Blekman, F (Blekman, F.); Britton, D (Britton, D.); Colling, D (Colling, D.); Daskalakis, G (Daskalakis, G.); Dewhirst, G (Dewhirst, G.); Dris, S (Dris, S.); Foudas, C (Foudas, C.); Fulcher, J (Fulcher, J.); Greder, S (Greder, S.); Hall, G (Hall, G.); Jones, J (Jones, J.); Leaver, J (Leaver, J.); MacEvoy, BC (MacEvoy, B. C.); Maroney, O (Maroney, O.); Nikitenko, A (Nikitenko, A.); Papageorgiou, A (Papageorgiou, A.); Raymond, DM (Raymond, D. M.); Ryan, MJ (Ryan, M. J.); Seez, C (Seez, C.); Sharp, P (Sharp, P.); Takahashi, M (Takahashi, M.); Timlin, C (Timlin, C.); Virdee, T (Virdee, T.); Wakefield, S (Wakefield, S.); Wingham, M (Wingham, M.); Zabi, A (Zabi, A.); Zhang, Y (Zhang, Y.); Zorba, O (Zorba, O.); Da Via, C (Da Via, C.); Goitom, I (Goitom, I.); Hobson, PR (Hobson, P. R.); Kyberd, P (Kyberd, P.); Munro, C (Munro, C.); Nebrensky, J (Nebrensky, J.); Reid, I (Reid, I.); Sharif, O (Sharif, O.); Taylor, R (Taylor, R.); Teodorescu, L (Teodorescu, L.); Watts, SJ (Watts, S. J.); Yaselli, I (Yaselli, I.); Hazen, E (Hazen, E.); Heering, AH (Heering, A. H.); Lazic, D (Lazic, D.); Machado, E (Machado, E.); Osborne, D (Osborne, D.); Rohlf, J (Rohlf, J.); Sulak, L (Sulak, L.); Rodriguez, FV (Rodriguez, F. Varela); Wu, S (Wu, S.); Cutts, D (Cutts, D.); Hooper, R (Hooper, R.); Landsberg, G (Landsberg, G.); Partridge, R (Partridge, R.); Vanini, S (Vanini, S.); Breedon, R (Breedon, R.); Case, M (Case, M.); Chertok, M (Chertok, M.); Conway, J (Conway, J.); Cox, PT (Cox, P. T.); Erbacher, R (Erbacher, R.); Gunion, J (Gunion, J.); Holbrook, B (Holbrook, B.); Ko, W (Ko, W.); Lander, R (Lander, R.); Pellett, D (Pellett, D.); Smith, J (Smith, J.); Soha, A (Soha, A.); Tripathi, M (Tripathi, M.); Vogt, R (Vogt, R.); Andreev, V (Andreev, V.); Arisaka, K (Arisaka, K.); Cline, D (Cline, D.); Cousins, R (Cousins, R.); Erhan, S (Erhan, S.); Felcini, M (Felcini, M.); Hauser, J (Hauser, J.); Ignatenko, M (Ignatenko, M.); Lisowski, B (Lisowski, B.); Matlock, D (Matlock, D.); Matthey, C (Matthey, C.); Mohr, B (Mohr, B.); Mumford, J (Mumford, J.); Otwinowski, S (Otwinowski, S.); Rakness, G (Rakness, G.); Schlein, P (Schlein, P.); Shi, Y (Shi, Y.); Tucker, J (Tucker, J.); Valuev, V (Valuev, V.); Wallny, R (Wallny, R.); Wang, HG (Wang, H. G.); Yang, X (Yang, X.); Zheng, Y (Zheng, Y.); Clare, R (Clare, R.); Fortin, D (Fortin, D.); Futyan, D (Futyan, D.); Gary, JW (Gary, J. W.); Giunta, M (Giunta, M.); Hanson, G (Hanson, G.); Jeng, GY (Jeng, G. Y.); Kao, SC (Kao, S. C.); Liu, H (Liu, H.); Pasztor, G (Pasztor, G.); Satpathy, A (Satpathy, A.); Shen, BC (Shen, B. C.); Stringer, R (Stringer, R.); Sytnik, V (Sytnik, V.); Wilken, R (Wilken, R.); Zer-Zion, D (Zer-Zion, D.); Branson, JG (Branson, J. G.); Dusinberre, E (Dusinberre, E.); Letts, J (Letts, J.); Martin, T (Martin, T.); Mojaver, M (Mojaver, M.); Paar, HP (Paar, H. P.); Pi, H (Pi, H.); Pieri, M (Pieri, M.); Rana, A (Rana, A.); Sharma, V (Sharma, V.); White, A (White, A.); Wurthwein, F (Wurthwein, F.); Affolder, A (Affolder, A.); Campagnari, C (Campagnari, C.); Hill, C (Hill, C.); Incandela, J (Incandela, J.); Kyre, S (Kyre, S.); Lamb, J (Lamb, J.); Richman, J (Richman, J.); Stuart, D (Stuart, D.); White, D (White, D.); Albert, J (Albert, J.); Bornheim, A (Bornheim, A.); Bunn, J (Bunn, J.); Chen, J (Chen, J.); Denis, G (Denis, G.); Galvez, P (Galvez, P.); Gataullin, M (Gataullin, M.); Legrand, I (Legrand, I.); Litvine, V (Litvine, V.); Ma, Y (Ma, Y.); Nae, D (Nae, D.); Newman, HB (Newman, H. B.); Ravot, S (Ravot, S.); Shevchenko, S (Shevchenko, S.); Singh, S (Singh, S.); Steenberg, C (Steenberg, C.); Su, X (Su, X.); Thomas, M (Thomas, M.); Timciuc, V (Timciuc, V.); van Lingen, F (van Lingen, F.); Veverka, J (Veverka, J.); Voicu, BR (Voicu, B. R.); Weinstein, A (Weinstein, A.); Wilkinson, R (Wilkinson, R.); Yang, X (Yang, X.); Yang, Y (Yang, Y.); Zhang, LY (Zhang, L. Y.); Zhu, K (Zhu, K.); Zhu, RY (Zhu, R. Y.); Ferguson, T (Ferguson, T.); Paulini, M (Paulini, M.); Russ, J (Russ, J.); Terentyev, N (Terentyev, N.); Vogel, H (Vogel, H.); Vorobiev, I (Vorobiev, I.); Cumalat, JP (Cumalat, J. P.); Ford, WT (Ford, W. T.); Johnson, D (Johnson, D.); Nauenberg, U (Nauenberg, U.); Stenson, K (Stenson, K.); Wagner, SR (Wagner, S. R.); Alexander, J (Alexander, J.); Cassel, D (Cassel, D.); Ecklund, K (Ecklund, K.); Heltsley, B (Heltsley, B.); Jones, CD (Jones, C. D.); Kuznetsov, V (Kuznetsov, V.); Patterson, JR (Patterson, J. R.); Ryd, A (Ryd, A.); Thom, J (Thom, J.); Wittich, P (Wittich, P.); Beetz, CP (Beetz, C. P.); Cirino, G (Cirino, G.); Podrasky, V (Podrasky, V.); Sanzeni, C (Sanzeni, C.); Winn, D (Winn, D.); Abdullin, S (Abdullin, S.); Afaq, MA (Afaq, M. A.); Albrow, M (Albrow, M.); Amundson, J (Amundson, J.); Apollinari, G (Apollinari, G.); Atac, M (Atac, M.); Badgett, W (Badgett, W.); Bakken, JA (Bakken, J. A.); Baldin, B (Baldin, B.); Bauerdick, LAT (Bauerdick, L. A. T.); Baumbaugh, A (Baumbaugh, A.); Baur, U (Baur, U.); Bhat, PC (Bhat, P. C.); Borcherding, F (Borcherding, F.); Burkett, K (Burkett, K.); Butler, JN (Butler, J. N.); Cheung, H (Cheung, H.); Churin, I (Churin, I.); Cihangir, S (Cihangir, S.); Demarteau, M (Demarteau, M.); Eartly, DP (Eartly, D. P.); Elias, JE (Elias, J. E.); Elvira, VD (Elvira, V. D.); Evans, D (Evans, D.); Fisk, I (Fisk, I.); Freeman, J (Freeman, J.); Gartung, P (Gartung, P.); Geurts, FJM (Geurts, F. J. M.); Glenzinski, DA (Glenzinski, D. A.); Gottschalk, E (Gottschalk, E.); Graham, G (Graham, G.); Green, D (Green, D.); Guglielmo, GM (Guglielmo, G. M.); Guo, Y (Guo, Y.); Gutsche, O (Gutsche, O.); Hahn, A (Hahn, A.); Hanlon, J (Hanlon, J.); Hansen, S (Hansen, S.); Harris, RM (Harris, R. M.); Hesselroth, T (Hesselroth, T.); Holm, SL (Holm, S. L.); Holzman, B (Holzman, B.); Iqbal, S (Iqbal, S.); James, E (James, E.); Johnson, M (Johnson, M.); Joshi, U (Joshi, U.); Klima, B (Klima, B.); Kowalkowski, J (Kowalkowski, J.); Kramer, T (Kramer, T.); Kwan, S (Kwan, S.); La Vallie, E (La Vallie, E.); Larwill, M (Larwill, M.); Los, S (Los, S.); Lueking, L (Lueking, L.); Lukhanin, G (Lukhanin, G.); Lusin, S (Lusin, S.); Maeshima, K (Maeshima, K.); McBride, P (McBride, P.); Murray, SJ (Murray, S. J.); O'Dell, V (O'Dell, V.); Paterno, M (Paterno, M.); Patrick, J (Patrick, J.); Petravick, D (Petravick, D.); Pordes, R (Pordes, R.); Prokofyev, O (Prokofyev, O.); Rasmislovich, V (Rasmislovich, V.); Ratnikova, N (Ratnikova, N.); Ronzhin, A (Ronzhin, A.); Sekhri, V (Sekhri, V.); Sexton-Kennedy, E (Sexton-Kennedy, E.); Shaw, T (Shaw, T.); Skow, D (Skow, D.); Smith, RP (Smith, R. P.); Spalding, WJ (Spalding, W. J.); Spiegel, L (Spiegel, L.); Stavrianakou, M (Stavrianakou, M.); Stiehr, G (Stiehr, G.); Suzuki, I (Suzuki, I.); Tan, P (Tan, P.); Tanenbaum, W (Tanenbaum, W.); Tkaczyk, S (Tkaczyk, S.); Veseli, S (Veseli, S.); Vidal, R (Vidal, R.); Wenzel, H (Wenzel, H.); Whitmore, J (Whitmore, J.); Womersley, WJ (Womersley, W. J.); Wu, WM (Wu, W. M.); Wu, Y (Wu, Y.); Yagil, A (Yagil, A.); Yarba, J (Yarba, J.); Yun, JC (Yun, J. C.); Acosta, D (Acosta, D.); Avery, P (Avery, P.); Barashko, V (Barashko, V.); Bartalini, P (Bartalini, P.); Bourilkov, D (Bourilkov, D.); Cavanaugh, R (Cavanaugh, R.); Drozdetskiy, A (Drozdetskiy, A.); Field, RD (Field, R. D.); Fu, Y (Fu, Y.); Gray, L (Gray, L.); Holmes, D (Holmes, D.); Kim, BJ (Kim, B. J.); Klimenko, S (Klimenko, S.); Konigsberg, J (Konigsberg, J.); Korytov, A (Korytov, A.); Kotov, K (Kotov, K.); Levchenko, P (Levchenko, P.); Madorsky, A (Madorsky, A.); Matchev, K (Matchev, K.); Mitselmakher, G (Mitselmakher, G.); Pakhotin, Y (Pakhotin, Y.); Prescott, C (Prescott, C.); Ramond, P (Ramond, P.); Rodriguez, JL (Rodriguez, J. L.); Schmitt, M (Schmitt, M.); Scurlock, B (Scurlock, B.); Stoeck, H (Stoeck, H.); Yelton, J (Yelton, J.); Boeglin, W (Boeglin, W.); Gaultney, V (Gaultney, V.); Kramer, L (Kramer, L.); Linn, S (Linn, S.); Markowitz, P (Markowitz, P.); Martinez, G (Martinez, G.); Raue, B (Raue, B.); Reinhold, J (Reinhold, J.); Askew, A (Askew, A.); Bertoldi, M (Bertoldi, M.); Dharmaratna, WGD (Dharmaratna, W. G. D.); Gershtein, Y (Gershtein, Y.); Hagopian, S (Hagopian, S.); Hagopian, V (Hagopian, V.); Jenkins, M (Jenkins, M.); Johnson, KF (Johnson, K. F.); Prosper, H (Prosper, H.); Wahl, H (Wahl, H.); Baarmand, M (Baarmand, M.); Baksay, L (Baksay, L.); Guragain, S (Guragain, S.); Hohlmann, M (Hohlmann, M.); Mermerkaya, H (Mermerkaya, H.); Ralich, R (Ralich, R.); Vodopiyanov, I (Vodopiyanov, I.); Adams, MR (Adams, M. R.); Betts, RR (Betts, R. R.); Gerber, CE (Gerber, C. E.); Shabalina, E (Shabalina, E.); Smith, C (Smith, C.); Ten, T (Ten, T.); Akgun, U (Akgun, U.); Ayan, AS (Ayan, A. S.); Cooper, A (Cooper, A.); Debbins, P (Debbins, P.); Duru, F (Duru, F.); Fountain, M (Fountain, M.); George, N (George, N.); McCliment, E (McCliment, E.); Merlo, JP (Merlo, J. P.); Mestvirishvili, A (Mestvirishvili, A.); Miller, MJ (Miller, M. J.); Newsom, CR (Newsom, C. R.); Norbeck, E (Norbeck, E.); Onel, Y (Onel, Y.); Schmidt, I (Schmidt, I.); Wang, S (Wang, S.); Anderson, EW (Anderson, E. W.); Atramentov, O (Atramentov, O.); Hauptman, JM (Hauptman, J. M.); Lamsa, J (Lamsa, J.); Barnett, BA (Barnett, B. A.); Blumenfeld, B (Blumenfeld, B.); Chien, CY (Chien, C. Y.); Kim, DW (Kim, D. W.); Maksimovic, P (Maksimovic, P.); Spangler, S (Spangler, S.); Swartz, M (Swartz, M.); Baringer, P (Baringer, P.); Bean, A (Bean, A.); Coppage, D (Coppage, D.); Grachov, O (Grachov, O.); Kim, EJ (Kim, E. J.); Murray, M (Murray, M.); Bandurin, D (Bandurin, D.); Bolton, T (Bolton, T.); Khanov, A (Khanov, A.); Maravin, Y (Maravin, Y.); Onoprienko, D (Onoprienko, D.); Rizatdinova, F (Rizatdinova, F.); Sidwell, R (Sidwell, R.); Stanton, N (Stanton, N.); Von Toerne, E (Von Toerne, E.); Baden, D (Baden, D.); Bard, R (Bard, R.); Eno, SC (Eno, S. C.); Grassi, T (Grassi, T.); Hadley, NJ (Hadley, N. J.); Kellogg, RG (Kellogg, R. G.); Kunori, S (Kunori, S.); Ratnikov, F (Ratnikov, F.); Skuja, A (Skuja, A.); Arcidiacono, R (Arcidiacono, R.); Ballintijn, M (Ballintijn, M.); Bauer, G (Bauer, G.); Harris, P (Harris, P.); Kravchenko, I (Kravchenko, I.); Loizides, C (Loizides, C.); Nahn, S (Nahn, S.); Paus, C (Paus, C.); Pavlon, S (Pavlon, S.); Roland, C (Roland, C.); Roland, G (Roland, G.); Sumorok, K (Sumorok, K.); Vaurynovich, S (Vaurynovich, S.); Veres, G (Veres, G.); Wyslouch, B (Wyslouch, B.); Bailleux, D (Bailleux, D.); Corum, S (Corum, S.); Cushman, P (Cushman, P.); De Benedetti, A (De Benedetti, A.); Dolgopolov, A (Dolgopolov, A.); Egeland, R (Egeland, R.); Franzoni, G (Franzoni, G.); Gilbert, WJ (Gilbert, W. J.); Grahl, J (Grahl, J.); Haupt, J (Haupt, J.); Kubota, Y (Kubota, Y.); Mans, J (Mans, J.); Pearson, N (Pearson, N.); Rusack, R (Rusack, R.); Singovsky, A (Singovsky, A.); Cremaldi, LM (Cremaldi, L. M.); Godang, R (Godang, R.); Kroeger, R (Kroeger, R.); Sanders, DA (Sanders, D. A.); Summers, D (Summers, D.); Bloom, K (Bloom, K.); Claes, DR (Claes, D. R.); Dominguez, A (Dominguez, A.); Eads, M (Eads, M.); Lundstedt, C (Lundstedt, C.); Malik, S (Malik, S.); Snow, GR (Snow, G. R.); Sobol, A (Sobol, A.); Iashvili, I (Iashvili, I.); Kharchilava, A (Kharchilava, A.); Alverson, G (Alverson, G.); Barberis, E (Barberis, E.); Boeriu, O (Boeriu, O.); Eulisse, G (Eulisse, G.); Musienko, Y (Musienko, Y.); Muzaffar, S (Muzaffar, S.); Osborne, I (Osborne, I.); Reucroft, S (Reucroft, S.); Swain, J (Swain, J.); Taylor, L (Taylor, L.); Tuura, L (Tuura, L.); Wood, D (Wood, D.); Gobbi, B (Gobbi, B.); Kubantsev, M (Kubantsev, M.); Schellman, H (Schellman, H.); Schmitt, M (Schmitt, M.); Spencer, E (Spencer, E.); Velasco, M (Velasco, M.); Baumbaugh, B (Baumbaugh, B.); Cason, NM (Cason, N. M.); Hildreth, M (Hildreth, M.); Karmgard, DJ (Karmgard, D. J.); Marinelli, N (Marinelli, N.); Ruchti, R (Ruchti, R.); Warchol, J (Warchol, J.); Wayne, M (Wayne, M.); Bylsma, B (Bylsma, B.); Durkin, LS (Durkin, L. S.); Gilmore, J (Gilmore, J.); Gu, J (Gu, J.); Herman, D (Herman, D.); Killewald, P (Killewald, P.); Knobbe, K (Knobbe, K.); Ling, TY (Ling, T. Y.); Elmer, P (Elmer, P.); Marlow, D (Marlow, D.); Piroue, P (Piroue, P.); Stickland, D (Stickland, D.); Tully, C (Tully, C.); Wildish, T (Wildish, T.); Wynhoff, S (Wynhoff, S.); Xie, Z (Xie, Z.); Apresyan, A (Apresyan, A.); Arndt, K (Arndt, K.); Banicz, K (Banicz, K.); Barnes, VE (Barnes, V. E.); Bolla, G (Bolla, G.); Bortoletto, D (Bortoletto, D.); Bujak, A (Bujak, A.); Garfinkel, AF (Garfinkel, A. F.); Lopez, OG (Lopez, O. Gonzalez); Gutay, L (Gutay, L.); Ippolito, N (Ippolito, N.); Kozhevnikov, Y (Kozhevnikov, Y.); Laasanen, AT (Laasanen, A. T.); Liu, C (Liu, C.); Maroussov, V (Maroussov, V.); Merkel, P (Merkel, P.); Miller, DH (Miller, D. H.); Miyamoto, J (Miyamoto, J.); Neumeister, N (Neumeister, N.); Rott, C (Rott, C.); Roy, A (Roy, A.); Sedov, A (Sedov, A.); Shipsey, I (Shipsey, I.); Parashar, N (Parashar, N.); Eppley, G (Eppley, G.); Lee, SJ (Lee, S. J.); Liu, J (Liu, J.); Matveev, M (Matveev, M.); Nussbaum, T (Nussbaum, T.); Padley, BP (Padley, B. P.); Roberts, J (Roberts, J.); Tumanov, A (Tumanov, A.); Yepes, P (Yepes, P.); Bodek, A (Bodek, A.); Budd, H (Budd, H.); Chung, YS (Chung, Y. S.); De Barbaro, P (De Barbaro, P.); Demina, R (Demina, R.); Eusebi, R (Eusebi, R.); Ginther, G (Ginther, G.); Gotra, Y (Gotra, Y.); Hocker, A (Hocker, A.); Husemann, U (Husemann, U.); Korjenevski, S (Korjenevski, S.); Sakumoto, W (Sakumoto, W.); Slattery, P (Slattery, P.); Tipton, P (Tipton, P.); Zielinski, M (Zielinski, M.); Bartz, E (Bartz, E.); Doroshenko, J (Doroshenko, J.); Halkiadakis, E (Halkiadakis, E.); Jacques, PF (Jacques, P. F.); Kalelkar, MS (Kalelkar, M. S.); Khits, D (Khits, D.); Lath, A (Lath, A.); Macpherson, A (Macpherson, A.); Perera, L (Perera, L.); Plano, R (Plano, R.); Rose, K (Rose, K.); Schnetzer, S (Schnetzer, S.); Somalwar, S (Somalwar, S.); Stone, R (Stone, R.); Thomson, G (Thomson, G.); Watts, TL (Watts, T. L.); Akchurin, N (Akchurin, N.); Carrell, KW (Carrell, K. W.); Gumus, K (Gumus, K.); Jeong, C (Jeong, C.); Kim, H (Kim, H.); Papadimitriou, V (Papadimitriou, V.); Sill, A (Sill, A.); Spezziga, M (Spezziga, M.); Washington, E (Washington, E.); Wigmans, R (Wigmans, R.); Zhang, L (Zhang, L.); Bapty, T (Bapty, T.); Engh, D (Engh, D.); Johns, W (Johns, W.); Keskinpala, T (Keskinpala, T.); Lopez, EL (Lopez, E. Luiggi); Neema, S (Neema, S.); Nordstrom, S (Nordstrom, S.); Pathak, S (Pathak, S.); Sheldon, P (Sheldon, P.); Vaandering, EW (Vaandering, E. W.); Webster, M (Webster, M.); Arenton, MW (Arenton, M. W.); Conetti, S (Conetti, S.); Cox, B (Cox, B.); Hirosky, R (Hirosky, R.); Imlay, R (Imlay, R.); Ledovskoy, A (Ledovskoy, A.); Phillips, D (Phillips, D., II); Powell, H (Powell, H.); Ronquest, M (Ronquest, M.); Smith, D (Smith, D.); Baek, YW (Baek, Y. W.); Bellinger, JN (Bellinger, J. N.); Bradley, D (Bradley, D.); Carlsmith, D (Carlsmith, D.); Crotty, I (Crotty, I.); Dasu, S (Dasu, S.); Feyzi, F (Feyzi, F.); Gorski, T (Gorski, T.); Grothe, M (Grothe, M.); Hogg, W (Hogg, W.); Jaworski, M (Jaworski, M.); Klabbers, P (Klabbers, P.); Lanaro, A (Lanaro, A.); Loveless, R (Loveless, R.); de Abril, MM (de Abril, M. Magrans); Reeder, D (Reeder, D.); Smith, WH (Smith, W. H.); Wenman, D (Wenman, D.); Atoyan, GS (Atoyan, G. S.); Dhawan, S (Dhawan, S.); Issakov, V (Issakov, V.); Neal, H (Neal, H.); Poblaguev, A (Poblaguev, A.); Zeller, ME (Zeller, M. E.); Yuldashev, BS (Yuldashev, B. S.)

MIT and TMU are the most influential research universities in the world

I hope to comment extensively on the new Times Higher Education - Thomson Reuters rankings in a while but for the moment here is a comment on the citations indicator.

Last year Times Higher Education and Thomson Reuters solemnly informed the world that the two most influential places for research were Rice University in Texas and the Moscow State Engineering Physics Institute (MEPhI).

Now, the top two for Citations: research influence are MIT, which sounds a bit more sensible than Rice, and Tokyo Metropolitan University. Rice has slipped very slightly and MEPhI has disappeared from the general rankings because it was realised that it is a single-subject institution. I wonder how they worked that out.

That may be a bit unfair. What about that paper on opposition politics in central Russia in the 1920s?

Tokyo Metropolitan University's success at first seems rather odd because it also has a very low score for Research, which probably means that it has a poor reputation for research, does not receive much funding, has few graduate students and/or publishes few papers. So how could its research be so influential?

The answer is that it was one of scores of contributors to a couple of multi-authored publications on particle physics and a handful of widely cited papers in genetics and also produced few papers overall. I will let Thomson Reuters explain how that makes it into a pocket or a mountain of excellence.

Waiting for the Rankings

There are hints from the twittersphere that the latest Times Higher Education - Thomson Reuters World University Rankings will have something interesting. Will Caltech remain at number one? Will a couple of institutions from the BRICs fall out of the elite? Will Japan remain on top in Asia?

At first sight it would be odd if there were any significant changes since last year. The rankings suck in data about thousands of students, faculty, publications, income and citations and it is unlikely that there could be dramatic changes in any of these over twelve months or less. Moreover there have been no methodological changes. So how could there be any changes worth a headline or two?

There is first of all, the influence of what might be called the dark rankings, those universities that do not appear in the top 200 or the top 400 but are nevertheless influential in that the contribute to the mean scores against which the elite places are benchmarked.

The overall scores and the indicator scores given by THE to the elite universities do not represent absolute numbers but the distance from the average of all universities in the Thomson Reuters database. If the database expands and the new arrivals tend on average to perform less well than those already there then the mean scores of the elite would rise even if everything else remained unchanged.

If all the indicator scores rose at exactly the same rate then it would make no real difference. But if they did not it could mean that an expanding database could cause rises or falls in the rankings without any significant changes. In 2011 and 2012 it was noticeable that the mean scores of universities included in the rankings were higher for citations than for any other indicator and that the gap increased between the two years. This was most probably because the gap between the elite and the also-rans in the database was greater for citations.

If there has been a further influx of new institutions into the database this year then it might further benefit those universities that perform better for citations than for the other indicators.

We should also not forget the impact of TR's "regional modification" which rewards universities for being in a country that preforms poorly for citations. This means that a university's citation impact score is divided by the square root of the citation impact for the country as a whole.

If there is a decline in the citations of papers from a particular country then a university whose performance remains unchanged would get a boost because it is being compared to a lower national average. Equally, a rise in national performance could lead to a fall in the scores of flagship universities.

So we shall have to wait until tonight to see what happens.

Corruption and the Rankings?

One of the more interesting things about the publication of the QS and Times Higher Education world rankings is the reaction of university heads to minor rises or falls. Often they -- or their advisors --  display a thorough ignorance of basic procedures which are explained quite clearly by the rankers.

The Director of the Indian Institute of Technology at Kanpur, Indranil Manna, is reported to have said 


' “The standard of teaching, research and job placement are not the criteria for the ranking instead it is based on the paying capabilities of the institutions. An amount of one lakh and fifty thousand dollars needs to be paid to get a good ranking in such lists,” he claimed.
“These global rankings are more of a business rather than based on academic performance of institutions,” he said.'


and


"On IIT-Kanpur ranking at 295 in the 'QS World University Rankings', Manna said the institute was placed on the position based on old information provided on the its website.
“The ranking should have been after a team would have come to IIT-Kanpur and seen how the institute works. There have been so many students of IIT-Kanpur who have achieved so much on the world stage,” he said."


The Director should know that QS does not directly measure the standard of teaching or job placement and makes no claim to do so. Research, however, is measured by a reputation survey and by citations per faculty. Neither of these are very satisfactory but they are certainly criteria for the ranking. As for QS using old information from the IIT website, that is the Institute's fault for not keeping its site up to date, not QS's.

I am not a fan of QS but the claim about paying one and a half lakh dollars for a good ranking is way over the top and I suspect in a few days there will be a statement about a misunderstanding by a junior reporter.

I have a feeling that the Director has been getting mixed up with the QS Stars program.

What should be really frightening to Indian educators and students  is that the Director is apparently on a five member.committee appointed to "understand the methodology of the ranking agencies".

Not a very good start.

Reactions to the QS Rankings

Once again the publication of the QS rankings has led to a mixed chorus of triumph, lamentation and soul searching. In most cases there is a plea for more money to prevent national champions from falling further or to maintain their high position. Some commentators offered implausible reasons for changes, although I suspect that in some cases, ascent or descent may be due at least in part to the academic and employer surveys.

Richard Adams of the Guardian reported that:

"Cambridge has slipped down an authoritative list of international university rankings in a league table of top universities published on Tuesday.

It was ousted from second place in the QS World University Rankings by Harvard University; both were behind the Massachusetts Institute of Technology in the list of the world's leading universities."

He reports that:

'John O'Leary, member of the QS global academic advisory board, said: "The UK invests below the OECD average in higher education, so it is unrealistic to expect its universities to continue to punch above their weight indefinitely.

"The current success of leading institutions shows how vital it is that the government matches the investments being made by other countries in order to maintain their world-class status." '

The Philippine Daily Inquirer noted:

"The country’s leading universities remain highly regarded in international academic circles, but most of them slipped in the latest ranking of the world’s top 800 universities by the ratings firm Quacquarelli Symonds (QS).

The University of the Philippines, Ateneo de Manila University and the University of Santo Tomas dropped in the 2013 QS World University Rankings, while De La Salle University maintained its place from last year. "

According to the Beirut Daily Star, cited in edarabia:

"Of the 11 Middle Eastern institutions on the list, the American University in Cairo was the only one to make an improvement from last year’s listing, jumping from 392nd to 348th.

AUB held steady at 250th in the world, while universities in Saudi Arabia, the United Arab Emirates, Oman and Qatar precipitously fell from their spots at the middle and bottom of the international listing.

The reason for the decline was due to years’ worth of political instability and unrest, a researcher at the group said.

“Middle Eastern institutions are producing less widely cited research than in previous years, which may be related to unrest in the region,” said Ben Sowter, a researcher with the ranking organization."

This is not very plausible. If a university produces less research in 2012, that will probably mean that publication will start to decline two years later and citations in another two years and that will not have an impact on the rankings until 2016 or 2017. If the short-lived Arab spring had any effect on university rankings, it was probably more likely because of its impact on international perceptions reflected in the academic and employer surveys. Notice that universities in politically stable Arab countries fell while the American University in Cairo rose.


Meanwhile, Lucy Townend from the Manawatu Standard in New Zealand reports that:

"New Zealand universities' slide down world rankings has tertiary education leaders uneasy - saying Government investment in the sector is falling short of what's needed for them to keep up.
 
But Tertiary Education Minister Steven Joyce refutes any funding shortfalls, and says the rankings reflect the increased competitiveness of the international university market. "


heraldscotland proclaims that:

"SCOTLAND has three universities ranked in the top 100 in the world, according to a new international league table.

SCOTLAND has three universities ranked in the top 100 in the world, according to a new international league table.

The top Scottish institution is Edinburgh, which came 17th overall and fifth in the UK after climbing four places from last year.

 

The second ranked institution was Glasgow University, which came 51st after a rise of three places.

St Andrews took 83rd place after climbing 10 places in the rankings."

The QS Rankings

The QS World University Rankings top 200 have just been published. The top 800 will be released later today.

The top ten are:

1.  MIT
2.  Harvard
3.  Cambridge
4.  University College London
5.  Imperial College London
6.  Oxford
7.  Stanford
8. Yale
9. Chicago
10. Caltech

The Shanghai Rankings 2004-2013

One sign of the reliability of the Shanghai Academic Ranking of World Universities is that at the top there is little change from year to year. It is difficult to get excited about Tokyo slipping one place to join University College London in 21st position although I must admit the Federal Institute of Technology in Zurich moving up two whole places is rather intriguing.

These rankings are best used to study changes over a few years. Since 2004, according to data provided by the Shanghai rankers, the following countries have increased their membership of the world's elite of the top 100 universities

Australia          +3
Israel                +2
USA                 +1
Switzerland      +1
Netherlands      +1
Denmark          +1
Belgium           +1

These countries have seen universities leave the top 100.

Germany        -3
Japan              -2
UK                 -2
Sweden          -1
Italy                -1
Austria           -1


At the very top there is no sign of the erosion of English speaking dominance (academically I think Israel can be classed as English speaking). If anything, it is being extended although with a shift from the UK to the US and Australia.

Looking at the top 500, which we might consider to include world class research universities, the picture is different. From 2004 to 2013 the following changes occurred.

China                      +26
Australia                  +5
Saudi Arabia            +4
South Korea             +3
Portugal                    +3
Brazil                       +2
New Zealand            +2
Spain                        +1
Sweden                     +1
Turkey                      +1
Malaysia                   +1
Slovenia                    +1
Iran                            +1
Egypt                         +1
Croatia                       +1
Chile                          +1
Serbia                        +1
Mexico                      +1


USA                         -21
Japan                        -16
Germany                    -5
UK                             -5
Italy                            -4
France                        -2
India                           -2
Switzerland                -1
Netherlands                -1
Denmark                    -1
Hungary                     -1



Here the big story is the relative decline of the US, Northern Europe, Japan and India and the rise of China and, to a lesser extent, Australia, Korea, Southwest Asia, Southern Europe except Italy and Latin America.

There is very little sign of any Asian renaissance outside Greater China and Korea and maybe the Middle East. India has actually lost ground over the last decade and there is now only one institution from the whole of South Asia and central Asia.

     

The Webometrics Methodology

Isidro F. Aguillo of Webometrics has kindly sent me a summary of the methodology:

- The ranking intends to measure global performance of the 
universities using the web only as a proxy. Web design is mostly 
irrelevant, web contents are key if the web policy intends to mirror 
all the university missions on the web.

- We have a MODEL for the weighting of the variables in the composite 
indicator. It is the traditional "impact factor" developed several 
decades ago in bibliometrics adapted to the web: A ratio 1:1 (50%:50%) 
between ACTIVITY and IMPACT.

- For measuring IMPACT (visibility?, impact?, quality?) there are 
three alternatives: prestige surveys (THE, QS), peers citations 
(Leiden, NTU, URAP) or link visibility (number of external inlinks or 
backlinks). We use this last option because in this way we acknowledge 
a larger diversity of activities and missions and (very important) by 
a huge amount of users, a truly global audience.

- Personally I have two important objectives with the ranking: First, 
I am a scholar (scholar.google.com/citations?user=SaCSbeoAAAAJ) who is 
paid by the Spanish government to make scientific research, so the 
ranking provides me with a lot of valuable data useful for analysis 
and papers. Second, I have a "political" agenda, that is supporting 
Open Access initiatives.

- So, for measuring ACTIVITY the key issue is considering the 
full-text documents, so Openness consists of the number of files in 
pdf, doc, ppt and similar formats.

- An important innovation in the 3 latest editions is the Excellence 
indicator that is not really web related but intends to acknowledge 
the research intensive institutions. The data is provided by Scimago 
and reflects the top 10% more cited papers in 21 disciplines.

The Webometrics Rankings

The July 2013 Webometrics rankings have just been published. The top five are:

1.  Harvard
2.  MIT
3.  Stanford
4.  UC Berkeley
5.  UCLA

In first place in various regions are:

Latin America: Sao Paulo
Europe: Oxford
Asia: National University of Singapore
Africa: Kwazulu Natal
Arab World: King Saud University
Oceania: Australian National University
Caribbean: University of Puerto Rico Mayaguez
Middle East: Tel Aviv
South Asia: IIT Bombay
Eastern and Central Europe: Lomonosov Moscow State University

The Forbes Rankings

Forbes has just released its Best College list, which is compiled by the Center of College Affordability and Productivity. This index reflects student needs and the indicators include quality of teaching, student debt and graduate employability.

Stanford is at the top, Harvard is eighth and Caltech 18th. The armed forces academies and small liberal arts colleges do well.

The top five are:

1.  Stanford
2.  Pomona College.
3.  Princeton
4.  Yale
5.  Columbia

Shopping Around

It seems that many universities are now targeting specific rankings. One example is Universiti Malaya which submits data to QS but so far has not taken part in the THE rankings.

Now there is a report about the University of Canberra:

"The University of Canberra will spend $15 million over the next five years on some of the world's top researchers as the university pushes to break into the world rankings by 2018
The university has budgeted $3 million a year to attract 10 ''high performing'' researchers in five specialist areas: governance, environment, communication, education and health.
The recruitment drive started last week with advertising in the London Times Higher Education supplement [sic], with the paper's ranking of ''young'' universities the target of UC's campaign, with 13 Australian universities already in their top 100.
''We've decided to aim for [that] one particular ranking, although that will probably mean we'll hit some of the targets for many of the rankings, because there are an overlapping set of criteria that are used,'' Professor Frances Shannon, the university's deputy vice-chancellor of research, said."

It looks like the university is aiming not just at the THE under-50 rankings but specifically at the citations indicator which rewards high levels of citations in fields that are usually not cited very much.

This may be another case where the pursuit of ranking glory undermines the overall quality of a university.

"The recruitment drive comes after the university was criticised this month for axing language courses to try to combat government funding cuts while continuing its sponsorship of the Brumbies rugby team."

A bad idea but not really new

From Times Higher Education

University teachers everywhere are subject to this sort of pressure but it is unusual for it to be stated so explicitly.

"A university put forward plans to assess academics’ performance according to the number of students receiving at least a 2:1 for their modules, Times Higher Education can reveal.
According to draft guidance notes issued by the University of Surrey - and seen by THE - academics were to be required to demonstrate a “personal contribution towards achieving excellence in assessment and feedback” during their annual appraisals.
Staff were to be judged on the “percentage of students receiving a mark of 60 per cent or above for each module taught”, according to the guidance form, issued in June 2012, which was prefaced by a foreword from Sir Christopher Snowden, Surrey’s vice-chancellor, who will be president of Universities UK from 1 August.
“The intention of this target is not to inflate grades unjustifiably but to ensure that levels of good degrees sit comfortably within subject benchmarks and against comparator institutions,” the document explained.
After “extensive negotiations” with trade unions, Surrey dropped the proposed “average target mark”, with replacement guidance instead recommending that staff show there to be “a normal distribution of marks” among students."

Serious Wonkiness

Alex Usher at HESA had a post on the recent THE Under-50 Rankings. Here is an except about the Reputation and Citations indicators.

"But there is some serious wonkiness in the statistics behind this year’s rankings which bear some scrutiny.  Oddly enough, they don’t come from the reputational survey, which is the most obvious source of data wonkiness.  Twenty-two percent of institutional scores in this ranking come from the reputational ranking; and yet in the THE’s reputation rankings (which uses the same data) not a single one of the universities listed here had a reputational score high enough that the THE felt comfortable releasing the data.  To put this another way: the THE seemingly does not believe that the differences in institutional scores among the Under-50 crowd are actually meaningful.  Hmmm.

No, the real weirdness in this year’s rankings comes in citations, the one category which should be invulnerable to institutional gaming.  These scores are based on field-normalized, 5-year citation averages; the resulting institutional scores are then themselves standardized (technically, they are what are known as z-scores).   By design, they just shouldn’t move that much in a single year.  So what to make of the fact that the University of Warwick’s citation score jumped 31% in a single year, Nanyang Polytechnic’s by 58%, or UT Dallas’ by a frankly insane 93%?  For that last one to be true, Dallas would have needed to have had 5 times as many citations in 2011 as it did in 2005.  I haven’t checked or anything, but unless the whole faculty is on stims, that probably didn’t happen.  So there’s something funny going on here."

Here is my comment on his post.

Your comment at University Ranking Watch and your post at your blog raise a number of interesting issues about the citations indicator in the THE-TR World University Rankings and the various spin-offs.

You point out that the scores for the citations indicator rose at an unrealistic rate between 2011 and 2012 for some of the new universities in the 100 Under 50 Rankings and ask how this could possibly reflect an equivalent rise in the number of citations.

Part of the explanation is that the scores for all indicators and nearly all universities in the WUR, and not just for the citations indicator and a few institutions, rose between 2011 and 2012. The mean overall score of the top 402 universities in 2011 was 44.3 and for the top 400 universities in 2012 it was 49.5.

The mean scores for every single indicator or group of indicators in the top 400 (402 in 2011) have also risen although not all at the same rate. Teaching rose from 37.9 to 41.7, International Outlook from 51.3 to 52.4, Industry Income from 47.1 to 50.7, Research from 36.2 to 40.8 and Citations from 57.2 to 65.2.

Notice that the scores for citations are higher than for the other indicators in 2011 and that the gap further increases in 2012.

This means that the citations indicator had a disproportionate effect on the rankings in 2011, one that became more disproportionate in 2012

It should be remembered that the scores for the indicators are z scores and therefore they measure not the absolute number of citations but the distance in standard deviations from the mean number of normalised citations of all the universities analysed. The mean is the mean not of the 200 universities listed in the top 200 universities in the printed and online rankings or the 400 included in the ipad/iphone app but the mean of the total number of universities that have asked to be ranked. That seems to have increased by a few hundred between 2011 and 2012 and will no doubt go on increasing over the next few years but probably at a steadily decreasing rate.

Most of the newcomers to the world rankings have overall scores and indicator scores that are lower than those of the universities in the top 200 or even the top 400. That means that the mean of the unprocessed scores on which the z scores are based decreased between 2011 and 2012 so that the overall and indicator scores of the elite universities increased regardless of what happened to the underlying raw data.

However, they did not increase at the same rate. The scores for the citations indication, as noted, were much higher in 2011 and in 2012 than they were for the other indicators. It is likely that this was because the difference between top 200 or 400 universities and those just below the elite is greater for citations than it is for indicators like income, publications and internationalisation. After all, most people would probably accept that internationally recognised research is a major factor in distinguishing world class universities from those that are merely good.

Another point about the citations indicator is that after the score for field and year normalised citations for each university is calculated it is adjusted according to a “regional modification”. This means that the score, after normalisation for year and field, is divided by the square root of the average for the country in which the university is located. So if University A has a score of 3.0 citations per paper and the average for the country is 3.0 then the score will be divided by 1.73, the square of 3, and the result is 1.73.  If a university in country B has the same score of 3.0 citations per paper but the overall average is just 1.0 citation per paper the final score will be 3.0 divided by the square root of 1, which is 1, and the result is 3.

University B therefore gets a much higher final score for citations even though the number of citations per paper is exactly the same as University A’s . The reason for the apparently higher score is simply that the two universities are being compared to all the other universities in their country. The lower the score for universities in general then the higher the regional modification for specific universities.  The citations indicator is not just measuring the number of citations produced by universities but also in effect the difference between the bulk of a country’s universities and the elite that make into the top 200 or 400.

It is possible then that a university might be helped into the top 200 or 400 by having a high score for citations that resulted from being better than other universities in a particular country that were performing badly.

It is also possible that if a country’s research performance took a dive, perhaps because of budget cuts, with the overall number of citations per paper declining, this would lead to an improvement in the score for citations of a university that managed to remain above the national average.

It is quite likely that -- assuming the methodology remains unchanged -- if countries like Italy, Portugal or Greece experience a fall in research output as a result of economic crises, their top universities will get a boost for citations because they are benchmarked against a lower national average.

Looking at the specific places mentioned, it should be noted once again that Thomson Reuters do not simply count the number of citations per paper but compare them with the mean citations for papers in particular fields published in particular years and cited in particular years.

Thus a paper in applied mathematics published in a journal in 2007 and cited in 2007, 2008, 2009, 2010, 2011 and 2012 will be compared to all papers in applied maths published in 2007 and cited in those years.

If it is usual for a paper in a specific field to receive few citations in the year of publication or the year after then even a moderate amount of citations can have a disproportionate effect on the citations score.

It is very likely that Warwick’s increased score for citations in 2012 had a lot to do with participation in a number of large scale astrophysical projects that involved many institutions and produced a larger than average number of citations in the years after publication. In June 2009, for example, the Astrophysical Journal Supplement Series published ‘The seventh data release of the Sloan Digital Sky Survey’ with contributions from 102 institutions, including Warwick. In 2009 it received 45 citations. The average for the journal was 13. The average for the field is known to Thomson Reuters but it is unlikely that anyone else has the technical capability to work it out. In 2010 the paper was cited 262 times: the average for the journal was 22. In 2011 it was cited 392 times: the average for the journal was 19 times.

This and similar publications have contributed to an improved performance for Warwick, one that was enhanced by the relatively modest number of total publications by which the normalised citations were divided.

With regard to Nanyang Technological University, it seems that a significant role was played by a few highly cited publications in Chemical Reviews in 2009 and in Nature in 2009 and 2010.

As for the University of Texas at Dallas, my suspicion was that publications by faculty at the University of Texas Southwestern Medical Center had been included, a claim that had been made about the QS rankings a few years ago.  Thomson Reuters have, however, denied this and say they have observed unusual behaviour by UT Dallas which they interpret as an improvement in the way that affiliations are recorded. I am not sure exactly what this means but assume that the improvement in the citations score is an artefact of changes in the way data is recorded rather than any change in the number or quality of citations.

There will almost certainly be more of this in the 2013 and 2014 rankings."

 

The Complete Efficiency Rankings

At last. Here are complete Efficiency Rankings measuring the efficiency with which universities turn inputs into citations. I am using the method of Professor Dirk van Damme which is to divide the scores for Citations: Research Influence in The THE World University Rankings by the scores for Research: Volume, Income and Reputation. Here is the method as cited in a previous post:

"The input indicator takes scaled and normalised measures of research income and volume into account, and also considers reputation, while the output indicator looks at citations to institutional papers in Thomson Reuters’ Web of Science database, normalised for subject differences.
Professor van Damme said that the results - which show that university systems outside the Anglo-American elite are able to realise and increase outputs with much lower levels of input - did not surprise him.
“For example, Switzerland really invests in the right types of research. It has a few universities in which it concentrates resources, and they do very well,” he said.
Previous studies have found the UK to have the most efficient research system on measures of citation per researcher and per unit of spending.
But Professor van Damme explained that under his approach, productivity - output per staff member - was included as an input.
“With efficiency I mean the total research capacity of an institution, including its productivity, divided by its impact. The UK is not doing badly at all, but other countries are doing better, such as Ireland, which has a very low research score but a good citations score,” he said.
Given the severity of the country’s economic crisis, Ireland’s success was particularly impressive, he said.
“I think it is really conscious of the effort it has to make to maintain its position and is doing so.”
Low efficiency scores for China and South Korea reflected the countries’ problems in translating their huge investment into outputs, he added."




1.  Tokyo Metroplitan
2.  Moscow Engineering Physics Institute
3.  Florida Institute of Technology
4.  Southern Methodist
5.  Hertfordshire
6.  Portsmouth
7= King Mongkut's University of Technology
7= Vigo
9.  Creighton
10.  Fribourg
11.  King Abdulaziz
12.  University of the Andes
13.  Trieste
14.  Renmin
15.  Medical University of Vienna
16.  Polytech University Valencia
17=  Beyreuth
18=  Montana
19.   Mainz
20.   Ferrara
21.   Drexel
22.   Valencia
23.   Linz
24.   Crete
25.   Colorado School of Mines
26.  Technical University of Dresden
27.   Innsbruck
28.  Nurnberg
29= Dauphine
29= Wake Forest
29= Maryland Baltimore County
32. St George's London
33.  William and Mary College
34. Hong Kong Baptist
35.  Basel
36.  Texas San Antonio
37.  Duisberg
38.  Lyon 1
39.  Wurzburg
40.  Charles Darwin
41.  Wayne State
42. Northeastern
43.  Bicocca
44. Royal Holloway
45.  Koc
46.  Georgia University of Health Science
47.  Modena
48.  Dundee
49.  Southern Denmark
50= IIT Roorkhee
50= Pompeu Fabra
52.  Graz
53= Oregon
53= Diderot
55.   Bielfeld
56.   Munster
57.  Waikato
58= Grenoble
59= East Anglia
60= Bonn
61=  Pavia
62.  ENS Lyon
63.  Eastern Finland
64.  Padua
65.  Brandeis
66.  Aberystwyth
67.  Tulane
68.  Tubingen
69= Warsaw
70= Sun Yat Sen
71= Keele
72.  Tromso
73.  Brunel
74.  Liege
75.  Queen Mary
76=  Vermont
77=  Trento
78.  Turin
79.  Jyvaskyla
80.  Carleton
81.  Kansas
82.  California Riverside
83.  SUNY Stony Brook
84=  George Washington
85=  Pisa
86.  Tasmania
87.  George Mason
88.  Boston College
89=  Oregon State
90=  Texas Dallas
91.  Trinity College Dublin
92= University Science and Technology China
92= Murdoch
92= Cinncinati
92= Galway
92= Yeshiva
97= Tufts
97= Minho
99. Miami
100.  Lehigh
101. Technical University Denmark
102= Rice
102= Iceland
104.  California Santa Cruz
104= Milan
106.  Monpellier 2
107.  Frankfurt
108= Bergen
109=  Strasbourg
110.  Victoria
111.  Rochester
112.  Cork
113.  Dartmouth
114.  Oklahoma
115.  Birkbeck
116.  Porto
117.  Canterbury
118= Newcastle  UK
118= Notre Dame
118= University College Dublin
121. Binghamton
122.  Aveiro
123=  Kiel
123= Sussex
125.  Temple
126.  Aachen
127=  Fribourg
127=  Queens Belfast
127= Colorado Boulder
130.  Iowa State
131. Tokyo Medical Dental
132= Autonomous Madrid
132= Swedish Agriculture
132= Tempere
135= Deakin
135= Barcelona
137= Stockholm
137= Stirling
139.  Laval
140.  Durham
141.  Bangor
142= Aberdeen
142= Vanderbilt
144.  Istanbul Technical
145.  Nanjing
146= Exeter
146= Emory
146= Leicester
149.  Southamton
150.  Paris Mines
151. Vrije Universiteit Brussel
152.  Polytechnic Milan
153. Kwazulu-Natal
154= Linkoping
154= Bilkent
154= Herriot-Watt
154= Bologna
158= Wyoming
158= Utah
158= Massey
161= Glasgow
161= Bern
163.  ENS Paris
164.  Zurich
165= Case Western Reserve
166= California Irvine
167= Tartu
168= Wellington
169= Salento
170.  South Carolina
171.  York UK
172.  Aalto
173= Curie
173= Macquarie
173= Boston
176= Delaware
177= Copenhagen
178= Hannover
179.  Norway University of Science and Technology
180.  Antwerp
181= Dalhousie
181= Renselaer Polytechnic Institute
183= Konstanz
184= Paris Sud
185.  Technical University Munich
186.  Lancaster
187.  Waseda
188.  Otago
189.  Arizona State
190= SUNY Albany
190= Gottingen
190= Autonomous Barcelona
193= Cape Town
194= St Andrews
195= Colorado State
195= Bath
195= Wollongong
198= Tsukuba
198= Simon Fraser
198= Liverpool
198= Umea
202= Geneva
202= Newcastle Australia
204= Universite Libre Bruxelles
204= Virginia
206= Lausanne
206= Louvain
208= Connecticut
208= Georgetown
208= York Canada
211.  EPF Lausanne
212= North Carolina State
212= Bristol
212= Aalborg
212= Free University Amsterdam
216= Indiana
216= Kentucky
218. Maryland College Park
219.  Karlsruhe Institute technology
220= University Technology Sydney
220= Iowa
222.  Charles
223.  Flinders
224.  Cardiff
225= Auckland
225= Oslo
227.  Pittsburgh
228= Heidelberg
228= Guelph
228= Washington State
228= Sheffield
232= Chinese University Hong Kong
232= Strathclyde
234= Ottawa
234= Gotherberg
234= Washington St Louis
237.  Medical South Carolina
238= McMaster
238= Brown
238= National Sun Yat Sen
238= Reading
242.  Ecole Polytechnique
243.  Helsinki
244= Quebec
244= National Central Taiwan     
246.  Bogazici
247= Southern California
247= Arizona
249.  Keio
250= Houston
250= Stellenbosch
250= Kings College London
250= Darmstadt
250= Western Australia
255= Pohang
255= IIT Bombay
257= Wageningen
257= Manitoba
259= South Australia
259= Nagoya
261= Leeds
261= UC Santa Barbara
261= Nijmegen
261= Jagiellon
265= New York University
265= Calgary
265= Ohio State
268.  Aarhus
269= Witwatersrand
269= North Carolina Chapel Hill
269= Michigan State
269= Fudan
273= Bochum
273= Munich
275= SUNY Buffalo
275= Adelaide
275= Sapienza
278= Utrecht
278= Edinburgh
278= Queensland University of Technology
281= Lund
281= Ghent
283.  Erasmus
284= Massachusetts
284= Illinois Chicago
284= Nottingham
287= Eindhoven
287= Amsterdam
289.  UC San Diego
290.  Birmingham
291= Western Ontario
291= Twente
293= Washington Seattle
293= Duke
295= Penn State
295= NUI Maynooth
297= Maastricht
297= Groningen
297= Columbia
297= Leiden
297= Georgia
302.  UC Davis
303= Southern Florida
303= Chalmers University of Technology
305= Minnesota
305= Essex
305= Manchester
305= Georgia Institute of Technology
309= Rutgers
309= Texas at Austin
311= Northwestern
311= Warwick
311= Vienna
311= MIT
315.  Johns Hopkins
316= Wisconsin Madison
316= Carnegie Mellon
318.  Alberta
319.  Pennsylvania
320= Hong Kong University of Science and Technology
320= Kyushu
322= Chicago
322= Vienna University of Technology
324= Queensland
324= Montreal
326.  British Columbia
327= Yale
327= Imperial College London
327= UCLA
327= Hebrew University of Jerusalem
327= Karolinska
332= Melbourne
332= Humboldt
332= National Tsinghua Taiwan
332= Cambridge
332= Harvard
332= Stanford
338= Monash 
338= Princeton
338= Caltech
338= Michigan
338= UC Berkeley
338= Cornell
344= Waterloo
344= KHT Sweden
344= Missouri
347.  University College London
348= Oxford
348= Middle East Technical University
350.  Yonsei
351= Toronto
351= Illinois Urbana Champagne 
351= Peking
351= Leuven
355= Zhejiang
355= Hokkaido
355= Hong Kong Polytechnic University
355= McGill
359= ETH Zurich
359= Tokyo Institute of Technology
361= Berlin
361= Uppsala
363= Korea
363= Sydney
365= Florida
365= New South Wales
367= Australian National
367= Tohoku
367= Purdue
367= Technion
371= Surrey
371= IIT Kharagpur
373= KAIST
373= Texas A and M
375. Virginia Polytechnic Institute
376= Osaka
376= Nanyang Technological University
376= Shanghai Jiao Tong
379.  LSE
380.  Sungkyunkwan
381.  Sharif University of Technology
382.  Tokyo
383= National Taiwan University of Science and Technology
383= National Autonomous University of Mexico
385= Kyoto
385= National University of Singapore
387.  Loughborough
388.  National Cheng Kung
389.  Tel Aviv
390= Hong Kong
390= Tsinghua
392.  Chinese University of Hong Kong
393.  National Taiwan
394.  National Chiao Tung
395.  Tilburg
396.  Delft
397.  Seoul National
398.  State University Campinas
399.  Sao Paulo
400.  Moscow State

Competition and Controversy in Global Rankings

My article,  'Competition and Controversy in Global Rankings' can be accessed here.

What about a Research Influence Ranking?

Keeping up with the current surge of global university rankings is becoming next to impossible. Still there are  a few niches that have remained unoccupied. One might be a ranking of universities according to their ability to spread new knowledge around the world. So it might be a good idea to have a Research Influence Ranking based on the citations indicator in the Times Higher Education -- Thomson Reuters World University Rankings.

Thomson Reuters are the world's leading collectors and analysts of citations data so such an index ought to provide invaluable data source for governments, corporations and other stakeholders deciding where to place research funding. Data for 400 universities can be found on the THE iPhone/iPad app.

The top place in the world would be jointly held by Rice University in Texas and Moscow State Engineering Physics Institute, closely followed by MIT and the University of California Santa Cruz.

Then there are the first places in various regions and counties. (MEPhI would be first in Europe and Rice in the US and North America.)

Canada
University of Toronto

Latin America
University of the Andes, Colombia

United Kingdom (and Western Europe)
Royal Holloway London

Africa
University of Cape Town

Middle East
Koc University, Turkey

Asia (and Japan)
Tokyo Metropolitan University

ASEAN
King Mongkut's University of Technology, Thailand

Australia and the Pacific
University of Melbourne

On second thoughts, perhaps not such a good idea.

Bad Mood Rising

In 2006 I tried to get an article published in the Education section of the Guardian, that fearless advocate of radical causes and scourge of the establishment, outlining the many flaws and errors in the Times Higher Education Supplement -- Quacquarelli Symonds (as they were then) World University Rankings, especially its "peer review". Unfortunately, I was told that they would be wary of publishing an attack on a direct rival. That was how University Ranking Watch got started.

Since then QS and Times Higher Education have had an unpleasant divorce, with the latter now teaming up with Thomson Reuters. New rankings have appeared, some of them to rapidly disappear -- there was one from Wuhan and another from Australia but they seem to have vanished. The established rankings are spinning off subsidiary rankings at a bewildering rate.

As the higher education bubble collapses in the West everything is getting more competitive including rankings and everybody -- except ARWU -- seems to be getting rather bad-tempered.

Rankers and academic writers are no longer wary about "taking a pop" at each other. Recently, there has been an acrimonious exchange between Ben Sowter of QS and Simon Marginson of Melbourne University. This has gone so far as to include the claim that QS has used the threat of legal action to try to silence critics.

"[Ben] Sowter [of QS] does not mention that his company has twice threatened publications with legal action when publishing my bona fide criticisms of QS. One was The Australian: in that case QS prevented my criticisms from being aired. The other case was University World News, which refused to pull my remarks from its website when threatened by QS with legal action.

If Sowter and QS would address the points of criticism of their ranking and their infamous star system (best described as 'rent a reputation'), rather than attacking their critics, we might all be able to progress towards better rankings. That is my sole goal in this matter. As long as the QS ranking remains deficient in terms of social science, I will continue to criticise it, and I expect others will also continue to do so."

Meanwhile the Leiter Reports has a letter from "a reader in the UK".

THES DID drop QS for methodological reasons. The best explanation is here: http://www.insidehighered.com/views/2010/03/15/baty
But there may have been more to it? Clearly QS's business practices leave an awful lot to be desired. See: http://www.computerweekly.com/news/1280094547/Quacquarelli-Symonds-pays-80000-for-using-unlicensed-software
Also I understand that the "S" from QS -- Matt Symonds -- walked out on the company due to exasperation with the business practices.  He has been airbrushed from QS history, but can be foud at:  https://twitter.com/SymondsGSB
And as for the reputation survey, there was also this case of blantant manipulation: http://www.insidehighered.com/news/2013/04/08/irish-university-tries-recruit-voters-improve-its-international-ranking
And of course there's the high-pressure sales: http://www.theinternationalstudentrecruiter.com/how-to-become-a-top-500-university/
And the highly lucrative "consultancy" to help universities rise up the rankings: http://www.iu.qs.com/projects-and-services/consulting/
There are "opportunities" for branding -- a snip at just $80,000 -- with QS Showcase: http://qsshowcase.com/main/branding-opportunities/
Or what about some relaxing massage, or a tenis tournament and networking with the staff who compile the rankings: http://www.qsworldclass.com/6thqsworldclass/
Perhaps most distribing of all is the selling of dubious Star ratings: http://www.nytimes.com/2012/12/31/world/europe/31iht-educlede31.html?pagewanted=all&_r=0
Keep up the good work. Its an excellent blog.

All of this is true although I cannot get very excited about using pirated software and the bit about relaxing massage is rather petty -- I assume it is something to do with having a conference in Thailand. Incidentally, I don't think anyone from THE sent this since the reader refers to THES (The S for Supplement was removed in 2008).

This is all a long way from the days when journalists refused to take pops at their rivals, even when they knew the rankings were a bit rum.

Times Higher Education Under 50s Rankings

Times Higher Education has now published its ranking of universities less than fifty years old.

The top five are:

1.  Pohang University of Science and Technology
2.  EPF Lausanne
3.  Korea Advanced Institute of Science and Technology
4.  Hong Kong University of Science and Technology
5.  University of California, Irvine

They are quite a bit different from the QS young universities rankings. In a while I hope to provide a detailed comparison.

Citation Cartels

An article by Paul Jump in Times Higher Education describes how Thomson Reuters have been excluding an increasing number of journals from their Journal Citation Reports for "anomalous citation patterns" which now includes not just self-citation but excessive mutual citation.

Surely it is now time for Thomson Reuters to stop counting self-citations for the Research Influence indicator in the THE World University Rankings. The threat of the self-citations of Dr El Naschie "of" Alexandria University has receded but there are others who would have a big impact on the rankings if they ever move to a university with a low volume of publications.

TR may not want to follow QS who no longer count citations for their rankings but excluding excessive mutual citation as well would put them one up again.

Uncanny Insight into Ranker Psychology

I just said that QS would announce its Young University Rankings now that THE has indicated the launch date for its rankings at Wellington College next week.

Actually it was just a few hours.

Anyway, here are the top five.

1.   Hong Kong University of Science and Technology
2.   Nanyang Technological University
3.   Warwick
4.   KAIST
5.   City University of Hong Kong

Prestigious Ranking Watch

Times Higher Education will be launching their Top 100 Under-50 Universities Rankings which, in case you have forgotten, is prestigious, at Wellington College, which is a school not a college, in another eight days.

Does anybody want to bet on the QS under-50 rankings appearing in a few days?

Meanwhile, the THE World Rankings will be published at the THE World Academic Summit in Singapore in October. See here. And yes, they are prestigious.

The QS Latin American Rankings

The QS Latin American Rankings show some interesting variations in methodology. The academic survey has a weight of 30%, compared to 40% in the World Rankings, and the employer survey a weight of 20%, compared to 10%.

Instead of 20% for citations per faculty there is 10% for papers per faculty and 10% for citations per paper. Since there are great variations according to the measures used to count research output and influence, as shown by the recent Leiden Ranking, this is very sensible.

Faculty-student ratio is reduced from 20% to 10% and international students and international faculty are removed. There is now 10% for proportion of staff with Ph Ds and 10% for web impact.

Here are the top five.

1.  Universidade de Sao Paulo
2.  Pontificia Universida Catolica de Chile
3.  Universidade Estadual de Campinas
4.  Universidad de Los Andes Colombia
5.  Universidad de Chile

The Global Gender Index

Times Higher Education has just published an article on the Global Gender Index produced in collaboration with Thomson Reuters. This consists of calculating the percentage of female academics among those universities included in the top 400 of the Times Higher Education World University Rankings and producing a percentage for each country.

This is a rather dubious exercise. The data is reported by the institutions themselves and, as the international unit of UK HE has pointed out, such data may not always be reliable. In addition, the universities that volunteer to be ranked by THE and Thomson Reuters may not necessarily be representative of the higher education sector in general. The global research orientated universities that make it into the top 400 may be even less so.

The report finds that everywhere women make up less than half the academic work force and that the numbers are lowest in Japan, followed by Taiwan. Numbers are nearly equal in Turkey.


Predictably, the article includes a call for universities to be ranked according to how far they have achieved gender equity among academic staff and a suggestion that East Asian countries should learn from Turkey.

There is a question that needs to be considered. If universities in countries like Taiwan and Japan are poised to overtake the West, as QS and THE are constantly warning us, should we be so eager to conclude that they have something, or indeed anything, to learn from Turkey or from Northern Europe?