University rankings as means of action: uses and abuses

In 1963, faculty and administration at the University of California, Berkeley objected vigorously when the campus’ radical student newspaper, Cal Reporter , took the initiative of publishing student-made evaluations of their courses and professors ( SLATE, 2003-2005). Despite this initial resistance, student assessments have gradually become essential parts of the internal accountability mechanisms of many universities, not only in the United States but in a growing number of countries around the world. Today, there are even Internet sites where any student can rate their professors, regardless of location . More generally, in the space of 20 years, universities that had hitherto enjoyed considerable autonomy are now required to justify their results and the use they make of public resources. Students are not the only ones asking for more responsibility from their schools, other stakeholders are making the same demands: public authorities, worried about rising costs, employers, who need competent graduates , and the general public, who wish to obtain more information on the quality of education and on the employment prospects of graduates.

Accreditations, cyclical reviews, external peer reviews, inspections, audits, performance contracts based on predetermined indicators, bench marking and research evaluations are some of the most common forms of accountability mechanisms. Some are set up by the institutions themselves, others are imposed on them from outside by donors, quality assurance bodies, committees of presidents and rectors, as well as by interested parties in the sense wide, from which, for example, university rankings are derived. There are now no less than 30 worthy rankings, ranging from general rankings of national universities, such as those of Maclean’s and USNews and World Report, to comprehensive international rankings, for example those of Times Higher Education Supplement ( THES ) and Shanghai Jiao Tong University (SJTU), to research-specific rankings, such as those of New Zealand and the United Kingdom, or even other targeted rankings whose objective is, for example, to identify the best connected or most politically active campuses, not to mention the countless rankings of Masters of Business Administration (MBA) and other pal are many professional establishments all over the world.

 Rankings of establishments, known in English as league tables , institutional rankings or even report cards (Gormley and Weimer, 1999), are constructed using objective and/or subjective data obtained from establishments or from the public domain, and giving rise to to a “quality measure” assigned to the comparison unit relative to its competitors. This unit usually designates a higher education institution, and primarily a university. However, post-secondary institutions or particular fields of study or programs may also be subject to rankings. This article focuses on university rankings.

 These classifications use a multitude of indicators that serve to determine the way the system is set up (input variables), its mode of operation and its internal efficiency (process variables), as well as its productivity and its impact (output variables) compared to the results of other universities and programs  [1]

Among the input variables, we can mention the autonomy in…. The media and other ranking organizations give more or less importance to the variables chosen to compare establishments, as shown in particular by the weightings they assign to the various indicators. Some rankings relate to a specific category of universities, which allows institutions with different missions and orientations to compete on equal terms  [2]

Maclean’s weekly news magazine, which publishes a…. Others relate to all institutions without distinction, and still others compare only specific programs and not the entire institution.

In some countries, the ranking exercise is carried out as part of the accreditation process, either by the body responsible for it, where it exists, or by the authority responsible for higher education. This mechanism may be limited to a single classification in three or four accreditation categories (in Argentina, for example), or may result in a detailed classification of establishments carried out by the competent body (as in Nigeria).

The proliferation of rankings and other awards has not gone unnoticed by the various stakeholders, and the reactions they elicit are rarely moderate. These rankings are often denounced by their many detractors as meaningless exercises, marred by numerous irregularities in data and methodology, they are boycotted by certain universities dissatisfied with the results and constitute a convenient way for the opposition to criticize the government in place. But if there’s one thing certain about these charts, it’s that they leave no one indifferent. As they grow in importance, even in developing countries, their accuracy, relevance and usefulness have become matters of concern (see, for example, Bowden, 2000; Clarke, 2002; Dill and Soo, 2005; Eccles, 2002). Are they completely inappropriate for measuring quality and should they be abandoned altogether? Can they be adapted to meet the information needs of developing countries? Do they have any interest in public action, for accountability purposes and for public information?

To answer these questions, we will study school rankings and similar instruments, with particular emphasis on the role and usefulness of these tools as mechanisms for informing the public and as indicators of quality of education provided. This article begins with an overview and typology of school rankings: their origins, evolution and characteristics. It continues with a study of the controversies that these rankings arouse: the basis and range of criticisms expressed, the value of the indicators generally used to assess quality and the conditions that can favor or disadvantage universities, particularly in international rankings. This analysis leads to the final section of the article, which examines the implications that university rankings can have on national policies and on institutional orientations and practices, in both developing and industrialized countries.