Rankings offer a limited view on universities
A ranking gives an incomplete picture of reality. The strength of higher education institutions specifically lies in their mutual diversity. Simplifying those institutions to a ‘list’ therefore in many cases does not do justice to universities.
An external benchmark can nevertheless be a valuable instrument. However, it is important to realise that no ranking system is entirely objective, as the ultimate ranking is the result of the many subjective choices of the creators of the list. Below are three limitations of rankings mentioned in, for example, the report of EUA, Global University Rankings and Their Impact.
1. Rankings focus on elite universities only and are not able to analyse the whole educational system
The criteria used in many rankings only distinguish the very best universities. An example of this is the number of winners of a Nobel Prize. The difference in number of Nobel Prize winners between number 1 and number 20 on the list can be considerable. But the same difference between, for example, number 60 and number 80 can be much smaller, even though they are also 20 positions apart. This means that in many rankings, scores below the 200th position are indistinguishable, as they are too close together. While universities may differ tens of positions in the rankings, the actual scores will sometimes only differ a few per cent. The below figure of the European Association of Universities shows how strongly scores drop in the top 100 of various rankings.
Drop in ranking scores in the top 100 of the ARWU, THE and QS Rankings. (Source: European University Association (2013), ‘Global University Rankings and Their Impact Report II’, page 18
2. Rankings favour certain disciplines
The Humanities and largely also the Social Sciences are not well represented in rankings. This is due to the fact that many rankings are based on the numbers of published articles in specific journals. Fields of study that are the focus of these academic journals, in particular medicine, natural sciences and engineering, benefit from this. The Humanities, and partly also the Social Sciences, often publish books rather than articles in academic journals. However, books are not considered in rankings.
3. Rankings use a number of indicators that are of a bad quality
In spite of this often heard criticism, reputation indicators continue to be widely used. Times New Higher has a specific ranking system based on reputation, and QS also uses reputation as an indicator. It happens from time to time that a university is nominated as excellent in a field of study, while the relevant institution does not have any teaching or research programmes in that field at all.
In many cases, only staff/student ratios are used to assess the quality of teaching. In reality, however, teaching quality is many times more complex than this ratio.
Additional information about rankings:
- The report Global University Rankings and their Impact, published by the European Association of Universities (2013).
- A group of global experts has drawn up the 'Berlin Principles on Ranking of Higher Education Institutions' (PDF), for the purpose of compiling reliable rankings.