Are the rankings biased?
"Listening to the public discourse one could be forgiven for thinking that the British higher education system is a failure. It is not. It is the envy of the world."
That is an unfortunate phrase. It used to be asserted that the National Health Service was the envy of the world.
She cites as evidence for university excellence the Times Higher Education World University Rankings which have three British universities in the world's top ten and twelve in the top one hundred. These rankings also, although she does not mention it here, put Oxford in first place.
There are now, according to IREG, 21 global university rankings. One wonders why a world-class scholar and head of a world-class university would choose rankings that regularly produce absurdities such as Anglia Ruskin University ahead of Oxford for research impact and Babol Noshirvani University of Technology its equal.
But perhaps it is not really surprising since of those rankings THE is the only one to put Oxford in first place. In the others it ranges from third place in the URAP rankings published in Ankara to seventh in the Shanghai Rankings (ARWU), Webometrics (WEB) and Round University Ranking (RUR) from Russia
That leads to the question of how far the rankings are biased in favor of universities in their own countries.
Below is a quick and simple comparison of how top universities perform in rankings published in the countries where they located and in other rankings.
I have looked at the rank of the top scoring home country university in each of eleven global rankings and then at how well that university does in the other rankings. The table below gives the overall rank of each "national flagship" in the most recent eleven global university rankings. The rank in the home country rankings is in red.
We can see that Oxford does better in the Times Higher Education (THE) world rankings where it is first than in the others where its rank ranges from 3rd to 7th. Similarly, Cambridge is the best performing UK university in the QS rankings where it is 4th. It is also 4th in the Center for World University Rankings (CWUR), now published in the UAE, and 3rd in ARWU. In the other rankings it does less well.
ARWU, the US News Best Global Universities (BGU), Scimago (SCI), Webometrics (WEB), URAP, the National Taiwan University Rankings (NTU), and RUR do not seem to be biased in favour of their country's flagship universities. For example, URAP ranks Middle East Technical University (METU) 532nd which is lower than five other rankings and higher than three.
CWUR used to be published from Jeddah in Saudi Arabia but has now moved to the Emirates so I count the whole Arabian peninsula as its home. The top home university is therefore King Saud University (KSU), which is ranked 560th, worse than in any other ranking except for THE.
The GreenMetric Rankings, produced by Universitas Indonesia (UI), have that university in 23rd place, which is very much better than any other.
It looks like THE, GreenMetric and, to a lesser extent QS, are biased towards their top home country institutions.
This only refers to the best universities and we might get different result looking at all the ranked universities.
There is a paper by Chris Claassen that does this although it covers fewer rankings.
THE | ARWU | QS | BGU | SCI | WEB | URAP | NTU | RUR | CWUR | GM | |
Oxford | 1 | 7 | 6 | 5 | 6 | 7 | 3 | 5 | 7 | 5 | 6 |
Tsinghua | 35 | 48 | 25 | 64 | 8 | 45 | 25 | 34 | 75 | 65 | NR |
Cambridge | 4 | 3 | 5 | 7 | 16 | 11 | 9 | 12 | 9 | 4 | NR |
Harvard | 6 | 1 | 3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | NR |
Barcelona | 201-250 | 201-300 | 156 | 81 | 151 | 138 | 46 | 64 | 212 | 103 | 180 |
METU | 601-800 | 701-800 | 471-480 | 314 | 489 | 521 | 532 | 601-700 | 407 | 498 | NR |
NTU | 195 | 151-200 | 76 | 166 | 342 | 85 | 100 | 114 | 107 | 52 | 92 |
Lomonosov MSU | 188 | 93 | 95 | 267 | 342 | 235 | 194 | 236 | 145 | 97 | NR |
KSU | 501-600 | 101-150 | 221 | 377 | NR | 424 | 192 | 318 | 460 | 560 | NR |
UI | 600-800 | NR | 277 | NR | 632 | 888 | 1548 | NR | NR | NR | 23 |
