That’s the title of a piece I had in the Chronicle of Higher Education in February. CHE is paywalled, but they kindly agree to let me republish here, after a suitable interval. The article (or at least a near final version) is over the fold.
Rank Delusions
Every year, U.S. News & World Report, Times Higher Education, and others update university rankings. Reactions are paradoxical. On the one hand, university administrators and faculty members scan the lists for evidence of small movements up or down. On the other hand, everyone knows that the top 10, or 20, or 50 names will be much the same as they have always been.
The Duke sociologist Kieran Healy points to a four-tier classification of leading universities made in 1911, and compares it to the most recent U.S. News ranking. Of the top 20 universities in the ranking today, 16 were in the top class in 1911, one (Notre Dame) was in the second class, and three (Duke, Rice, and Caltech) had not yet been established under their current names.
The United States is not unusual in this respect. In most countries with an established higher-ed tradition, the list of high-status universities has changed little over decades or even centuries. There have been some modest shifts in the relative status of different kinds of universities (for example, private versus public in the United States), and there are some impressive new institutions in Asia and other areas of rapid economic growth, but the impacts are marginal.
Broadly speaking, the 1911 list would not raise any eyebrows if it were used as the basis for the next U.S. News ranking. For those seeking to answer the question “What makes a great university?” the answer appears to be “having been great 100 years ago.”
Now compare the Dow Jones Industrial Average for 1911, which included such companies as the American Smelting and Refining Company (now Asarco), U.S. Rubber (now Uniroyal), and U.S. Steel. Some of those companies have vanished altogether, and others have survived as subsidiaries of a larger enterprise, but only General Electric is still included in the Dow Jones index. Most of today’s Dow Jones companies did not even exist in 1911.
What accounts for the remarkable stability of university rankings in comparison to the instability of big business, and for that matter, other nonprofits? More important, what implications does this have for university management and higher-education policy?
Several features of universities are important in explaining these outcomes. First, unlike other enterprises, universities almost never die and rarely merge. The 14 universities that formed the Association of American Universities, in 1900, are all still in existence, as are all those admitted since then.
Second, and directly related, universities are what are called, in the literature on industrial organization, “single-plant firms.” The vast majority have one (or at most two) main campuses, with a few peripheral offshoots. Apparent exceptions like the University of California system are in reality a set of distinct universities, linked only by notionally shared governance.
Those structural facts put an upper bound on the feasible size of a university. A single campus can’t accommodate more than about 40,000 undergraduate students without running into diseconomies of scale, such as constraints on the size of lecture halls. The biggest state universities reached that size in the 1970s, and their enrollments have remained broadly stable ever since. Elite private universities operate on a much smaller scale, typically 3,000 to 5,000 students, and most have maintained that size since the 1950s.
Taken together, those facts rule out many of the mechanisms by which markets reward success and punish failure. A successful university doesn’t typically create new campuses or even greatly expand its enrollments. (Some American universities are attempting to test that proposition by establishing offshore campuses such as those of Yale in Singapore and NYU in the UAE. If only they’d learned from the experience of their Australian counterparts who followed the same path in the 1990s, with results ranging from disappointing to disastrous.) Conversely, poor performance may create stresses of various kinds, but almost never leads universities to close down, or even to radically contract.
As a result, growth in the university system has occurred primarily through the creation of new universities, or through the upgrading of vocational-training institutions such as teachers’ colleges. At least initially, the new entries are almost always at the lower levels of the status hierarchy. The creation of a new research university, such as the University of California at Merced, is a rare event.
Those facts are enough to explain part of the difference between the relatively stable ranking of universities and the ever-changing rankings of top companies. With no departures, and limited possibilities for growth, the only way that universities can change their ranking is through a change in the (perceived) quality of their research and teaching. This is necessarily a slow process.
Moreover, universities are nonprofit enterprises that nonetheless generate substantial operating surpluses. In the absence of shareholders, the surplus generated by a university is available to improve the university’s standing, for example by hiring star professors, establishing new research centers, or adding facilities to attract students.
But while those structural features explain why the relative status of universities doesn’t change much from year to year or decade to decade, they don’t explain the near-constancy of the rankings over scales of a century or more.
In statistical terms, we can think of university status as a process characterized by mean reversion. That is, if a high-status university performs poorly for some time, perhaps because of poor leadership or bad hiring decisions, it is likely to recover the lost ground over time. Conversely, a lower-status university that does well for a few years will find it difficult to maintain its enhanced status.
The crucial factor in explaining mean reversion is the existence of exceptionally durable assets of various kinds, the most important of which are human and reputational. A long-established high-status university has a large body of alumni, Ph.D. graduates, former faculty members, and research collaborators. Apart from obvious benefits such as alumni donations, that group can be looked to as a source of legacy students, opportunities for graduate placements, and senior hires keen to return to their former affiliation.
One way to test that idea is to look for other examples of status competition where rankings remain stable over long periods. An interesting case is that of European soccer leagues. Unlike American sport leagues, these mostly lack a draft or salary cap. Mobility is supposed to be achieved through a system of promotion and relegation, in which the winners in lower-division competitions move up, while the bottom-placed teams in the higher division move down. In practice, however, promoted teams usually struggle, while those relegated one year often return to the higher division the next.
Moreover, while most teams are privately owned, few of the owners seek to extract profits. Rather, returns are plowed back into the team and used to attract better players. That in turn produces winning records, which attract more fans and more revenue. Once attracted, fans, like alumni in the university context, are commonly lifelong assets.
Not surprisingly, the results are the same as in competition between universities. Most of the European leagues, notably including those of Italy, the Netherlands, and Spain, are dominated by the same two or three clubs, decade after decade. The Scottish competition provides a typical, if somewhat extreme, example. In 118 years of competition, two clubs (Rangers and Celtic, known collectively as the Old Firm) have won 99 times between them.
The Scottish League also provides a good example of mean reversion. A financial scandal in 2012 led to the Rangers’ being declared insolvent and ejected from the competition. A reformed club was admitted to the Third Division (roughly the equivalent of the bottom tier of minor-league baseball), but with its top management, most of its money, and many star players gone. Crucially, however, the new Rangers retained the membership and fan base of the old one. It is rapidly ascending the status ladder and is set to resume its rivalry with Celtic in a season or two.
What should we learn from all this?
Most obviously, there is not much point in worrying about university rankings, whoever may issue them. Differences from year to year in a given ranking, or between different rankings, will inevitably be dominated by random noise. But even if changes in rankings reflect actual differences in performance, mean reversion ensures that this will mostly wash out over time. For the kinds of decisions for which rankings should matter, such as which university to attend, year-to-year variations are of no significance.
Second, it seems unlikely that university presidents, or other top managers, can make much difference in the way their institutions perform. As much as a decade at the helm is still simply too short to produce any sustained shift in rankings. Conversely, the departure of presidents and other top administrators, even under a cloud of disgrace, seems to have little or no impact on the status of the institutions concerned.
But the big question is whether the stable hierarchy is beneficial or harmful to the teaching and research mission of the university system as a whole. If harmful, what can be done about it?
As regards research, the advantages of stratification are obvious. The institutions at the top of the status hierarchy have continued to produce the bulk of research in leading journals, to earn Nobel Prizes and similar awards, and so on. Nevertheless, there are plenty of cases where this self-perpetuating elite might benefit from the challenge of outside perspectives.
For undergraduate education, American experience has shown that a highly stratified system works poorly. Competition for status encourages high-ranked institutions to restrict enrollments and to provide a high-quality experience to a small number of students, which inevitably implies high tuition fees. Since new entrants to the system inevitably enter with lower status, a steep and stable hierarchy implies that an increasing proportion of students attend poorly funded institutions that struggle to provide a quality education.
What, if anything, can be done to flatten the hierarchy and increase mobility within it? Sporting leagues have adopted solutions such as salary caps and draft systems for the recruitment of new players. Analogs could be imagined in the university context, but seem unlikely to command much support.
A more plausible response is a shift in public-financing priorities. If, as we’ve seen, success or failure in the status race is largely preordained, financing systems designed to reward and “incentivize” success are misconceived. Support should be allocated on the basis of need rather than used to amplify historical advantage. In this respect, President Obama’s initiative to expand access to community college is a step in the right direction.
John Quiggin is a fellow in economics at the University of Queensland; a columnist for The Australian Financial Review; a blogger for Crooked Timber; and the author of Zombie Economics: How Dead Ideas Still Walk Among Us (Princeton University Press, 2010).
John Quiggin 
Perhaps the Australian tradition of going to the most suitable university in your home city rather than looking around the country leads to the relatively equal status of the various tiers of university here, ie the “sandstone” (Sydney, Melbourne, UQ etc), “technical” (UTS, RMIT, QUT etc) and “redbrick” (Macquarie, La Trobe, Griffith etc). Another factor would be the policy of providing distance education through the regional universities which increases their scale and standing.
Whatever the case, the fact that a relatively large proportion of Australian undergraduates have the opportunity to attend a university of at least moderately high standing must be seen as a good thing.
Celtic and Rangers, of course, are respectively the Catholic and Protestant clubs of Glasgow. They hate each other worse than Shias and Sunnis hate each other, and almost as much as rival Trotskyist parties hate each other. Their fans stick with them, for generation after generation, for reasons that have little to do with soccer. This is where your mean reversion comes from.
You’re back at the Fin? Have you and Stutchbury kissed and made up?
@Uncle Milton
So the protestants got themselves into a financial scandal, eh? Why am I not surprised?
*runs away*
@Uncle Milton
No, I need to get CHE to fix my tag line.
Thank you, JQ; a fine article to show up the sillyness of the one-fits-all corporate approach and the importance of knowing the conditions under which a specific incentive scheme works.
Under a fully deregulated system, a university is trading on its name and traditions, and on the contacts students make on campus. If noone knows your university at the time you apply for work, your CV’s value is implicitly diminished; on the other hand, if it is a well-regarded, well recognised university, your CV jumps a few notches up the ladder. Sad but true.
Wealthy people send children to expensive private schools because of name recognition, and the contacts the children will make. The same goes for universities. For children who are more independent of their parents, in Australia I’d say that when they are of age, the kids gravitate to universities which are convenient for them, the nearest capital city campus for example.
The only winners out of the deregulated system are the top tier of the corporate hierarchy, the Vice Chancellors as an example. They make a bucket of money for doing what the vice chancellors of old managed to do on a reasonable but modest (in comparative terms) salary. The multiplier from VC to Lecturer Level A (or whatever the bottom bracket is called now) is substantially larger than the 1980’s before all this nonsense really kicked off.
There are some great changes which have occurred on campuses, but as far as I can see, most of that is technology-driven; surely these changes would have occurred under a $150K VC as under a $1.6M VC.
Which is underlying and unsaid reason Minister Pyne and his mates are pushing for deregulation, ostensibly so we can get “world class” (whatever that means) universities, but really to push the price of admission up so the sons and daughters of the rich can go from private school and on to university without any danger of ever mixing with the hoi poloi.
The “world class” which appears in vision and/or mission statements rankles me as well. I’m looking forward to going to the shop for a world class burger with some world class fries. The expression is, as you ask bjb, meaningless syllables casually strung together to give the impression of a lofty goal. “World class” regains its cachet only when preceding “VC salary”.
“Second, it seems unlikely that university presidents, or other top managers, can make much difference in the way their institutions perform”
La Trobe is a counter example. It was almost as prestigious as Monash at the start of the 90s. Now it is almost broke and has little prestige. So you can certainly hire people that will wreck your university. The other way is much more difficult.
The University of New South Wales has been running a staff survey in which one of the questions asks staff members to express a view about the world ranking the University should aspire to achieve: top 10, top 25 (or possibly top 20, I can’t remember), top 50, top 100, or another self-selected option.
Somebody, somewhere, must have dreamed that it was a sensible question to ask and would produce usable information. I wish I knew what disordered process of thought produced that result.
@J-D
Mania.
@conrad
I disagree. The leaders of Monash did their best to wreck the place also, pouring hundreds of millions into the moneypit of Monash global, getting done for plagiarism and so on. The problem was the emergence of a policy framework that ensured mismanagement across the entire sector.
The same policy framework benefitted all members of the Go8, however well or badly managed, and severely harmed 1970 vintage universities like La Trobe, Griffith and Macquarie. The unitechs were also winners and the regionals were losers.
As in the US case, what matters is the date and circumstances of foundation, not the quality of management.
Macquarie University (gum tree university) seems to be on the mend. It is a very attractive university in several fields – earth science, environmental sciences, linguistics to name only those for which it has been highly regarded for a long time and still is highly ranked.
Rankings of individual faculties or departments is, IMO, useful and meaningful to students as well as potential staff. These rankings (formalised ways of communicating reputation and perceptions) are linked to the work of the academics in specific areas rather than ‘the institution’ as a whole. Brand advertising is irrelevant in this context.
Ranking universities as a whole is a little like ranking portfolios of financial securities offered by financial institutions (funds). While an individual may be able to buy a fraction of a portfolio of securities sold by a financial institution, a student does not ‘buy’ a fraction of the portfolio of all subjects offered by all departments within a university at the same time or during the period of their studies and an academic does not work in all departments of a university at the same time or throughout their working life.
I put the word buy in single quotation marks to signal a more fundamental problem with treating education as a consumption or investment activity.
John Ralston Saul pointed out (and I am sure he is not the first or the last) that the education of the citizen is not solely about preparation for employment. It is also about preparation for enlightened and engaged citizenship, especially in a democracy. It is about preparation for social engagement at all levels and for living a satisfying and useful life. It is not possible to reduce all this to commercial calculation. Indeed, it is imperative not to do so.
Professor Quiggin says: “For those seeking to answer the question ‘What makes a great university?’ the answer appears to be ‘having been great 100 years ago’.”
In my childhood there was a rather cruel anti-Irish joke about the Dublin policeman who, upon being asked by a tourist the way to a certain destination, replied: “I can tell you how to get there, but if I were you, I wouldn’t start from here.” Perhaps this Hibernian cop was the spiritual ancestor of the university ranking system.
@Ikonoclast
The education of citizens is about preparation for enlightened and engaged citizenship, especially in a democracy. But universities (mostly) aren’t and never were. The education of citizens is about preparation for social engagement at all levels and for living a satisfying and useful life. But universities (mostly) aren’t and never were. Universities (mostly) aren’t and never were institutions for the education of citizens; if you wanted institutions for the education of citizens, you’d need to subject universities to changes so implausibly fundamental that you’d probably do better to start from scratch (for one thing, institutions for the education of citizens would need to be designed as universal ones — why should institutions for the education of citizens have a selective intake?). The university system does what it is (and always was) actually designed to do, moderately well. It is advisable (if not imperative) to distinguish propaganda from truth.
@J-D
Did I limit my comment to universities? If you read it carefully you will see I did not. However, my comment is applicable to universities also. Universities have not traditionally gone in for only narrow technical vocational education though some modern schools (like Law and Business Mangement) now tend to do so. There is broader teaching than mere teaching for vocation in universities. There always has been and always should be. You have a very narrow and unrealistic view of what universities are and what they do. You comments are also uninformed by any historical knowledge. It seems clear your views are formed by an adulthood entirely lived in the neoliberal era. You know and can conceive of nothing else other than the current dominant neocon ideology.
@Ikonoclast
No, you did not limit your comment to universities. If you read my comment carefully, you will see that I did not limit my comment to universities either: I referred to the possibility of creating from scratch non-university institutions for the education of citizens. (Note also that my imagining of things that don’t exist indicates, contrary to your ignorant assertions — what makes you think you know all about the contents of my mind, or about the period I grew up in? — my conceptual ability is not limited to existing institutions and ideologies but extends to radically different possibilities.) But my comment was primarily concerned with universities because universities were the topic of John Quiggin’s blog post.
My reading of the historical data is that the primary (although far from sole) attraction of a university education to university students, from their medieval foundations until now, is and has been the improvement of employment prospects (not only through narrow technical education, I should perhaps add, although on my reading the curriculum of medieval universities was narrowly technical). This doesn’t mean that I’m against the idea of creating and maintaining institutions that are primarily dedicated to the expansion of human understanding and the highest achievements of the human spirit. I’m in favour of that. Such institutions exist (like the University of the Third Age) and a good thing, too. Actual universities, however, contemporary and historical, don’t fall into that category and never did.
If your interpretation of the available historical information is different from mine, please feel free to expand on it. I’m interested in opportunities to learn more. But don’t make the erroneous step of reasoning from the premise that you know some things that I don’t to the conclusion that I know nothing.
@Ikonoclast
It is a shame that Fed Ed Minister Pyne & his fellow L-NP MPs, including the current Treasurer, do not have this at the core of their deliberations & policies when it comes to education in Aus. It is high time that all Govts realised a business model can’t & won’t fit education which deals with the many slippery variables involving & surrounding human beings.