The Doctored Rankings Problem
In US News’ college rankings, explicit deception has been dangerously festering for years. USC got caught sending doctored data a few months ago. Columbia University was recently also exposed, by one of its own professors, for doctoring data. Last year, a Dean from Temple University got prosecuted for submitting fake data. There are plenty of reasons to believe that these efforts aren’t unique. Anecdotes, often from former elite college administrators, about gaming the rankings are quite plentiful. In fact, Columbia and USC are but symptoms of a much broader disease. Consumers are being misled.
One of the roles we have delineated to government in our economic system is to protect consumers from explicit deception. But, in the case of college rankings, the government has failed to meet its responsibility. This must change.
In this article, I explain how exactly rankings data is doctored by top elite colleges, I explain why lying might be even more pervasive than we thought, I explain why honest rankings matter, and finally I suggest what the DOJ & FTC can do to fix this rampant cheating.
Columbia University’s Lies: A Rankings Scandal Explored
Columbia University was recently caught in a massive rankings scandal. Put plainly, over the course of many years, Columbia’s administrators habitually lied about and misrepresented their data to juice their US News Ranking. Partly as a result of their cheating, although not exclusively, Columbia has steadily risen in the U.S. News & World Report Ranking over the last two decades, and the university even ascended all the way to #2 in last year’s ranking. Columbia should have known better. Cheaters never prosper.
In hindsight, climbing to the number two spot can be understood by metaphor as Icarus’ flight. In the end, Columbia flew too close to the sun. With the attention, scrutiny, and pressure such a high ranking created, it was only a matter of time before someone exposed the fraud. Finally, Columbia’s wax wings melted under the heat of scrutiny.
Enter Michael Thaddeus, a tenured Math Professor at Columbia. An ethical broom at his own school, earlier this year, Michael Thaddeus published a blog post exposing all the ways in which Columbia’s ranking is juiced. As Michael Thaddeus said, “[Columbia’s] ascendancy may largely be founded, not on an authentic presentation of the university’s strengths, but on a web of illusions.” Once Michael Thaddeus opened the kimono, everyone took a peak. The New York Times then published a long piece explaining the rankings scandal. Other news outlets have also covered this issue.
In response, Columbia has denied that anything improper took place. Columbia claims to be conducting an internal investigation to gather the facts. This has been going on for a while, and recently Columbia even announced that it won’t participate in this year’s U.S. News ranking while it figures out what is going on with its data.
I’ll bite. The excuse of an internal investigation is likely complete bull. Of course, Columbia is backed into a corner. If they participate in the rankings and change their numbers to be more honest, they’ll have to implicitly admit they lied before. Plus, they’ll fall in the rankings, and this fall from grace will draw a ton of attention. If they participate and continue to lie, they’ll only invite more scrutiny and possibly legal trouble. My theory is that this is why they’re sitting this year out. Columbia wants to see if they can get away with not being a part of the rankings.
But, how do I know they lied? I read Mr. Thaddeus’ report. So, let’s be very explicit about what happened.
How Columbia Lied
For US News rankings work, elite universities have to truthfully fill out surveys about their data. This data is then put through the US News formula which assigns weights to certain criteria. The end result is a single number assigned to a college. This number is where the college ranks. The central idea behind the US News ranking is that many different elite and non-elite colleges can be compared in one unified forum. But of course, the US News ranking is path-dependent on honest data. If some colleges start faking the data, then no matter the formula, the ranking isn’t fair to those who are honest.
There are many areas where Columbia continually misrepresented its numbers, and if you’re interested in a more detailed breakdown, you can read Mr. Thaddeus’ report here. I’ll focus on just three.
Firstly, one criteria for the US News rankings is instructional spending per student. Columbia falsely included spending by its hospital, on things entirely unrelated to instruction, in this category. As Mr. Thaddeus writes, “Columbia’s stated instructional expenditures are implausibly large and include a substantial portion of the $1.2 billion that its medical center spends annually on patient care.”
Secondly, another criteria for the rankings is graduation rates & outcome measures. Columbia excluded the graduation rates and outcome measures of its transfer students in this area. As Mr. Thaddeus confirms, “We show that Columbia’s performance on some, perhaps even most, of these measures would plunge if its many transfer students were included.”
Thirdly, a different criteria for the rankings is the percentage of faculty with PhDs or terminal degrees. Columbia reports that 100% of its faculty satisfy this criterion. As Mr. Thaddeus explains, “The 100% figure claimed by Columbia cannot be accurate. Among 958 members of the (full-time) Faculty of Columbia College, listed in the Columbia College Bulletin online, are included some 66 persons whose highest degree, if any, is a bachelor’s or master’s degree.”
Why is Lying Likely to be Pervasive?
As I’ve previously described, the relationship between the elite colleges and the US News can be understood as a hub-and-spoke cartel. This structure occurs when a hub (the US News) coordinates the actions of its spokes (the elite colleges). In the case of the US News rankings, there is strong evidence that institutional spending & admissions policies at elite colleges shift depending on what the US News’ ranking criteria values. Yet, it is also true that the elite colleges have often been caught trying to lie to the US News. What explains these facts?
The history of doctored data at many colleges actually only supports the hub-and-spoke theory. First, if colleges didn’t care about rankings, why would they go through the trouble to lie? Second, in many other hub-and-spoke arrangements, the spokes often lie to their hub in an attempt to unilaterally seek advantage. As Joe Harrington Jr. explains, “Crucial to the effectiveness of collusion is monitoring for compliance. If a firm does not anticipate being monitored then it is likely to cheat.”
In the elite college context, in publicly available data like enrollment or test scores, elite colleges find it hard to lie to the US News. Therefore, there is much more conformity around scarce enrollments. However, in areas where checking is harder, elite colleges likely routinely lie (as the USC and Columbia examples show).
As Mr. Thaddeus concludes, “Even on its own terms, the ranking is a failure because the supposed facts on which it is based cannot be trusted. Eighty percent of the U.S. News ranking of a university is based on information reported by the university itself. This information is detailed and subtle, and the vetting conducted by U.S. News is cursory enough to allow many inaccuracies to slip through. Institutions are under intense pressure to present themselves in the most favorable light. This creates a profound conflict of interest, which it would be naive to overlook.”
This conflict of interest means that there is a high likelihood that others are also lying, even if they haven’t been found out yet.
Why this Matters?
I personally think that the US News Rankings are entirely arbitrary. But, even I cannot deny the simple fact that the US News Rankings exert tremendous influence on student decisions. Quantifying the impact of the a college’s US News rank, a Harvard Business School study finds that, “a one-rank improvement leads to a 1-percentage-point increase in the number of applications to that college.” Students clearly make decisions on where to go based on the US News’ ranking criteria.
Therefore, it is of importance that the data presented by elite colleges is honest, no matter how arbitrary the formula & criteria that US News uses. If we continue to allow doctored data to become more and more prevalent, the tool so many students use to approximate quality will lead them further and further astray.
An honest rankings is particularly important because if elite private colleges like Columbia and USC extract a dishonest advantage in the rankings, these entitled schools make it even more difficult for high quality public schools like Michigan and UVA, who are less likely to cheat because of public scrutiny, to rank as highly as they otherwise would. This leads to further distorted behavior. For example, rankings pressure incentivizes public schools to increase admissions for out-of-state and international students to boost revenues to compete with colleges that are lying. This hurts bright in-state students who now have fewer seats available to them. Further, the brightest students might be more attracted to dishonest schools with higher rankings, again hurting the ability of public schools to attract the best talent.
Essentially, these high rankings create huge advantages which tend to compound over time. Higher rankings means more alumni donations, general prestige, and larger endowments. All of these things mean even higher rankings in the future. There is a flywheel at work, and honest schools are disproportionately hurt. Bad behavior in reporting, institutional spending, and admissions policies becomes the norm. We need to fix this.
How to Clean Up the US News Rankings?
The DOJ has already demonstrated its ability to prosecute deception & fraud of the US News rankings when they prosecuted Moshe Porat of Temple University for wire fraud. As the DOJ announced, “Moshe Porat … was convicted today after trial on charges that he conspired and schemed to deceive the school’s applicants, students, and donors into believing that the school offered top-ranked business degree programs, so that they would pay tuition and make donations to Temple.”
The DOJ needs to bring more cases of this type. Currently, there is far more fraud than prosecutions. Who from Columbia will be held responsible? Who from USC? What about Claremont McKenna? Who will be held responsible at all the schools where fraud is occurring but is not being reported by journalists? The DOJ should launch a thorough investigation into this type of practice. They should subpoena records from the US News and they should cross verify those submissions against the data submitted to the Department of Education, IRS, and others.
But, even if the DOJ won’t ramp up enforcement, the FTC should. As the FTC proclaims on its website, “Section 5(a) of the FTC Act provides that “unfair or deceptive acts or practices in or affecting commerce . . . are . . . declared unlawful.” Furthermore, as the FTC elaborates, “‘Deceptive’ practices are defined … as involving a material representation, omission or practice that is likely to mislead a consumer acting reasonably in the circumstances.”
Certainly, this is an area where a material representation is likely to mislead a consumer acting reasonably. After all, if a student opens the US News ranking, this student is likely to reasonably believe that a certain college has fairly topped the US News criteria. This student may decide to go to a certain college for that reason.
In 2020, the FTC went after a different ranking in a different industry on the basis of deceptive practices. So, why not this ranking, and why not this industry? Someone please, for the love of god, demand the records. Let’s finally see if these emperors are wearing clothes or not.
Conclusion
When students get caught cheating in college, they’re academically sanctioned. But what happens when colleges get caught cheating students? Often, nothing happens. This is wrong. It must change, and the FTC and DOJ can do something about it. Its time they act.
Please post this article on Reddit, Twitter, Hacker News, or anywhere else it can help promote discourse and awareness.