Domestic Affairs

Disregard university rankings

By Patrick Mattimore (chinadaily.com.cn)
Updated: 2011-06-02 09:27

There is an opinion in Wednesday's edition of China Daily (not necessarily reflecting the views of the editors) critical of the latest rankings of Chinese universities. The writer suggests that the indices used in the rankings are questionable. One index she cites is the number of hits a particular college generates on the Internet.

The commentator allows that the subjective criteria should be replaced with indices that are absolutely objective.

I agree with the writer that the rankings are broken. I don't believe, however, that it's worth trying to fix them by adopting objective criteria.

Speaking to a group of listeners at the global forum on "Rankings and Accountability in Higher Education" held last month in Paris, Phil Baty, deputy editor of the Times Higher Education magazine, said that "rankings are rather crude; they can’t be completely objective." The Times publishes an annual world university ranking.

American universities have had a long and uneasy relationship with rankings.

Robert Morse, Director of Data Research at US News & World Report heads a team that compiles the mother of all rankings, the best colleges issue of USN&WR magazine. Morse wrote several years ago that the key ranking issue in the US is not the data or the methodology used to arrive at a school's ranking. It's doing ordinal rankings in the first place.

Magazines like US News have a benign explanation as to why they rank schools: to help students make one of the most important decisions of their lives.

A cynic might suggest a more compelling interpretation. The college issues are frequently the magazines' best-selling issue of the year. That fact has allowed magazine publishers to become driving forces in the college admissions industry in the US.

Although no collusion scandal has yet arisen in the rankings industry, it is not hard to imagine the day when a Yale University admissions counselor, for example, offers a college rankings magazine official a bribe to get a higher ranking than Harvard. The stakes have become too high for colleges to ignore their rankings and the temptations are too great for data gatherers to accept the kinds of payoffs typical of many industries.

Many things can be ranked but universities should not be among them. For example, there are 15 players on each team in the NBA finals between the Dallas Mavericks and the Miami Heat. The 30 players could be assigned ordinal ranks based upon the average number of points they have scored per game or their individual heights. In a swimming competition, the finishers can be placed in the order in which they finish.

Some things can be partially ranked. The winner of the NBA championship can and will be ranked number one in the NBA. But, after declaring a winner, it's problematic to rank order the rest of the playoff teams. Is the team that lost in the finals necessarily number two? Well, a case could me made that if the eventual champion had a tougher matchup with a division rival in the playoffs, that team should be ranked ahead of the runner-up. The waters get even muddier and more subjective if one tries to rank teams based upon performances earlier in the playoffs.