A serious effort to measure the quality of a university education in Ontario would be prohibitively expensive. More likely, universities and the government will devise performance indicators that are easy and cheap to measure -- retention rates, for example, or job placement rates.
Frances Woolley is a professor of economics at Carleton University, where she teaches public finance. Her recent Economy Lab posts can be found here.
In a highly influential report, Harvey Weingarten and Fiona Deller argue that greater differentiation between Ontario universities will lead to higher quality academic programs. This begs the question: what does "quality of an academic program" mean?
Quality could mean international prestige. According to the Times Higher Education University World Rankings, UBC is the 22nd best university in the world, and University of Toronto is the 19th. That ranking is based primarily on the amount of research produced, and the influence that research has. Yet the presence of a Nobel Prize winner down the hall and around the corner is irrelevant to the first year student being taught by an instructor who plays no part in the research enterprise. International prestige is no guarantee of a quality undergraduate program.
Quality could mean the quality of instruction -- but this begs the question: what is good quality instruction? Most commonly used measures of instructional quality are deeply flawed.
The Macleans rankings assess instructional quality by dividing the number of students by the number of full-time faculty members. But a part-time contract instructor can bring real-world experience -- and valuable industry connections -- that a full-time professor might lack. Moreover, there is little evidence that students learn more in smaller classes.
Teaching evaluations are a more direct, but equally imperfect, measure of instructional quality. Studies have found that students generally give higher evaluations to professors who give them higher grades, raising questions about what the evaluations are actually measuring, and suggesting that any attempt to place more weight on student evaluations could exacerbate grade inflation.
Another metric of instructional quality is retention rates -- do students drop out? The value of a university degree is, in part, that it signals a person's ability to see things through, to persevere and accomplish their goals. If students cannot fail, a university degree's value as a signal of ability diminishes.
A final interpretation of "the quality of an academic program" is that a high quality program is one in which a student learns things that are useful. But what is useful? Humanities degrees are widely condemned as useless, but a student studying the humanities will learn to write clearly and think analytically. In the long run, those skills have a high return. As a former student once said to me, "My MA in Economics got me my job, but my undergraduate degree in philosophy got me my promotions."
As a professor and as a parent, I know that the quality of education matters. But a good quality measure would require, for example, an outside expert reading over Eco 100 final exams, to see if the questions asked were clear and reasonable, or sitting in on a professor's lecture, and giving her helpful feedback on her instructional style. Given the current budget situation in Ontario, the chances of resources being devoted to providing a serious assessment of the quality of education are basically nil.
What is more likely, is that universities and the Ontario government will devise performance indicators that are easy and cheap to measure -- retention rates, for example, or job placement rates.
The problem with incentives, however, is that they work too well: universities will game the system. Suppose, for example, that the quality of a university degree is measured by the percentage of students who are employed one year after graduation. Studies have found that some groups have an easier time getting jobs in the Canadian labour market than other groups, all else being equal. So a university wanting to maximize the percentage of students finding jobs could add programs that tend to attract the more hirable groups, and cancel programs that attract the less hirable groups.
Sound apocryphal? The experience of other jurisdictions suggests that once incentives are put in place, universities react swiftly and strongly -- but not always in predictable or desirable ways.
Economy Lab, winner of the 2011 Eppy Award for best business blog. Follow Economy Lab on twitter
Frances Woolley