Skip to main content
A scary good deal on trusted journalism
Get full digital access to globeandmail.com
$0.99
per week for 24 weeks SAVE OVER $140
OFFER ENDS OCTOBER 31
A scary good deal on trusted journalism
$0.99
per week
for 24 weeks
SAVE OVER $140
OFFER ENDS OCTOBER 31
// //

A serious effort to measure the quality of a university education in Ontario would be prohibitively expensive. More likely, universities and the government will devise performance indicators that are easy and cheap to measure -- retention rates, for example, or job placement rates.

Frances Woolley is a professor of economics at Carleton University, where she teaches public finance. Her recent Economy Lab posts can be found here.



In a highly influential report, Harvey Weingarten and Fiona Deller argue that greater differentiation between Ontario universities will lead to higher quality academic programs. This begs the question: what does "quality of an academic program" mean?



Quality could mean international prestige. According to the Times Higher Education University World Rankings, UBC is the 22nd best university in the world, and University of Toronto is the 19th. That ranking is based primarily on the amount of research produced, and the influence that research has. Yet the presence of a Nobel Prize winner down the hall and around the corner is irrelevant to the first year student being taught by an instructor who plays no part in the research enterprise. International prestige is no guarantee of a quality undergraduate program.

Story continues below advertisement



Quality could mean the quality of instruction -- but this begs the question: what is good quality instruction? Most commonly used measures of instructional quality are deeply flawed.



The Macleans rankings assess instructional quality by dividing the number of students by the number of full-time faculty members. But a part-time contract instructor can bring real-world experience -- and valuable industry connections -- that a full-time professor might lack. Moreover, there is little evidence that students learn more in smaller classes.



Teaching evaluations are a more direct, but equally imperfect, measure of instructional quality. Studies have found that students generally give higher evaluations to professors who give them higher grades, raising questions about what the evaluations are actually measuring, and suggesting that any attempt to place more weight on student evaluations could exacerbate grade inflation.



Another metric of instructional quality is retention rates -- do students drop out? The value of a university degree is, in part, that it signals a person's ability to see things through, to persevere and accomplish their goals. If students cannot fail, a university degree's value as a signal of ability diminishes.



A final interpretation of "the quality of an academic program" is that a high quality program is one in which a student learns things that are useful. But what is useful? Humanities degrees are widely condemned as useless, but a student studying the humanities will learn to write clearly and think analytically. In the long run, those skills have a high return. As a former student once said to me, "My MA in Economics got me my job, but my undergraduate degree in philosophy got me my promotions."



As a professor and as a parent, I know that the quality of education matters. But a good quality measure would require, for example, an outside expert reading over Eco 100 final exams, to see if the questions asked were clear and reasonable, or sitting in on a professor's lecture, and giving her helpful feedback on her instructional style. Given the current budget situation in Ontario, the chances of resources being devoted to providing a serious assessment of the quality of education are basically nil.



What is more likely, is that universities and the Ontario government will devise performance indicators that are easy and cheap to measure -- retention rates, for example, or job placement rates.

Story continues below advertisement



The problem with incentives, however, is that they work too well: universities will game the system. Suppose, for example, that the quality of a university degree is measured by the percentage of students who are employed one year after graduation. Studies have found that some groups have an easier time getting jobs in the Canadian labour market than other groups, all else being equal. So a university wanting to maximize the percentage of students finding jobs could add programs that tend to attract the more hirable groups, and cancel programs that attract the less hirable groups.



Sound apocryphal? The experience of other jurisdictions suggests that once incentives are put in place, universities react swiftly and strongly -- but not always in predictable or desirable ways.



Economy Lab, winner of the 2011 Eppy Award for best business blog. Follow Economy Lab on twitter

Your Globe

Build your personal news feed

  1. Follow topics and authors relevant to your reading interests.
  2. Check your Following feed daily, and never miss an article. Access your Following feed from your account menu at the top right corner of every page.

Follow the author of this article:

View more suggestions in Following Read more about following topics and authors
Report an error Editorial code of conduct
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

If you do not see your comment posted immediately, it is being reviewed by the moderation team and may appear shortly, generally within an hour.

We aim to have all comments reviewed in a timely manner.

Comments that violate our community guidelines will not be posted.

UPDATED: Read our community guidelines here

Discussion loading ...

To view this site properly, enable cookies in your browser. Read our privacy policy to learn more.
How to enable cookies