Skip to main content

Traffic makes its way to Ambassador Bridge that connects Canada to the United States at Windsor, Ontario. Trying to analyze the benefits of a free-trade agreement, for example, requires more than just adding up imports and exports.

Mark Spowart/THE CANADIAN PRESS

Good economic analysis is surprisingly hard, as in many (but not all) cases it is impossible to run controlled experiments. To analyze the effects of a policy or event, economists must filter out everything unrelated to that event. They don't ask "how is the world different after that event?" but rather "how would the world be different had that event not taken place?" In economics speak, we need to consider the counterfactual.

Bad economic analysis, on the other hand, is quite straightforward; take a policy change, eyeball economic data before and after the change and jump to a conclusion. Three recent issues in the news are illustrative of this.

One example is Jim Stanford's piece on why the Canadian-U.S. Free Trade Agreement was a failure for Canada. Stanford includes a number of metrics to support his argument, such as: "[i]n the mid-1980s, 19 per cent of all U.S. imports came from Canada. Today, our share of their imports is just 14 per cent."

Story continues below advertisement

This statistical examination, while interesting on the surface, is ultimately meaningless. We cannot draw any conclusions from the data, as we need to ask "what would our share of imports be in the absence of the free trade deal". A great deal has changed since 1985, including the emergence of China and India as world trading powers. It is naive to think that international trade would have remained the same had we not signed a trade deal with the U.S.

We need to control for factors outside of the trade deal and consider the counterfactual where the deal had not been signed. In a 2004 study, economist Daniel Trefler does just this and finds a small, but positive, economic impact from the deal, including a 14 per cent increase in plant-level labour productivity. By failing to consider the counterfactual, Mr. Stanford comes to the wrong conclusion.

A second example is illustrated in Corporate tax revenues higher despite lower rate, as Bill Curry discusses how both supporters and detractors of the tax cuts will find evidence of their positions:

"Looking over the last four years of data, advocates of the tax cut could argue that cutting the rate has had virtually no impact on federal revenues. Critics of the tax cut could compare the latest numbers to the 2007-08 fiscal year and argue that revenues are down $8.9-billion."

Both positions are inherently flawed, as both fail to consider the counterfactual of what would have happened without the tax cuts. The lower corporate rate does effect investment, output and pre-tax corporate profits and therefore tax revenue (a fact often missed by detractors of the cut), but there are factors other than the tax cut that alter corporate revenue (a fact missed by those who would use the revenue change to prove the tax cuts pay for themselves). The best estimates economists have show that corporate tax revenues would be higher in Canada with higher federal rates, but the relationship is far from 1-to-1 (raising rates by 10 percent does not lead to 10 per cent more revenue for the government).

Finally, David Suzuki, in An economy is a means, not an end:

"But spending money on things that anyone would see as negative, or even horrendous, contributes to positive GDP growth – including disastrous oil spill cleanups, car accidents…"

Story continues below advertisement

This fallacious line of thinking has a name, the broken window fallacy. It is a fallacy because we need to consider how those resources would have been used if they were not cleaning up oil spills and how that activity would have increased economic activity. It is a fallacy because it fails to consider the counterfactual.

To truly understand the economic impact of a policy, we need to control for other factors which may have changed unrelated to that policy. If we do not consider the counterfactual, we are likely to be led astray.

Report an error Editorial code of conduct
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

Read our community guidelines here

Discussion loading ...

Cannabis pro newsletter
To view this site properly, enable cookies in your browser. Read our privacy policy to learn more.
How to enable cookies