Skip to main content

Cardiac death rates 100% more than predicted? Sometimes numbers aren’t exactly what they seem

When the Canadian Institute for Health Information (CIHI) published a report card on the country’s cardiac centres, it estimated the death rate at the Peter Munk Cardiac Centre (PMCC), the country’s premiere heart surgery facility, should be 1 per cent.

In reality, it was almost double that rate – nearly 2 per cent.

That may not seem like a big difference to a lay person. But surgeons at the PMCC were outraged. The administration of the University Health Network wanted answers. So, too, did donors and patients.

Story continues below advertisement

“When CIHI reported results, it looked like our mortality was twice what it should be,” said Barry Rubin, a vascular surgeon and director of the Munk Centre. “That’s not good. And we knew it was wrong.”

In a new paper, published in Tuesday’s edition of the medical journal CMAJ Open, a group of researchers, including Dr. Rubin, explain why it was misleading.

The detailed explanation has to do with the complex mathematic of modelling and risk adjustment, and the differences between administrative and clinical data.

A simpler version is that the way CIHI predicted mortality did not adequately take into account the complexity of patients.

About 2.4 million Canadians live with heart disease; it is the second leading cause of death. Tens of thousands of patients a year undergo procedures such as angioplasty, bypass surgery and valve replacement.

The procedures are remarkably safe and successful. But they are not without risk. Outcomes data is collected and then used to predict risk for future patients. Large variances between expected and actual outcomes raise red flags.

That risk varies a lot based on a patient’s condition. For example, doing triple bypass on a seemingly healthy patient whose blocked arteries were discovered during routine testing is very different from doing triple bypass on a patient who had a massive heart attack and had to be revived using CPR.

Story continues below advertisement

The Peter Munk Cardiac Centre, as with other programs located in teaching hospitals, tends to attract the top surgeons as well as the most difficult cases.

Paradoxically, this means the best programs tend to have the highest mortality rates.

CIHI, which collects and publishes data on almost all aspects of Canadian health care, principally uses administrative data – basic information on patients (anonymized, of course), procedures and outcomes.

But there are other, more sophisticated models. One of them, which the PMCC subscribes to, was created by the Society of Thoracic Surgeons.

It captures far more detailed information on patients, notably seven medical conditions that predict worse outcomes after heart surgery: heart failure, abnormal heart rhythm, recent heart attack, very low blood pressure (shock), recent cardiopulmonary resuscitation or a heart pump, or kidney failure.

In the study published in CMAJ Open, the researchers looked at data on 1,635 PMCC patients, 32 of whom died within 30 days of surgery. The CIHI model predicted a mortality rate of 1.03 per cent. The STS model predicted 1.96 per cent – exactly the same as in reality.

Story continues below advertisement

It may seem unusual for a heart centre to publicly advertise its mortality rate. But Dr. Rubin said this is important for patients to know because they should be making decisions about surgery based on the most complete information available.

“What we want is an accurate portrayal of risk,” Dr. Rubin said. “The purpose of measuring is not to say how good we are, it’s to identify problems and fix them.”

The other reason for data transparency is that it matters in attracting the best staff. Surgeons are acutely aware of their outcomes, and models that underestimate risk can lead them to avoid higher risk patients.

“We want to be the place that takes care of the sickest,” Dr. Rubin said. “But we shouldn’t be penalized for doing so.”

CIHI, to its credit, acknowledges the shortcomings of the way it measures, and even participated in the new research. But switching from administrative to clinical data would require more time, more effort and more money, and only some centres collect the more detailed clinical data.

While this may seem like a purely technical issue, there is actually an important lesson here for the public and policy makers: Report cards are great, but what and how you measure matters a lot.

That’s an important caveat in an era where we are obsessively measuring, encouraging patients to use data to make wise choices, and even moving to provide funding based on outcomes.

As the new research states dryly, “Caution is warranted.”

Report an error Editorial code of conduct
Comments

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • All comments will be reviewed by one or more moderators before being posted to the site. This should only take a few moments.
  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed. Commenters who repeatedly violate community guidelines may be suspended, causing them to temporarily lose their ability to engage with comments.

Read our community guidelines here

Discussion loading ...

Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.
Cannabis pro newsletter