Skip to main content

The Globe and Mail

Rogers says software glitch led to massive wireless outage

A man walks by a Rogers store at Yonge and Eglinton, Toronto, August 27, 2013.

Gloria Nieto/The Globe and Mail

Rogers Communications Inc. said a software glitch created a big spike in "signalling traffic" that caused one of the worst wireless network outages in the company's history.

Canada's largest wireless carrier determined that root cause on Thursday roughly 18 hours after implementing a fix that restored voice and text services for customers across the country.

The vast majority of the company's 9.42 million wireless subscribers lost those services Wednesday, starting around 6:00 p.m. ET. The software fix was employed around 10:30 p.m., which enabled a full restoration of services a short time later. Data services were not affected by the outage.

Story continues below advertisement

Although the problem lasted for less than five hours, the furor that erupted over the service interruption, particularly on social media sites like Twitter, reflects how dependent Canadians have become on their smartphones.

A software glitch on a switch caused the surge in signalling traffic on one of two networks the company runs in parallel to provide cellphone service. While one network carries voice traffic, a second network is used for "signalling" or the instructions for handling calls. For instance, signalling is used to ensure that a call does not drop if a customer is moving while talking.

"There was a big spike in signalling traffic and what happened is the mobile switches didn't handle it properly, and they basically restarted. And when they restarted, it caused the service interruptions," said Bob Berner, executive vice-president of network and chief technology officer.

Some switches took longer than others to reset. The software fix should prevent any further service interruptions because it changes how switches handle a signalling surge, the company said. There is no indication that the outage was caused by hacking or a security breach.

"We've been doing this for 28 years and we've never seen anything like this happen before. I don't expect it to happen again," added Mr. Berner.

Telecom consultant Mark Goldberg questioned whether the federal telecom regulator should be collecting and sharing information on network outages to assist consumers.

Canadians are among the world's heaviest users of smartphones and text messaging is extremely popular. Canadians send more than 270 million text messages per day, with a total of 96.5 billion texts sent in 2012, according to the Canadian Wireless Telecommunications Association.

Story continues below advertisement

"It has become part of people's lives," added Mr. Berner. "If you just watch people anywhere on the street, everyone is always doing something on their wireless device."

Moreover, an increasing number of Canadians are ditching their home phones altogether with roughly 12.8 per cent of Canadian households considered "wireless only" in 2011, according to the latest data from the CRTC. There are also fewer payphones available for Canadians to use when cellphone networks fail.

Rogers CEO Nadir Mohamed, meanwhile, apologized to customers for the inconvenience.

"I recognize this service interruption was unacceptable for our customers," Mr. Mohamed said. "We worked as quickly as possible to restore service and it was gradually restored over the course of the evening."

Rogers and its Fido subsidiary said they would credit postpaid wireless subscribers for one day of service because of the outage. It is unclear on how much that would cost the company in lost revenue.

Since the Rogers' corporate website received a surge in traffic on Wednesday evening, some customers complained they were unable to pay bills or check their Internet usage. The company is advising customers to call if they have concerns about their bills.

Story continues below advertisement

Rogers also advised customers to use land lines if they needed to reach police and other emergency services on Wednesday evening.

Many customers took to social media to complain about losing service, and some of those complaints were misdirected. Glenn Rogers, who lives in Brooklyn, N.Y., and uses the @rogers handle, saw his Twitter feed inundated by angry customers.

"The wrath of a thousand Canadians is a mighty sight," he tweeted. He added: "Thanks for all the kind tweets Canada. May they never take your phone or internet from you again."

Report an error Licensing Options
About the Authors
National news reporter

  More

Brent Jang is a business reporter in The Globe and Mail’s Vancouver bureau. He joined the Globe in 1995. His former positions include transportation reporter in Toronto, energy correspondent in Calgary and Western columnist for Report on Business. He holds a Bachelor of Commerce degree from the University of Alberta, where he served as Editor-in-Chief of The Gateway student newspaper. Mr. More

Comments

The Globe invites you to share your views. Please stay on topic and be respectful to everyone. For more information on our commenting policies and how our community-based moderation works, please read our Community Guidelines and our Terms and Conditions.

Please note that our commenting partner Civil Comments is closing down. As such we will be implementing a new commenting partner in the coming weeks. As of December 20th, 2017 we will be shutting down commenting on all article pages across our site while we do the maintenance and updates. We understand that commenting is important to our audience and hope to have a technical solution in place January 2018.

Discussion loading… ✨