Bernard Savoie thought he'd planned for every kind of computer crack-up that could possibly hit his system.
For one thing, the manager of computer operations and telecommunications at Moncton-based Co-op Atlantic, had built-in redundancy to keep things humming should a drive fail in his database server.
That common precaution wasn't enough. Mr. Savoie learned some hard lessons about information backup and recovery recently when the unthinkable happened: Not one but two drives failed within minutes. Co-op Atlantic had no computers for the next 18 hours -- a serious lapse for a $500-million-a-year wholesale and retail grocery business.
"In theory it could happen, and it did," Mr. Savoie says ruefully. "Before we could resynchronize our order-processing systems and validate that everything was running appropriately, we were looking at about an 18-hour period where we really were not able to process orders. . . . What took all the time was rebuilding the disks in such a way that we could be sure of the integrity of the data."
Bernard Savoie, manager of computer operations and telecommunications at Moncton, N.B.-based Co-Op Atlantic holds a handheld computer that can scan items and give information and statistics on stock. (The Globe and Mail)
Related Articles
Though not devastating for the company's business, which has 135 retail and wholesale stores in Atlantic Canada, it was serious enough to prompt an information technology overhaul.
The crash also provided lessons that could benefit other businesses. Hewlett-Packard Co., which was in the process of designing new data backup and recovery products for small and medium-sized businesses (SMBs), went to school on the Co-op Atlantic experience, says Paul Brousseau, small and medium-sized business marketing manager at Mississauga-based HP Canada.
"In the medium-sized business space, we're dealing with companies that are like Co-op Atlantic in that they have some backup strategy. But we know it's essentially sporadic," he says. "Often it's not a defined part of their IT strategy, even though the implications of downtime are huge. . . . If you don't have the expertise in-house, then it's very difficult to protect yourself."
Difficult, but far from impossible, says Thomas Slodichak of Burlington, Ont.-based WhiteHat Inc., which provides IT security services to companies of all sizes.
Mr. Slodichak, WhiteHat's chief security officer, says that small businesses first need to shed the "it can't happen here" attitude. He recommends that smaller companies adopt the sort of disaster-recovery testing exercise that's common among larger enterprises, where "on a specific day you unplug the server, make data unavailable and say, 'Okay, what are you going to do now?' " The most common mistake among SMBs is a lack of attention or funding for that kind of activity, he says. "The key message is create a strategy and act on it."
Joel Martin, vice-president for software at market researcher IDC Canada, sees remote storage of mission-critical information as the greatest challenge facing SMBs in terms of business continuity. It's also a growth market for suppliers, as increasingly tech-reliant SMBs stockpile information.
"It's an opportunity for the small IT shops as well as for the IBMs, the HPs and the Dells," Mr. Martin says. "The other obvious companies that should be coming up with these programs for SMBs are the communications companies that provide broadband. They know how much data small business uses, and they should have a great opportunity to promote remote backup solutions.
"In fact," he says, "we are starting to see this happen with Microsoft MSN, with Google, with Yahoo, where they're offering gigabytes of e-mail storage."
He also notes that International Business Machines Corp. is pushing computer lines with the capacity to mirror the hard drive and maintain data integrity in the event of failure.
Storage is also a major component of the HP package unveiled earlier this year for small businesses.
"We're dealing with customers who are backing up all their information on their PCs, on their notebooks," Mr. Brousseau says. "They are hugely vulnerable. If that PC fails, if there's an employee error, if there's a virus attack -- you name it -- they're at tremendous risk of downtime."
As a result of researching incidents such as Co-op Atlantic's backup failure, HP is offering several packages for small and medium-sized businesses starting in the $3,500 range.
On the client side, there's a desktop management service -- essentially outsourcing the management of the desktop and backing up all information remotely for quick retrieval after a system failure. A service called electronic vaulting enables an organization to take its most critical data and put it in a secure, remote site. The Smart Office client protection and network security solution includes a security vulnerability assessment service. Proactive Plus continuously monitors the company's backup systems to help relieve the load on a small business's IT staff.
In Moncton, as a result of the crash, Mr. Savoie replaced disk storage components and installed new monitoring software. "Not only do we have RAID technology [redundant array of independent disks], we also have additional drives that are just sitting there waiting to come into action.
"With unlimited resources, I would probably look at a redundant database server sitting alongside our main production server," he adds. "But there are cost considerations with that, as you have to acquire an additional licence for your database, operating systems and so forth.
"We haven't had another [system failure], but we have that good warm feeling that we've made all the right investments."
