News

Your network has slowed to a crawl. E-mail is down. Blank monitors stare back at workers trying to log in to the system. Any of these scenarios can send people into panic mode. According to network experts, however, even the worst-case scenarios could be false alarms.

Those in the know have no end of stories about companies that spend thousands of dollars bringing in tech support, adding bandwidth and replacing equipment to fix their networks or keep them in top form—only to find that a cable was unplugged or there was a free software patch that would have solved it all. "Once the fear factor sets in, people try to pull things apart and replace them without understanding the cause of the problem—and that's a crime," says John Breakey, chief executive officer of Oakville, Ontario-based systems integrator Unis Lumin. "The problem often boils down to something as simple as the $2 connector that plugs into the wall."

Brantz Myers, director of industry marketing for Cisco Systems Canada in Toronto, attributes the biggest mistakes to overanalyzing a problem. "Everyone looks for too complicated a solution when they panic. But look at this way: If your stereo isn't working, you know something isn't turned on or plugged in. The same logic should apply with networks."

Even though the problem might be simple, discovering what it is takes an unflappable attitude, a logical brain, a good eye and the right test tools—a valuable combination that seems to be in short supply.

Tony Fortunato, senior network specialist for the Technology Firm in Georgetown, Ontario, says he is one of only a handful of professional network troubleshooters whose full-time job is network "sleuthing." He spends his days working with companies to root out the source of network problems that have stymied others.

Like other experts, Fortunato says 70% of network problems can be solved simply by walking around the premises, following cables. "In the business world they say follow the money. In the network world, it's follow the cable," agrees Myers.

For the other 30%, you need specialized tools that go beyond simply telling you if something is on or off, or how fast your network is. One such device is a protocol analyzer, a sophisticated tool that provides a good picture of packet details as they travel over the network. "A lot of people can test if their network is running and at what capacity," Fortunato notes. "But that's like the idiot lights on your car—you know it's not working, but you have no idea what's really wrong or why. The protocol analyzer is like a therapy tool. When two devices don't talk, it lets me plug in to listen to the conversation."

Eugene Suarez is a product specialist with Mississauga, Ontario-based Fluke Networks, developers of network test tools. He says many organizations simply don't invest in the proper testing before bringing in the big guns. "I'm always surprised how many businesses spend millions on their infrastructure, but won't spend a few thousand on tools or troubleshooting services that can analyze network performance and find the real cause of the problem," Suarez says.

So what's a company to do if the network's on the fritz? Not everyone has a testing expert on hand. Nor do many businesses have the tools to figure out what's really going on when the network isn't performing up to par. Taking a walking tour around your operation is a good place to start. If the problem isn't readily visible, bring in a troubleshooter.

It might cost a couple of thousand dollars for top-notch sleuthing tools and/or services, but chances are your network will be up and running at full speed with a few small fixes. And if you do need major upgrades, it's a small price to pay to know you're doing the right thing.

Here's a sampling of some of the most spectacular network blunders witnessed by experts.

Wandering wireless link
Operators at a production plant that had implemented a wireless network couldn't figure out why its network went down for a few minutes many times during the day. After a few months of head-scratching, it was discovered that the designated site for the route bridge (that is, the central point of contact for the network) was a crane that left the building several times during the day. Every time the crane visited the supply yard, operations were shut down. A simple reconfiguration fixed the problem.

Do-it-yourself routing
Businesses often use "non-routable" IP addresses for internal equipment so they won't be visible to the outside world. Experts say that a common problem for small to mid-sized business do-it-yourselfers is setting up web and e-mail servers with non-routable IP addresses. Once they do that, no one from the outside can communicate. And that's definitely bad for business.

Look for the freebies
An organization couldn't get its high-speed network to live up to its promise. After $100,000 was spent on extra bandwidth and equipment, the network was slower than ever. A few simple tests showed that an older application on the system wasn't able to handle high-speed traffic, and the more bandwidth they gave it, the slower it got. The fix was a software patch—and it was free.

Closet skeletons
One business setting up a WiFi network decided that in the interests of security, it would be smart to put the wireless access point in a wiring closet. Little did workers realize that wiring closets are often concrete rooms with steel reinforcements and are designed to stop radio signals. Needless to say, connection with the access point was foiled from the start. Another company found that water from the air-conditioning system had been dripping into the wiring closet housing networking equipment. After cleaning up the mess (the equipment wasn't damaged), workers put in a heater to dry everything out. However, the heater was right below the switches—which shut down automatically if thermal readings were too high.

The people factor
Experts have lost count of the times they've been called in to troubleshoot network slowdowns and discovered that network traffic is jammed because employees are downloading movies or playing online games. This is the cause of much corporate embarrassment when signing off on costly tech-support bills.

Another common problem is UPS (uninterruptible power supply) bars. These heavy-duty devices are usually placed on lower shelves of racking systems—right at knee height, in fact. Many times they're turned off because operators inadvertently hit a switch setting as they walk by. Seems simple, but it's enough to kill your network connections—and it's often the last place someone looks.

Plugging problems
A bank technician configuring PCs used his computer as a source for software. When it came time to rollout the application to 22,000 computers, all the network traffic was trying to connect to his PC, which flooded the network. The bank spent upwards of $50,000 on network upgrades before discovering that it all boiled down to a simple misconfiguration.

An unsupervised networking student doing a routine service on a U.S. bank's network saw an empty port on a switch and thought it was the perfect place to plug in a network cable. Bad idea. Instead of connecting the server to the network, he had plugged it into the console board. This tiny mistake brought the bank's e-mail server down for more than five hours during the business day.

Cable killers
A portion of one distribution company's network operations had gone down and connection with the main production server had been lost. It turned out that a concerned employee felt the room that housed the server was too warm so propped open the door to let in some air. Trouble was, the worker used the server cable to tie the doorknob and pulled it out of the switch.

An automotive manufacturing plant found that with every shift change, half of the plant went down because of excess noise on the network cables. The problem? A safety-minded foreman spotted a cable on the ground, picked it up and wrapped it around an arc welder. It was the cable that connected the two halves of the plant-floor operations, and the interference from the welder killed the link.

WiFi washout
An international U.S. airport had deployed WiFi to allow travellers to check their e-mails. It didn't work well from the start; connections were constantly being dropped. Thinking the equipment was at fault, the airport operators invested an additional $50,000 in upgrades and new cabling only to find the problem was still there. When the network specialist was finally called in, he only had to look at the wireless access points to get the real picture: they were located just a metre above a bank of microwave ovens in the cafeteria. Microwaves act like big, square, jamming devices for signals. Every time someone turned on an oven, it cooked the WiFi signal.