The Globe and Mail

The history of humankind is apt to be a digitized rather than fossilized legacy. We've built a world where bits and bytes of data exist like zillions of atoms.

Electronic information is found everywhere in today's world, collected through what's estimated to be more than a billion electronic devices, including phones, cameras, scanners, data sticks, personal computers and servers. Every scrap of personal and business activity might remain stored on record for an eternity.

The sheer volume of digitized data is astounding. A recently published IDC white paper, entitled The Expanding Digital Universe and sponsored by IT storage company EMC Corp., offers an illustration of this digital data explosion.

In an attempt to quantify the volume, the researchers suggest defining a byte of digital data as one character on a page. They reckon there was enough digital data in 2006 to theoretically fill 12 separate stacks of novels, each of which would extend the 93 million miles from the Earth to the sun. By 2010, the accumulation of digital data would further extend these 12 stacks of books to reach from the sun to Pluto and back.

The report estimates that electronic business information alone accounts for about 25 per cent of the world's digital data, with the remainder made up mostly of music, videos, digital television signals and pictures.

By 2010, IDC expects this business portion will grow to 30 per cent -- a result of greater use of computing systems by smaller business, regulations that will mandate data archiving and privacy, and the increasing digitizing of data from more processes, including medical imaging, customer support information and many other things.

E-mail alone accounts for about 3 per cent of digital data or six exabytes (one exabyte being a million million megabytes).

This expanding universe of digitized bits is getting out of control -- and threatening to have a devastating impact on businesses.

The biggest problem is one of data management. More specifically, it's the challenge faced by companies to control who views business data and to whom it's being distributed. Privacy and security must be ensured and competitive intelligence must not be leaked.

The challenge is also one of digital data containment. Companies find they can't enclose the ocean of digital information being gathered. The IDC report makes the point that in 2007 the volume of information created and replicated will, in fact, surpass the storage capacity available to keep it.

Storage media are expected to grow 35 per cent annually from 2006 to 2010. Unfortunately, the volume of digital information created and replicated is expected to increase by 57 per cent a year during this same forecast period.

Too much digital information creates the problem of organizing it in a way that makes it useful. Many businesses have more digital data than they can intelligently work with and often can't extract what they need when they need it or create business intelligence from it.

Again, the IDC report makes a key point concerning the struggle by many organizations in manage digitized content. The researchers contend that a company with 1,000 knowledge workers loses $5.7-million annually just in the time spent reformatting information for use among different applications. Another $5.3-million is lost annually by these same companies who can't find the information they've gathered.

The truth is that the volume of digitized data may, in fact, be crippling to many businesses and particularly to the smaller ones. All this data is overwhelming.

Charles King, an IT market analyst and consultant in Hayward, Calif., describes the growing digital universe as the "too much of a good thing" conundrum.

"Say, you love ice cream and over time achieve a state where you can eat as much ice cream as you like. If you overindulge or eat unwisely, you might reach a point where access to all that ice cream deadens your senses or even negatively affects your health," Mr. King says. The electronic gathering and storage of digital data is at that point and out of control, he adds.

"A couple of years ago, people began talking about technology reaching a state where systems management costs would increasingly outweigh the costs of hardware and software," he says.

"Information management is difficult and expensive now, but the world envisioned by the (IDC) report takes our current situation down a path where businesses will be able to afford equipment to store more and more information that, in turn, will be increasingly difficult and expensive to manage."

Without effective storage tools and strategies, businesses will reach a point where the financial and logistical burden of managing information outweighs its value to business, Mr. King says.

At that point, the value of information is diminished, if not rendered useless, simply because the sheer volume of what's being collected can't be managed.

The situation is an even greater problem for smaller businesses. As with most IT issues, the biggest challenge of a small business is the lack and cost of IT skills. These are companies that typically don't employ full-time IT staff and instead depend on workers with general skills to help keep things up and running.

"As data continues to accumulate, sorting it out becomes increasingly difficult and expensive," Mr. King says. "The 'out of sight, out of mind' approach to storage management taken by many small businesses -- and even some larger organizations -- is akin to simply tossing things into a closet or garage.

"It only works so long as you never need access to that material. At the point where you do, digging through the rubble, let alone sorting things out and putting them right, costs time and money that many or most small businesses find painful."

And you'd probably need a digital archaeologist to make business sense of it.

Dan McLean is editor-in-chief of publisher ITWorldCanada.com.

dmclean@itworldcanada.com