What were you doing as the clock struck midnight on January 1, 2000? Enterprise Data has been busy since then.
If you had a job managing enterprise systems, you were probably toasting yourself for having upgraded those system in time 
for the big deadline. I’m sure you deserved that glass of bubbly. After all, you prevented your company from getting stuck on legacy applications that might not have worked after 1/1/2000. Perhaps unwittingly, you also helped usher in a golden age of enterprise resource planning (ERP) software. Y2K concerns during the late 1990s fueled a surge of investment in ERP software and services. The resulting flurry of implementations forever changed the corporate IT environment. Many large global corporations launched projects with budgets of $50 million to $100 million – and this at a time when a dollar was still worth fifty cents. It’s interesting to note that those ERP projects didn’t require anywhere near the rigorous ROI justification we see for today’s (typically much smaller) ERP implementations. And in fact, these projects really didn’t seem to yield much of an ROI within the first five years. But I suspect that the dramatic increases in U.S. economic productivity over the past five years (right through the recent recession and into the current recovery) are largely due to the agile business processes enabled by strong ERP platforms. In other words, those initial ERP investments – though based on weak ROI justifications – are now generating an enormous ROI 10-to-12 years later. So, if you implemented an ERP in the late 1990s, you averted the Y2K crisis – and delivered ongoing benefits to your organization.But, are you ready for Y2K Part Two?
Too Much of a Good Thing?
All those gigantic ERP implementations triggered another crisis looming on the horizon. Here’s why.After companies purchased their new ERPs, they transferred countless gigabytes of enterprise data into these systems, and launched an unprecedented wave of data collection that continues today. Most major companies now store their data in massive relational databases that can be accessed by employees using standard systems across the enterprise.
Having centralized, accessible corporate data is a good thing. Data is the raw ingredient of analysis. Analysis drives strategic decisions. Strategic decisions lead to greater profits, heightened competitive advantage – and higher productivity.
But you can have too much of a good thing. And after about 10 years of frenzied activity in moving data into ERP systems, IT departments are now dealing with the effects of what can best be described as data cholesterol.
Today’s ERP Crisis: Data Cholesterol
Data cholesterol is a condition in which the excessive buildup of data leads to sluggishness across your production systems. It 
extends to non-production copies and affects the way all data is managed.Just as too much cholesterol in the human body can lead to serious health problems, data cholesterol hinders the function of enterprise systems. It causes slower response times for customer service requests and report queries. It prolongs testing and reporting. It ripples through everything you do in IT, forcing you to use more labor to support your infrastructure. And, it exposes the corporation to needless litigation.
None of the leading enterprise software vendors ever provided an easy way for customers to archive or purge their data. Because they were so focused on creating integrated repositories – on making it easy to get the data in – they never bothered to consider that customers might not want that data forever. And, the complexities of ERP data models with their referential integrity make it very difficult to pull out data without breaking something.
That’s why so many corporations now suffer from data cholesterol. Even mid-sized companies are amassing databases of over one terabyte that are growing at 30-70% each year.
There are several reasons why databases of this size are not ideal (despite the fact that Moore’s Law keeps bringing down the price of hardware):
- The current legal and regulatory environment creates potential liabilities around keeping data longer than is legally necessary. (Think, “million-dollar lawsuits.”)
- Maintaining predictable application performance is much tougher with large data volumes. (Think, “application outages, frustrated employees, and lost customers.”)
- Although CPU and storage costs continue to decline, it’s no longer easy to free up IT budget to throw hardware at the problem. (Think, “board-room arguments.”)
- At a time when CIOs must do more with less, the labor cost associated with managing terabyte-sized application environments is growing. (Think, “begging your CFO for more money.”)
- The proliferation of production data in non-production systems (for development, testing, and training purposes) exposes companies to data privacy liability. (Think, “devastating PR consequences.”)
Why Companies Are Turning to EDM
The painful combination of tighter IT budgets, data cholesterol buildup, and strict regulatory requirements has driven savvy companies to begin focusing on how they manage their data. That’s why they’re using the principles of enterprise data management (EDM) as they implement their data governance functions.
EDM focuses on creating accurate, consistent, lean data content and integrating it into business applications. Today’s EDM solutions address the critical issues of data growth risk management, data privacy compliance, nimble test data management, e-discovery, and application upgrades, migrations, and retirements.
In our next post, we’ll explore the four most common goals of EDM and how to achieve them.
