Data Archiving: Why It Slipped Your ERP Vendor’s Mind
- Details
- Published on Wednesday, 04 May 2011 09:03
- Written by Marc Hebert
If you’ve ever struggled with the process of purging, archiving, or extracting data for a midsized or large enterprise, you may have wondered aloud, “Why didn’t my ERP vendor provide a better way to do this?”
It’s a fair question. If we can put a man on the moon—and if we can design ERP systems that make sure everyone involved in
a lunar expedition gets paid on time and receives an accurate pension—then why is it so darn difficult to archive data?
As a longtime veteran of the ERP industry, I believe I can shed some light on this topic. (Just promise not to shoot the messenger.)
The Apps that Changed ERP Forever
It was 1988. I had just joined the Applications Division at Oracle. From my very first day of work, I could tell that everything was about to change.
In 1986, Oracle had started developing a new generation of business applications that would transform the playing field. Up until that point, ERP systems had been proprietary. They were either written for IBM mainframes (as SAP’s accounting software suite was), or for one of the various mini-computer platforms (remember those?).
Oracle’s new software would propel ERP one step closer to truly open systems. How? It was written for a new generation of hardware platforms—such as Sequent Computers and Pyramid Computers—that ran on a generic version of the UNIX operating system. Adding to the flexibility, the Oracle relational database could also run on proprietary platforms. Thus, companies using the new Oracle Financials suite with an Oracle database could run their software on virtually anything.
Suddenly, enterprises truly had a choice of hardware and software. Although IBM had invented the relational database, Oracle was running with it.
That wasn’t the only pro-customer change. The user interface in these new applications was vastly improved. Previous generations of ERP had offered users a traditional mainframe green-screen. You would fill the screen with data and press a button. Since there was no field-level validation, errors would only come back to you after you’d processed an entire screen.
Oracle’s new apps, on the other hand, offered much richer interaction between computer and user. And because they ran on a relational database, users could query data on demand, rather than using the cumbersome nightly reporting associated with mainframe computing. These innovations naturally enabled much greater productivity—not to mention, user satisfaction.
Giddy with Computing Power
I could wax on endlessly about the progress we made in those days. Oracle’s new apps—and the competing products that soon followed—really did make life better and easier for enterprises.
But here’s the rub: as we built those applications (and, frankly, rushed them to market), we all became a bit transfixed by their unprecedented power. All we cared about was helping users get as much data as possible into those applications so that they could take advantage of this power.
Oh, it crossed our minds that due to the complex parent-child relationships between data tables, there would be no easy way to pull data out of relational databases without breaking something. But it was a complicated problem, so we pushed it to the back burner. Oracle never did provide tools for archiving data, and PeopleSoft didn’t offer much. (SAP does include archiving tools, but from what I understand, the process isn’t much fun.)
We charged ahead. Despite all the innovations, it took about 10 years for ERP to become a substantial market. Not knowing what to do about mainframe applications that were hard-coded with the Y2K date problem, laggards finally got serious about adopting new ERP systems in the late 1990s.
Y2K came and went without incident, of course. But by then, uptake of ERP applications was dramatic. And there was no stopping the drive to load these systems with vast amounts of corporate data.
Y2K, Part Two?
More than a decade later, we’re realizing that our now-aging ERP applications are hard-coded with a more serious problem: it’s really hard to get data out of them.
As many companies seek to retire those initial ERP applications—and, in some cases, the mainframe apps and proprietary platforms that preceded them—they’re not sure how to transition gracefully while maintaining access to all their data. This data still has value, not to mention liability potential, so archiving it takes careful thought.
Since your ERP vendor didn’t invest in this kind of thought, the task has fallen to you. Where to begin? We’ll deal with the concept of a corporate data repository in an upcoming post.
Read more – Estuate EDM Checklist