Data cleansing, like maintaining your car, fixing the roof, and painting the porch needs to be done if you expect your systems to function properly. Why is data cleansing important for ERP systems?
- System bloat: Old data such as obsolete parts, long lost customers, orders that have been open and untouched since 1972 take up space on your system, clutter up reports and slow down processing.
- Incorrect planning: We understand the concept of garbage-in, garbage-out. If you have incorrect data driving your planning system, what kind of a plan are you getting? Incorrect data manifests itself in two painful ways; excess inventory and late customer orders.
- System problems: Bad data can bring an ERP system to its knees. In my IT days, we had a planner accidentally set the Max quantity to 1 on a item (paint) that had demands in the millions of milliliters. As a result, our MRP process ran hours longer than it should have and ended up crashing because it ran out of order numbers!
Even with the best tools, data cleansing is a difficult, tedious task. Those of us who have had to do it, know we don’t want to go through that process again if we can avoid it. So the question becomes how do we prevent our systems from sliding into disarray again. Let’s take the discussion out of IT for a second. My home workshop had a well earned reputation for being a legendary mess. When I set off to do a job, I used to need to set aside an hour or two just to find the tools I needed. Sometimes I couldn’t find the tool I needed and ended up using the wrong tool with less than stellar results. Every once in a while, I’d clean up and for a few days, my workshop was organized bliss. After a few weeks, however, old habits (and my son) snuck in and before I know it, pandemonium ruled. Finally, borrowing from lean six sigma training I received in my manufacturing days, I set up a shadow board where the outline of my tools let me know where each tool was supposed to be. When I finished a job and that spot was empty, I knew I had forgotten to but something back. The result? My tools have stayed organized. I can start working on my projects immediately and don’t need to spend the first hour tracking down missing tools.
Ongoing data cleaning is not about setting up shadow boards in our data. It’s about putting tools and processes in place to ensure that data quality is maintained over time so we don’t need to go off on these massive data cleansing exercises. So what capabilities do you need to maintain the quality of your ERP data?
- Visibility: You can’t correct what you can’t see. Sometimes data quality issues exist in other locations in the supply chain. For example, how sure can we be that our Contract Manufacturer’s BOM is the same as we provided? We need tools that provide visibility to the key areas of our supply chain.
- Powerful, configurable, reporting technology: Every business has special rules that control what is correct and what is incorrect within their data. Any tool used for data cleansing must allow you to modify the reports to reflect your business rules.
- Alerting: The most important thing you can do to maintain data quality is to clean up data issues as they occur. Experience has shown, that it’s much more effective to have tools monitoring the data that let us know a problem has occurred than to proactively check a series of reports looking for data problems.
- Simulation: Sometimes we will identify a problem in our data, especially the rules that drive planning, where it’s not clear what the setting actually should be. With simulation capabilities, we can try different settings to see the effect they have on everything from detailed planning to key corporate metrics.
When is the last time you looked at the quality of your data? Have you gone through a data cleansing exercise? Do you have tools and policies in place to ensure your data quality is maintained? Post a comment and let us know!
TrackBack
• Digg This
• Add to del.icio.us
Tags: Enterprise resource planning (ERP)
Posted in Best practices
You can leave a response, or trackback from your own site.
Hi John,
I completely agree. Everyone knows that garbage in = garbage out, and if you don’t take the time to insure data quality, it can also impact the timeliness and effectiveness of the data. Not to mention, it can also have a downstream (or upstream) impact on the systems that you are integrating data with. Solving the problem in one system can cause a ripple effect on other systems. The effort of maintaining data quality today needs to account for how the data in your ERP system interacts with other systems.
What have you found in your experiences with handling historical data when going through the data cleansing exercise? For instance, what do you do with data for products that are no longer being sold (but might have been replaced by newer parts), or reorganization of your data?
Ming
Thanks for your question Ming.
Depending on your organization, you will likely experience diminishing returns as you tackle historical data. Current data, (Orders, supples, ordering rules, inventory) have an immediate impact on your business. If you have an incorrect planning parameter you could order significantly more than you need.
Often, you use historical data for forecasting…but this information is really an input that is combined with multiple other factors used for developing a forecast. Yes, it would be good to have clean, accurate historical data, but the cost of cleaning that data would be prohibitive.
I’d like to circle back to my initial position; data clean up of any sort is expensive. The ideal situation is to put policies and processes in place that ensure on-going data accuracy. Over time, this data will become historical data and if accurate now, will be accurate in the future.
Thanks for the response, John. In regards to developing the forecast — how much emphasis do you normally place on the sales forecast? Ever since the market downturn last September, we have seen that customer behavior has become more unpredictable, and the information from the field has become more relevant than ever.
Circling back to your point about garbage-in = garbage-out and having the right processes in place — it’s especially important to identify the inputs to your planning system and insure that you have the right data coming in along with context for the information.
From my perspective, one of the key places to implement the data quality process is in the sales forecast (note: I work for a company that specializes in this area which makes me somewhat biased). Having a trusted sales forecast with quality data can really make a difference in driving the planning process and providing good visibility to the entire organization.
The integration of this data with the other systems ensures that you don’t have silos of data and can make good planning decisions. Siloed data, regardless of how good it is, provides a limited perspective and can often lead to bad decisions — i.e. The classic case where Volvo tried to rid itself of its excess inventory of green cars, only to end up with an additional inventory.
The companies that we work with have a variety of forecasting approaches. What seems to work best is a rationalization of the various forecast inputs through collaboration (Sales, marketing, customers) to develop a consensus forecast.
There are numerous approaches to weighing the various forecast inputs. One customer actually scored the historical forecast inputs from each contributor against actual sales and gave that source a higher weighting when developing the consensus forecast. If sales provided the closest estimate over the past three months then the sales forecast would have a higher weighting.
I’ll throw Ming’s question out to the rest of the readers. How much emphasis do you place on each of the different forecast inputs?