Is Forecasting Fatally Flawed?

addtoany linkedin

Believe it or not, I didn’t plan the alliteration. But that is my central point: So much of actual demand is unplanned. Which is fine as long as it is near to what was expected in terms of items purchased, period in which purchased, and the customer/region in which the purchase took place. But this does not appear to be the situation in many cases. So is forecasting fatally flawed? Lora Cecere has been writing about forecasting, principally within the CPG industry for many years. She has worked in industry, for a software vendor, and most recently as a highly respected analyst. In a recent blog Lora states that   Mean Absolute Percentage Error (MAPE) for a one month lag was 31 percent + 12 percent.  Data eight years ago for the same companies was an average of 36 percent + 10 percent MAPE. This made me sit up and listen.  Especially when she went on to quote from her research while at AMR Research that Based on AMR Research correlations, a six percent forecast improvement could improve the perfect order by 10 percent and deliver a 10-15 percent reduction in inventory. In other words, there is a lot of benefit to getting the forecast right.  But a range of highly respected CPG companies cannot do better than 31 percent MAPE, with a range of 19 percent to 43 percent?  That caught my attention.  Mostly because I am more familiar with the High-Tech/Electronics industry which has much shorter product life cycles than CPG and therefore more volatile or variable demand patterns. Of course it is difficult to be precise with industry classifications. Does Consumer Electronics fall into CPG, High-Tech/Electronics, or both?  However we slice it, things like cell phones, tablets, cameras, etc have shorter product life cycles, greater seasonal variations in demand, and greater demand variability than do nearly all categories of CPG such as soap, washing powder, etc.  In Consumer Electronics, and more generally High-Tech/Electronics I hear from companies that they seldom get their forecast accuracy, as measured by MAPE, above 50 percent, which is consistent with my observations about the characteristic differences with CPG.  Higher demand variability/volatility would imply a lower forecast accuracy. Before anyone jumps down my throat, especially Lora, let my state unequivocally that everyone MUST forecast and that all companies should be demand driven.  But … But where is the discussion about how best to satisfy the missing 31 percent demand in the case of CPG and 50 percent in the case of High-Tech/Electronics?  Where is the discussion about the profitable response to the demand that is not anticipated? I feel as we are only having half the conversation.  The half about forecasting.  But if the best we can do is improve forecast accuracy from 64 percent to 69 percent over eight years in an industry segment with relative stable demand, I think we should be talking about supply chain agility and responsiveness.  What amazes me is that since the early 1990’s we have been applying optimization engines, typically Linear Programming (LP), to the supply side.  Ignoring for the moment the inherent issue of using linear models to represent highly non-linear systems, if you are basing your optimizations on inputs that are best 69 percent correct, are you not focusing on the wrong problem?  Should you not be focusing on systems that enable you to detect true demand early and determine the best way to satisfy the unanticipated demand using the competing requirements of profitability and customer service?  Of course you will need a supply chain that can execute in an agile and responsive manner consistent with your decision. Here is the rub: All our resources are limited. Time. Cash. People. So in this zero-sum game, where are you going to apply your energies?  Spending eight years to improve the forecast by five percent, or working on the manner in which you satisfy the unanticipated demand in the most timely and profitable manner?

Discussions

L Abenroth
- March 24, 2011 at 12:13pm
The situation is undoudtedly much worse than is indicated by MAPE analysis. Forecasting methods always assume gaussian distributions at some level, including MAPE methods. There is a lot of literature on this topic and it the evidence is overwhelmingly dismissive of gaussian econometrics. Mandelbrot, et al show that much of what I though to be sound industry analysis is fundamentally flawed. What is equally troubling to me is that most professional forecasters know it but can't or won't stop using these techniques.
Ron Freiberg
- March 25, 2011 at 4:26pm
Obviously the answer is to put most of your resources into trying to satisfy anticipated demand as opposed to improving your forecasting techniques. From my experience, forecasts are nothing more that spin on what a Sales group thinks they are going to sell and in reality it comes down to nothing more than a guess as to how much better or worse a future period of business is going to be over a prior period. In the companies I have been associated with, seasonal volatility can swing up to +/- 500% so the objective becomes putting a business plan together that allows the company to ship 95% on time completed orders while using a forecast that may be only 50% accurate at best. To cover this extra 45% of inaccuracy we all employ hedge material plan, demand driven strategies and flexible manufacturing techniques to get to the 95% performance level.
David Shirey
- March 25, 2011 at 10:22pm
Yes, Forecasting is always flawed because we are trying to apply linear methods to a non-linear world. So why do we use it? Because it is much easier to develop a forecast and work continually although futilely to refine it than it is to find creative ways to quickly get raw materials and to shorten your production time. I can't see most companies abandoning conventional forecasts to instead hope that they can fulfill demand quickly enough. Most places don't have that much confidence in themselves. And today, it's more important to be safe than to be good. Although maybe that's always been true.
Trevor Miles
- March 28, 2011 at 10:37am
To L Abenroth, I agee. Being exposed the effect that randomness has on the efficiency of a process was a revelation for me, let alone that most patterns are not 'normal' or Gaussian. But I think ti sis as relevant to supply as it is to demand.
Trevor Miles
- March 28, 2011 at 10:44am
To Ron Freiberg: My point is that you should put as much effort into to satisfy UNanticipated demand rather than anticipated demand. I would also never suggest that companies abandon traditional forecasting. they need something against which to plan capacities and write long term contracts.

More disturbingly I have lots of anecdotal evidence that the inquiry-to-quote-to-order is often as long and many times longer than the order-to-delivery cycle. In other words companies can reduce their full cycle without changing much of the physical supply chain.
Trevor Miles
- March 28, 2011 at 10:46am
To David Shirley: I am really not suggesting that companies abandon forecasting. All I am suggesting is that they redirect some of their efforts to the supply side.
Katherine Hartman
- May 12, 2011 at 7:30pm
It's nice to know that my company is not alone in its constant struggle to improve our forecast accuracy. As the Sales & Marketing Manager at DTx Inc., I feel the pressure every day. Unfortunately, I know the perfection for which I strive can never be achieved as forecasts are inherently imperfect. For DTx, the most noticeable improvement was a result of implementing Sales & Operations Planning methodologies. Not only have our forecasts improved but our ability to respond and execute to those forecasts has improved dramatically. We still have a long way to go, but as Trevor suggests, I believe that forecasts are just a part of the picture. We've invested in the big picture (S&OP) and now we're reaping the benefits. Thanks for honestly broaching this topic. You might be interested in our blog as well. http://blog.dtx.com which is dedicated to providing tips, tricks and resources to OEMs and a variety of industries.

Leave a Reply

CAPTCHA