9.5 Methods of Forecasting Accuracy
Accurate forecasting is crucial for effective decision-making in various business contexts. To ensure the reliability of forecasts, it’s essential to measure their accuracy against actual outcomes. Several metrics can be employed to gauge the precision of forecasts, each with its own strengths and limitations. In this section, we’ll delve into three commonly used accuracy metrics: Bias, Mean Absolute Deviation (MAD), Mean Absolute Percent Error (MAPE) and Mean square error (MSE).
- Bias: Bias measures the average forecast error and indicates whether the forecast is consistently overestimating (positive bias) or underestimating (negative bias) the actual demand.
Bias= [latex]\dfrac {1}{n}\sum_{t = 1}^{n} (A_t - F_t)[/latex]
Where:
- Ft is the forecast for period t.
- At is the actual demand for period t.
- n is the number of periods.
Example: For three months, the forecasts were 100, 110, and 105 units, and the actual demands were 105, 108, and 104 units, respectively. The bias is calculated as:
Bias= = [latex]\dfrac{(105-100) + (108 - 110) + (104-105)}{3}[/latex] = [latex]\dfrac{5 -2 -1}{3}[/latex] = [latex]\dfrac {2}{3}[/latex]
This positive bias indicates that, on average, the forecast is slightly underestimating the demand.
Advantages:
- Simple to calculate.
- Provides a clear indication of systematic over or under forecasting.
Disadvantages:
- Doesn’t provide insights into the magnitude of errors.
- Positive and negative errors can offset each other, leading to a misleadingly low bias.
- Mean Absolute Deviation (MAD): MAD measures the average magnitude of forecast errors, regardless of their direction.
MAD = [latex]\dfrac {1}{n}\sum_{t = 1}^{n} |A_t - F_t|[/latex]
Example: Using the same data as above, the MAD is:
MAD= [latex]\dfrac{|105-100| + |108 - 110| + |104-105|}{3}[/latex] = [latex]\dfrac{5 + 2 + 1}{3}[/latex] = 2.67
Advantages:
- Provides a clear measure of the average error magnitude.
- Insensitive to the direction of errors.
Disadvantages:
- Doesn’t provide insights into over or under forecasting.
- Can be sensitive to outliers.
- Mean Absolute Percent Error (MAPE): MAPE expresses the forecast errors as a percentage of actual values, providing a relative measure of accuracy.
MAPE= [latex]\dfrac {1}{n}\sum_{t = 1}^{n} |\dfrac{A_t - F_t}{A_t}|*100[/latex]
Example: Using the same data, the MAPE is:
MAPE = [latex]\left( \left| \frac{100-105}{105} \right| + \left| \frac{110-108}{108} \right| + \left| \frac{105-104}{104} \right| \right) \div 3 \times 100\%[/latex]
[latex]\qquad = \left( \dfrac{0.0476 + 0.0185 + 0.0096}{3} \right) \times 100\%[/latex] = 2.86%
Advantages:
- Provides a relative measure of error, making it easier to compare across different scales or products.
- Useful for benchmarking and performance tracking.
Disadvantages:
- Can be skewed by periods with very low actual values.
- Not always suitable for situations where actual demand can be zero.
- Mean Squared error (MSE):
[latex]MSE = \dfrac{1}{n} \sum_{t = 1}^{n} (A_t - F_t)^2[/latex]
Where:
- MSE is the Mean squared error
- Ft is the forecast for period t.
- At is the actual demand for period t.
- n is the number of periods.
The Mean Squared Error (MSE) is a widely used metric in various fields, including statistics, machine learning, and forecasting, due to several advantages:
Emphasis on Larger Errors: Squaring the errors gives more weight to larger errors, which can be advantageous when you want to penalize significant deviations from the true values. This aligns with the intuition that large errors are often more costly or important in many applications.
Statistical Properties: MSE is related to various statistical concepts, such as variance and bias. This makes it interpretable from a statistical perspective, which is important in many scientific and research applications.
Universality: MSE can be used in a wide range of applications, from simple linear regression to complex machine learning models. Its universality makes it a versatile metric for evaluating forecasting accuracy.
While MSE is suitable for measuring accuracy, it does not provide information about the bias in forecasts, which is why it is often used in conjunction with other metrics for a more comprehensive evaluation.
Example: For three months, the forecasts were 100, 110, and 105 units, and the actual demands were 105, 108, and 104 units, respectively. The MSE is calculated as:
[latex]\text{MSE} = \dfrac{(100-105)^2 + (110-108)^2 + (105-104)^2}{3}[/latex]
[latex]\qquad = \dfrac{25 + 4 + 1}{3}[/latex] = 10
In conclusion, while each of these metrics offers valuable insights into forecast accuracy, it’s essential to consider their individual strengths and limitations. Often, a combination of these metrics provides the most comprehensive view of forecast performance.
