Mdthbs

My task is to forecast future 1 month stock required for retail store, at a daily basis. How do I decide whether
MAPE, SMAPE and MASE
is a good metrics for the scenario?

In my context, over-forecast is better than under-forecast.

You are forecasting for stock control, so you need to think about setting safety amounts. In my opinion, a quantile forecast is far more important in this situation than a forecast of some central tendency (which the accuracy KPIs you mention assess).

You essentially have two or three possibilities.

1. 
   

   Directly forecast high quantiles of your unknown future distribution. There are more and more papers on this. I'll attach some below.

   
   
   

   Regarding your question, you can assess the quality of quantile forecasts using hinge loss functions, which are also used in quantile regression. Take a look at the papers by Ehm et al. (2016) and Gneiting (2011) below.

   
   


2. 
   

   Forecast some central tendency, e.g., the conditional expectation, plus higher moments as necessary, and combine these with an appropriate distributional assumption to obtain quantiles or safety amounts. For instance, you could forecast the conditional mean and the conditional variance and use a normal or negative-binomial distribution to set target service levels.

   
   
   

   In this case, you can use a forecast accuracy KPI that is consistent with the measure of central tendency you are forecasting for. For instance, if you try to forecast the conditional expectation, you can assess it using the MSE. Or you could forecast the conditional median and assess this using the MAE, wMAPE or MASE. See Kolassa (2019) on why this sounds so complicated. And you will still need to assess whether your forecasts of higher moments (e.g., the variance) are correct. Probably best to directly evaluate the quantiles this approach yields by the methods discussed above.

   
   


3. 
   

   Forecast full predictive densities, from which you can derive all quantiles you need. This is what I argue for in Kolassa (2016).

   
   
   

   You can evaluate predictive densities using proper scoring rules. See Kolassa (2016) for details and pointers to literature. The problem is that these are far less intuitive than the point forecast error measures discussed above.

   
   

What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? is likely helpful, and also contains more information. If you are forecasting for a single store, I suspect that the MAPE will often be undefined, because of zero demands (that you would need to divide by).

References

(sorry for not nicely formatting these)

Ehm, W.; Gneiting, T.; Jordan, A. & Krüger, F.
Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings (with discussion).
Journal of the Royal Statistical Society, Series B, 2016 , 78 , 505-562

Gneiting, T.
Quantiles as optimal point forecasts.
International Journal of Forecasting, 2011 , 27 , 197-207

Kolassa, S.
Why the "best" point forecast depends on the error or accuracy measure.
International Journal of Forecasting, 2019

Kolassa, S.
Evaluating Predictive Count Data Distributions in Retail Sales Forecasting.
International Journal of Forecasting, 2016 , 32 , 788-803

The following are more generally on quantile forecasting:

Trapero, J. R.; Cardós, M. & Kourentzes, N.
Quantile forecast optimal combination to enhance safety stock estimation.
International Journal of Forecasting, 2019 , 35 , 239-250

Bruzda, J.
Quantile smoothing in supply chain and logistics forecasting.
International Journal of Production Economics, 2019 , 208 , 122 - 139

Kourentzes, N.; Trapero, J. R. & Barrow, D. K.
Optimising forecasting models for inventory planning.
Lancaster University Management School, Lancaster University Management School, 2019

Ulrich, M.; Jahnke, H.; Langrock, R.; Pesch, R. & Senge, R.
Distributional regression for demand forecasting -- a case study.
2018

Bruzda, J.
Multistep quantile forecasts for supply chain and logistics operations: bootstrapping, the GARCH model and quantile regression based approaches.
Central European Journal of Operations Research, 2018