Technical expertise, human judgment, and the time spent by an analyst are often believed to be key factors in determining the accuracy of forecasts obtained with the use of a time series forecasting method. A control experiment was designed to empirically test these beliefs. It involved the participation of experts and persons with limited training. Forecasts were generated for 25 time series with the use of the Box-Jenkins, Holt-Winters and Carbone-Longini filtering methods. Results of the nonparametric tests used to compare the forecasts confirmed that technical expertise, judgmental adjustment, and individualized analyses were of little value in improving forecast accuracy as compared to black box approaches. In addition, simpler methods were found to provide significantly more accurate forecasts than the Box-Jenkins method when applied by persons with limited training.