There are two versions of Henri Theil's U statistic for summarising
forecasting accuracy. If a forecasting method produces a set, {P_i}, of
predicted _changes_ (often proportional changes, but not predictions of
_levels_ of the series), these can be compared with corresponding set of
actual changes, {A_i}. The earlier (1958) statistic was
U = sqrt[sum{(P_i-A_i)^2}]/[sqrt{sum((P_i)^2)}+sqrt{sum((A_i)^2)}]
and the later (1966) statistic is the simpler, and generally preferred,
U = sqrt[sum{(P_i-A_i)^2}]/sqrt{sum((A_i)^2)}
If the forecasts are perfect, Theil's U is 0. The naive model predicting no
change will give a U value of 1, so better models will give U values
between 0 and 1.
See: H.Theil (1966), Applied Economic Forecasting, North Holland.
Advantages claimed for Theil's coefficient over alternative measures of
forecasting accuracy such as the Mean Absolute Deviation or the (Root)
Mean Square Deviation are its scale free character - as with the Mean
Absolute Percentage Error - while its derivation using changes rather than
levels avoids the inflated view of accuracy often conveyed when forecasting
levels of series with strong trends using measures such as, or similar to,
R^2. In practice it is generally sensible to look at a range of
measures when judging the accuracy of forecasts.
Mike Fuller
Canterbury Business School, University of Kent, Canterbury, Kent, CT2 7PE, UK
Tel +44 1227 827729 direct line; 827726 messages; 764000 switchboard
Fax +44 1227 761187; email M.F.Fuller@ukc.ac.uk