Publications

PDF - Estimation of Alternative Forecasting Models ()
Date : Nov 20, 2003
Estimation of Alternative Forecasting Models

Section IV

IV.1. Tests for Nonstationarity

The first step in the estimation of the alternative models is to test for nonstationarity. Three alternative tests are used, i.e., the augmented Dickey-Fuller (ADF) test, Phillips Perron (PP) test and the KPSS test. If there is a conflict between the ADF and PP tests, this is resolved using the KPSS test. If at least two of the three tests show the existence of a unit root, the series is considered as nonstationary. The tests for nonstationarity are reported using weekly data from April 1997 to September 2002. Unit root tests are also conducted for a longer time span using monthly data from early 1990s onwards since Shiller and Perron (1985) and Perron (1989) note that when testing for unit roots, the total span of the time period is more important than the frequency of observations. In the same vein, Hakkio and Rush (1991) show that cointegration is a long-run concept and hence requires long spans of data rather than more frequently sampled observations to yield tests for cointegration with higher power. Since the inferences from monthly data conform to those from weekly data, the monthly results are not reported.

Table 1.1A reports the augmented Dickey-Fuller and Phillips Perron tests for the five interest rates under study – call money rate, 15-91 days Treasury Bill rate, and 1, 5, and 10-year government securities (residual maturity). Table 1.1B reports the same tests for variables used in multivariate models while Table 1.2 gives the results of the KPSS test for all the variables used in this study. The results of these three tests are summarised in Table 1.3 and show that except for the week-to-week inflation rate, all the variables can be treated as nonstationary. Testing for differences of each variable confirms that all the variables are integrated of order one.

IV.2. Estimation of Univariate and Multivariate Models

The univariate best-fit models (Tables 2A-2E) for the first-differenced interest rates are estimated as follows for the period April 1997 to December 2002:

Call money rate:

ARMA (2,2); ARMA(2,2)

 

GARCH(1,1)

Treasury Bill rate –

ARMA(3,0); ARMA(3,0)-ARCH(1)

15-91 days:

 

Government Securities –

ARMA(1,0); ARMA(1,0)-

1-year:

GARCH(1,1)

Government Securities –

 

5-years:

ARMA(2,0); ARMA(2,0)-ARCH(2)

Government Securities –

 

10-years:

ARMA(1,0); ARMA(1,0)-ARCH(1)

These models are reported in Tables 2A-2E and are used to generate out-of-sample forecasts from January through September 2002.

Three kinds of multivariate models are estimated – vector autoregressive (VAR) models, vector error correction models (VECM), and Bayesian vector autoregressive (BVAR) models. First, a VAR model is estimated. Second, its error correction representation is derived. Finally, alternative BVAR models are estimated using the optimal lag length determined for an unrestricted VAR.

To estimate a VAR, it is important to first determine if the variables included in a VAR are also cointegrated. If the variables are indeed cointegrated, the VAR model can be estimated in level-form. Accordingly, we first test for cointegration between the variables for each of the interest rates. The optimal lag length for each VAR system is determined by the Akaike Information Criterion, Schwartz Bayesian Criterion and the likelihood ratio test.

Selection of Variables

To estimate the multivariate models, the variables are selected for each model on the basis of economic theory and out-of-sample forecast accuracy. Several factors can impact interest rates. Furthermore, their impacts may differ depending upon the maturity spectrum of the interest rates. For instance, for short-term/medium-term rates, factors that might impact interest rates include monetary policy; liquidity, demand and supply of credit, actual and expected inflation, external factors such as foreign interest rates and change in foreign exchange reserves, and the level of economic activity. For long-term interest rates, demand and supply of funds and expectations about government policy might be relatively more important.

Some of these factors also emerge from the stylized model developed by Dua and Pandit (2002) under covered interest parity condition. The equation for the real interest rate derived from their model can be expressed as a function of expected inflation, foreign interest rate, forward premium, and variables to denote fiscal and monetary effects. Based on this model, the inflation rate, foreign interest rate, forward premium and a variable to gauge monetary policy are included in the forecasting model. In addition to these variables, the following are also included: yield spread (discussed below); liquidity in the monetary system; and a variable to measure credit conditions. Other variables such as CRR, foreign exchange reserves, exchange rate, stock prices, advance, turnover, 3 and 6-months US TB rate (secondary market) and reserve money, were also tried. Since these did not improve the forecast accuracy in any of the equations, these results are not reported. The repo rate is also considered. A detailed comparison between models including Bank Rate and repo rate is given in Tables 7A-7E.

There are, of course, other variables that might impact interest rates such as current and future economic activity and expectations of government policy as mentioned above. However, since the models reported in this study are estimated using weekly data, the selection of variables was obviously circumscribed and, therefore, all of these effects could not be incorporated.

Nevertheless, some of these effects are captured in financial spreads that are measured by differences in the yields on financial assets. These spreads exist due to differences in liquidity, risk and maturity that can also be influenced by factors such as taxes and portfolio regulations. Cyclical changes in any of these factors can arise from monetary policy shifts leading to changes in financial spreads. The most commonly used financial spread is the yield spread whose role in predicting future changes in interest rates is documented in several articles including Campbell and Shiller (1991), Froot (1989), and Sarantis and Lin (1999).

The slope of the yield curve – the difference between the long-term interest rate and the short-term interest rate, measures the yield spread. According to the expectations hypothesis of the term structure, this yield differential provides an indication of the expected future inflation rate (Mishkin, 1989). It also provides a signal about growth in future output. For instance, tight monetary policy and high interest rates can imply a declining yield curve and thus a slowdown in future output growth.

Thus, variables included in the models are: yield spread (10 year Government Security rate minus 3-month Treasury Bill rate); inflation (calculated from Wholesale Price Index using week-to-week changes and year-on-year changes); liquidity in the system (described in Annexure II); credit; Bank Rate/repo rate (indicator of monetary policy); foreign interest rates (Libor 3 months and 6 months); and forward premia (3 months and 6 months). Details of data sources and definitions are given in Annexure II.

The specific variables included in the various models are given below:

Model A:

Call money rate: inflation (week-to-week); Bank Rate; yield spread; liquidity, foreign interest rate (3-month Libor), forward premium (3-months)

Model B:

Treasury Bill rate (15-91 days): inflation (year-on-year), Bank Rate; yield spread, liquidity, foreign interest rate (3-month Libor), forward premium (3-months)

Model C:

Government Security 1 year: inflation (year-on-year), Bank Rate; yield spread, liquidity, foreign interest rate (6-month Libor), forward premium (6-months)

Model D:

Government Security 5 years: inflation (year-on-year), Bank Rate; yield spread, credit, foreign interest rate (6-month Libor), forward premium (6-months)

Model E:

Government Security 10 years: inflation (year-on-year), Bank Rate; yield spread, credit, foreign interest rate (6-month Libor), forward premium (6-months)

In the present context, it is worth noting that the week-to-week inflation rate (weeki+1 relative to weeki) produces better forecasts for the call money rate than year-on-year inflation (weeki+52 relative to weeki) while for all other interest rates, year-on-year inflation produces superior forecasts. This may be because the call money rate is more responsive to week-to-week changes.

The cointegration results are reported in Table 3. A caveat here is that the cointegrating equations are estimated over a short span (five and a half years) and therefore cannot capture the long-run properties of the model. The purpose of estimating the equations is to establish the existence of a cointegrating relationship and thus justify estimating the VAR in levels. Nevertheless, we estimate the error correction model and examine the predictive ability of the variables using Granger causality tests. These results are reported in Table 4 and show that all the variables significantly Granger cause the various interest rates, thus justifying their inclusion in the model.

In addition to the level VAR and VECM models, several Bayesian vector autoregressive models are also estimated. We begin with the prior recommended by Doan (1992), w=0.2, d=1, k=0.5. Four more priors are used to select the optimal prior – i.e., the combination of hyperparameters that yields the most accurate forecasts. Tighter priors compared to Doan (1992) for k=0.5 are: w=0.1, d=1; w=0.1, d=2; and w=0.2, d=2. A looser prior relative to Doan (1992) is obtained by increasing the interaction parameter, k, e.g., k=0.7, w=0.2, d=1.

Tables 5A through 5E report the Theil statistics for the out-of-sample forecasts from January 2002 to September 2002 for all the models while Tables 6A through 6E give the corresponding root mean square errors. The ‘optimized’ BVAR model for k=0.5, i.e., one that has the lowest overall U statistic is tabulated along with the other models while the remaining BVAR models are tabulated under ‘alternative’ models. Figures 1A through 1E show the out-of-sample forecasts from the univariate models. Figures 2A through 2E depict the out-of-sample forecasts from the multivariate models while Figures 3A through 3E provide a comparison of the ‘best’ univariate model vs. the ‘best’ multivariate model.

Figures 4A through 4E provide insight into multi-horizon forecasts made at the end of January 2002 for up to September 2002. This shows how a real-time forecaster would have performed at the end of January 2002 in predicting interest rates up to September 2002.

IV.3. Main Findings

Call Money Rate (Tables 5A and 6A, Figures 1.1A-1.3A, 2.1A-2.3A, 3.1A-3.3A and 4A)

  • ARMA-GARCH model yields more accurate forecasts than the best-fit ARIMA model.
  • ARMA-GARCH model outperforms all alternative (univariate and multivariate) models for very short-term forecasts (up to 9-weeks ahead). The model U statistic is < 1 for almost all forecast horizons, which indicates that the model strongly outperforms the random walk.
  • Level VAR (LVAR) model provides more accurate forecasts relative to the naïve and other univariate models for more than 9 weeks forecast horizon.
  • LVAR model generally provides more accurate forecasts than the Vector Error Correction Model (VECM).
  • VECM yields the most inaccurate forecasts.
  • BVAR models perform better than LVAR for longer-term forecasts, over 20 weeks ahead.
  • Of the BVAR models, the model with a loose prior (w=0.2, d=1 with k fixed at 0.5) outperforms the alternatives. Allowing k to increase (thus increasing the interaction) improves forecast accuracy. This model is superior to the random walk model for over 8-week-ahead forecasts as reflected in the Theil U statistic.
  • The univariate models and VECM generally exhibit an increase in RMSE, i.e., a decrease in forecast accuracy (Table 6A) with an increase in the forecast horizon. On the other hand, the level VAR model almost consistently shows decrease in RMSE while the BVAR models show some improvement in accuracy at the very long end. This is also reflected in Figures 2A, 3A and 4A.

Thus, for the call money rate, an ARMA-GARCH model is best suited for very short-term forecasting while a BVAR model with a loose prior can be used for longer-term forecasting.

Treasury Bill Rate – 15-91 days (Tables 5B and 6B; Figures: 1.1B-1.3B, 2.1B-2.3B, 3.1B-3.3B and 4B)

  • ARMA model produces marginally more accurate forecasts compared to the ARMA-ARCH model. However, since the U statistic is greater than or close to 1 for all forecast horizons, the forecast performance is not superior to that of a random walk.
  • For all univariate models (including the random walk) there is deterioration in accuracy with an increase in the forecast horizon (Table 6B).
  • The LVAR model outperforms the VECM model consistently.
  • The LVAR model also beats the BVAR models in terms of forecast accuracy.
  • Performance of all BVAR models is reasonable and generally improves on loosening the prior. In the extreme case, with a very loose prior, the BVAR model converges to LVAR.

Therefore, for the 15-91 day Treasury Bill rate, the LVAR models produce the most accurate short- and long-term forecasts.

Government Securities – 1-year (Tables 5C and 6C, Figures 1.1C-1.3C, 2.1C-2.3C, 3.1C-3.3C and 4C)

  • ARMA model is generally more accurate than ARMA-GARCH.
  • LVAR model almost consistently outperforms VECM forecasts.
  • Performance of BVAR forecasts is satisfactory for short- and long-term forecasts and is almost consistently better than that of LVAR.
  • Of the BVAR models, the model with w= 0.2, d=1 and k=0.5 performs best.
  • All models are inaccurate for forecasts 16 through 22 weeks ahead. This can be attributed to the fluctuations in the interest rate from March to May 2002 (from 5.37 per cent to 7.22 per cent).

Thus, for 1-year government securities, BVAR models outperform the alternatives at the short and long end.

Government Securities – 5-year (Tables 5D and 6D, Figures 1.1D-1.3D, 2.1D-2.3D, 3.1D-3.3D and 4D)

  • ARMA model is generally more accurate than ARMA-GARCH. Accuracy of both models improves relative to the random walk for forecast horizons over 24 weeks.
  • All models are inaccurate for forecasts 17 through 23 weeks ahead, which can be attributed to fluctuations in the interest rate from 6.43 per cent to 7.29 per cent.
  • LVAR and BVAR models produce inaccurate forecasts, generally worse than those from a random walk.
  • VECM yields the most accurate forecasts and is almost consistently better than the random walk.
  • ARMA-ARCH model is more accurate than LVAR and the BVAR models.
  • The poor performance of all the models with the exception of VECM is highlighted in Figure 4D.

For 5-year government securities, the BVAR models do not perform well. Overall, VECM outperforms all the alternative models. VECM also generally outperforms the alternatives at the short and long forecast horizons.

Government Securities – 10- year (Tables 5E and 6E, Figures 1.1E-1.3E, 2.1E-2.3E, 3.1E-3.3E and 4E)

  • Introducing ARCH effects in the ARMA model does not improve forecast accuracy.
  • LVAR produces the most accurate short-term and long-term forecasts, better than all other models.
  • VECM is generally out-performed by LVAR and BVAR models.
  • Performance of all BVAR models is reasonable and generally improves on loosening the prior. In the extreme case, with a very loose prior, the BVAR model converges to LVAR, which in this case is the preferred model.
  • All models consistently out-perform the random walk.
  • The accuracy of all the univariate models deteriorates with the increase in theforecast horizons (Table 6E).
  • LVAR and BVAR models generally show improvement in accuracy with the increase in the forecast horizons (Table 6E).
  • Figures 2E, 3E, and 4E reinforce the superiority of LVAR and BVAR models.

Therefore, for 10-year government securities, forecasting performance of all the models is satisfactory. The model that produces the most accurate forecasts is LVAR, or, in other words, a BVAR with a very loose prior. LVAR model produces the most accurate short- and long-term forecasts.

Thus, generally, BVAR models perform well and are able to beat the naïve forecast most of the time.

In the multivariate analysis above, the Bank Rate is used to capture the effect of monetary policy. Other variables included are: inflation, liquidity, credit, spread, Libor 3 and 6-months, forward premia 3 and 6-months. In the above models, we now examine, if the repo rate can be used in place of the Bank Rate, i.e., if the repo rate is a better predictor of interest rates compared to the Bank Rate. Tables 7A-7E report the out-of-sample forecast accuracy (reflected in a decrease in U) for both these rates as measured by the Theil statistic. The tables show that the improvement (if any) in accuracy from using the repo rate is marginal at best. The maximum improvement occurs in the TB 15-91 and that too by less than 10%. The Bank Rate can therefore be used as a satisfactory indicator of monetary policy.