On accuracy of survey forecasts of US mortgage spread

The cyclical variation behavior of the mortgage spread has motivated some studies to investigate its relationship to economic activity. Indeed, recent empirical findings indicate that the mortgage spread is a determinant/predictor of economic activity. We define the mortgage spread as the difference between the 30-year mortgage and 10-year Treasury rates and ask whether the Blue Chip (consensus) forecasts of these series are accurate for 1988–2015. Our findings indicate that the Blue Chip forecasts of both the 30-year mortgage and 10-year Treasury rates are biased, fail to outperform the random walk benchmark, and are directionally inaccurate. However, the Blue Chip forecasts of the mortgage spread are generally unbiased, outperform the random walk benchmark, and are directionally accurate—thus of value to a user. Given such evidence, a natural extension for future research is to explore whether the predictive information content of Blue Chip forecasts of the mortgage spread is a better predictor of output growth and whether it helps improve Blue Chip forecasts of output growth. Subjects: Macroeconomics; Economic Forecasting; Property & Real Estate Finance


PUBLIC INTEREST STATEMENT
The literature indicates that the US mortgage spread is a determinant/predictor of economic activity. Motivated by such findings, we define mortgage spread as the difference between 30year mortgage and 10-year Treasury rates and ask whether experts' forecasts of these series are accurate for 1988-2015. Long-term interest rates are difficult to forecast. Both theory and empirical evidence suggest that the best forecast of future rate is today's rate (a naïve forecast). In line with this, we show that experts' forecasts of 30-year mortgage and 10-year Treasury rates fail to outperform naïve forecasts and are directionally inaccurate. However, experts' forecasts of mortgage spread (calculated as the difference between experts' forecasts of 30-year mortgage and 10-year Treasury rates) outperform naïve forecasts and are directionally accurate. Such evidence is important since accurate forecasts of mortgage spread have the potential to be a better predictor of economic activity than the actual spread.

Introduction
A strand of research asks whether financial spreads contain useful predictive information for output growth and inflation (Aretz & Peel, 2010;Estrella & Hardouvelis, 1991;Estrella & Mishkin, 1997;Stock & Watson, 2003). Estrella (2005) constructs a formal macroeconomic model to provide the theoretical rationale for predictive information of the term spread (defined as the difference between yields on long and short maturity debt). A survey of the literature by Wheelock and Wohar (2009) points to the term spread as a predictor of output growth and recessions, although the findings show variations across countries and time. 1 More recent studies have focused on the mortgage spread as a determinant/predictor of economic activity. Walentin (2014, p. 62) identifies three channels through which mortgage spreads can influence economic activity: "(i) house prices and residential investment through the user cost of housing, (ii) as one relevant rate in the consumption/savings decision and, (iii) the post-interest disposable income of any household with a mortgage." He cites Hubbard and Mayer (2009), Guerrieri and Lorenzoni (2011) and Hall (2011a, 2011b who have alluded to the relationship and then goes on to add to the literature by providing empirical evidence indicating that mortgage spread shocks have economically and statistically significant impacts on economic activity. The study by Hännikäinen (2016) provides further evidence by showing that the mortgage spread has useful predictive information for both real GDP and industrial production.
As noted by Darrat, Dickens, and Al-Khazali (2006), the mortgage industry uses the 10-year Treasury rate as the benchmark for pricing fixed rate mortgages. As such, we follow Hännikäinen (2016) and define the mortgage spread as the difference between the 30-year mortgage rate and the 10-year Treasury rate and ask whether the Blue Chip forecasts of these series are accurate for 1988-2015. Theory suggests that the best forecast of the long-term interest rate is the random walk forecast, defined as the most recent rate known at the time of the forecast (Pesando, 1979;Reichenstein, 2006). Consistent with this theory and related existing empirical evidence (Baghestani, 2006(Baghestani, , 2008(Baghestani, , 2009aBrooks & Gray, 2004), our findings indicate that Blue Chip forecasts of both the 30-year mortgage rate and the 10-year Treasury rate are biased, fail to outperform the random walk benchmark, and are directionally inaccurate. 2 However, Blue Chip forecasts of the mortgage spread are generally unbiased, outperform the random walk benchmark, and are directionally accurate. As noted by Stekler (1990, 1991), a forecast is of value if it accurately predicts directional change. Accordingly, our findings indicate that, unlike Blue Chip forecasts of 30-year mortgage and 10-year Treasury rates, Blue Chip forecasts of the mortgage spread are of value to a user since they are directionally accurate. Such results are generally consistent with existing evidence on survey forecasts of financial spreads (Baghestani 2005b(Baghestani , 2009b(Baghestani , 2010a. Put together, our findings in this study contribute to the literature on the survey forecasts of financial spreads, with respect to comparing the accuracy of mortgage spread forecasts and the accuracy of the forecasts of the variables (mortgage rate and treasury rate) used to calculate the mortgage spread. Section 2 presents the methodology by first describing Blue Chip and random walk forecasts and then discussing the forecast evaluation test results. Section 3 concludes.

Methodology
As a long-running survey, Blue Chip asks approximately 50 analysts at major US banks, corporations, and consulting firms for their forecasts of major economic and financial indicators. The survey is conducted around the beginning of every month with the participants providing their forecasts for the current quarter and the following four quarters. Blue Chip utilizes the individual responses to calculate the consensus (mean) forecasts, which are then published in Blue Chip Financial Forecasts. 3 Since the survey is conducted monthly, there are three sets of forecasts for each quarter. In this study, we focus on the one-, two-, three-, and four-quarter-ahead Blue Chip consensus forecasts of the 30-year mortgage rate and the 10-year Treasury rate made around the beginning of the second month of the quarter.
In generating the random walk forecasts of the 30-year mortgage rate (and the 10-year Treasury rate), we make use of the latest actual weekly rate available at the time of the survey published in Blue Chip Financial Forecasts along with the quarterly forecasts. The latest weekly rate for the surveys conducted in the second month of quarter t is the rate for the third week in the first month of quarter t. More specifically, let A t + f be the actual rate in quarter t + f and R t + f be the random walk forecast of A t + f made in the second month of quarter t (with the forecast horizon f = 1, 2, 3, and 4 quarters). 4 Then, the random walk forecast R t + f = Aw t (where Aw t is the latest weekly rate known at the time of the survey).
As noted above, we follow Hännikäinen (2016) and calculate the actual mortgage spread as the difference between the actual 30-year mortgage rate and the actual 10-year Treasury rate. We then calculate the Blue Chip (random walk) forecast of the mortgage spread as the difference between the Blue Chip (random walk) forecast of the 30-year mortgage rate and the Blue Chip (random walk) forecast of the 10-year Treasury rate.
In evaluating Blue Chip forecasts of the 30-year mortgage rate, 10-year Treasury rate, and mortgage spread, we focus on answering the following three questions: (1) Are Blue Chip forecasts unbiased?
(2) Are Blue Chip forecasts more informative than the random walk benchmark?
(3) Are Blue Chip forecasts directionally accurate? Figure 1 plots the actual rates of the 30-year mortgage and 10-year Treasury for 1988Q1-2015Q4. As indicated, the 30-year mortgage has a mean rate of 6.77% with a high (low) rate of 10.82 (3.36), and the 10-year Treasury has a mean rate of 5.12% with a high (low) rate of 9.21 (1.64). Figure 2 plots the actual mortgage spread for 1988Q1-2015Q4. Again, as indicated, the spread has a mean rate of 1.65% with a high (low) rate of 2.59 (1.23). We focus on Blue Chip and random walk forecasts of these series that are made in the first quarter of 1988 through the fourth quarter of 2014. As such, the sample periods for the one-, two-, three-, and four-quarter-ahead forecasts are, respectively, 1988Q2-2015Q1, 1988Q3-2015Q2, 1988Q4-2015Q3, and 1989Q1-2015Q4, with 108 observations for each forecast horizon.

Are Blue Chip forecasts unbiased?
We start with specifying the following test equations for unbiasedness, 30-year mortgage rate: Mean = 6.77% High = 10.82 Low = 3.36 10-year Treasury rate: Mean = 5.12% High = 9.21 Low = 1.64 where A t+f and A t−1 are the actual series for, respectively, quarters t + f and t − 1, and P t+f is the Blue Chip forecast of A t+f made in quarter t. We subtract A t-1 from both the actual and forecast series in Equation (1) in order to alleviate problems arising from the non-stationarity of the series (Fair & Shiller, 1990). The forecast is unbiased when the individual null hypotheses that α = 0, β = 1, and α′ = 0 cannot be rejected (Holden & Peel, 1990). With the forecasts made in quarter t, the error term u t+f follows an f-order moving-average process under the null hypothesis of unbiasedness. Since forecast errors are generally heteroscedastic, we utilize the Newey and West (1987) procedure to estimate the above test equations in order to correct for both heteroscedasticity and serial correlation over f quarters.
Rows 1-4 of Table 1 (Table 2) report, respectively, the OLS estimates of Equations (1) and (2) with the correct standard errors for the one-, two-, three-, and four-quarter-ahead Blue Chip forecasts of the 30-year mortgage rate (10-year Treasury rate). As indicated by superscript "a," these forecasts fail to be unbiased since we reject the individual null hypotheses that α = 0, β = 1, and α′ = 0.
Rows 1-4 of Table 3 report, respectively, the OLS estimates of Equations (1) and (2) with the correct standard errors for the one-, two-, three-, and four-quarter-ahead Blue Chip forecasts of the mortgage spread. As can be seen, the one-, two-, and three-quarter-ahead forecasts are all unbiased since we cannot reject the individual null hypotheses that α = 0, β = 1, and α′ = 0. However, the four-quarter-ahead forecasts fail to be unbiased since we reject the null hypotheses that α = 0 and β = 1 both individually and jointly. Further inspection reveals that the one-through four-quarterahead Blue Chip forecasts of the mortgage spread in rows 1-4 are all free of systematic bias since we cannot reject the null hypothesis that α′ = 0. This conclusion is consistent with the mean forecast error (ME) and mean absolute forecast error (MAE) estimates reported in rows 1-4 of Table 3. That is, the ME estimates, ranging from 0.027 to 0.066, are small compared to the MAE estimates, ranging from 0.148 to 0.219.

Are Blue Chip forecasts more informative than the random walk benchmark?
In answering, we estimate the following encompassing test equation,  where, again, A t+f and A t-1 are the actual series for, respectively, quarters t + f and t − 1, and P t+f (R t+f ) is the Blue Chip (random walk) forecast of A t+f made in quarter t. A significant and positive estimate of γ B with an insignificant estimate of γ R indicates that the Blue Chip forecast contains more predictive information than the random walk benchmark; the converse is also true. Blue Chip and random walk forecasts contain distinct predictive information when the estimates of γ B and γ R are both positive and significant. Blue Chip and random walk forecasts contain similar predictive information when the estimates of both γ B and γ R are insignificant. Notes: A t + f and A t − 1 are the actual 30-year mortgage rates for, respectively, quarters t + f and t − 1, and P t + f (R t + f ) is the Blue Chip (random walk) forecasts of A t + f made in quarter t; f is the forecast horizon. ME is the mean forecast error, and MAE is the mean absolute forecast error. The correct (Newey-West) standard errors are in parentheses. P1 is the pvalue for testing the joint null hypothesis of unbiasedness (α = 0 and β = 1). The sample periods for f = 1, 2, 3, and 4 are, 1988Q2-2015Q1, 1988Q3-2015Q2, 1988Q4-2015.3, and 1989Q1-2015Q4, with 108 observations for each forecast horizon. Superscript "a" indicates significance at the 10% or lower level of significance.

Panel A. Unbiasedness test results
Row No.

Table 2. Forecast accuracy test results: 10-year Treasury rate
Notes: See the notes in Table 1. A t + f and A t − 1 are the actual 10-year Treasury rates for, respectively, quarters t + f and t − 1, and P t + f (R t + f ) is the Blue Chip (random walk) forecasts of A t + f made in quarter t. Superscript "a" indicates significance at the 10% or lower level of significance.

Panel A. Unbiasedness test results
Row No. Rows 5-8 of Table 1 (Table 2) report, respectively, the OLS estimates of Equation (3) with the correct standard errors for the one-, two-, three-, and four-quarter-ahead forecasts of the 30-year mortgage rate (10-year Treasury rate). As can be seen, for the one-, two-, and three-quarter-ahead forecasts, the estimates of γ B are insignificant but the estimates of γ R are both positive and significant, meaning that random walk forecasts are more informative than Blue Chip forecasts. For the four-quarter-ahead forecasts, the estimates of γ B and γ R are both insignificant, indicating that Blue Chip and random walk forecasts contain similar predictive information.
Rows 5-8 of Table 3 report, respectively, the OLS estimates of Equation (3) with the correct standard errors for the one-, two-, three-, and four-quarter-ahead forecasts of the mortgage spread. As can be seen, the estimates of both γ B and γ R in row 5 are positive and significant, meaning that the one-quarter-ahead Blue Chip and random walk forecasts contain distinct predictive information. However, the estimates of γ B in rows 6-8 are both positive and significant but the estimates of γ R are insignificant, indicating that the two-, three-, and four-quarter-ahead Blue Chip forecasts contain more predictive information than the random walk benchmark.

Are Blue Chip forecasts directionally accurate?
We follow the literature and employ the test procedure first introduced by Merton (1981) and Henriksson and Merton (1981) and later refined by Schnader and Stekler (1990), among others. Table 4 presents the two-by-two contingency table in which the actual change is (A t+f − Aw t ) and the predicted change is (P t+f − Aw t ), with four elements representing the numbers of correct directional  Table 1. A t + f and A t − 1 are the actual mortgage spreads for, respectively, quarters t + f and t − 1, and P t+f (R t + f ) is the Blue Chip (random walk) forecast of A t + f made in quarter t. Superscript "a" indicates significance at the 10% or lower level of significance.

Table 4. Contingency table
Notes: With Aw t measuring the most recent rate known at the time of the survey, ΔA t + f = (A t + f − Aw t ) is the actual change, and ΔP t + f = (P t+f − Aw t ) is the predicted change. The sign (+) represents an upward move, and the sign (−) represents a downward move in the series. The numbers of correct (incorrect) sign predictions are n 1 and n 2 (n 3 and n 4 ).

Actual Upward Downward
Correct directional predictions n 1 : predictions (n 1 and n 2 ) and incorrect directional predictions (n 3 and n 4 ). As such, the overall directional accuracy rate is π All = (n 1 + n 2 )/n, with n representing the sample size. In addition, the proportion of correctly predicted upward moves is π Up = n 1 /(n 1 + n 3 ), and the proportion of correctly predicted downward moves is π Down = n 2 /(n 2 + n 4 ). Table 5 reports these statistics for the Blue Chip forecasts of the 30-year mortgage rate in rows 1-4, the 10-year Treasury rate in rows 5-8, and the mortgage spread in rows 9-12. In testing the null hypothesis of no directional association between actual and predicted changes, we use Fisher's exact test and the chi-square tests with and without Yate's continuity correction (Sinclair, Stekler, & Kitzinger, 2010). As can be seen, we cannot reject this null hypothesis for the forecasts in rows 1-8, meaning that the one-, two-, three-, and four-quarter-ahead Blue Chip forecasts of the 30-year mortgage rate and the 10-year Treasury rate fail to be directionally accurate. These test results are consistent with the low overall accuracy rates (π All ), ranging from 0.41 to 0.55. Further results indicate that, for each forecast in rows 1-8, the proportion of correctly predicted upward moves (π Up ), ranging from 0.59 to 0.86, is high, but the proportion of correctly predicted downward moves (π Down ), ranging from 0.24 to 0.37, is quite low. As such, it appears that the directional inaccuracy of the Blue Chip forecasts of the 30-year mortgage rate and the 10-year Treasury rate is due to inaccurately predicting the downward moves. Baghestani and Kherfi (2008) report a similar finding using the data from the Michigan Surveys of Consumers. They argue that "When interest rates are relatively stable, consumers may be more concerned with the explicit cost of borrowing since the declines in interest rates are perceived to be too small for the implicit cost to matter. Therefore, it becomes less important for them to accurately predict the downward moves in interest rates" (p. 727).

Table 5. Directional accuracy test results
Notes: See the notes for Table 4. The overall directional accuracy rate is π All = (n 1 + n 2 )/n, the proportion of correctly predicted upward moves is π Up = n 1 /(n 1 + n 3 ), and the proportion of correctly predicted downward moves is π Down = n 2 / (n 2 + n 4 ). Superscript "b" indicates that the p-values of Fisher's exact test and the chi-square tests with and without Yate's continuity correction are all below 0.10, and thus the null hypothesis of no directional association between the actual and predicted change is rejected. P2 is the chi square test p-value for examining the null hypothesis of "no asymmetry," which is defined as the proportion of incorrectly predicted upward moves (1 − π Up ) equals the proportion of incorrectly predicted downward moves (1 − π Down ). For the one-, two-, three-, and four-quarter-ahead Blue Chip forecasts of the mortgage spread in rows 9-12, the overall accuracy rate (π All ), ranging from 0.60 to 0.69, happens to be high. As indicated by superscript "b," we reject the null hypothesis of no directional association for these forecasts, meaning that they are directionally accurate and thus are of value to a user. Further results for every forecast in rows 9-12 indicate that the proportion of correctly predicted downward moves (π Down ), ranging from 0.67 to 0.81, is much more than the proportion of correctly predicted upward moves (π All ), ranging from 0.43 to 0.61. In testing the null hypothesis of no asymmetry, which is defined as the proportion of incorrectly predicted upward moves (1 − π Up ) equals the proportion of incorrectly predicted downward moves (1 − π Down ), we use the chi-square test described in Berenson, Levine, and Rindskopf (1988, sec. 11.4.1). With the chi-square test p-value of 0.570 greater than 0.10 in row 9, we cannot reject the null hypothesis of "no asymmetry," meaning that the one-quarterahead Blue Chip forecast is of value to a user who assigns the same loss (cost) to both incorrect upward and downward moves in the mortgage spread. However, with the chi-square test p-values (in rows 10-12) lower than 0.10, we reject the null hypothesis of "no asymmetry." As such, we conclude that the two-, three-, and four-quarter-ahead forecasts are of value to a user who assigns high (low) cost to incorrect downward (upward) moves in the mortgage spread.

Conclusions
Recent studies have focused on the mortgage spread as a determinant/predictor of economic activity. Walentin (2014) shows that mortgage spread shocks have economically and statistically significant impacts on economic activity, and Hännikäinen (2016) shows that the mortgage spread has useful predictive information for both real GDP and industrial production. We define the mortgage spread as the difference between the 30-year mortgage rate and the 10-year Treasury rate and ask whether the Blue Chip forecasts of these series are accurate for 1988-2015. Our findings indicate that Blue Chip forecasts of both 30-year mortgage and 10-year Treasury rates are biased, fail to outperform the random walk benchmark, and are directionally inaccurate. However, the Blue Chip forecasts of the mortgage spread are generally unbiased, outperform the random walk benchmark, and are directionally accurate -thus of value to a user.
A strand of research focuses on using survey data to generate accurate forecasts and/or improve professional forecasts of economic and financial indicators (Baghestani, 2005a(Baghestani, , 2015Baghestani & AbuAl-Foul, 2017;Christiansen, Eriksen, & Møller, 2014;Dees & Soares Brinca, 2013;Kellstedt, Linn, & Hannah, 2015;Lahiri, Monokroussos, & Zhao, 2016). Given the evidence presented in this study, a natural extension for future research is to explore whether Blue Chip forecasts of the mortgage spread are more useful than the actual mortgage spread in predicting output growth, and whether Blue Chip forecasts of the mortgage spread can help improve Blue Chip forecasts of output growth. Further future analysis may involve controlling for mortgage-related developments, such as those during the US recession of 2007-2009, and exploring the reason(s) behind the accuracy of mortgage spread forecasts and the inaccuracy of the forecasts of the variables used to compute the spread (i.e. mortgage rate and treasury rate). Notes 1. Term spreads have also been found useful in predicting directional change in home sales (Baghestani & Kaya, 2016). In another study, Baghestani and Toledo (2017) show that the Blue Chip forecasts of the Australian-US (UK-US) term spread differential helps predict directional change in the Australian-US (UK-US) exchange rate. 2. There are studies which have produced more accurate forecasts than the random walk. For example, Baghestani (2010b) utilizes the information in expected inflation to generate the 10-year Treasury rate forecasts that are superior to the random walk benchmark. Also, Baghestani (2017) shows that consumer survey data on both mortgage interest rates and expected inflation can help produce more accurate 30-year mortgage rate forecasts than the random walk benchmark.