1. Introduction

The most common estimator used in applied econometrics is that of ordinary least squares simply due to its “ideal” theoretical characteristics. However, Clive Granger used nonsense correlations introduced by Yule (1926) and a framework based on balanced and unbalanced equations to illustrate some “non-ideal” outcomes; Granger explored the fact that in the presence of stationary and non-stationary time series, the standard ordinary least squares inference could be misleading.

Our purpose is to illustrate the meaning of the conduction of a reliable statistical analysis using some of Granger’s groundbreaking ideas. In section 2, we offer a background to understand some fundamental principles. In sections 3 and 4, we present our simulation results of the empirical bias of OLS. In section 5, we share some final thoughts.

2. Background

Granger coined the term cointegration to describe a regression in which a pair of variables maintain a genuine long-run relationship instead of a nonsense one, which he referred to as spurious. Looked upon as a long-memory uncovering process, the invention of cointegration was preceded by Granger’s concerns about nonsense correlations introduced by Yule (1926) and a framework based on balanced and unbalanced equations. As a result of such endeavors, Granger was awarded the 2003 Nobel prize in Economic Sciences.

The path to the Nobel was not an easy one, in fact it was quite the opposite. In the following quote Granger (2010, p. 3) explains the difficulties he faced publishing his findings: “Econometrica rejected the paper for various reasons, such as wanting a deeper theory and some discussion of the testing question and an application. As I knew little about testing I was very happy to accept Rob’s offer of help with the revision. I re-did the representation theorem and he produced the test and application, giving a paper by Granger and Engle which evolved into a paper by Engle and Granger, whilst I was away for six months leave in Oxford and Canberra. This new paper was submitted to Econometrica but it was also rejected for not being sufficiently original. I was anxious to submit it to David Hendry’s new econometrics journal but Rob wanted to explore other possibilities. These were being considered when the editor of Econometrica asked us to re-submit because they were getting so many submissions on this topic that they needed our paper as some kind of base reference.”

To introduce his viewpoint, Yule (1926) proposed the correlation coefficient between mortality per 1,000 persons in England and Wales and the proportion of Church of England Marriages per 1,000 of all marriages as an example. The correlation obtained was 0.9512! Yule (1926) conducted a reliable statistical analysis to the extent: 1) he was aware that the correlation between two variables is a meaningful measure of its linear relationship only if its means are constant over time; 2) he identified the high autocorrelation of the selected variables, and 3) he concluded that the underlying probabilistic assumption of the correlation coefficient was violated and, therefore, the obtained measure was likely misleading.¹ In this regard, Johnston and DiNardo (1997, p. 10) added: “no British politician proposed closing down the Church of England to confer immortality on the electorate.”

As an extension of his work on spurious regressions during the early 1980s, Granger investigated what he named “balanced models”, defined as follows (Granger, 1993, p. 12): “An equation which has X _t having the same dominant features as Y _t will be called balanced. Thus, if X _t is I(0), Y _t is I(1), the equation will not be balanced. It is a necessary condition that a specified model be balanced for the model to be satisfactory.” As a rule of thumb in applied economics, we point out that the level of a variable is I(1), and its first difference is I(0). The specification of the Granger’s Ordinary Least Squares (OLS) bivariate regression is the following:

Having in mind “the balanced equation law”, we would expect that Ut has the same property of Y _t , and so the third assumption of the classical linear regressions model will not be obeyed, meaning we would expect some low Durbin-Watson statistics, a well-known symptom of spurious regression.

To explain a nonsense regression, Granger used to quote a book written long ago by a coauthor of Yule, M. G. Kendall. We have two AR(1) processes:

with β₁ = β₂ = β, that is Z1 _t and Z2 _t both obeyed the same autoregressive model, and Ut are “white noise” innovations independent of each other at all pairs of time. The sample correlation (R) between n consecutive terms of Z1 _t and Z2 _t has the following variance (Kendall, 1954, p. 113):

where n is the sample size. According to Granger (2003, p. 558), if β “is near one and n not very large, then the var(R) will be quite big, which can only be achieved if the distribution of R values has large weights near the extreme values of -1 and 1, which will correspond to” a “significant” coefficient value in a bivariate regression of Z1 _t on Z2 _t .

The idea of cointegration between a pair of variables I(1), which generates a linear combination I(0), was a natural extension of Granger’s concerns on spurious regression and balanced models. Suppose Y _t and W _t are integrated of order one, and we run the following regression:

Y _t and W _t are cointegrated if

<mml:msub> <mml:mrow> <mml:mi>Y</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>t</mml:mi> </mml:mrow> </mml:msub> <mml:mo>-</mml:mo> <mml:msub> <mml:mrow> <mml:mi>γ</mml:mi> </mml:mrow> <mml:mrow> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> <mml:msub> <mml:mrow> <mml:mi>W</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>t</mml:mi> </mml:mrow> </mml:msub>

eliminates its common stochastic trend. In this particular case,

<mml:msub> <mml:mrow> <mml:mi>γ</mml:mi> </mml:mrow> <mml:mrow> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub>

is called the “cointegrating” parameter, and U _t is I(0) -implying a stable and unique long-run equilibrium relationship between Y _t and W _t .

Lastly, a permanent concern of Professor Granger was the generation of spurious results. Posthumously, he sent us an elegant warning (White and Granger, 2011, p. 15): “When looking for causal relations between stochastic trends, it is important to recognize that Reichenbach’s principle of common cause (Reichenbach, 1956) does not necessarily apply. Briefly, Reichenbach’s principle holds that if we see correlation between two series, then one series must cause the other, or there must be an underlying common cause. With stochastic trends, however, observed correlations may be spurious, as pointed out long ago by Yule (1926).” Their message should guide our empirical work: a high correlation between a pair of variables should not lead us to assume a cause-effect relationship or the existence of a common cause, but rather to recognize it as a result of expressing the statistical properties of the variables at stake in addition to the sensitivity of the statistic used.

3. The empirical bias of OLS in the presence of non-stationary time series

In their seminal paper, Granger and Newbold (1974) performed a mini Monte Carlo as a way “to find evidence of spurious regressions” (Granger, 2003, p. 558).² They generated pairs of independent random walks without drift, each of length 50 (T = 50), and run 100 bivariate regressions. Considering the knowledge at the beginning of the seventies, in their simulation it was expected that roughly 95 percent of |t| values on the estimated parameter would be less than 1.96, but it was the case only in 23 occasions.³ Since the random walks were unrelated by design, the t values were misleading, and so, using their simulation results Granger and Newbold (1974, p. 115) suggested a new critical value of 11.2 when assessing the significance of the coefficient at the 5% level. At present we know that this suggestion is inconvenient, in the sense that Phillips (1986, p. 318) demonstrated that it has not asymptotic sense to apply the correction based on the standardized statistic, that is,

<mml:mfenced> <mml:mrow> <mml:mfrac> <mml:mrow> <mml:mo>|</mml:mo> <mml:mi>t</mml:mi> <mml:mo>|</mml:mo> </mml:mrow> <mml:mrow> <mml:msqrt> <mml:mi>T</mml:mi> </mml:msqrt> </mml:mrow> </mml:mfrac> </mml:mrow> </mml:mfenced>

.

Inspired by Granger and Newbold (1974) we designed a Monte Carlo simulation. In our case, each random walk has 100 terms -approximately the same length of the variables contained in the database of Nelson and Plosser (1982) that we will use later-, and the number of replications was the nowadays standard 10,000. Box 1 shows our (EViews) program.

Box 1. Code to run a Monte Carlo simulation using random walks ’Create a workfile undated, range 1 to 10,000 !reps = 10000 for !i=1 to !reps genr innovationrw1{!i}=@nrnd genr innovationrw2{!i}=@nrnd smpl 1 1 genr rw1{!i}=0 genr rw2{!i}=0 smpl 2 100 genr rw1{!i}=rw1{!i}(-1)+innovationrw1{!i} genr rw2{!i}=rw2{!i}(-1)+innovationrw2{!i} smpl 1 100 matrix(!reps,2) results equation eq{!i}.ls rw1{!i}=c(1)*rw2{!i} results(!i,1)=eq{!i}.@coefs(1) results(!i,2)=eq{!i}.@tstats(1) d innovationrw1{!i} d innovationrw2{!i} d rw1{!i} d rw2{!i} d eq{!i} next ’Copy and paste “results” to Excel ’In EViews type: data NONSENSE TsNONSENSE ’Copy and paste from Excel to EViews NONSENSE TsNONSENSE

It is necessary to specify that the “!reps” command sets the number of replicas; in the following lines, we create (“genr”) a pair of stationary time series that contain pseudo-random draws from a standard normal distribution. Initially, we focused on the first observations (“smpl 1 1”) to fix the initial conditions of the two random walks. We then generated the rest of its values by stacking the innovations generated. To create a new object, specifically speaking a matrix with two columns, we invoke the command “matrix”, we estimate the regression using OLS (“.ls”), and we deposit the values of the estimated coefficients and its corresponding t values in “results”. It is necessary to erase the inputs of the exercise for obvious reasons (“d” from delete), and to use the “next” command to end the code. Finally, with the last three suggestion lines, we change the format of the estimated coefficients and t values as columns of a matrix to variables (data), using the spreadsheet as a “scale” software.

The following figure shows statistics about the values of the NONSENSE estimated coefficients (y-axis frequency and x-axis coefficient values) and the table its t values tabulation.

In figure 1, the horizontal axis constitutes the bias in estimating the slope coefficient -to the extent the true value is zero- and the vertical axis is its relative frequency. In other words, we would expect coefficients with a value around zero and non-significant t values in the 95 percent of the regressions since, by design, drift-free independent random walks were used. But the results obtained were not as expected: in 7,800 cases the values of the coefficients exceeded the value of |0.2| and we barely obtained in 1,490 occasions t values less than |2|. Hence, there is a problem if we apply the “standard OLS inference” (Granger et al. 2001, p. 900). In our days we know that the heart of the matter is the use of standard errors which underestimate the true variation of the OLS estimators; unfortunately, there is not a practical solution, that is, an alternative mechanical estimator of the standard errors useful to construct an adjusted t statistic (Sun, 2004).

4. The empirical bias of OLS in the presence of stationary time series

According to Phillips (2003, p. 2), “a primary limitation on empirical knowledge is that the true model for any given data is unknown and, in all practical cases, unknowable. Even if a formulated model were correct, it would still depend on parameters that need to be estimated from data”.

Using a particular type of stationary time series Greene (2003, pp. 44-45) proposed the following Monte Carlo experiment: 1) Generate two stationary random variables, W _t and X _t ; 2) Generate U _t = 0.5W _t , and Y _t = 0.5X _t + U _t ; 3) Run 10,000 regressions, and 4) Randomly select 500 samples of 100 observation and make a graph. As we read in other books frequently, his conclusion was simple (Greene, 2003, p. 45): “note that the distribution of slopes has a mean roughly equal to the ‘true values’ of 0.5”. It is convenient to note the following:

1. Our preference for an unbiased estimator stems from the “hope” that a particular estimate will be close to the mean of the estimator’s sampling distribution; but “it is possible to have an ‘unlucky’ sample and thus a bad estimate” (Kennedy, 2003, p. 16 and p. 31). This handicap of empirical sciences explains the origin of the cruel story of the three econometricians who go duck hunting. The first shoots about a foot in front of the duck, the second about a foot behind, and the third yells, “We got him!”
2. We knew the true model, so we avoid the “alias o bias matrix” (Draper and Smith, 1998, chapter 10). But since this ideal situation is never present in empirical research, we should take advantage of the following axiom (Granger, 1993, p. 2): “any model will be only an approximation to the generating mechanism...It follows that several models can all be equally good approximations. How can one judge the quality of the approximation means further discussion.”
3. Professor Greene explored a very extreme time series in the sense that W _t , X _t , and Y _t can be written as first order autoregressive processes with a coefficient equal to zero or, in other words, because the correlation coefficients between Wt and Wt-k is equal to zero for all k ≠ 0, and similarly for X _t and Y _t . Indeed, it is impossible to imagine economic variables with a similar data generating process. In this sense, we recommend never lose sight of the statistical properties of the dependent variable and the regressors because (Patterson, 2000, p. 317) “the properties of the OLS estimator of the coefficients and the distribution of this estimator depend crucially upon these properties.”

In Granger et al. (2001) and Granger (2003, p. 560), it was shown that spurious regressions can also occur, “although less clearly”, with stationary time series. They generated two AR(1) processes as in our equations (2) and (3) with 0 < β₁ = β₂ = β ≤ 1, and run regressions following equation (1) with sample sizes varying between 100 and 10,000 -to suggest that the nonsense regression issue is not a small sample property. Table 2 is a summary of their results:

According to Granger (2003, p. 557), Yule (1926) is a much-cited paper but not sufficiently understood. The content of Table 2 supports this point of view in the sense that, as in Yule (1926), the clue is the degree of the autocorrelation of involved variables in a regression. It is not necessary to have random walks to generate nonsense exercises. It is just great, with AR(1) processes with coefficients between 0.25 and 0.9, and distinct sample sizes, Granger et al. (2001) were in a position to put all the applied econometricians in check.

In their seminal (Campbell and Perron, 1991, p. 147) paper, Nelson and Plosser (1982, p. 147) analyzed 27 variables in “natural logs except for the bond yield”: real GNP, nominal GNP, real per capita GNP, industrial production, employment, unemployment rate, GDP deflator, consumer prices, wages, real wages, money stock, velocity, bond yields, and common stock prices. Theoretically speaking, there is a mixture of stationary and non-stationary time series. So as not to enter into an unnecessary discussion, the next figure shows all the variables and Table 3 its autocorrelation coefficients.

To run four simulations, we use as a point of reference the minor values of the autocorrelations contained in Table 3. In each case, for example, we run the following general program:

Box 2. Code to run a Monte Carlo simulation using I(0) time series ’Create a workfile undated, range 1 to 10,000 !reps = 10000 for !i=1 to !reps genr innovationx1{!i}=@nrnd genr innovationx2{!i}=@nrnd smpl 1 1 genr x1{!i}=0 genr x2{!i}=0 smpl 2 100 genr x1{!i}=(0.754)*x1{!i}(-1)+innovationx1{!i} genr x2{!i}=(0.754)*x2{!i}(-1)+innovationx2{!i} smpl 1 100 matrix(!reps,2) results equation eq{!i}.ls x1{!i}=c(1)*x2{!i} results(!i,1)=eq{!i}.@coefs(1) results(!i,2)=eq{!i}.@tstats(1) d innovationx1{!i} d innovationx2{!i} d x1{!i} d x2{!i} d eq{!i} next ’Export “results” to Excel ’In EViews: data NONSENSE tsNONSENSE ’Copy and paste from Excel to EViews NONSENSE tsNONSENSE

The user should only change the autoregressive values in the program (lines 10 and 11). The following figure and table show statistics about the values of the estimated coefficients and its t values (y-axis frequency and x-axis values). Following Patterson (2000, pp. 317-26), we improved its titles.

Before commenting on our results, we want to focus on a critical aspect of the dissemination of sciences. In 2011 Crocker and Cooper, editors of the prestige Science, wrote a note entitled “Addressing scientific fraud”, showing evidence that fraud is a common practice. In this regard, as the first line of defense, Crocker and Cooper (2011, p. 1182) ask for “greater transparency with data”.

At least in economics, Crocker and Cooper (2011) hit the nail on the head. To cite but one terrible example, Duvendack, Palmer-Jones, and Reed (2015, pp. 181-182) uncovered the following: “What can we learn from our analysis of replication studies? Most importantly, and perhaps not too surprisingly, the main takeaway is that, conditional on the replication having been published, there is a high rate of disconfirmation. Over the full set of replication studies, approximately two out of every three studies were unable to confirm the original findings. Another 12 percent disconfirmed at least one major finding of the original study, while confirming others (Mixed?). In other words, nearly 80 percent of replication studies have found major flaws in the original research.”

Hence, we used an open-access database of a seminal paper and explained our computational code step by step. We can now begin to analyze our results.

In Figure 3, the horizontal axis measures the bias in estimating the slope coefficient, and the vertical axis is its relative frequency. Once again, in our simulations, we obtained unexpected coefficient values and t values. For Granger (2003, p. 560), the implication of these results is that “applied econometricians should not worry about spurious regressions only when dealing with I(1), unit root, processes.” But his caveat has been practically ignored by a large portion of the specialized literature. Excellent econometric literature -both textbooks and journal articles- exposes OLS without looking at the statistical properties of the variables we are dealing with, visualizes non-stationary time series in some moments as dominated by a deterministic trend and in others by a stochastic one, and last but not least, limits the problem of spurious regressions to the case of integrated variables.⁴

5. Final messages from “measurement in economics” and to perform a “reliable statistical analysis”

Measurement in economics is not a unified field but fragmented in subfields such as econometrics, index theory, and national accounts (Boumans, 2007, p. 3). It is not an exaggeration to point out that currently, econometric analysis constitutes a central measuring instrument used in our science. It is worth mentioning that, as other measuring instruments, initially functioned as an “artifact of measurement”, but rapidly became an “analytical device” (Klein, 2001 and Morgan 2001). In this regard, the guru of measurement in economics warned us that (Boumans 2005, p. 121): “a relevant problem of instruments used to make unobservables visible is how to distinguish between the facts about the phenomenon and the artifacts created by the instrument.”

Therefore, it is always necessary to keep in mind that the output of the econometric analysis shapes our scientific study of objects. Granger’s path-breaking ideas already showed us the dangers of carrying out a regression that does not take into account the statistical characteristics of the analyzed variables and of ignoring the fact that the properties of the OLS estimator depend crucially upon these characteristics. We should be cautious when drawing qualitative conclusions based on a standard OLS inference carried out in the context of a regression analysis. Indeed, it is urgent that the specialized literature completely assimilates Granger’s contributions regarding the risk of obtaining spurious results in economics.