A Monte Carlo simulation framework on the relative performance of system estimators in the presence of multicollinearity

Abstract The correctness and reliability of findings and\recommendations of empirical studies conducted by social and economic researchers depend largely on the efficiency of the econometrics methodologies employed in such studies. Of particular interest are such studies which are centered on the Sustainable Development Goals (SDG) considering the relevance of such studies to the total wellbeing of the world populace. In view of this, there is always a need for theoretical review of econometrics methodologies commonly used by researchers with a view to providing researchers with research updates on the theoretical standing of these methodologies. In this study, we set up a Monte Carlo Experiment (MCE) to evaluate the relative performance of various estimators of a simultaneous equation model in the presence of varied levels of multicollinearity. The model was estimated with a simulated data set of sample size 30 over 100 replications. The parameter estimates obtained from the six estimators considered were evaluated using RMSE criteria. Our result revealed that irrespective of the level of multicollinearity in our model, ILS and OLS yielded best estimates of the parameters. On the contrary, the system estimators all performed poorly in the presence of multicollinearity. Also, 2SLS, LIML and 3SLS estimators yielded virtually identical estimates. By our findings, in the presence of multicollinearity, estimators OLS and ILS performed best and should therefore be preferred above the multi-equation estimators.


PUBLIC INTEREST STATEMENT
The correctness and reliability of findings from empirical studies conducted by social and economic researchers depend largely on the efficiency of the econometrics methodologies employed in such studies. Of particular interest are such studies, which are centered on the Sustainable Development Goals (SDG) considering the relevance of such studies to the total wellbeing of the world populace. In view of this, there is a need for continuous contribution to literature on econometrics methodologies commonly used by researchers with a view to providing researchers with research updates on these methodologies. To this end, in this study we set up a Monte Carlo Experiment (MCE) to evaluate the relative performance of various estimators of a simultaneous equation model in the presence of varied levels of multicollinearity.

Introduction
The correctness and reliability of findings and recommendations of empirical studies conducted by social and economic researchers depend largely on the efficiency of the econometrics methodologies employed in such studies. Of particular interest are such studies, which are centered on the Sustainable Development Goals (SDG) considering the relevance of such studies to the total wellbeing of the world populace. In view of this, there is always a need for theoretical review of econometrics methodologies commonly used by researchers with a view to providing researchers with research updates on the theoretical standing of these methodologies. In this study, we set up a Monte Carlo Experiment (MCE) to evaluate the relative performance of various estimators of a simultaneous equation model in the presence of varied levels of multicollinearity. In the last few decades, Monte Carlo Simulation (MCS) has found extensive use in the fields of operational research and nuclear physics, where there are varieties of problems beyond the available tools of theoretical mathematics. They have been employed sporadically in numerous other fields of science, including Chemistry, Biology, Econometrics and Medicine. Problems handled by Monte Carlo methods are of two types. These are probabilistic or deterministic type according to whether or not they are directly concerned with the behavior and outcome of random processes.
Although the basic procedure of the Monte Carlo method is the manipulation of random numbers, these should not be employed prodigally. Each random number is a potential source of added uncertainty in the final result. Olubusoye (2001) warned researchers using the Monte Carlo method of the problem of using inherently correlated normal deviates to simulate the disturbance term in a simultaneous equation context. Consequently, it is necessary to scrutinize each part of a Monte Carlo experiment to see whether that part cannot be replaced by exact theoretical analysis contributing no uncertainty and also to ensure that experimental results are not confounded by the use of inherently deficient random deviates.
The Monte Carlo experimentalist needs wide experience in the use of some tools of pure mathematics and statistics, especially the theory of probability, in order that he may discern those connections between apparently dissimilar problems which suggest sophisticated Monte Carlo methods. He has to exercise ingenuity in distorting and modifying problems in the pursuit of variance-reducing techniques. He has to be competent at statistical and inferential procedures in order to extract the most reliable conclusions from his artificially generated data. The tools of his trade include computing machinery, and he must be familiar with numerical analysis. As in all experimental works, a feel of the problems is a great and sometimes essential asset.
In this paper, we present a Monte Carlo Simulation (MCS) framework used in studying the relative efficiency of system estimators in the presence of multicollinearity.

Literature review
The principle of asymptotic theory is the utilization of distributional properties in large samples to find estimates of unknown population parameters and draw inference about them. Some of the properties include consistency, asymptotic efficiency, and asymptotic normality. In econometrics, attention is often focused on evaluating the behavior of estimators in small sample and asymptotic theory fails most of the time to furnish enough useful information about these estimators. This inevitably creates a problem in trying to choose from among competing estimators.
One way of studying the small sample properties of estimators is to utilize the Monte Carlo method. The Monte Carlo method has been applied not only to the choice of alternative estimators but also in determining the impact of sample size, serial correlation, multicollinearity, and other factors on the different possible estimators in a given study. This approach creates a "laboratory environment" where controlled experiments on estimators are performed. So far, the sample properties of the various econometric techniques have been studied from simulated data in Monte Carlo studies and not with direct application of the techniques to actual observations. This approach is due to the fact that actual observations on economic variables are usually infected by multicollinearity, autocorrelation, errors of measurement and most other econometric "diseases" and in some cases simultaneously. Studies on small sample properties of estimators are usually based on the assumption of the simultaneous occurrence of all these problems. By Monte Carlo approach, the econometrician can generate data sets and stochastic terms, which are free of all but one of the problems listed above and therefore generate data resembling those obtained from controlled experiments. Wagner (1958) employed Monte Carlo sampling technique to compare small sample properties of limited information single Equation (LISE), least squares, and instrumental variable estimates. Two versions of two-equation models were used to generate 100 sets of observations over 20 time periods. With these observations and various statistical techniques, estimates of the parameters of an over-identified equation were obtained and compared. The analysis of the results of the 100 sets of data produced for each of the model indicates that the least squares generally give more biases but less variable estimates than the LISE method. Baldev (1980) considered four alternative forms of two-parameter normal and non-normal error distributions. He also reported on a Monte Carlo study of the small-sample properties of OLS, 2SLS, 3SLS, and FIML estimators. On the basis of 1,000 replications of sample size 20 in two experiments on an over-identified model, he found that the small-sample rankings of econometric estimators of both structural coefficients and forecasts of endogenous variables according to parametric and non-parametric measures of bias, dispersion and dispersion including bias do not change for any of the four error distributions considered. Kloek and Van Dijk (1978) applied Monte Carlo methods to obtain estimates of posterior moments of structural and reduced form parameters of Bayesian simultaneous equation systems. They noted that the Monte Carlo methods allow the analyst to make use of several types of exact and stochastic prior information. The method was also used to compute marginal posterior distributions of some parameter of interest. Lurie and Hartley (1972) employed Monte Carlo methods to handle distributional problems involving order statistics that cannot be treated analytically. Kmenta and Joseph (1963) carried out a Monte Carlo experiment to examine the small sample properties of ordinary least squares, indirect least squares, Hochs and Kleins estimates of the parameters of the Cobb-Douglas production function. A perfectly competitive group of firms in a single industry is considered in nine situations, which differ in the behavior of the disturbances, the variability of inputs and the position of the average firm. In each case, 200 samples of size 20 and 200 samples of size 100 were selected to approximate the sampling distribution of the various estimators. Nagar (1960) also studied the small sample properties of the simultaneous equation estimators using the Monte Carlo approach. Four methods of estimation considered were least squares, 2SLS, unbiased and minimum second moment. The study showed that the least squares of the four methods possesses the smallest second order sampling moments about the true parameter value in a majority of cases, while 2SLS showed the smallest bias in all cases. Carlin et al. (1992) proposed an adaptive Monte Carlo integration technique known as Gibbs sample as a mechanism for implementing a conceptually and computationally simple solution for multivariate space-modeling, forecasting and smoothening. They allowed for possibilities of non-normal errors and non-linear functions in the state equation, the observational equation or both.
Also, Essi (1997), employed the Monte Carlo approach to examine the consequences of wrong specification of error terms on the robustness of the estimators of non-linear singleequation econometric model. His study focused on the consequences of making a wrong choice i.e. assuming a multiplicative error-based model instead of an additive error-based model, and vice-versa; in the process of estimating an intrinsically nonlinear Cobb-Douglas model with parameters θ 1 ; θ 2 and θ 3 : Nwabueze (2000) considered a single-equation study of the choice of estimators of nine autocorrelated models also using the Monte Carlo technique. The study confirmed the least squares as the best out of the four estimators considered. Olubusoye (2001) using a two-equation simultaneous econometric model studied the consequences of the violation of the assumption of zero correlation between pairs of random stochastic terms used in Monte Carlo experiments. He concluded that pairs of normal deviates should be screened and those inherently correlated should not be used in generating stochastic error terms to be used in Monte Carlo experiments.
Despite the widespread use of the Monte Carlo technique, it suffers from two main deficiencies namely: specificity and impression. Specificity means that results obtained are largely dependent on the specification of the model(s) used in a given Monte Carlo experiment while imprecision means that the results are not precise. The impact of these deficiencies can be reduced to a reasonable extent. For instance, Response Surface Methodology may be used to reduce specificity deficiency while Control Variate Technique may be used to reduce the problem of imprecision. The implication of these deficiencies is two-fold. One is that simulation results are most of the time suggestive rather than conclusive since the extent of generalization of sampling experiments compares to their analytical counterparts is smaller. Secondly, Monte Carlo results are not end-product in econometric theory in general, or in the investigation of finite sample distribution in particular, but stands somewhere in-between. Hendry (1984) asserted that: "Monte Carlo experimentation can efficiently complement analysis to establish numerical analytic formulae which jointly summarizes the experimental findings and known analytical results in order to help interpret empirical evidence and to compute outcomes at other points within the relevant parameter space. The accuracy obtainable depends on the given budget constraints and the relevant price of capital labour. The later determining the point at which simulation is substituted for analysis and the former the overall precision of the exercise."

Theoretical framework
where μÑ 0; σ 2 À � and it also satisfies other classical assumptions for least squares estimation.
Numerical values are assigned to all the parameters embodied in the vector θ: The variance σ 2 is also assigned a numerical value, and on the basis of the assumed σ 2 ; normal deviates are selected and used in generating μ; the disturbance term A random sample size T of Xis selected and the numerical values of F X; θ ð Þare computed.
The vector Y is then obtained by computing F X; θ ð Þ þ μ The regression of Y on X is performed to produce estimate θ of θ: This procedure is replicated many times using the same sample size making it possible for the sampling distribution of b θto be constructed. The empirical distribution so obtained is then utilized in evaluating the precision of θ and in making other comparisons especially of the relative performance of different estimators of θ

Empirical framework
Assume the following model where y 0 s are the endogenous variables x 0 s are the exogenous variables u 0 s are the disturbance terms Equation (2) may be expressed in matrix form as In the linear simultaneous equations econometric model containing Gendogenous (jointly dependent) variables y 1 ; . . . ; y G and K predetermined variables X 1 ; X 2 ; . . . ; X k ; the Gstructural equatons at time tmay in general be written as: In matrix form, (3) can be written as: where B is a GxG matrix of coefficients of current endogenous variables, Γ a GxKmatrix of coefficient of predetermined variables y t ; X t and u t are column vectors of G and K elements respectively. The system of equations is complete since the number of endogenous variables is equal to the number of structural equations in the model. The system of equations as written in Equation (4) is a structural specification of the model. Also, the system of equations jointly determines at time t, the G endogenous variables, in terms of the predetermined variables and the values assumed by the stochastic disturbance term. (4) has G þ K parameters β i1 ; β i2 ; . . . ::; β iG ; γ i1; γ i2; . . . ::; γ ik : In practice, some of the parameters are usually specified to be zero. Otherwise, all the equations in the model will look alike statistically, and no equation could be identified. Constant terms are assumed to be included in the model by specifying one of the predetermined variables to be identically unity.

Each equation in Equation
Major assumptions about the model are: (1) The first basic assumption of the model is that the vector of sample observations on Ymay be expressed as a linear combination of the sample observation as the explanatory X variables plus a disturbance vector.
The assumptions about u are as follows: (1) For an arbitrary period t E u t ð Þ ¼ 0 (1) Variance-Covariance matrix of u is defined as: Equation (7) gives a GnxGn symmetric matrix whose elements are given by: where, (a) implies existence of homoscedasticity (b) implies absence of autocorrelation (c) implies that covariances of the same periods are equal to σ it � : This means that contemporaneous covariances in different equations are independent of time.
(d) Implies that covariances at different times in different equations or non-contemporaneous covariances are zero.
(1) X matrix is of full column rank i.e. ρ X ð Þ ¼ K (2) X is a non-stochastic matrix.
where ¼ À β À 0 Γ is the matrix of reduced form coefficients, and, V t ¼ β À 0 u t is the residual vector of the reduced form equations with ; Ω ð Þ These two sets of parameters are related as follows: The matrix is of order GxK and thus contains GK elements. The β and Γ matrices contain at most G 2 + GK elements.
The MCS framework presented below was adopted to assemble a data set that is conformable to the assumption specified for the model and the imposed levels multicollinearity: (i) We set sample sizes T at 30: (ii) The following numerical values are arbitrarily assigned to each of the structural parameters of the model. (i) Values of the disturbance terms U 1t and U 2t are specified for each sample points. We employed a two-stage process in generating values of the random disturbance terms from a Normal (0, 1) distribution. The two stages involved here are described below.
(a) Independent series εε t of random normal deviates Ñ 0; 1 ð Þwere drawn from a pool of random normal deviates. The length of each pair is determined by the sample size designed.
(b) The generated series were transformed into a series of random disturbances to guarantee conformity with the variance-covariance matrix Ω predetermined for the model. The method presented by Nagar (1960) for transformation of independent series of standard random deviates into series of random deviates with zero mean and a specified variancecovariance matrix was used for this purpose. This is described below.
The disturbance terms were generated using the Nagar (1960) approach. Since Ʃ is a positive definite matrix, it can be decomposed by a non-singular upper triangular matrix P such that Then S 11¼þ ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi σ 11 À S 2 21 À � q The two random disturbance series are thus formed using (vi) The endogenous variables were generated from the values already obtained for the X 0 s and U 0 s and the values assigned to the structural parameters. This is most conveniently done using the reduced form of the model.
Given the two-equation model Rearranging the model, we have, This can be written as Where, Rewriting Equation (15), we have Equation (17)

Results and discussion
By the Order and Rank conditions of identification, the two equations in the model are exactly identified. This makes the prospect of unique estimates of the parameters realizable. We employed 2 multicollinearity scenarios in studying the relative performance of the estimators. These are: (i) Low multicollinearity-Where correlation among the exogenous variables are insignificant at 5% level.
(ii) High multicollinearity-Where correlation among the exogenous variables are significant at 1% level.
To empirically utilize the simulation framework, a finite data set of sample size T = 30 over 100 replications (R = 100) was simulated to facilitate an exploration of the two multicollinearity scenarios highlighted above. Table 1 highlights averages of the parameter estimates generated for the 100 replications considered for the chosen sample size. The true values of the parameters are presented in parenthesis. Best estimators are those whose average estimates are closest to the true parameter value. For parameter β 12 whose true value is 1.8, the best estimates are obtained from ILS for both low and high multicollinearity. Similarly, for parameter β 21 whose true value is 0.4, the best estimates are obtained from ILS for both low and high multicollinearity.
As confirmed by Johnston (1991), when an equation is just identified, estimates of parameters obtained by 2SLS, LIML, ILS and 3SLS should be identical. The results obtained in this study show that 2SLS, LIML and 3SLS estimators yielded virtually identical estimates as can be seen from Table  1. The other three estimators: OLS, ILS and FIML yielded results that are clearly different from those of 2SLS, LIML and 3SLS. Consequently, we classified our original six estimators into 4 categories for the purpose if this analysis. These categories are: OLS-Ordinary Least Squares estimator; L23-Limited Information Maximum Likelihood estimator, Two Stage Least Squares estimator and Three Stage Least Squares estimator; ILS-Indirect Least Squares estimator; and FIML-Full Information Maximum Likelihood estimator. Table 2 presents further evaluation of the parameter estimates obtained through our Monte Carlo Experiment. Root Mean Square Error of Estimates (RMSE) was used to evaluate the relative performance of our estimators in the presence of multicollinearity as stated above. RMSE criterion takes into consideration the bias as well as the variance of estimates in evaluating the performance of the estimators.
For parameterβ 12 , in the case of low multicollinearity, ILS performed best with the lowest RMSE of 0.4347 followed by OLS with the RMSE of 1.2881. Also, for high multicollinearity, ILS performed best with the lowest RMSE of 1.2087 followed by OLS with the RMSE of 1.2796. Similar result is obtained for parameterβ 21 . For parameter γ 11 ; in the case of low multicollinearity, OLS performed best with the lowest RMSE of 2.7177. Also for high multicollinearity, OLS performed best with the lowest RMSE of 4.2254. Similar result like that of γ 11 was obtained for parameters γ 12 andγ 22 . For parameterγ 23 , and the case of low multicollinearity, ILS performed best with the lowest RMSE of 7.3309. Also, for high multicollinearity, ILS performed best with the lowest RMSE of 8.1863. In summary, ILS followed by OLS yielded best estimates for β 12 and β 21 which are the parameters of the endogenous variables. On the contrary for the parameters of the exogenous variables, OLS is best forγ 11 , γ 12 and γ 22 while ILS is best forγ 23 . These observations are surprisingly the same for both low and high measures of multicollinearity.

Conclusion and recommendations
We set up a Monte Carlo Experiment (MCE) to evaluate the relative performance of various estimators of a simultaneous equation model in the presence of varied levels of multicollinearity. Our model was estimated with a simulated data set of sample size 30 over 100 replications. The parameter estimates obtained from the six estimators considered were evaluated using Root Means Squares of Estimates (RMSE) criteria. Our result revealed that irrespective of the level of multicollinearity in our model, ILS followed by OLS yielded best estimates for β 12 and β 21 which are the parameters of the endogenous variables. On the contrary for the parameters of the exogenous variables, OLS is best forγ 11 , γ 12 and γ 22 while ILS is best forγ 23 . However, the system estimators all performed poorly. Also 2SLS, LIML and 3SLS estimators yielded virtually identical estimates. By our findings, in the presence of multicollinearity, estimators OLS and ILS performed best and should therefore be preferred above the multi-equation estimators.