Guided tour on Nonlinear General Method of Moments

Simultaneous Probit models

In this guided tour I will demonstrate how to estimate a nonlinear simultaneous equation system by user-defined nonlinear GMM, on the basis of the example of the following system of simultaneous Probit models.


with I() the indicator function: I(true) = 1, I(false) = 0, and


Only y1 and y2 are observed as dependent variables. The errors u1 and u2 are independent of the explanatory variables x1 and x2.

It is explained in detail in NLGMM.PDF that this model gives rise to the following conditional moment functions:




The conditional expectations of these three moment functions, given x1 and x2, are zero.

Assuming that the vector x2 does contain a constant 1, we may use z = (x1', x2')' as the vector of instrumental variables for each of these moment functions. In particular, the parameters are identified by the unconditional moment restrictions


I strongly recommend that you read my lecture notes on the Method of Moments first before you use the nonlinear GMM module!

Specification of the moment functions in EasyReg

Similar to nonlinear regression and user-defined maximum likelihood, we have to build up the conditional moment functions recursively, starting from the initial selection of model variables. See NLTRANS.PDF for a list of all available transformations.

Suppose we have four variables: the dummy variables Y1 and Y2, and the variables X1 and X2. Let in model (1), y1 = Y1, x1 = X1, and in model (2), y2 = Y2, x2 = (1,X2)'.

In this example I will use artificial data, with sample size 500. The X1 and X2 have been drawn independently from the standard normal distribution. The Y1's and Y2's have been generated independently such that P[Y1=1|X1] = Probit(X1) and P[Y2=1|X1] = Probit(X2). Thus, the actual parameter values are:

The data file involved is included in this tour, as NLGMMDATA.TXT, in EasyReg space delimited data format, and should be declared cross-section data.

After selecting Y1, X1, Y2, X2 (in this order) as the model variables, including instrumental variables, the following window appears.

EasyReg Window

EasyReg adds automatically the constant 1 to the list.

As explained in NLGMM.PDF, the moment functions are build up recursively as follows.



The programming procedure is similar to nonlinear least squares, and are therefore not repeated here.

EasyReg Window

Once you are done with the specification of the conditional moment functions, click "Select the moments". Then the following window appears.

EasyReg Window

The last three X variables are the conditional moment functions. The last X variable should be a conditional moment function, but other conditional moment functions may appear anywhere in this list.

Double click the X variables corresponding to the conditional moment functions, and click "Selection OK". Then the following window appears.

EasyReg Window

You have to choose a set of instrumental variables for each of the moment functions. The instrumental variables may be different for different moment functions. However, in the case under review I recommend to choose a common set of instrumental variables, namely X1, X2 and 1.

Double click the instrumental variables X1, X2 and 1, and click "Selection OK". Then the window changes to:

EasyReg Window

Double click the instrumental variables X1, X2 and 1 again, and click "Selection OK". Then the window changes to:

EasyReg Window

Double click the instrumental variables X1, X2 and 1. We are now done with the specification of the instrumental variables. Click "Selection OK". Then the following window appears.

EasyReg Window

This is the complete nonlinear GMM model. EasyReg has appended the list of X variables with nine unconditional moment functions. You may wish to store this model in a template file. Since I have already stored another nonlinear GMM model in template file TEMPLATE_NLGMM.001, the currrent model will be stored in template file TEMPLATE_NLGMM.002 if the "Store" option is chosen.

Click "Model is OK".

EasyReg Window

You have to specify the lower and upper bounds of the parameters. I have chosen -10 and 10, except for b(5), which is the correlation coefficient of the model errors and is therefore bounded between -1 and 1. Note that the values -1 and 1 would imply an exact relation between the errors in (1) and (2):

u2 = -u1 if b(5) = -1,
u2 = u1 if b(5) = 1.

If so, X(12) will be zero, and consequently the transformations X(14)=X(13)/X(12), X(15)=X(6)/X(12) and X(16)=X(9)/X(12) will generate an "overflow" error. Therefore the lower and upper bounds of b(5) have been chosen -0.999 and 0.999.

Click "Bounds OK". Then the following window appears.

EasyReg Window

Initially the start values of the parameters are drawn from the uniform distribution on the hypercube formed by the lower and upper bounds. In this case, however, we can do better, because model (2) is a standard Probit model, and can easily be estimated by maximum likelihood via EasyReg module LIMDEP (via "Menu > Single equation models > Discrete dependent variable models"). The random start values of b(2) and b(3) have therefore been replaced with the corresponding maximum likelihood estimates b(2) = -0.0217871, b(3) = 0.9702533:

EasyReg Window

Click "Start". After the first iteration round I have restarted the simplex iteration a few times from the last parameter values. It took about 15 minutes to complete these iteration rounds.

EasyReg Window

Click "Done".

EasyReg Window

Click "Continue". EasyReg will then compute the optimal weight matrix, W, (see my lecture notes on the Method of Moments), and jump back to the previous window:

EasyReg Window

You have to re-estimate the parameters to get the efficient GMM estimators. Since the start values are already consistent estimators, the new iteration round will be faster than the previous one.

EasyReg Window

This is the final iteration result. Click "Done". Then the following window re-appears.

EasyReg Window

Click "Continue". Then EasyReg will calculate the variance matrix of the parameter estimates, and generate the output. This may take a minute. Just wait. Once EasyReg is done, the output appears.

EasyReg Window

The GMM estimation results are pretty close to the true values b(1) = 1, b(2) = 0, b(3) = 1, b(4) = 0, b(5) = 0. In particular, the parameters with true value zero are all insignificant.

If the GMM model is over-identified (as in this case) the GMM objective function times the sample size is a chi-square test statistic for testing the null hypothesis that the unconditional moment restrictions hold. The degrees of freedom are the number of unconditional moment restrictions minus the number of parameters: 9 - 5 = 4 in this case. As expected, this null hypothesis is not rejected.

EasyReg Window

Finally, I have tested the null hypothesis

using the option "Wald test of linear parameter restrictions". As expected, the null hypothesis is not rejected at any reasonable significance level.

EasyReg Window

Remark: If you use this nonlinear GMM module for time series, and you need lagged variables, you have to create them in advance via "Menu > Input > Transform variables".


Nonlinear GMM
GMM model variables, including instrumental variables:
X(1) = Y1
X(2) = X1
X(3) = Y2
X(4) = X2
X(5) = 1

First available observation: t = 1 
Last available observation: t = 500 
= chosen

NLGMM model:
Model variables and moment restrictions
Conditional moment restriction 1:
E[X(24)|b,Z] = 0, 
with instrumental variables Z:

Corresponding unconditional moment restrictions:
E[X(27)] = 0, where X(27)=X(24)*X(2)
E[X(28)] = 0, where X(28)=X(24)*X(4)
E[X(29)] = 0, where X(29)=X(24)*X(5)
Conditional moment restriction 2:
E[X(25)|b,Z] = 0, 
with instrumental variables Z:

Corresponding unconditional moment restrictions:
E[X(30)] = 0, where X(30)=X(25)*X(2)
E[X(31)] = 0, where X(31)=X(25)*X(4)
E[X(32)] = 0, where X(32)=X(25)*X(5)
Conditional moment restriction 3:
E[X(26)|b,Z] = 0, 
with instrumental variables Z:

Corresponding unconditional moment restrictions:
E[X(33)] = 0, where X(33)=X(26)*X(2)
E[X(34)] = 0, where X(34)=X(26)*X(4)
E[X(35)] = 0, where X(35)=X(26)*X(5)

-10 <= b(1) <= 10
-10 <= b(2) <= 10
-10 <= b(3) <= 10
-10 <= b(4) <= 10
-.999 <= b(5) <= .999

The GMM objective function has been minimized using the simplex method of
Nelder and Mead. The algorithm involved is a Visual Basic translation of
the Fortran algorithm involved in:
W.H.Press, B.P.Flannery, S.A.Teukolsky and W.T.Vetterling, 'Numerical
Recipes', Cambridge University Press, 1986, pp. 292-293

Estimation results:
Parameters GMM estimate   t-value
b(1)           0.949129    11.967
b(2)          -0.023036    -0.364
b(3)           0.956066    11.617
b(4)          -0.031779    -0.269
b(5)          -0.149770    -1.307

[The two-sided p-values are based on the normal approximation]

GMM objective function:         0.004168
n:                                   500

If the moment restrictions hold, the value of the GMM objective function,
times the sample size n, will be asymptotically Chi-square(4) distributed:
Test statistic = 2.084092
p-value = 0.72030
Significance levels:        10%         5%
Critical values:           7.78       9.49
Conclusions:             accept     accept

Wald test:

b(1)           0.949129    11.967(*)
b(2)          -0.023036    -0.364(*)
b(3)           0.956066    11.617(*)
b(4)          -0.031779    -0.269(*)
b(5)          -0.149770    -1.307(*)
(*): Parameters to be tested

Null hypothesis:
1.b(1)+0.b(2)+0.b(3)+0.b(4)+0.b(5) = 1.
0.b(1)+1.b(2)+0.b(3)+0.b(4)+0.b(5) = 0.
0.b(1)+0.b(2)+1.b(3)+0.b(4)+0.b(5) = 1.
0.b(1)+0.b(2)+0.b(3)+1.b(4)+0.b(5) = 0.
0.b(1)+0.b(2)+0.b(3)+0.b(4)+1.b(5) = 0.

Null hypothesis in matrix form: Rb = c, where
R =
 1. 0. 0. 0. 0.
 0. 1. 0. 0. 0.
 0. 0. 1. 0. 0.
 0. 0. 0. 1. 0.
 0. 0. 0. 0. 1.
and c =
Wald test statistic:                    4.07
Asymptotic null distribution:  Chi-square(5)
  p-value = 0.53868
  Significance levels:        10%         5%
  Critical values:           9.24      11.07
  Conclusions:             accept     accept

This is the end of the guided tour on nonlinear General Method of Moments