[S] Suggested minor alteration to nlminb.control--UPDATE

Terry Elrod (Terry.Elrod@UAlberta.ca)
Fri, 8 May 1998 08:03:15 -0600


My suggested fix to nlminb.control has no effect on the running of nlminb. A detailed examination of the nlminb code shows that the value of diff.g passed to nlminb from nlminb.control is not (no longer?) used by nlminb.

An alternative that does work is to take advantage of nlminb's provision for a user-defined function to calculate the gradient. This function may use a numerical approximation with the size of the increment under the user's control. This is less convenient, and user-defined functions for numerical approximations are likely to be less sophisticated than nlminb's. (For example, the latter's will shift automatically to central differences when the error in the gradient becomes too high.) I hope MathSoft will restore to nlminb its provision for setting the size increment.

Terry Elrod
--------
Terry Elrod; Assoc. Prof. of Marketing; 3-23 Fac. of Business; Univ. of Alberta;
Edmonton AB; Canada T6G 2R6
email: Terry.Elrod@Ualberta.ca; tel: (403) 492-5884; fax: (403) 492-3325
Web page: http://www.ualberta.ca/~telrod/
--------

----ORIGINAL MESSAGE----

THE ERROR:

In S-Plus 4.0, release 3, the help file for nlminb.control states that a parameter diff.g may be set by the user, but setting this parameter within a call to nlminb generates the error: "Error in call to nlminb.control(): Argument diff.g= not matched." However, examination of the code for nlminb reveals that it does check to see if diff.g has been set by the user through a call to nlminb.control.

WHO MIGHT CARE:

diff.g allows the user to set the size of increment to a parameter that is used in numerical approximations to the gradient. That is, the gradient for a parameter p is approximated by the calculation: ( f(p + diff.g) - f(p) ) / diff.g. The default value for diff.g is .Machine$double.eps^(1/2), which works out to 1.5e-008 on my machine.

Users inexperienced with optimization algorithms such as nlminb should leave diff.g alone, but being able to alter its default value can be helpful on occasion. For some problems, the default is too small. For example, I am writing a procedure to fit a particular form of generalized nonlinear model which, however, contains only a few parameters that cannot be estimated by glm. I solve the problem by using a principle known as "concentrating the likelihood"; i.e., by having nlminb find estimates for only the nonlinear parameters, and nest calls to glm within the function definition for nlminb. Thus what would otherwise be a 10 parameter problem for nlminb is reduced to a 3 parameter problem when only 3 of the 10 parameters cannot be estimated by glm. This principle of concentrating the likelihood leads to much more stable estimation. However, it also can, as in this example, lead to an increase the numerical error in assessments of the loglikelihood function within nlminb. Henc!
!
e the need sometimes to increase the size of diff.g. Otherwise the signal-to-noise ratio in nlminb's approximation of the gradient can be too low.

diff.g only affects how numerical gradients are calculated. If you supply an analytical gradient to nlminb, then diff.g is irrelevant.

THE FIX:

A simple alteration to nlminb.control returns to users the ability to set the diff.g parameter. The current function definition is:

nlminb.control <-
function(eval.max = NULL, iter.max = NULL, abs.tol = NULL, rel.tol = NULL, x.tol = NULL,
step.min = NULL, step.max = NULL, scale.init = NULL, scale.upd = NULL, scale.fac = NULL,
scale.tol = NULL, scale.mod = NULL, sing.tol = NULL, sing.step = NULL, rel.err = NULL)
list(eval.max = eval.max, iter.max = iter.max, abs.tol = abs.tol, rel.tol = rel.tol, x.tol =
x.tol, step.min = step.min, step.max = step.max, scale.init = scale.init, scale.upd =
scale.upd, scale.fac = scale.fac, scale.tol = scale.tol, scale.mod = scale.mod,
sing.tol = sing.tol, sing.step = sing.step, rel.err = rel.err)

The simple change is to:

nlminb.control<-
function(eval.max = NULL, iter.max = NULL, abs.tol = NULL, rel.tol = NULL,
x.tol = NULL, step.min = NULL, step.max = NULL, scale.init = NULL,
scale.upd = NULL, scale.fac = NULL, scale.tol = NULL, scale.mod = NULL,
sing.tol = NULL, sing.step = NULL, rel.err = NULL, diff.g = NULL)
list(eval.max = eval.max, iter.max = iter.max, abs.tol = abs.tol, rel.tol =
rel.tol, x.tol = x.tol, step.min = step.min, step.max = step.max,
scale.init = scale.init, scale.upd = scale.upd, scale.fac = scale.fac,
scale.tol = scale.tol, scale.mod = scale.mod, sing.tol = sing.tol,
sing.step = sing.step, rel.err = rel.err, diff.g = diff.g)

The only alterations are the addition of the argument diff.g = NULL to the argument list, and the addition of diff.g = diff.g at the end of the single expression in the function body.

I follow the recommendation of MathSoft of not actually altering S-Plus source code, but instead place my altered nlminb.control function into a personal library ("myfuncs") which is added to the search list ahead of the S-Plus directories by the expression: library(myfuncs, T).

In my call to nlminb.control I also SLIGHTLY increase the values for abs.tol, rel.tol, x.tol as well as for diff.g. These additional changes slightly relax the convergence criteria for nlminb, which may also be called for when, as here, there is more than the usual amount of error in the calculation of the loglikelihood function.

WARNING:

Although nlminb.control allows relaxation of the convergence criteria for nlminb, which may sometimes be called for, it is too easy to fall into the trap of relaxing these criteria too casually and by too much.

Terry Elrod
--------
Terry Elrod; Assoc. Prof. of Marketing; 3-23 Fac. of Business; Univ. of Alberta;
Edmonton AB; Canada T6G 2R6
email: Terry.Elrod@Ualberta.ca; tel: (403) 492-5884; fax: (403) 492-3325
Web page: http://www.ualberta.ca/~telrod/
--------

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news