Ecological Archives E085-119-A1

Lorenzo Ciannelli, Kung-Sik Chan, Kevin M. Bailey, and Nils Chr. Stenseth. 2004. Nonadditive effects of the environment on the survival of a large marine fish population. Ecology 85:3418–3427.

Appendix A. A description of the method of penalized least squares..

The method of penalized least squares balances the goodness of model fit with the smoothness of the function estimators. For simplicity, assume that f2 vanishes and there is only one g so that there are two unknown functions denoted as f and g. Model identification requires that all functions be centered, that is, the sample mean of each function is zero over the data points. Formally, the model is estimated by minimizing where RSS is the residual sum of squares and the roughness of a function is measured by the integral of its squared  second derivative; the smoothness coefficients are positive numbers that quantify the trade-off of badness of fit with the roughness of the functions. For fixed smoothness coefficients, the minimization admits a closed-form solution with the estimators of f and g being natural cubic splines, i.e., piecewise cubic polynomials that are linear at the two extremes and twice continuously differentiable (Wahba 1990, Green and Silverman 1994). In particular, for zero smoothness coefficients, the function estimators are generally extremely wiggly as the functions try to follow every details of the data pattern including their random aspects. On the other hand, with infinite smoothness coefficients, the functions become straight lines. Thus, the determination of the smoothness coefficients is pivotal. Two popular approaches to determining the are the methods of cross validation (CV) and generalized cross validation (GCV) (Wahba 1990, Green and Silverman 1994, Wood 2000). The method of CV determines the by minimizing the predictive residual sum of squares; the predictive residual of a specific data case is defined as the observed response minus the predicted value based on the model fitted to all data except the data case under prediction. The method of CV then, in principle, requires repeating the model estimation as many times as the number of data cases, and hence is generally computationally intensive. For "linear"estimators, the predictive residuals can be related to the residuals from the overall model fitted to all data, in which case the method of CV can be done efficiently. (This is the case for the TGAM when the threshold line is known). In fact, in such cases, the predictive residuals can be further approximated using an appealing scheme, resulting in the method of generalized cross validation (GCV; see Wahba 1990, Green and Silverman 1994). Notice that within a parametric framework and under some regularity conditions the method of CV is asymptotically equivalent to the Akaike Information Criterion (AIC; Stone 1977, Tong 1990). However, the methods of CV and GCV are applicable in both parametric and nonparametric settings. Indeed, the method of CV can be shown to be consistent under very general conditions (Wahba 1990, Chan and Tong 2001).



Chan, K. S., and H. Tong. 2001. Chaos: a statistical perspective. Springer Verlag, New York, New York, USA.

Green, P. J., and B. W. Silverman. 1994. Nonparametric regression and generalized linear models: a roughness penalty approach. Chapman and Hall, New York New York, USA.

Stone, M. 1977. An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society, Series B, Methodological 39:44–47.

Tong, H. 1990. Non-linear time series: a dynamical system approach. Oxford University Press, Oxford, UK.

Wahba, G. 1990. Spline models for observational data (SIAM).

Wood, S. J. R. 2000. Modelling and smoothing parameter estimation with multiple quadratic penalties. Journal of the Royal Statistical Society B 62(2):413–428.

[Back to E085-119]