Appendix C. A description of the conditional Akaike information criterion (cAIC).
In our analyses, we define one set of groups based on system in order to control for structured non-independence, while other groups (e.g.,. resource trophic level, ecosystem type, consumer response mechanism) are investigated as explanatory factors of interest. For the second set of groups, we use multilevel parameter estimates to investigate how explanatory factors affect the response variable. Vaida and Blanchard (2005) have argued that when random effects models are used in this way to make inferences about specific groups, the Akaike information criterion is not the appropriate metric for model comparison. Instead, they propose a conditional Akaike information criterion (cAIC).
The difference between AIC and cAIC follows from the appropriate scope of inference. The AIC describes the ability of the model to predict new data, drawn from new groups which are themselves drawn from the modeled distribution. The cAIC describes the ability of the model to predict new data drawn from the same groups as the original data. In the former case, the focus is on the population from which the groups are drawn, while in the latter case the focus is on the particular groups in the analysis. The difference in focus requires different likelihood terms and different degrees of freedom (Vaida and Blanchard 2005). The formula for AIC and cAIC can be written as:
Where the marginal likelihood is that typically returned by model-fitting functions, and K is the number of fixed parameters and variance components. The conditional likelihood used for cAIC is the likelihood of the data conditional on the fitted random effects estimates. The value gives the “effective degrees of freedom” for a mixed model, as described in Hodges and Sargent (2001). The differences between these formulae derive from the differences in focus described above. The marginal likelihood integrates over all possible random effects estimates, while the conditional likelihood is conditional on the particular random effects that best fit the data. Likewise, K counts only the population-level parameters as degrees of freedom, while estimates degrees of freedom directly from the amount of structure in the data. Further explanation of the derivation of these quantities can be found in Vaida and Blanchard (2005).
We present cAIC results as primary because our intent in these analyses is to explore the explanatory power of different factors. For example, we would like to know how response magnitude varies by resource trophic level, and we are therefore making an inference specifically about each trophic level. However we also present AIC results for comparison because the grouping of the data by system is consistent with a population-level focus, i.e. we would like to control for the effect of system but we are not making any inferences about particular systems. Results using cAIC and AIC were qualitatively similar, and we calculated both criteria using a finite-sample correction term. The formulae with finite-sample correction are:
Hodges, J. S. and D. J. Sargent. 2001.Counting degrees of freedom in hierarchical and other richly-parameterised models. Biometrika 88:367–379.
Vaida, F. and S. Blanchard. 2005. Conditional Akaike information for mixed-effects models. Biometrika 92:351–370.