Skip to content

Check the gradient of gamm4() submodel fits at the optimum - #665

Open
avehtari wants to merge 8 commits into
masterfrom
gamm-conv-check2
Open

avehtari wants to merge 8 commits into
masterfrom
gamm-conv-check2

Conversation

@avehtari

Copy link
Copy Markdown
Member

This PR is part of #479. #630 covers additive submodels without group-level terms.

Summary

gamm4::gamm4() fits the lme4 part of a submodel with smooth and group-level terms, but it does not call lme4::checkConv(). So projpred's convergence check of these submodels uses only the convergence code and the warnings of the optimizer. This PR adds gamm_check_conv(), which applies the gradient rule of lme4::checkConv() after gamm4(), with the tolerance of check.conv.grad. A fit that fails the rule gives the lme4 warning, and the projection warns that the submodel fits might not have converged.

The new check does not test the Hessian, and it uses the unscaled gradient if it cannot compute the scaled gradient. With responses without noise, as in the projection, the finite-difference Hessian of large smoothing parameters can be not positive definite although the fit converged. In a study of 240 gamm4() fits to such responses, the full lme4::checkConv() gave a nonzero code for 17 of 80 Gaussian fits. The new check gave none for these 80 fits. A code comment lists the other differences from lme4::checkConv().

Tests

New tests in test_submodel_fitters.R compare the new check with lme4::checkConv() in glmer() and lmer(). They cover the special cases (Hessian not positive definite, NA gradient, fit at a boundary, actions "ignore" and "stop") and the projection warning. They also check that converged fits give no warning. The fits use Gaussian data without noise, non-integer binomial responses (as the existing Poisson test does), and the smooth term of a linear function. A prediction test in test_proj_predfun.R now muffles the new warning. Without the change, the new tests give 1 failure and 2 errors. The full suite passes (R 4.6.1, lme4 2.0.6, gamm4 0.3.0) with the same warnings as master.

This PR was assisted by Claude.

@avehtari avehtari added the bug Bugs. label Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Bugs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant