Skip to content

Fit binomial and Poisson multilevel submodels to non-integer data with lme4 < 2.1-1, and require lme4 >= 2.0-1 - #655

Merged
avehtari merged 48 commits into
masterfrom
lme4-nonint-family
Nov 19, 2026
Merged

avehtari merged 48 commits into
masterfrom
lme4-nonint-family

Conversation

@avehtari

@avehtari avehtari commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

lme4 < 2.1-1 (all CRAN versions) computes the glmer() objective of the binomial and poisson families from the integer density. lme4 rounds non-integer binomial data. For non-integer Poisson responses, the objective is infinite, so the group-level standard deviations stay at 1. The projection gives such data to the submodel fitters. lme4/lme4#1012 fixes this in lme4 2.1-1.

For other family names, lme4 calls the R function aic of the family. So with lme4 < 2.1-1, fit_glmer_callback() and fit_gamm_callback() now pass a copy of binomial() or poisson() named binomial_projpred or poisson_projpred. Its aic uses lgamma(), as lme4 2.1-1 does. For integer data, it gives the same value as the standard aic. Side effects: with lme4 2.0-x, the fit estimates a sigma() that it does not use. projpred converts the binomial factor response of the augmented-data projection to 0/1. simulate() does not work for these submodel fits.

With the family copy, lme4 1.1-x multiplies VarCorr() by sigma(), so this PR requires lme4 >= 2.0-1. It removes the nAGQ = 0 workaround for rounded binomial data (#616). It also replaces #642, which added a gamm4() stage-1 optimizer for the same problem.

Also:

  • init_refmodel() no longer warns that the projection of multilevel Poisson models may take very long. With this PR, a Poisson glmer() fit takes 0.1 to 0.4 s instead of 1.9 to 2.1 s (see "Effect").
  • The fix does not remove the problem of Optional submodel fitter MASS::glmmPQL() #353 for binomial models. With the correct objective, the Optional submodel fitter MASS::glmmPQL() #353 Bernoulli demo still gives the group-level SD 0. The search does not select (1 | grpGL) in the first 19 terms. The cause is the marginal likelihood. It assumes binomial noise around the reference predictions, which have no noise, so with little information per group the SD estimate is 0. The augmented-data projection (also of the cumulative and categorical families) has the same problem. With the observation weights of the submodel fit multiplied by 100 (only as a probe), the SD is 0.27. The group-level term then has a larger KL gain than the noise predictor that the search selects. So init_refmodel() now gives one warning for multilevel models with a binomial response or with the augmented-data projection. The warning gives this cause and the latent projection as a possible remedy (not for brms::categorical()). The warning does not apply to datafit objects.
  • ?projpred-package says that mgcv::gam() with method = "REML" or "ML" stops with an error for non-integer Poisson responses.

Effect

Poisson glmer(y ~ x + (1 | g)), 30 groups of 20 observations, response = true mean. Group-level SD from VarCorr():

true SD lme4 2.0-6 / 2.1-0 this PR, lme4 2.0-6 / 2.1-0 lme4 2.1-1
0.3 1.0000 0.2443 0.2443
1 1.0000 0.9465 0.9465
2 1.0000 1.9112 / 1.9113 1.9113

gamm4() fits show the same pattern: SD 1 before, the lme4 2.1-1 value with this PR. Time per glmer() fit: 0.1 to 0.4 s with this PR, 1.9 to 2.1 s before.

Tests

New tests in test_submodel_fitters.R and test_refmodel.R. Tests that needed lme4 2.1-1 now run with all lme4 versions, and test_augdat.R again compares the traditional and augmented-data projections with group-level terms. On master, the new and changed tests give 20 failures and 4 errors (lme4 2.0-6) and 26 failures and 4 errors (lme4 2.1-0). On this branch, the full suite with NOT_CRAN=true passes with lme4 2.0-6, 2.1-0 and 2.1-1 (R 4.6.1).

Commits a04b654, 39eef29 and 3e0370c (after the PR post) change the warning. Its test in test_refmodel.R expects the warning for the binomial traditional, binomial augmented-data and cumulative augmented-data projections. It expects no warning for the Poisson, latent, no-group-term and cvrefbuilder cases. With the R/refmodel.R of f2c8850, the test gives 3 failures. The full suite on 39eef29 with NOT_CRAN=true and lme4 2.0-6 (R 4.6.1, testthat 3.3.2) passes: 61,376 expectations, 0 failures. None of the 20 warnings comes from the changed code. 3e0370c changes only the wording of the warning, a comment and NEWS.md; test_refmodel.R passes on it. The change does not depend on the lme4 version.

This PR was assisted by Claude.

avehtari and others added 30 commits November 6, 2026 18:07
…t roxygen2 documents the projection methods as S3 methods
# Conflicts:
#	NAMESPACE
#	NEWS.md
#	R/divergence_minimizers.R
#	R/projpred-package.R
…, and test the optimizer of the optimx refit
@avehtari avehtari added the bug Bugs. label Nov 17, 2026
@avehtari
avehtari merged commit b25cea2 into master Nov 19, 2026
4 of 6 checks passed
@avehtari
avehtari deleted the lme4-nonint-family branch November 19, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Bugs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants