Repository navigation
Fit binomial and Poisson multilevel submodels to non-integer data with lme4 < 2.1-1, and require lme4 >= 2.0-1 - #655
Merged
Merged
Conversation
# Conflicts: # NEWS.md
…and unix from the dependencies
… base R operator %||%
…t roxygen2 documents the projection methods as S3 methods
…ger data and lme4 < 2.1-1
…d of hardcode it" This reverts commit 4e1c9ef.
# Conflicts: # NAMESPACE # NEWS.md # R/divergence_minimizers.R # R/projpred-package.R
…, and test the optimizer of the optimx refit
# Conflicts: # NEWS.md # R/divergence_minimizers.R
…document the GAM REML limit
…e cause of the problem
… the cvrefbuilder case
avehtari
force-pushed
the
lme4-nonint-family
branch
from
November 18, 2026 11:03
70249e0 to
3e0370c
Compare
aloctavodia
approved these changes
Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lme4 < 2.1-1 (all CRAN versions) computes the
glmer()objective of thebinomialandpoissonfamilies from the integer density. lme4 rounds non-integer binomial data. For non-integer Poisson responses, the objective is infinite, so the group-level standard deviations stay at 1. The projection gives such data to the submodel fitters. lme4/lme4#1012 fixes this in lme4 2.1-1.For other family names, lme4 calls the R function
aicof the family. So with lme4 < 2.1-1,fit_glmer_callback()andfit_gamm_callback()now pass a copy ofbinomial()orpoisson()namedbinomial_projpredorpoisson_projpred. Itsaicuseslgamma(), as lme4 2.1-1 does. For integer data, it gives the same value as the standardaic. Side effects: with lme4 2.0-x, the fit estimates asigma()that it does not use. projpred converts the binomial factor response of the augmented-data projection to 0/1.simulate()does not work for these submodel fits.With the family copy, lme4 1.1-x multiplies
VarCorr()bysigma(), so this PR requires lme4 >= 2.0-1. It removes thenAGQ = 0workaround for rounded binomial data (#616). It also replaces #642, which added agamm4()stage-1 optimizer for the same problem.Also:
init_refmodel()no longer warns that the projection of multilevel Poisson models may take very long. With this PR, a Poissonglmer()fit takes 0.1 to 0.4 s instead of 1.9 to 2.1 s (see "Effect").MASS::glmmPQL()#353 for binomial models. With the correct objective, the Optional submodel fitterMASS::glmmPQL()#353 Bernoulli demo still gives the group-level SD 0. The search does not select(1 | grpGL)in the first 19 terms. The cause is the marginal likelihood. It assumes binomial noise around the reference predictions, which have no noise, so with little information per group the SD estimate is 0. The augmented-data projection (also of the cumulative and categorical families) has the same problem. With the observation weights of the submodel fit multiplied by 100 (only as a probe), the SD is 0.27. The group-level term then has a larger KL gain than the noise predictor that the search selects. Soinit_refmodel()now gives one warning for multilevel models with a binomial response or with the augmented-data projection. The warning gives this cause and the latent projection as a possible remedy (not forbrms::categorical()). The warning does not apply todatafitobjects.?projpred-packagesays thatmgcv::gam()withmethod = "REML"or"ML"stops with an error for non-integer Poisson responses.Effect
Poisson
glmer(y ~ x + (1 | g)), 30 groups of 20 observations, response = true mean. Group-level SD fromVarCorr():gamm4()fits show the same pattern: SD 1 before, the lme4 2.1-1 value with this PR. Time perglmer()fit: 0.1 to 0.4 s with this PR, 1.9 to 2.1 s before.Tests
New tests in
test_submodel_fitters.Randtest_refmodel.R. Tests that needed lme4 2.1-1 now run with all lme4 versions, andtest_augdat.Ragain compares the traditional and augmented-data projections with group-level terms. On master, the new and changed tests give 20 failures and 4 errors (lme4 2.0-6) and 26 failures and 4 errors (lme4 2.1-0). On this branch, the full suite withNOT_CRAN=truepasses with lme4 2.0-6, 2.1-0 and 2.1-1 (R 4.6.1).Commits a04b654, 39eef29 and 3e0370c (after the PR post) change the warning. Its test in
test_refmodel.Rexpects the warning for the binomial traditional, binomial augmented-data and cumulative augmented-data projections. It expects no warning for the Poisson, latent, no-group-term and cvrefbuilder cases. With theR/refmodel.Rof f2c8850, the test gives 3 failures. The full suite on 39eef29 withNOT_CRAN=trueand lme4 2.0-6 (R 4.6.1, testthat 3.3.2) passes: 61,376 expectations, 0 failures. None of the 20 warnings comes from the changed code. 3e0370c changes only the wording of the warning, a comment andNEWS.md;test_refmodel.Rpasses on it. The change does not depend on the lme4 version.This PR was assisted by Claude.