Skip to content

Fix LOO validated search if the PSIS weights of all draws of a cluster underflow to 0 - #666

Open
avehtari wants to merge 7 commits into
masterfrom
loo-zero-weight-clusters
Open

avehtari wants to merge 7 commits into
masterfrom
loo-zero-weight-clusters

Conversation

@avehtari

Copy link
Copy Markdown
Member

Summary

In cv_varsel() with cv_method = "LOO" and validate_search = TRUE, get_p_clust() reweights the reference-model draws with the PSIS-LOO weights of the left-out observation. If all draws of a cluster have weight 0, for example after underflow with a non-converged chain, the cluster center is 0/0 = NaN. The NaN response removes all rows of the submodel data, and cv_varsel() stops with "matrix multiplication: incompatible matrix dimensions: 0x1 and 2x1".

With this PR, such a cluster keeps projection weight 0, and its center, dispersion and predictive variance use equal weights of its draws. The results for the other clusters do not change, apart from rounding. If all projected draws have weight 0 (possible only with thinning), get_p_clust() stops with an error that names the workarounds.

Tests

New tests in test_get_p_clust.R, test_wcol_moments.R and test_loo_fold_projection.R cover clusters with weight sum 0 and the new error. The test in test_loo_fold_projection.R runs cv_varsel() with Poisson and Gaussian reference models for which the PSIS-LOO weights underflow. One case in test_loo_weight_perturbation.R changes its expected result. On master, these test files give 79 failures and 1 error (the error above). The full suite passes (R 4.6.1, lme4 2.0.6) with the same warnings as master.

This PR was assisted by Claude.

@avehtari avehtari added the bug Bugs. label Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Bugs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant