Skip to content

[QUESTION] predict() on a bootstrapped triangle leaves out process variance, and VotingChainladder inherits that #1639

Open

Description

@scks1

On a triangle from BootstrapODPSample, fit() adds the simulated process variance to ultimate_ through MethodBase._include_process_variance (methods/base.py L154). predict() doesn't call it, so predict(sims).ultimate_ is each draw's point estimate. VotingChainladder._get_ultimate builds its ultimate from est.predict(X, sample_weight).ultimate_ (workflow/voting.py L427–429), so a voted bootstrap never carries process variance, even with two identical members.

import chainladder as cl, numpy as np
raa = cl.load_sample("raa")
sims = cl.BootstrapODPSample(n_sims=2000, random_state=42).fit_transform(raa)
sd = lambda ult: float(np.std((ult - sims.latest_diagonal).sum("origin").values))
model = cl.Chainladder().fit(sims)
print("fit      ", round(sd(model.ultimate_), 1))
print("predict  ", round(sd(model.predict(sims).ultimate_), 1))
vot = cl.VotingChainladder([("a", cl.Chainladder()), ("b", cl.Chainladder())]).fit(sims)
print("voting   ", round(sd(vot.ultimate_), 1))

On main (c113fd9) this prints a total IBNR standard deviation of 18939.9 for fit, 17337.3 for predict and 17337.3 for voting.

Shapland (2016, §3.2, p. 12), which the BootstrapODPSample docs cite, describes England and Verrall (2002) adding gamma process variance to the future incremental values so the bootstrap gives a distribution of possible outcomes and not just of point estimates. fit() follows that and predict() doesn't.

  1. Is leaving process variance out of predict() intended? The 2020 refactor commit d916f4f says "Bootstrap process variance now carrys through to predict method", but on main only fit() adds it.
  2. Should VotingChainladder combine draws that include process variance, and should it mix them instead of averaging them? When Shapland combines models (§6, p. 47), "the weights by year were used to randomly sample the specified percentage of iterations from each model", so the combined distribution is a mixture of the members' draws. _get_ultimate takes a weighted average of the member ultimates draw by draw, and for each origin that spread is narrower than the mixture's once the members differ.

(This issue was written with Árbol, an AI assistant, under my direction.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Scope

    None yet

    Effort

    None yet

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions