Repository navigation
Conversation
Pyright Type CompletenessView the full Project (full
Other symbols referenced but not exported by
Symbols without documentation:
Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 1)
Patch symbol details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The (3.14, pandas3, ubuntu-latest) job failed with "The hosted runner lost communication with the server" partway through Run tests, and it left no log. Every other job passed, including 3.14 on pandas 2.3.3. Could you re-run that job when you get a chance? I don't have permission to re-run checks on this repo. (This comment was written with Árbol, an AI assistant, under my direction.) |
henrydingliu
left a comment
There was a problem hiding this comment.
responding to the questions in the PR summary
R also switches to Mack when the slope's p-value is above 0.05. I did not add that. The largest fill left on clrd is 3,341,035 (CumPaidLoss, 24-36, extrapolated back from four sigmas that fall from 14.7 to 0.00059). Its p-value is 0.048, so R would keep it too.
anyone using sigma_ should understand that smaller triangles or triangles with lots of missing data is going to cause issues. i don't think we need to nanny
R warns when it switches to Mack. This PR does not warn, since clrd would warn on many triangles at once. Should it?
since the sigma calculations are happening on every development even when people are just trying to get to a normal chainladder result, i don't want to warn on a corner case on an unused statistic. if we had a toggle to turn sigma calcs on or off (defaulting to off), then it becomes reasonable to warn this switching behavior when someone intentionaly turns on the sigma calcs.
mack_interpolation still returns 9.9999e-321 where a sigma is 0, because of the same 1e-320 stand-in. Turning those into 0 would make them NaN under "mack" and could change Mack results, so I left it out.
i'm not following. so is mack interpolation not doing anything?
|
|
||
| # One positive sigma: the fill falls back to Mack's approximation. | ||
| brotherhood = cl.Development().fit(tri.loc["Brotherhood Mut Ins Co", "prodliab"]) | ||
| assert np.nanmax(brotherhood.sigma_.set_backend("numpy").values) < 1e3 |
There was a problem hiding this comment.
please double check my logic flow here.
we are here to test mack's approximation for sigma_ when loglinear regression doesn't have enough valid data. have we tested mack's approximatio itself somewhere? i.e. are any of our hardcoded mack tests covering the approximation for sigma?
There was a problem hiding this comment.
Your logic is right, and the Brotherhood case didn't test it. Its raw sigmas are NaN, NaN, 7.67, 0, then NaN. Mack's approximation needs two positive sigmas just before a gap, so it fills nothing there; the case only checks that the 1.8e308 fills are gone. I fixed its comment and added Great Amer Grp / medmal, whose sigmas are 14.98, 52.90, then a gap: the fallback fills 36-48 with 14.978, and the test checks that against sigma_interpolation="mack" on the same triangle. On main that fill is 1.2e-142. The approximation itself is covered by printed values: test_mack1999_hardcode checks mortgage's filled last sigma (16.887 against 16.89 in Mack 1999), and test_cdr_matches_merz2008 checks mw2008's filled last sigma^2 (0.040 against 0.04). test_mack1994_hardcode only reaches it through the oldest origin's standard error.
(This comment was written with Árbol, an AI assistant, under my direction.)
| """ | ||
| Fill undefined sigmas by regressing log(sigma) on development period. | ||
|
|
||
| As in R's ChainLadder (``estimate.sigma``), only positive sigmas enter |
There was a problem hiding this comment.
since we are pivoting away from pinning any results to R chainladder implementation, we should also provide independent rationale for design choices, rather than referring back to R chainladder.
There was a problem hiding this comment.
Rewritten without R. A zero sigma has no logarithm, and the 1e-320 stand-in put a point near -737 on the log scale that outweighed every other point. A line through two points fits both exactly, so nothing is left to judge it by; on clrd IncurLoss one such line gave a fill of 1,921,341 from sigmas of 0.0268 and 11.12.
(This comment was written with Árbol, an AI assistant, under my direction.)
| fill_ = xp.nan_to_num(xp.exp(reg.x * slope + intercept) * (1 - estimated)) | ||
| use_mack = xp.sum(w, axis=self.axis, keepdims=True) < 3 | ||
| if use_mack.any(): | ||
| with warnings.catch_warnings(): |
There was a problem hiding this comment.
i didn't see a rationale for this warning suppression in the PR summary. any specific issue this is trying to solve?
There was a problem hiding this comment.
The default path now calls mack_interpolation, which divides by the square of the sigma two periods back. Where that sigma is 0, numpy warns "divide by zero" and "invalid value" in cells whose fill comes out as 0 anyway. Without a filter, a default fit would gain those two warnings whenever one of its triangles takes the fallback and any of them has a zero sigma. I replaced the blanket RuntimeWarning filter with np.errstate(divide="ignore", invalid="ignore") around that one call. On clrd the default path now gives the same warnings as with the old filter, under numpy and sparse.
(This comment was written with Árbol, an AI assistant, under my direction.)
|
Thanks. I'll leave out the p-value fallback and the warning. On "is mack interpolation not doing anything?": it does fill. It fills a missing sigma from the two sigmas just before it, which covers the usual missing last sigma. My bullet was only about one detail: where a sigma is exactly 0, (This comment was written with Árbol, an AI assistant, under my direction.) |
are you referring to this line? assuming yes, then i'm wondering if the brotherhood test is valid, i.e. if we plan to change that line, wouldn't that likely change brotherhood test? while reading through the mack interpolation, the docstring makes me wonder what usefulness may remain of this mack-interpolation fallback if we change that line to not creep 0 to 1e-320 |
loglinear_interpolation replaced a zero sigma with 1e-320 and fitted its log (-736.8) at full weight, so a single zero-variation period could push the extrapolated sigmas to 1.8e308. Only positive sigmas now enter the fit, as in R's ChainLadder estimate.sigma, and zero sigmas are not filled. Where fewer than three positive sigmas remain, the fill falls back to Mack's approximation, which R's Mack.S.E also does for one point and for two points (undefined slope p-value). Closes casact#1609. Developed with an AI assistant (Árbol, running Muse).
Give the log-linear fill's reasons in the docstring: a zero sigma has no logarithm, and a line through two points leaves no residual to judge it by. Scope the fallback's error suppression to numpy's divide and invalid flags, which mack_interpolation sets where an earlier sigma is zero. Add a case where the fallback fills a sigma and compare it with sigma_interpolation="mack", and correct the comment on the case where Mack has nothing to fill from. Developed with an AI assistant (Árbol, running Muse).
a274a90 to
eb71331
Compare
|
Yes, that's the line. The Brotherhood test passes with and without it. The fill only uses that value squared or to the fourth power, and
On the docstring: Mack's rule matches a two-point log-linear extrapolation only while the sigmas fall. Mack (1993, p. 217) offers log-linear regression or, more simply, keeping the ratio of successive sigmas constant as long as they decrease, which gives I can correct the (This comment was written with Árbol, an AI assistant, under my direction.) |
Summary of Changes
WeightedRegression.loglinear_interpolationfills the sigmas thatDevelopmentcannot estimate by regressing log(sigma) on development period. It replaced a zero sigma with 1e-320 and fitted its log (-736.8) at full weight, so one zero-variation period could push the fills to 1.8e308 (details in #1609).This PR changes only that function:
mack_interpolationinstead. One point gives no slope, and a line through two points fits both exactly, so nothing is left to judge it by. With a cutoff of two, as in the issue, clrd IncurLoss still got a fill of 1,921,341 extrapolated from two sigmas, 0.0268 and 11.12.A regression test on clrd CumPaidLoss covers Brotherhood Mut Ins Co / prodliab (one positive sigma, where Mack has nothing to fill from and the old 1.8e308 fills become NaN), Great Amer Grp / medmal (two positive sigmas, where the fallback's 36-48 fill must equal the
sigma_interpolation="mack"value) and Buckeye Ins Grp / othliab (five positive sigmas and two zeros). For Buckeye the test refits the five positive sigmas withnp.polyfitand checks that the 12-24 and 108-120 fills lie on that line; the 12-24 fill is 7.1819 (it was 2.50e137).Related GitHub Issue(s)
Closes #1609
AI/LLM Usage
I used Árbol, an AI assistant running on Muse, under my direction to trace the cause, write the change and the test, run the comparisons and draft this description. I reviewed the change and the results.
Additional Context for Reviewers
Before and after on main ee2c24d, default settings:
total_mack_std_err_mack_std_err_sigma_interpolation="mack"results are byte-identical on every triangle checked.chainladder/developmentandchainladder/methodsgive 240 passed with this change and 238 passed on main (the difference is the two new test runs). The new test fails on main.Submitter's Checklist
[FIX],[FEAT],[DOCS],[TST],[CHORE], or[BRK]).Reviewer's Checklist