Skip to content
303 changes: 303 additions & 0 deletions docs/user_guide/development.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7747,6 +7747,309 @@
").fit(abc_adj)\n",
"model.coef_.round(4)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-intro",
"metadata": {},
"source": [
"### Calendar-period effects\n",
"\n",
"A one-off event that changes every payment made in one calendar period shows up\n",
"along a single diagonal of the `Triangle`. The link ratios into that diagonal are\n",
"too high, and `Chainladder` carries them into the projection of every origin. The\n",
"synthetic triangle below has origins 2011 to 2020 and the same expected payment at\n",
"each development age for every origin, except that every payment made in calendar\n",
"year 2017 is 50% higher. It is built through 2029, so the payments still to come\n",
"after 2020, and with them the true unpaid amount for each origin, are known."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-data",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"# Incremental payment at each development age, the same for every origin\n",
"pattern = [300, 250, 150, 100, 70, 50, 40, 20, 15, 5]\n",
"cells = pd.DataFrame(\n",
" [\n",
" (str(origin), str(origin + lag), paid)\n",
" for origin in range(2011, 2021)\n",
" for lag, paid in enumerate(pattern)\n",
" ],\n",
" columns=[\"origin\", \"valuation\", \"paid\"],\n",
")\n",
"expected = cl.Triangle(\n",
" cells, origin=\"origin\", valuation=\"valuation\", columns=\"paid\", cumulative=False\n",
").dropna()\n",
"\n",
"# Every payment made in calendar year 2017 is 50% higher\n",
"square = expected + expected[expected.valuation.year == 2017] * 0.5\n",
"\n",
"# The triangle known at the end of 2020, and the true unpaid amount by origin\n",
"shocked = square[square.valuation.year <= 2020].incr_to_cum()\n",
"true_ibnr = square[square.valuation.year > 2020].sum(\"development\")\n",
"\n",
"\n",
"def ibnr_error(model, truth=true_ibnr):\n",
" # Relative error of each origin's IBNR, for origins with unpaid amounts\n",
" estimate = model.ibnr_.to_frame(origin_as_datetime=False).iloc[:, 0]\n",
" actual = truth.to_frame(origin_as_datetime=False).iloc[:, 0]\n",
" return (estimate / actual - 1).dropna()\n",
"\n",
"\n",
"def show(df):\n",
" # Adding 0.0 prints rounding noise of -0.0 as 0.0\n",
" return df.round(3) + 0.0"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-square-intro",
"metadata": {},
"source": [
"The full triangle shows what this example knows and a real reserving exercise\n",
"does not. The shaded cells in italics, below the 2020 diagonal, are paid after\n",
"2020, and each origin's shaded cells add up to its true unpaid amount. The extra\n",
"2017 payments run along one diagonal above them."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-square",
"metadata": {},
"outputs": [],
"source": [
"payments = square.to_frame(origin_as_datetime=False)\n",
"paid_later = (\n",
" square[square.valuation.year > 2020]\n",
" .to_frame(origin_as_datetime=False)\n",
" .reindex_like(payments)\n",
" .notna()\n",
")\n",
"later_style = \"background-color: #d9d9d9; color: #000000; font-style: italic\"\n",
"payments.style.format(\"{:,.0f}\").apply(\n",
" lambda _: np.where(paid_later, later_style, \"\"), axis=None\n",
").set_caption(\"Incremental payments; shaded cells in italics are paid after 2020\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-test-intro",
"metadata": {},
"source": [
"Before fitting a model, test for a calendar-period effect.\n",
"`Triangle.valuation_correlation` runs the test in Appendix H of {cite}`mack1994`.\n",
"Within each development age it marks every link ratio as larger or smaller than\n",
"the median, and on each diagonal it counts the smaller of the two groups, `z`.\n",
"With no calendar-period effect, large and small ratios mix on every diagonal.\n",
"`total=True` sums `z` over the diagonals and compares the sum with its expected\n",
"value plus or minus 1.645 standard deviations (for `p_critical=0.1`).\n",
"`total=False` runs a binomial test on each diagonal, labeled by the valuation at\n",
"which its link ratios end. The {doc}`gallery example\n",
"</gallery/plot_valuation_correlation>` compares the two forms."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-test",
"metadata": {},
"outputs": [],
"source": [
"vc_total = shocked.valuation_correlation(p_critical=0.1, total=True)\n",
"print(\"total z:\", vc_total.z.iloc[0, 0])\n",
"print(\"no-effect range:\", np.round(np.ravel(vc_total.range), 2))\n",
"shocked.valuation_correlation(p_critical=0.1, total=False).z_critical"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-test-result",
"metadata": {},
"source": [
"The total `z` of 3 is below the range, so the test rejects the hypothesis of no\n",
"calendar-period effect at the 10% level. By diagonal, it flags the link ratios\n",
"ending in 2017 and those ending in 2020, but not those ending in 2018 or 2019,\n",
"although the extra 2017 payments sit in the starting amounts of all three."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-heatmap-intro",
"metadata": {},
"source": [
"A heatmap of the link ratios shows the same pattern. `heatmap` colors each ratio\n",
"by its rank within its development age, from blue for the smallest to red for\n",
"the largest, and ratios that print alike share a color once they are rounded to\n",
"the four decimals shown. The ratios ending in 2017 form a red diagonal, and most\n",
"ratios below it are blue. The oldest origin's last two ratios are red as well:\n",
"every ratio at those ages starts after 2017, and the oldest origin's are the\n",
"largest of them, for the reason given for 2019 below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-heatmap",
"metadata": {},
"outputs": [],
"source": [
"# low and high keep every color pale enough for dark text\n",
"shocked.link_ratio.round(4).heatmap(low=0.5, high=0.5)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-diagonals",
"metadata": {},
"source": [
"A link ratio equal to the median of its age counts as neither larger nor smaller.\n",
"At 12-24 and 24-36 most ratios are untouched by the event and equal, so the\n",
"median is the untouched value and those ratios don't count.\n",
"\n",
"- All six ratios ending in 2017 are larger than their medians, because the extra\n",
" payments are in their ending amounts. Six out of six, in either direction, has\n",
" a probability of 0.031 with no calendar-period effect.\n",
"- A ratio that starts in 2017 or later is smaller than an untouched one, because\n",
" the extra payments are in its starting amount. On the 2020 diagonal six ratios\n",
" are smaller and none is larger, again a probability of 0.031. The other three\n",
" are untouched (the 12-24 and 24-36 ratios) or alone at their age (108-120).\n",
"- On the 2019 diagonal five ratios are smaller and one is larger: the 96-108\n",
" ratio of origin 2011. Only two origins have a 96-108 ratio, and both start after\n",
" 2017, so the median falls between them. Origin 2011 made its 2017 payment at a\n",
" later age, where payments are smaller, so its starting amount carries less of\n",
" the extra (20 against 25 for origin 2012) and its ratio is the larger of the\n",
" two. One against five has a probability of 0.219, above the 0.1 threshold. The\n",
" 2018 diagonal has two larger ratios for the same reason."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-drop",
"metadata": {},
"source": [
"Dropping link ratios with `drop_valuation` is a common response. Link ratios are\n",
"labelled by the valuation of their starting cell, so the ratios into and out of the\n",
"2017 diagonal are dropped with `drop_valuation=[\"2016\", \"2017\"]`. This removes the\n",
"distorted ratios but leaves the extra 2017 payments in the cumulative amounts of\n",
"the later ages, so the projection is still off."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-compare",
"metadata": {},
"outputs": [],
"source": [
"errors = pd.DataFrame({\n",
" \"Chainladder\": ibnr_error(cl.Chainladder().fit(shocked)),\n",
" \"drop_valuation\": ibnr_error(\n",
" cl.Chainladder().fit(\n",
" cl.Development(drop_valuation=[\"2016\", \"2017\"]).fit_transform(shocked)\n",
" )\n",
" ),\n",
"})\n",
"show(errors)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-ptf",
"metadata": {},
"source": [
"The PTF framework handles this with a calendar (valuation) term {cite}`barnett2000`.\n",
"Inside the estimator, `valuation` is coded as the number of years since the\n",
"earliest valuation, so calendar year 2017 is `valuation == 6`. An\n",
"indicator for that one diagonal absorbs the event, and its coefficient is the log\n",
"of the 50% increase."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-indicator",
"metadata": {},
"outputs": [],
"source": [
"one_off = cl.BarnettZehnwirth(\n",
" formula=\"C(origin) + C(development) + I(valuation == 6)\"\n",
").fit(shocked)\n",
"print(\"exp(coefficient):\", np.exp(one_off.coef_.iloc[-1, 0]).round(4))\n",
"errors[\"BarnettZehnwirth\"] = ibnr_error(\n",
" cl.Chainladder().fit(one_off.transform(shocked))\n",
")\n",
"show(errors)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-future",
"metadata": {},
"source": [
"The future cells get the indicator's value for their own valuation, which is zero\n",
"here, so the projection assumes the event does not happen again. If the same event\n",
"is scheduled for a future calendar period, include that period in the indicator.\n",
"For a recurrence in 2022 (`valuation == 11`):"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "calendar-effect-recurring",
"metadata": {},
"outputs": [],
"source": [
"recurring = cl.BarnettZehnwirth(\n",
" formula=\"C(origin) + C(development) + I((valuation == 6) | (valuation == 11))\"\n",
").fit(shocked)\n",
"\n",
"# True unpaid amounts if the 2022 payments are also 50% higher\n",
"recur_square = square + square[square.valuation.year == 2022] * 0.5\n",
"recur_ibnr = recur_square[recur_square.valuation.year > 2020].sum(\"development\")\n",
"show(\n",
" pd.DataFrame({\n",
" \"one-off indicator\": ibnr_error(\n",
" cl.Chainladder().fit(one_off.transform(shocked)), recur_ibnr\n",
" ),\n",
" \"shared indicator\": ibnr_error(\n",
" cl.Chainladder().fit(recurring.transform(shocked)), recur_ibnr\n",
" ),\n",
" })\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "calendar-effect-note",
"metadata": {},
"source": [
"```{note}\n",
"A shared indicator applies the same multiplicative factor to every cell on the\n",
"future diagonal. That matches an event that scales payments in proportion, as in\n",
"this example. An event that adds a fixed amount per claim or per policy grows\n",
"with the number of claims or policies on that diagonal, not with the expected\n",
"payment of each cell, and a log-link factor can then misstate it badly. Check how\n",
"the event scales before sharing its coefficient with a future period.\n",
"```"
]
}
],
"metadata": {
Expand Down
33 changes: 33 additions & 0 deletions docs/user_guide/triangle.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2369,6 +2369,39 @@
"triangle.valuation_correlation().z_critical"
]
},
{
"cell_type": "markdown",
"id": "triangle-valuation-correlation-friedland",
"metadata": {},
"source": [
"The test finds no calendar-period effect in `raa`. Chapter 7, Exhibit III of\n",
"{cite}`friedland2010` applies the development method to U.S. private passenger\n",
"auto data under four scenarios, two of them with increasing case outstanding\n",
"strength. Strengthening case reserves is a calendar-period change: it raises\n",
"reported claims, which include case outstanding, but not paid claims. The\n",
"`friedland_uspp` sample holds all four scenarios."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "triangle-valuation-correlation-uspp",
"metadata": {},
"outputs": [],
"source": [
"uspp = cl.load_sample(\"friedland_uspp\")[[\"Reported Claims\", \"Paid Claims\"]]\n",
"uspp.valuation_correlation(p_critical=0.1, total=True).z_critical"
]
},
{
"cell_type": "markdown",
"id": "triangle-valuation-correlation-uspp-result",
"metadata": {},
"source": [
"The test flags reported claims in the two scenarios with increasing case\n",
"outstanding strength, and nothing else."
]
},
{
"cell_type": "markdown",
"id": "d5bf6b74",
Expand Down
Loading