diff --git a/docs/user_guide/development.ipynb b/docs/user_guide/development.ipynb index edebee380..ce29eac40 100644 --- a/docs/user_guide/development.ipynb +++ b/docs/user_guide/development.ipynb @@ -7747,6 +7747,309 @@ ").fit(abc_adj)\n", "model.coef_.round(4)" ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-intro", + "metadata": {}, + "source": [ + "### Calendar-period effects\n", + "\n", + "A one-off event that changes every payment made in one calendar period shows up\n", + "along a single diagonal of the `Triangle`. The link ratios into that diagonal are\n", + "too high, and `Chainladder` carries them into the projection of every origin. The\n", + "synthetic triangle below has origins 2011 to 2020 and the same expected payment at\n", + "each development age for every origin, except that every payment made in calendar\n", + "year 2017 is 50% higher. It is built through 2029, so the payments still to come\n", + "after 2020, and with them the true unpaid amount for each origin, are known." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-data", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "\n", + "# Incremental payment at each development age, the same for every origin\n", + "pattern = [300, 250, 150, 100, 70, 50, 40, 20, 15, 5]\n", + "cells = pd.DataFrame(\n", + " [\n", + " (str(origin), str(origin + lag), paid)\n", + " for origin in range(2011, 2021)\n", + " for lag, paid in enumerate(pattern)\n", + " ],\n", + " columns=[\"origin\", \"valuation\", \"paid\"],\n", + ")\n", + "expected = cl.Triangle(\n", + " cells, origin=\"origin\", valuation=\"valuation\", columns=\"paid\", cumulative=False\n", + ").dropna()\n", + "\n", + "# Every payment made in calendar year 2017 is 50% higher\n", + "square = expected + expected[expected.valuation.year == 2017] * 0.5\n", + "\n", + "# The triangle known at the end of 2020, and the true unpaid amount by origin\n", + "shocked = square[square.valuation.year <= 2020].incr_to_cum()\n", + "true_ibnr = square[square.valuation.year > 2020].sum(\"development\")\n", + "\n", + "\n", + "def ibnr_error(model, truth=true_ibnr):\n", + " # Relative error of each origin's IBNR, for origins with unpaid amounts\n", + " estimate = model.ibnr_.to_frame(origin_as_datetime=False).iloc[:, 0]\n", + " actual = truth.to_frame(origin_as_datetime=False).iloc[:, 0]\n", + " return (estimate / actual - 1).dropna()\n", + "\n", + "\n", + "def show(df):\n", + " # Adding 0.0 prints rounding noise of -0.0 as 0.0\n", + " return df.round(3) + 0.0" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-square-intro", + "metadata": {}, + "source": [ + "The full triangle shows what this example knows and a real reserving exercise\n", + "does not. The shaded cells in italics, below the 2020 diagonal, are paid after\n", + "2020, and each origin's shaded cells add up to its true unpaid amount. The extra\n", + "2017 payments run along one diagonal above them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-square", + "metadata": {}, + "outputs": [], + "source": [ + "payments = square.to_frame(origin_as_datetime=False)\n", + "paid_later = (\n", + " square[square.valuation.year > 2020]\n", + " .to_frame(origin_as_datetime=False)\n", + " .reindex_like(payments)\n", + " .notna()\n", + ")\n", + "later_style = \"background-color: #d9d9d9; color: #000000; font-style: italic\"\n", + "payments.style.format(\"{:,.0f}\").apply(\n", + " lambda _: np.where(paid_later, later_style, \"\"), axis=None\n", + ").set_caption(\"Incremental payments; shaded cells in italics are paid after 2020\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-test-intro", + "metadata": {}, + "source": [ + "Before fitting a model, test for a calendar-period effect.\n", + "`Triangle.valuation_correlation` runs the test in Appendix H of {cite}`mack1994`.\n", + "Within each development age it marks every link ratio as larger or smaller than\n", + "the median, and on each diagonal it counts the smaller of the two groups, `z`.\n", + "With no calendar-period effect, large and small ratios mix on every diagonal.\n", + "`total=True` sums `z` over the diagonals and compares the sum with its expected\n", + "value plus or minus 1.645 standard deviations (for `p_critical=0.1`).\n", + "`total=False` runs a binomial test on each diagonal, labeled by the valuation at\n", + "which its link ratios end. The {doc}`gallery example\n", + "` compares the two forms." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-test", + "metadata": {}, + "outputs": [], + "source": [ + "vc_total = shocked.valuation_correlation(p_critical=0.1, total=True)\n", + "print(\"total z:\", vc_total.z.iloc[0, 0])\n", + "print(\"no-effect range:\", np.round(np.ravel(vc_total.range), 2))\n", + "shocked.valuation_correlation(p_critical=0.1, total=False).z_critical" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-test-result", + "metadata": {}, + "source": [ + "The total `z` of 3 is below the range, so the test rejects the hypothesis of no\n", + "calendar-period effect at the 10% level. By diagonal, it flags the link ratios\n", + "ending in 2017 and those ending in 2020, but not those ending in 2018 or 2019,\n", + "although the extra 2017 payments sit in the starting amounts of all three." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-heatmap-intro", + "metadata": {}, + "source": [ + "A heatmap of the link ratios shows the same pattern. `heatmap` colors each ratio\n", + "by its rank within its development age, from blue for the smallest to red for\n", + "the largest, and ratios that print alike share a color once they are rounded to\n", + "the four decimals shown. The ratios ending in 2017 form a red diagonal, and most\n", + "ratios below it are blue. The oldest origin's last two ratios are red as well:\n", + "every ratio at those ages starts after 2017, and the oldest origin's are the\n", + "largest of them, for the reason given for 2019 below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-heatmap", + "metadata": {}, + "outputs": [], + "source": [ + "# low and high keep every color pale enough for dark text\n", + "shocked.link_ratio.round(4).heatmap(low=0.5, high=0.5)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-diagonals", + "metadata": {}, + "source": [ + "A link ratio equal to the median of its age counts as neither larger nor smaller.\n", + "At 12-24 and 24-36 most ratios are untouched by the event and equal, so the\n", + "median is the untouched value and those ratios don't count.\n", + "\n", + "- All six ratios ending in 2017 are larger than their medians, because the extra\n", + " payments are in their ending amounts. Six out of six, in either direction, has\n", + " a probability of 0.031 with no calendar-period effect.\n", + "- A ratio that starts in 2017 or later is smaller than an untouched one, because\n", + " the extra payments are in its starting amount. On the 2020 diagonal six ratios\n", + " are smaller and none is larger, again a probability of 0.031. The other three\n", + " are untouched (the 12-24 and 24-36 ratios) or alone at their age (108-120).\n", + "- On the 2019 diagonal five ratios are smaller and one is larger: the 96-108\n", + " ratio of origin 2011. Only two origins have a 96-108 ratio, and both start after\n", + " 2017, so the median falls between them. Origin 2011 made its 2017 payment at a\n", + " later age, where payments are smaller, so its starting amount carries less of\n", + " the extra (20 against 25 for origin 2012) and its ratio is the larger of the\n", + " two. One against five has a probability of 0.219, above the 0.1 threshold. The\n", + " 2018 diagonal has two larger ratios for the same reason." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-drop", + "metadata": {}, + "source": [ + "Dropping link ratios with `drop_valuation` is a common response. Link ratios are\n", + "labelled by the valuation of their starting cell, so the ratios into and out of the\n", + "2017 diagonal are dropped with `drop_valuation=[\"2016\", \"2017\"]`. This removes the\n", + "distorted ratios but leaves the extra 2017 payments in the cumulative amounts of\n", + "the later ages, so the projection is still off." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-compare", + "metadata": {}, + "outputs": [], + "source": [ + "errors = pd.DataFrame({\n", + " \"Chainladder\": ibnr_error(cl.Chainladder().fit(shocked)),\n", + " \"drop_valuation\": ibnr_error(\n", + " cl.Chainladder().fit(\n", + " cl.Development(drop_valuation=[\"2016\", \"2017\"]).fit_transform(shocked)\n", + " )\n", + " ),\n", + "})\n", + "show(errors)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-ptf", + "metadata": {}, + "source": [ + "The PTF framework handles this with a calendar (valuation) term {cite}`barnett2000`.\n", + "Inside the estimator, `valuation` is coded as the number of years since the\n", + "earliest valuation, so calendar year 2017 is `valuation == 6`. An\n", + "indicator for that one diagonal absorbs the event, and its coefficient is the log\n", + "of the 50% increase." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-indicator", + "metadata": {}, + "outputs": [], + "source": [ + "one_off = cl.BarnettZehnwirth(\n", + " formula=\"C(origin) + C(development) + I(valuation == 6)\"\n", + ").fit(shocked)\n", + "print(\"exp(coefficient):\", np.exp(one_off.coef_.iloc[-1, 0]).round(4))\n", + "errors[\"BarnettZehnwirth\"] = ibnr_error(\n", + " cl.Chainladder().fit(one_off.transform(shocked))\n", + ")\n", + "show(errors)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-future", + "metadata": {}, + "source": [ + "The future cells get the indicator's value for their own valuation, which is zero\n", + "here, so the projection assumes the event does not happen again. If the same event\n", + "is scheduled for a future calendar period, include that period in the indicator.\n", + "For a recurrence in 2022 (`valuation == 11`):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "calendar-effect-recurring", + "metadata": {}, + "outputs": [], + "source": [ + "recurring = cl.BarnettZehnwirth(\n", + " formula=\"C(origin) + C(development) + I((valuation == 6) | (valuation == 11))\"\n", + ").fit(shocked)\n", + "\n", + "# True unpaid amounts if the 2022 payments are also 50% higher\n", + "recur_square = square + square[square.valuation.year == 2022] * 0.5\n", + "recur_ibnr = recur_square[recur_square.valuation.year > 2020].sum(\"development\")\n", + "show(\n", + " pd.DataFrame({\n", + " \"one-off indicator\": ibnr_error(\n", + " cl.Chainladder().fit(one_off.transform(shocked)), recur_ibnr\n", + " ),\n", + " \"shared indicator\": ibnr_error(\n", + " cl.Chainladder().fit(recurring.transform(shocked)), recur_ibnr\n", + " ),\n", + " })\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "calendar-effect-note", + "metadata": {}, + "source": [ + "```{note}\n", + "A shared indicator applies the same multiplicative factor to every cell on the\n", + "future diagonal. That matches an event that scales payments in proportion, as in\n", + "this example. An event that adds a fixed amount per claim or per policy grows\n", + "with the number of claims or policies on that diagonal, not with the expected\n", + "payment of each cell, and a log-link factor can then misstate it badly. Check how\n", + "the event scales before sharing its coefficient with a future period.\n", + "```" + ] } ], "metadata": { diff --git a/docs/user_guide/triangle.ipynb b/docs/user_guide/triangle.ipynb index 0577473fa..e20eaab07 100644 --- a/docs/user_guide/triangle.ipynb +++ b/docs/user_guide/triangle.ipynb @@ -2369,6 +2369,39 @@ "triangle.valuation_correlation().z_critical" ] }, + { + "cell_type": "markdown", + "id": "triangle-valuation-correlation-friedland", + "metadata": {}, + "source": [ + "The test finds no calendar-period effect in `raa`. Chapter 7, Exhibit III of\n", + "{cite}`friedland2010` applies the development method to U.S. private passenger\n", + "auto data under four scenarios, two of them with increasing case outstanding\n", + "strength. Strengthening case reserves is a calendar-period change: it raises\n", + "reported claims, which include case outstanding, but not paid claims. The\n", + "`friedland_uspp` sample holds all four scenarios." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "triangle-valuation-correlation-uspp", + "metadata": {}, + "outputs": [], + "source": [ + "uspp = cl.load_sample(\"friedland_uspp\")[[\"Reported Claims\", \"Paid Claims\"]]\n", + "uspp.valuation_correlation(p_critical=0.1, total=True).z_critical" + ] + }, + { + "cell_type": "markdown", + "id": "triangle-valuation-correlation-uspp-result", + "metadata": {}, + "source": [ + "The test flags reported claims in the two scenarios with increasing case\n", + "outstanding strength, and nothing else." + ] + }, { "cell_type": "markdown", "id": "d5bf6b74",