Repository navigation
[FEAT] Check Development parameter values in fit and read_json - #1606
Conversation
DevelopmentBase and MethodBase gain a _validate_params hook that checks nothing by default. Development overrides it: average must be 'volume', 'simple', 'regression' or 'geometric', n_periods must be -1 or a positive integer, and sigma_interpolation must be 'log-linear' or 'mack'. A bad value raises a ValueError naming the parameter. fit calls the hook with the triangle, so lists are checked element by element and dicts and callables after they resolve against it. read_json calls it with no triangle when it loads an estimator or a pipeline step. Before this, average='bogus' loaded and then raised a KeyError in fit, n_periods=-5 or 0 fitted silently as -1, and sigma_interpolation='bogus' left sigma_ as NaN. Closes casact#1569. Developed with an AI assistant (Árbol, running Muse).
Pyright Type CompletenessView the full Project (full
Other symbols referenced but not exported by
Symbols without documentation:
Patch (exported symbols added or changed by this PR): 25.0% fully typed (2 / 8)
Patch symbol details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1606 +/- ##
==========================================
+ Coverage 94.84% 95.00% +0.15%
==========================================
Files 99 100 +1
Lines 6424 6461 +37
Branches 811 820 +9
==========================================
+ Hits 6093 6138 +45
+ Misses 212 204 -8
Partials 119 119
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Developed with an AI assistant (Árbol, running Muse).
Developed with an AI assistant (Árbol, running Muse).
|
can you check the codecov? is it cupy again? |
|
It isn't cupy this time. The three lines are new code that no test reaches: That check hides a bug. sklearn's I've pushed d420695, which calls the hook only on One naming question: the hook has the same name as sklearn's private (This comment was written with Árbol, an AI assistant, under my direction.) |
|
please disregard my name suggestion and use something that doesn't clash with sklearn.
let's keep the behavior but you can raise an issue to discuss deprecation
my guess is that it may have worked at one point. does average=1.0 even work now? if average=1.0 doesn't work, then we can reasonably conclude that it's not in use and can be removed from docstring
sounds like distinct scope |
scikit-learn's BaseEstimator defines _validate_params, so the chainladder hook takes a name scikit-learn does not use. read_json still calls the hook only on DevelopmentBase and MethodBase estimators, the classes that define it. Developed with an AI assistant (Árbol, running Muse).
Development fitted n_periods=0 as -1 before the parameter check, and the check now accepts 0 again so that behavior is unchanged. Negative values other than -1 still raise. A test checks that n_periods=0 gives the same ldf_ and sigma_ as n_periods=-1 on raa. Developed with an AI assistant (Árbol, running Muse).
The docstring said a numeric average sets the exponent of the regression weights, but average=1.0, 1.5 and other numbers raise a KeyError on main, because the weight lookup in weighted_regression.py accepts only the string names. The docstring now lists only those names. Developed with an AI assistant (Árbol, running Muse).
| callable is checked value by value; see | ||
| ``DevelopmentBase._validate_params``. | ||
| """ | ||
| averages = ("volume", "simple", "regression", "geometric") |
There was a problem hiding this comment.
i was originally thinking the allowable values would be declared as a Literal constant, which can then be checked against via get_args. open to other ideas if that's not a reliable pattern
| ``DevelopmentBase._validate_params``. | ||
| """ | ||
| averages = ("volume", "simple", "regression", "geometric") | ||
| for value in self._assumption_values(self.average, X): |
There was a problem hiding this comment.
somewhere in this module or development/base.py, there's also checking for length of the parameters. do you think that should be in scope for this method? (doesn't have to be this PR)
| "params, name", | ||
| [({"average": "bogus"}, "average"), ({"n_periods": -5}, "n_periods")], | ||
| ) | ||
| def test_read_json_rejects_invalid_params(params, name): |
There was a problem hiding this comment.
do you think we'll eventually get to a sizable suite of I/O tests for development estimators? does it make sense to start a test_io.py?
| Subclasses override it; the base class checks nothing. | ||
| """ | ||
|
|
||
| def _assumption_values(self, value, X: Triangle | None, axis: int = 3) -> list: |
There was a problem hiding this comment.
the way this is currently written probably belongs in common?
|
|
||
| return cl.__dict__[json_dict["__class__"]]().set_params(**json_dict["params"]) | ||
|
|
||
| def _estimator_from_json(class_name, params): |
There was a problem hiding this comment.
is this a from_dict is disguise?
| class MethodBase(BaseEstimator, EstimatorIO, Common): | ||
| _estimator_type = "chainladder" | ||
|
|
||
| def _validate_params(self, X: Triangle | None = None) -> None: |
There was a problem hiding this comment.
we can leave this for a future PR if there's nothing to implement
|
|
||
|
|
||
| class DevelopmentBase(BaseEstimator, TransformerMixin, EstimatorIO, Common): | ||
| def _validate_params(self, X: Triangle | None = None) -> None: |
There was a problem hiding this comment.
the pattern that I was obssessed with is validate_X in MethodBase, i.e. it doesn't even have to be explicitly called by the subclasses. but Development doesn't call super().fit(). So do we still need to declare an empty method here? sorry for the extra work
Average and SigmaInterpolation are module-level Literal aliases. Development.__init__ annotates average and sigma_interpolation with them, and _check_param_values reads the allowed values with typing.get_args, so the annotation, the check and the error message share one list. Developed with an AI assistant (Árbol, running Muse).
_assumption_values resolves an assumption the same way _validate_assumption does and is not specific to development estimators, so it moves from DevelopmentBase to Common. Behavior is unchanged. Developed with an AI assistant (Árbol, running Muse).
No method estimator checks its parameters yet, so MethodBase does not declare the hook. read_json now calls _check_param_values on any estimator that defines it; scikit-learn 1.7.2 defines no method of that name. Developed with an AI assistant (Árbol, running Muse).
Development.fit does not call DevelopmentBase.fit, so an empty hook on the base class added nothing. The hook now lives only on Development, and its docstring says how fit and read_json call it. Developed with an AI assistant (Árbol, running Muse).
The three read_json tests added for casact#1569 move from test_development.py to a new chainladder/development/tests/test_io.py, unchanged, so later I/O tests for development estimators have one place to go. Developed with an AI assistant (Árbol, running Muse).
|
Thanks for the review. Pushed, one commit per change so any can be dropped:
On the inline comments:
(This comment was written with Árbol, an AI assistant, under my direction.) |
Summary of Changes
Following #1569,
Developmentgains a_check_param_values(X=None)method.fitcalls it, and so doesread_jsonfor a single estimator and for each pipeline step, on any estimator that defines it. The name avoids scikit-learn's own_validate_paramsand_check_params. It checks:averagemust be'volume','simple','regression'or'geometric', the values of a newAverageLiteralalias read withget_args.n_periodsmust be -1 or a positive integer; 0 stays an alias for -1.sigma_interpolationmust be'log-linear'or'mack', the values of aSigmaInterpolationalias.A bad value raises a
ValueErrornaming the parameter. Lists are checked element by element, and dicts and callables are checked infitonce resolved, through a newCommon._assumption_valuesnext to_validate_assumption. Before,average='bogus'loaded and then raised aKeyErrorinfit,n_periods=-5fitted as -1, andsigma_interpolation='bogus'leftsigma_as NaN.The
averagedocstring no longer offers a numeric exponent:fitpassed numbers through as weight exponents from v0.7.7 until 59b7f9e (first released in v0.8.15), and on main they raise aKeyError.Tests in
test_development.pycover eight bad values infit,n_periods=0fitting like -1, and valid lists, dicts and callables. A newchainladder/development/tests/test_io.pychecks two bad values inread_json, thatread_jsonstill loadsTrend,BerquistShermanandParallelogramOLF, and that it loads a dict assumption without a triangle.Related GitHub Issue(s)
Closes #1569
AI/LLM Usage
This change was made with Árbol, an AI assistant running on Muse, working under my direction. It wrote the change and the tests, ran the checks and drafted this description. I reviewed the diff and the results before submitting.
Additional Context for Reviewers
Left for follow-ups: the
read_jsonchecks for a non-chainladder__class__and for nested estimators, a discussion of deprecatingn_periods=0, and checks on the length of per-link-ratio parameters.Other modules and
Development's other parameters are left for later PRs. This PR doesn't depend on #1568's PR. It merges cleanly into main. It conflicts with #1389 on adjacent lines ofDevelopment.fit, and it and #1610 both add tests at the end oftest_development.py, so whichever merges second needs a small rebase. The development, utilities and JSON tests pass with the uv.lock versions, and ruff is clean.Submitter's Checklist
[FIX],[FEAT],[DOCS],[TST],[CHORE], or[BRK]).Reviewer's Checklist