Skip to content

Remove custom pandas assert wrappers and use pandas.testing directly - #2951

Open
DresdenGman wants to merge 2 commits into
pvlib:mainfrom
DresdenGman:fix-2263-remove-pandas-assert-wrappers
Open

DresdenGman wants to merge 2 commits into
pvlib:mainfrom
DresdenGman:fix-2263-remove-pandas-assert-wrappers

Conversation

@DresdenGman

@DresdenGman DresdenGman commented Nov 20, 2026 •

Copy link
Copy Markdown
  • Closes Simplify or remove assert_*_equal wrappers around pandas.testing functions #2263
  • I am familiar with the contributing guidelines
  • I attest that all AI-generated material has been vetted for accuracy and is in compliance with the pvlib license
  • Tests added
  • Updates entries in docs/sphinx/source/reference for API changes.
  • Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
  • New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
  • Pull request is nearly complete and ready for detailed review.
  • Maintainer: Appropriate GitHub Labels (including remote-data) and Milestone are assigned to the Pull Request and linked Issue.

Summary

Remove the assert_series_equal and assert_frame_equal wrappers in tests/conftest.py and use pandas.testing functions directly, with explicit tolerances at every call site. Simplify assert_index_equal to a thin compatibility shim (see below).

The wrappers wrapped the three pandas.testing assert functions with pandas 1.1-era version logic. _check_pandas_assert_kwargs silently rewrote caller tolerances — check_less_precise (truthy) became atol=rtol=1e-3, otherwise atol=rtol=1e-5 — overwriting any atol/rtol the caller passed. The pandas < 1.1 branch is dead code (the minimum supported pandas is 1.5.0). Following maintainer AdamRJensen's direction in the issue ("I think we should just get rid of our own wrappers, and use pandas"), the series and frame wrappers are removed entirely.

Tolerance translation (behavior-preserving)

  • Bare call → atol=1e-5, rtol=1e-5 (the wrapper's defaults; pandas' own defaults are 1000× tighter, so bare removal would silently tighten tests).
  • Truthy check_less_precise (values True/2/3/4/5 in the suite) → atol=1e-3, rtol=1e-3.
  • Call sites that passed an explicit atol/rtol keep the effective values 1e-5/1e-5, not the written ones — the wrapper overwrote them. This affects 6 sites: tests/spectrum/test_irradiance.py:45 (atol=1e-2), :62 (atol=1e-4), :90 (rtol=1e-4); tests/test_albedo.py:51,64 (atol=1e-5); tests/test_iam.py:461 (rtol=1e-6). The full test run passes at 1e-5, confirming these tolerances are genuinely sufficient.

Why assert_index_equal is kept as a shim

check_freq was added to pandas.testing.assert_index_equal only in pandas 3.1.0, and the CI minimum environment pins pandas 1.5.0, so passing check_freq=False unconditionally would raise TypeError on older pandas. The wrapper is therefore simplified to a version gate only — it passes check_freq=False on pandas >= 3.1.0 and nothing otherwise — with a comment noting it can be removed once pvlib requires pandas >= 3.1.0. The 17 index call sites use it with explicit atol=1e-5, rtol=1e-5 and no check_freq kwarg.

Other changes

  • Replace tests/test_conftest.py (which only tested the removed wrappers) with a single regression test pinning the shim's version gate: it monkeypatches pd.__version__ to 1.5.0 / 3.1.0 and asserts check_freq is forwarded only on pandas >= 3.1.0.
  • Whatsnew entry in docs/sphinx/source/whatsnew/v0.16.2.rst.

Test report

Affected test files (34 files, 372 call sites), run against the pre-change baseline — identical results, zero regressions:

  • Non-iotools: 1065 passed, 35 skipped, 0 failed
  • iotools (-m "not remote_data"): 68 passed, 75 deselected, 0 failed
  • New shim regression tests: 2 passed

flake8 --diff on the changed lines is clean.

Closes #2263

Remove the assert_series_equal and assert_frame_equal wrappers around
pandas.testing asserts and replace all call sites with direct
pandas.testing calls carrying explicit tolerances. The wrappers silently
rewrote caller tolerances (check_less_precise -> atol=rtol=1e-3,
otherwise atol=rtol=1e-5, overriding any atol/rtol the caller passed)
and carried dead pandas<1.1 version logic; the minimum supported pandas
is 1.5.0. Simplify assert_index_equal to a thin compatibility shim that
passes check_freq=False only on pandas >= 3.1.0, where the keyword
exists; full removal of that wrapper must wait until pvlib requires
pandas >= 3.1.0 (the CI minimum environment pins pandas 1.5.0). Add a
regression test pinning the shim's version gate.

Tests: 1135 passed (1133 affected-suite tests identical to the pre-change
baseline plus 2 new shim tests), 35 skipped, 75 deselected
(remote_data), 0 failed. flake8 --diff on the changed lines is clean.
@cwhanse

cwhanse commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

this looks useful and is appreciated. however, we require the PR checklist at the head of the PR leading comment. Please add the checklist.

- test_sodapro.py: drop unused pvlibDeprecationWarning import (F401)
- test_location.py: reformat the six clearsky expected-DataFrame
  constructions to a flake8-clean layout (E126/E202/E203/E131 were
  pre-existing style issues surfaced by --diff on the touched hunks);
  all six frames verified numerically identical via assert_frame_equal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify or remove assert_*_equal wrappers around pandas.testing functions

2 participants