Skip to content

fix(spec): a row the data empties reaches the solver, as in core linopy - #1164

Open
FabianHofmann wants to merge 18 commits into
masterfrom
ccr-1868f888-l1tlxg-1147
Open

FabianHofmann wants to merge 18 commits into
masterfrom
ccr-1868f888-l1tlxg-1147

Conversation

@FabianHofmann

@FabianHofmann FabianHofmann commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1147.

Changes proposed in this Pull Request

Note

The following content was generated by AI.

Depends on #1183 (merged into this branch; merge #1183 first). The spec layer now follows its rule: linopy never silently drops a present constraint row. A row that the data leaves without a variable term is built like any other row and reaches the solver as constant sense constant. If it cannot hold, e.g. 0 == 5, the solver reports the model infeasible. from_spec no longer refuses, warns about or drops such a row.

  • The issue's example builds in both cases. With load[0] = 0 the row 0 == 0 is kept and the model solves optimal. With load[0] = 5 the row 0 == 5 is kept and the solve ends infeasible, instead of a build-time SpecDataError.
  • Present but empty (kept): a sum whose terms are all absent or where:-masked (also a sum_back window, a sum over a dimension without members, a shift without edge= inside a sum), the vacated end of a shift with edge=0, a coefficient of 0, a masked variable that reads its missing: number.
  • Absent (no row, label -1): a where: on the constraint, an absent value read outside a sum (a missing: absent parameter, a masked missing: absent variable, a shift without edge=, an unmapped at()). This is the line core linopy draws: a NaN constant takes the row, a sum of absent terms is 0. missing: semantics are unchanged.
  • Code: _check_live (master's refusal) is replaced by _present, which only skips a constraint whose rows are all absent (fix(spec): a constraint of masked variables alone adds no rows #1154's case). Core add_constraints masks the absent rows, as before.
  • Also fixed: master's _check_live looked only at the side it gathers variables onto. It refused sum(y, over=k) + 1 <= x where only the sum was empty, although x is live there. The live-other-side case pins this.
  • Removed from the earlier revision: EmptiedRowWarning, the DROPPED_ROW_TOL check and its error quoting, the pruning of terms the data scales by 0 (Context.zeroes, _without_zeroed_terms), the objective re-run (_forget_terms) and the product hunk from 7ea5b20. fix: a constraint row without a nonzero term reaches the solver #1183 makes them unnecessary.
  • Docs: spec-data.md, overview.md, the spec notebook and the from_spec docstring describe the new behaviour. linopy now matches mathspec here: at cc531151, mathspec says such a row exists once the data is attached (docs/reference/language/absence.md, docs/about/relations-as-linear-maps.md). So the docs no longer call this a difference, and the spec: rows emptied by absence are dropped silently; revisit strict refusal #993 links are gone.
  • Release notes: one bug-fix line. The "Limits" line loses its row sentence. fix(spec): a window sum over absent terms is 0, like sum #1174's line loses its "refused" clause. fix(spec): a constraint of masked variables alone adds no rows #1154's line now says "where every row of it is absent", since an all-masked sum now gives present rows.
Validation and reviewer notes
  • History: merged origin/master (conflicts in release notes, builder.py and context.py, resolved by keeping both sides), then merged origin/fix/1170-keep-empty-rows (fix: a constraint row without a nonzero term reaches the solver #1183, 926e6f4), then one rework commit and one review follow-up commit (docs, tests). Nothing was force-pushed. The two merge commits name the local branch rework/1164-follow-1183.
  • PYTHONHASHSEED=0 pytest -n 4 linopy --doctest-modules test with MATHSPEC_EXAMPLES at mathspec cc531151: 8256 passed, 91 skipped, 1 xfailed. Spec tests alone: 703 passed, 1 skipped, 1 xfailed.
  • ruff check ., ruff format --check . (0.16.10): clean. mypy . as CI runs it (Python 3.12, wheel with [dev,spec]): 3 errors in test/test_spec_operators.py:1099-1100, identical on master (from fix(spec): an internal grouped sum onto a sparse key no longer warns of a dense grid #1192).
  • Tests in test/test_spec_builder.py:
    • test_a_row_the_data_empties_reaches_the_solver (11 cases × dense/sparse): the row at t=0 keeps its label, ncons counts it, and the solve is optimal where the row holds and infeasible where it cannot.
    • test_a_row_a_missing_number_empties_reaches_the_solver: missing: numbers, holding and failing. A UserWarning fails the test, so master's "trivially true" warning cannot return.
    • test_a_dimension_with_no_members_empties_a_summed_row: 0 <= 0 and 0 >= 5 are each one row.
    • test_a_row_of_nothing_but_an_absent_window_reaches_the_solver_like_sum replaces fix(spec): a window sum over absent terms is 0, like sum #1174's refusal test.
    • test_an_absent_row_the_data_would_empty_stays_absent (× dense/sparse): a where: and a bare vacated shift keep t=0 absent, so 0 == 5 never reaches the solver.
    • The example sweep no longer expects operators/shift_by_parameter.yaml to raise.
    • Regressions, with this PR's test_spec_builder.py run against each tree: 24 fail on the real base (master 912a8c9 + fix: a constraint row without a nonzero term reaches the solver #1183, merged tree f97a50c), 32 on master. The 8 extra on master are zero-coefficient-holds, zero-coefficient, written-order and zero-coefficient-sum (× dense/sparse). They pass on master + fix: a constraint row without a nonzero term reaches the solver #1183, so they test fix: a constraint row without a nonzero term reaches the solver #1183's core change, not this PR; they stay as spec-level guards. The absent-row guards pass everywhere, as expected.
  • Departure from the issue's "Expected Behavior": it asked to drop a holding row with a warning and to refuse a failing one at build time. The maintainer rule from fix: a constraint row without a nonzero term reaches the solver #1183 replaces both: the row is kept, and the solver judges it.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release-notes.md of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.

Generated by Claude Code

A row that the data empties of every variable term was dropped with a
"trivially true" warning only when every variable declared a missing:
number. A row emptied by a shift's stated edge, e.g.
sum(shift(x, along=t, offset=1, edge=0), over=k) == load at the first t,
was refused even when it read 0 == 0.

Count such a row as valued when the spec states every value it can be
left with: variables without where: (or with a missing: number), shifts
with an edge= number, and data that is never absent. A valued row whose
sides differ is now refused as one that cannot hold, naming the first
comparison (e.g. 0 == 5). A row emptied by an absent term, such as a
shift without edge=, a window, a lookup or a sum over no member, is still
refused as before.

Fixes #1147

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8L63c3o8u6QQvr6TGN
@codspeed

codspeed Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing ccr-1868f888-l1tlxg-1147 (0d6be46) with master (912a8c9)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

A shift whose offset parameter can be absent, and a window, keep an
emptied row refused even under edge=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8L63c3o8u6QQvr6TGN
Port of 1ea671d from spec/prune-unbuilt onto master.
A sum of absent terms counts as 0.
A row the data leaves without a variable term reads constant sense constant.
It is dropped with linopy.EmptiedRowWarning where it holds and refused, quoting the row, where it cannot.
A term the data scales by zero is pruned, so 0 * x >= 5 is refused instead of built.
The objective is evaluated again without that pruning when it would lose every variable.
1ea671d's _variable hunk is left out: it made an objective over a fully masked variable unsolvable.
The product of two terms converts one constant side only, as in 7ea5b20.
A row emptied of variable terms compared its constants exactly, so 0.1 + 0.2 == 0.3 was refused.
It now holds within DROPPED_ROW_TOL, the tolerance Model.restrict uses for rows it drops.
An integral constant is quoted without a fraction, e.g. 0 == 5.
The refusal hint points to the data that empties the row.
Tests cover a left side emptied against a right side that keeps variables, and an objective the data zeroes, also through a named expression.
A row the data leaves without a variable term is dropped with EmptiedRowWarning where it holds and refused where it cannot.
The overview and the spec data guide described the old refusal.
EmptiedRowWarning is listed with the other warnings.
@FabianHofmann FabianHofmann changed the title fix(spec): drop a row emptied by a stated edge value when it holds fix(spec): check a row the data empties against its constants Nov 18, 2026
@FabianHofmann
FabianHofmann marked this pull request as ready for review November 18, 2026 11:12
linopy dropped a labelled row whose terms were all zero or absent, so
0 * x >= 5 solved "optimal". The matrices, ncons and duals now keep every
row with a label, and the LP writer gives such a row the single term
+0.0 times the first variable.

solve and to_file warn once about such rows whose right-hand side cannot
hold. The fix also corrects Xpress IIS row names on the LP path, a
persistent update of a row without a stored term, and dualize of an
empty row.
…sible

GLPK writes INTEGER EMPTY or INFEASIBLE (FINAL) for a proven infeasible
problem. linopy did not map these, so the status was unknown, the solution
was parsed anyway and the status became ok. An empty row that cannot hold
skips GLPK's presolve and exposed this.
Only the LP writer adds the 0 * first-variable term to a row without a term.
The public Constraint.to_polars returns the same frame as on master.
solve warns after sanitize_zeros, so it also names rows that become empty.
The solver-grid test runs on a dense model; one HiGHS case covers sparse.
linopy skips terms with a zero coefficient everywhere, and the solver
handles near-zero ones. Model.solve no longer prunes |c| <= 1e-10 in
place; passing sanitize_zeros warns (FutureWarning) and has no effect.
Constraint(s).sanitize_zeros() stays as an explicit step.

Remove the 'cannot hold' warning with its Model.to_file wrapper: the
solver reports such a model infeasible.
The method only pruned |c| <= 1e-10 in place: exact zeros are skipped
everywhere, densify_terms() compacts expression terms, and the solver
handles near-zero coefficients. Tests that built rows without a term
through it now build them directly.
…1183

# Conflicts:
#	doc/release-notes.md
#	linopy/spec/builder.py
#	linopy/spec/context.py
Follow #1183: spec no longer refuses, warns about or drops a present row
without a variable term. It builds the row, and the solver judges it, so
0 == 5 makes the model infeasible. Absent rows stay absent: a where: mask,
absence spread outside a sum, a masked variable or a vacated shift.

This removes the drop and warn machinery of this PR: EmptiedRowWarning,
the DROPPED_ROW_TOL check, the pruning of terms the data zeroes and the
objective re-run it needed. It also removes master's _check_live refusal.
@FabianHofmann FabianHofmann changed the title fix(spec): check a row the data empties against its constants fix(spec): a row the data empties reaches the solver, as in core linopy Nov 19, 2026
…opy now does

mathspec at cc53115 says such a row exists once the data is attached, so
linopy matches it. Drop the "differs" and "mathspec skips" claims and the
#993 links.
Pin that a live variable on the other side no longer gets the row refused,
that a missing: number row warns no more, and run the absent-row guard on
the sparse backend too.
Shorten the release notes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

spec: a row emptied by the data is refused even when it holds

2 participants