Repository navigation
feat(smt): encode features valued by an expression over other features - #1617
Merged
Merged
Conversation
…e with explore Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
A feature valued by an expression over other features keeps its variable in every state but is never framed: each state defines it by the expression over the state's values, and a read within a move substitutes the expression over the move's current environment, through features so valued in turn. An unresolvable footprint, a narrowing declared type, an assignment to such a feature and a release of one are refused naming the reason. A result of a feature the interpreter holds as a sequence of one value is spelled as such. Co-Authored-By: jason.han <hanhuijun@gmail.com>
The six conformance fixtures holding such a feature state a solver budget of 20 moves, within which every run of theirs ends, so cvc5 decides their fixed outcomes within the default timeout as z3 does. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…e referee The positional-redefinition fixture lists the redefined out parameter, which the interpreter holds as a one-element sequence, among its outcomes; the referee's oracle reads a Sequence value, and the two-copy spelling shares the result spelling with the witness decoder. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…e; refuse a derived value that may be undefined A merged environment (after an if, after each loop iteration) now keeps the written mark, so a loop condition reading a feature valued by an expression sees the iteration's write of what it reads rather than the state's value. A feature valued by an expression that may have no value (a side condition such as a divisor, or a read of a feature holding none) is refused: the interpreter settles it when the performance completes and fails then even unread, which the relation did not carry. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…eatures Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # docs/project/spec-compliance.md # internal/exec/smt/encode.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The SMT engine refused every action holding a feature valued by an expression over other features (
attribute y : Integer = x + 1): the states carried only values written, so before the refusal the encoder heldyat its initial value and provedy = 1where the interpreter computesy = x + 1at every read (explore:y ∈ {2, 3},z ∈ {12, 13}on the reproducer). The refusal was sound but left all such actions outside SMT coverage — the referee stood at 17 encoded / 53 refused, the six most recent refusals being the fixtures that exposed the gap.This PR encodes the binding and keeps refusing only what the encoding does not cover.
Design
Representation: a per-state defined term, plus substitution where a move's writes make the state's value stale.
y = exprkeeps its state variabley@k(so results, witnesses,-check-property, enumeration and the two-copy sensitivity queries read it exactly as any other feature), but it is never framed byfeatureValues: every statek = 0 … Kassertsy@k = ⟦expr⟧(s_k), the expression evaluated over the state's own feature variables (Encoding.deriveValues). A chain (v = y * 2) definesv@kovery@k.ybefore any write usesy@k; after a write in the same move (env.written, kept across the merge ofifbranches and of loop iterations) the read is substituted by⟦expr⟧over the move's current symbolic environment (env.evaluate), so a statement that writesxand then readsyin one body sees the fresh value, as the interpreter's read-time evaluation (deriveBodyCell) does; chains substitute recursively. Guards are evaluated over the post-body environment, so a guard readingyafter a write ofxsees the derived term too.Encoding.checkDerived): the interpreter settles every still-tracking binding when the performance completes (freezeBodyCells) and fails then even if nothing read it (y : Rational = 10 / x; assign x := 0ends inerror: division by zero), a failure the relation does not carry. Every encoded derived feature is therefore always defined.Result.Inputsno longer lists it); state 0 asserts the same defined term over the initial values, which is where-check-inputfree features and-check-assumemeet it — an assumption overyconstrainsexprover the released inputs it reads (tested).out p : Integerparameter, see below) is spelled as such (Output.Sequence,[2]), so the solver's outcome identities matchexplore's.Loops and
-check-unroll. Nothing specific: a loop condition is evaluated over the current environment at every unrolled iteration, which gives ity@kbefore the body writes and the substituted term after; the past-bound flag is untouched.-check-unrollonly changes how many iterations are evaluated.Result and
-check-propertyreads. Both read the state variables: a property overyat stepkreadsy@k, which the defined term ties tox@k; a result readsy@K.Outputs()is unchanged apart from the sequence spelling.What is refused, each naming the feature and its reason
Footprint.Dynamic: a chain through a computed object);attribute y : Natural = x + 1): the interpreter checks the bound value against the type at each read, which the encoding does not perform;attribute y : Rational = 10 / x, or one reading an optional feature): settled by the interpreter at completion, failing then even unread;assign y := 7): the interpreter then holds the written value until the next write, a mark the encoding does not carry;-check-inputreleasing a derived feature (free a feature it reads instead).y = w + 1; w = y * 2) never reaches the encoder: the interpreter refuses it at the start of the performance (cyclic feature value dependency: y), which-engine smtreports as not covered; pinned byTestDerivedFeatureRefusals/cycle.Finding left alone: a bare
outparameter is held as a one-element sequenceThe interpreter holds
out yBase : Integer = 5(no multiplicity, noattributekeyword) as the sequence[5]and spells it so inexploreand in run results;out yBase : Integer[1] = 5andout attribute r : Integerare scalars, and anoutparameter without a value of its own that redefines one by position (out yRace : Integer;overout yBase : Integer = x + 1) is a scalar even whenyBaseis[2]. That isinternal/exec/runtimebehaviour, unchanged here; the encoder now spells such a result as the interpreter does (Output.Sequence) so the referee compares like with like, andaction_order_positional_bound_read_of_racing_writer.expected.jsonlistsyBaseas aSequenceamong its outcomes (it listedyBaseunderorderDependentbut not underoutputs, which the referee's sensitivity check requires). Whether a bareoutparameter should be scalar-valued is the maintainer's call.Specification basis
KerML 1.1 §7.3.4.2 Feature values (a feature-value binding
= exprvalues the feature by the result ofexpr; an initial value:= expris evaluated once), §7.3.4.5 (a redefining feature valued by nothing of its own is bound as the redefined one is) and §7.4.2 (parameters redefine positionally) — the interpreter implements the binding as read-time evaluation (deriveBodyCell). Moves the SMT model-checking row ofdocs/project/spec-compliance.md(coverage of feature values;🧩 Extensionunchanged) and the order-dependence row'ssmt/encode.gopointer (it named the refusal).Behaviour changes
sysml -engine smt(and%engine smt, the gRPC engine) now encodes actions holding features valued by an expression over other features; the refusala feature valued by an expression over other features is computed as it is read; the states carry only values writtenis gone. No fixture, golden or test pinned that message (only the changelog fragmentbound-feature-read-marks.fixed.mdand the compliance row mentioned it).a feature valued by an expression whose reads the lowering cannot resolve is not encoded;a feature valued by an expression that may have no value (a side condition, or a feature it reads holding none) is not encoded: the interpreter settles it when the performance completes, failing then even unread;…, with a declared type narrowing its sort, is not encoded: the value is checked against the type at each read;assignment to <f>: a feature valued by an expression over other features holds the value written until written again; …;-check-input <f>:action <a> values it by an expression over other features; free a feature it reads instead.Result.Inputsof ansmtanswer no longer lists a derived feature among the inputs (it was listed pinned at its starting value).[v]asexplorespells it.Fallout
.expected.jsonsolverBudget: {"moves": 20}on the six fixtures holding a derived feature: every run of theirs ends within 8 moves; at the default 40 moves cvc5 1.3.4 answersunknownto the referee's fixed-outcome query within the default 10 s solver timeout (z3 decides it in 3–7 s; cvc5 decides it in 3 s at 20 moves). This is the knob the corpus README documents for exactly that case; the referee's completion query still proves every run ends within 20..expected.jsonaction_order_positional_bound_read_of_racing_writerlistsyBase(Sequenceof oneInteger) in both outcomes, which the referee's sensitivity check requires of every feature the encoding holds.yBaseof the positional fixture is the extra feature compared)sensitivity_test.go)oracleValuereadsSequencevaluesyBaseHow it was verified
TestDerivedFeaturesAgreeWithExplorerefused the six fixtures at the first commit; it now encodes each, enumerates the solver's completed outcomes, replays every witness and asserts the solver's value set per feature equalsexplore's, withy ∈ {2, 3},z ∈ {12, 13},x ∈ {1, 2},w = 1pinned on the reproducer; the seventh case holding a derived feature is a calculation with no flow (ErrMalformedFlow, logged, not a pass).TestDerivedFeatureInGuardAndProperty: a guard reads the derived value after the write (y > 3takes the branch), a chained feature follows (v = 12), a loop condition over the derived value sees each iteration's write (while y < 4 { assign x := x + 1; }completes atx = 3; y = 4; before the merge kept the written mark it was bounded), a property over the derived feature is proved (y == x + 1) and another violated (v < 12), and an assumption overyconstrains a released inputn(violated without it, proved with it).TestDerivedFeatureRefusals: the assignment, release, narrowing-type and may-be-undefined refusals, and the interpreter's cycle refusal.TestRefereeCorpuswhole corpus under z3 4.8.12 (counts above). The derived tests also pass under cvc5 1.3.4 (OPENSYSML_SMT=cvc5).internal/exec/smtruns in the suite (quick tier and themodelrace shard) withOPENSYSML_REQUIRE_SMT=1set workflow-wide, so an absent solver fails rather than skips;solve.Discoverprefers z3, which the workflow installs. The dedicated "solver gate" steps that grep for--- SKIPcoverinternal/exec/solveandinternal/frontend/replunder z3 and cvc5 but not this package — so under cvc5 the SMT engine's own tests are not gated in CI (they pass locally).go build ./...,go vet ./...,gofmt -l .empty,go test ./...,make docs-check,python3 scripts/check-doc-ids.py,python3 scripts/changelog.py check.Checklist
make testandmake lintpass locallychanges/unreleased/<slug>.<section>.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (compliance rows need nothing: the census is counted at docs build)F4,K5) in the body, docs, or changelogLink to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/176617ea9ec046fc987b09bb26763c0e
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/176617ea9ec046fc987b09bb26763c0e?variant=devin
Requested by: @HuiJun