Skip to content

fix(replay): log the learned rate grids and the battery temperature forecast the plan reads - #6243

Open
chalfontchubby wants to merge 3 commits into
mainfrom
fix/replay-rate-grid-temperature
Open

chalfontchubby wants to merge 3 commits into
mainfrom
fix/replay-rate-grid-temperature

Conversation

@chalfontchubby

Copy link
Copy Markdown
Collaborator

Opened by Claude on behalf of @chalfontchubby.

Summary

Forward replay (tests/replay_forward.py) replans a day of predbat.log from a debug yaml. Two inputs that the plan now reads were not in the log, so a replay that ran past them planned with different battery ceilings from live:

  • Learned battery rate grids (feat(battery): learn charge and discharge rate grids (SoC x battery temperature) #6212). These are refitted once a day from the history scan, which a replay does not run. A new Replay input: rate grids learned changed {...} line is written whenever either grid object is replaced (and once a day). The replay installs it.
  • Battery temperature forecast. This is rebuilt every cycle from the temperature history, which a replay does not have. Until now the replay kept the yaml's forecast fixed while its clock moved on. A new Replay input: battery temperature forecast changed, whole degrees every 5 minutes from HH:MM [[deg, n], ...] line now records it.
    • The line holds the whole degree the rate grids read at each step, not the raw reading. The plan reads the forecast only through rate_grid_curve, which gives a reading and its whole degree the same curve (an exact 0 stays 0). A test checks this across freezing and past both ends of the grid. Logging raw readings would mean a new line nearly every cycle, because the trend moves each step.
    • A new line is written only when moving the last line on to now (shift_grid_temperatures) would not give the live forecast, plus once a day. The replay rebuilds each run's forecast with the same function, so the two agree by construction.
    • Both lines carry past runs the replay drops and across midnight. Before the first line arrives, the yaml's own forecast moves with the clock, which also helps older logs.

PV calibration (#6231/#6233) needs no change: it runs upstream in the Solcast component, and the existing PV forecast line already logs the calibrated forecast.

The two "last logged" copies are added to DEBUG_EXCLUDE_LIST, so they stay out of debug dumps.

Testing

  • New round-trip tests write lines with the live logging code, replay them, and check the replay ends up with the same forecast and grids as live. They cover midnight, a dropped run, a carried line, a short forecast and an emptied one.
  • ./run_all --quick passes apart from web_pv_calibration_chart. That test also fails on unmodified main today: test 10 expects 2026-11-17 00:00 as the next scan and gets 03:00, so it depends on the wall clock. It is not touched here.
  • replay_roundtrip_day and debug_cases pass, and all pre-commit lint hooks pass.

Known limit

On the day the clocks go back, a line carried past a dropped run within the repeated hour can be taken for the previous day's. The existing rates and dispatch carry has the same limit.

🤖 Generated with Claude Code

chalfontchubby and others added 2 commits November 16, 2026 23:05
…orecast the plan reads

Since #6212 the plan reads each step's charge and discharge ceilings from the
rate grids, by that step's forecast battery temperature. Neither reached the
log, so a forward replay kept the yaml's learned grids through the daily refit
and held the yaml's temperature forecast (minutes from its own moment) still
while the clock moved on.

- "Replay input: rate grids learned changed {...}": both learned grids on a
  change, rows as runs; zero_row is derived on read as fit_rate_grid does.
- "Replay input: battery temperature forecast changed, whole degrees ...":
  the whole degree rate_grid_curve reads at each step (the plan reads the
  forecast only through it, so degrees plan exactly as readings), written
  only when moving the last line on to now would not give the live forecast.
- The replay applies both, carries them past dropped runs and across
  midnight, and moves the yaml's own forecast on with the clock until the
  first line.
- The two replay signatures are left out of debug dumps.

PV calibration (#6231/#6233) needs nothing: the PV line already logs the
calibrated forecast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… compared over their own length

Review follow-ups: the rate-grid change check compares the learned grid objects
instead of encoding both grids every cycle; a battery temperature forecast
shorter than the horizon no longer logs a line every cycle; the replay's
temperature line is set only once there are runs to replay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@springfall2008 springfall2008 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated comment from the triage bot.

Reviewed 68841aaa against main. No correctness defects found: 1 inline finding (low, documentation), none in this body.

What I checked, and why I believe the design holds:

  • Degrees plan as readings. battery_temperature_prediction has only two consumers - Prediction.run_prediction() and the kernel mirror in prediction_kernel.py - and both reach it only through rate_curves() -> rate_grid_curve(), which applies grid_temperature_index() itself. So the exact-0 branch (zero_row), the -1..1 non-zero mapping to +/-1, and the clamping past both ends of the grid all agree on both sides. The new equivalence test covers None, manual-only, combined and single-row grids across both ends of the range.
  • The logged grids can reach the plan. Every Prediction takes its grids from base.rate_grids(), which rebuilds when the learned object identity changes; apply_logged_rate_grids() installs new objects, so the install is not dead state. The "grids are replaced, never mutated" premise the signature comparison relies on holds at the one place they are assigned (battery_rate_grid_learned[kind] = learned).
  • Ordering in the live loop. log_replay_rate_grids() runs after fetch_inverter_data() (which performs the refit) and before the plan, so the logged grids are the ones that cycle planned with. The temperature logger reads battery_temperature_prediction after fetch_sensor_data() refreshed it.
  • Carry, dropped runs and midnight. Both new stores are keyed on is not None, so a logged {"charge": None, "discharge": None} and an emptied [] forecast propagate (they are changes, not absences). roll_over_midnight() shifts replay_temperature_line and apply_run() re-anchors a carried line by at most one day; both start values derive from minutes_now, which update_time() rounds to a multiple of PREDICT_STEP, so shift_grid_temperatures()' floor division is always exact rather than approximate.
  • replay_clock() renders midnight_utc + minutes_now, and minutes_now is recomputed against a midnight carrying the current offset, so the logged HH:MM round-trips on ordinary days; the clocks-back case is the one the PR body already calls out.
  • Debug dump. Both new attributes are in DEBUG_EXCLUDE_LIST, and both needed to be (one holds references to the learned grids).
  • Tests are red without the change (the helpers and appliers do not exist on main), and they cover midnight, a dropped run, a carried line, a short forecast and an emptied one. Both runners were already registered in TEST_REGISTRY.

Comment thread apps/predbat/fetch.py
if degrees and shift_grid_temperatures(logged[2], self.minutes_now - logged[1], len(degrees) * PREDICT_STEP, PREDICT_STEP) == current:
return
self.replay_temperature_logged = (self.midnight_utc, self.minutes_now, degrees)
self.log("Replay input: battery temperature forecast changed, whole degrees every {} minutes from {} {}".format(PREDICT_STEP, self.replay_clock(self.minutes_now), value_runs(degrees)))

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated comment from the triage bot.

This is the only Replay input: ... changed line that does not go through log_replay_changed(), and it can repeat many times in a day - which makes tools/debug-journal.md's documented rule ("the signature includes midnight_utc, so each line logs once a day at most") wrong for it.

log_replay_changed() puts midnight_utc in the signature, so rates, sessions, dispatches, cars, inverter state and the new rate grids are all content-signature compared and forced to re-log once per day. This one instead content-compares the shifted previous line against the current forecast. That is required for the suppression to work (a plain signature of degrees would differ every cycle), but it has two consequences worth writing down:

  • A repeated line here means "some step's degree differs from the shifted last line", not "the value changed" and not "a new day". A future log reader applying the journal's rule will misread it.
  • The shared day-rollover and warn-once behaviour in log_replay_changed() does not reach this line, so a later fix to either will silently miss it - which is how the journal rule became wrong here in the first place.

Failure scenario: someone triaging a replay mismatch counts hundreds of battery temperature forecast changed lines and concludes the forecast moved materially hundreds of times that day, or treats a repeated line as a midnight boundary and mis-orders the day's runs.

Cheapest fix: amend the journal sentence (and/or this docstring) to say this line is compared against its own shifted predecessor rather than a signature, so unlike every other Replay input: ... changed line it may repeat within a day.

What I could not confirm: the real-world frequency, as I did not run the code. predict_battery_temperature() seeds step 0 from the newest reading with a backward difference (history[0] - history[step]) while later steps use the 24h-ago profile, so consecutive forecasts are not exact shifts; the per-step difference is roughly the size of the recent trend, small while the battery temperature is flat and larger during a warm-up.

@springfall2008 springfall2008 removed the BOT_REVIEW Trigger an autotriage label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants