Skip to content

feat/spot-rates New spot rate sensor & documentation - #6250

Open
gcoan wants to merge 1 commit into
springfall2008:mainfrom
gcoan:spot-rates
Open

gcoan wants to merge 1 commit into
springfall2008:mainfrom
gcoan:spot-rates

Conversation

@gcoan

@gcoan gcoan commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Generic spot rate settings: metric_spot_import / metric_spot_export

Discussion item #5285

Requirement

metric_octopus_import / metric_octopus_export currently accept any sensor that publishes its rates in raw_today / raw_tomorrow attributes, and that is how the Nord Pool, Czech and Frank Energie examples work. But the name implies UK Octopus only, and the required data format was not documented anywhere (you had to infer it from the examples).

What changed

New settings (apps.yaml)

  • New metric_spot_import and metric_spot_export configuration entires in apps.yaml for feeding custom spot rates into Predbat. They read rates only from a sensor's raw_today / raw_tomorrow attributes. They are added to APPS_SCHEMA.
  • They are read independently of the Octopus handling. Agile detection (futurerate_adjust_auto) and the automatic AgilePredict region lookup only look at metric_octopus_*, so a spot sensor can never trigger either. futurerate.py is unchanged.
  • Slot keys are detected from the first slot, using a copy of fetch_octopus_rates()'s logic, so a sensor gives identical rates under either name.
Slot keys Unit Predbat
start, end, value currency/kWh ×100
from, till, price currency/kWh ×100
from, to, rate pence/cents as is
valid_from, valid_to, value_inc_vat pence/cents as is

Code

  • fetch.py: new fetch_spot_rates() and spot_rates_warn_once(). New elif branches in the import and export chains of fetch_sensor_data(). Precedence is rates_*_octopus_url, then metric_octopus_*, then metric_spot_*, then Energi Data Service, then Strømligning, then rates_*. A one-off warning is logged if both metric_octopus_* and metric_spot_* are set.
  • octopus.py: fetch_octopus_rates() gains an optional spot_arg parameter, used only for the deprecation warning below. The rates it returns and the logs it writes are unchanged (checked against the original for every attribute layout).
  • octopus.py: OctopusAPI.automatic_config() no longer auto-configures metric_octopus_import / metric_octopus_export when the user's apps.yaml sets a lower-ranked source for that direction: metric_spot_*, metric_energidataservice_* or metric_stromligning_*. Previously the auto-discovered Octopus entry outranked them silently. For a skipped import, metric_standing_charge is also left alone. A note is logged once, worded like set_arg_auto()'s existing "keeping your apps.yaml setting" message.
  • predbat.py: spot_rates_warned set, initialised in reset().

Docs

  • energy-rates.md: new section on the spot sensor data format (key and scaling table, example attribute, the "feeding cents makes prices 100× too high" pitfall), the precedence order, and the auto-config behaviour. The Czech example now uses the new names. Frank Energie now has an apps.yaml example, and its template publishes raw_today instead of prices (docs only: anyone who copied the old template keeps working under metric_octopus_export).
  • apps-yaml.md: metric_spot_import / metric_spot_export listed, with the old names flagged.

Deprecation

Using raw_today / raw_tomorrow attributes through metric_octopus_import / metric_octopus_export still works but is now deprecated. Once such rates have loaded, a one-off warning names the sensor and the new setting to use. It does not fire for:

  • genuine Octopus sensors (the Predbat Octopus component's own sensors and the Octopus Energy integration publish rates)
  • sensors using the prices attribute
  • the Octopus day-rate event sensors
  • metric_octopus_gas

No removal date is set.

Behaviour change for existing users

Only the automatic_config() guard: a user running the Octopus component with auto-configure and metric_spot_*, metric_energidataservice_* or metric_stromligning_* in apps.yaml will now get their own source used, where before the Octopus one silently won. Nothing changes for anyone else.

Tests

New tests/test_fetch_spot_rates.py, registered in unit_test.py as two tests:

  • fetch_spot_rates (13 cases): all four key layouts and their scaling, raw_tomorrow, missing/empty/None sensors, rates / prices / Octopus event sensors not being read, io_adjusted never touched, identical rates through fetch_octopus_rates(), a metric_spot_* sensor never detected as Agile and giving no AgilePredict region, the one-off helper, and the deprecation warning (fires once per direction, silent for real Octopus / prices / no spot_arg / failed loads).
  • octopus_auto_rates (6 cases): the auto-config guard across spot, Strømligning and Energi Data Service, one note across re-runs, user metric_octopus_* regular expression still replaced as before, and no apps.yaml snapshot available.

No existing tests were amended.

Not covered by a unit test: the precedence order and the both-names-set warning inside fetch_sensor_data(), which can't be reached from the test fixture without a larger refactor.

Checklist

  • ./run_all --test fetch_octopus_rates --test fetch_spot_rates --test octopus_auto_rates all pass
  • ./run_pre_commit passes

@github-actions github-actions Bot added the BOT_REVIEW Trigger an autotriage label Nov 17, 2026

@springfall2008 springfall2008 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated comment from the triage bot.

Reviewed at a4d606a (PR head, matches headRefOid). 5 findings inline, 1 in the body. The new metric_spot_* read path, the key/scaling table, the deprecation warning and the automatic_config() guard all behave as the PR body describes; what follows is one unwired consumer, one untested wiring change, and three duplication/diagnostic points. Nothing was run - this is a reading of the checked-out head only.

apps/predbat/compare.py:229 - the Compare feature does not know about metric_spot_import / metric_spot_export

compare.py has a rate-source chain that enumerates every other source - rates_import_octopus_url, metric_octopus_import, metric_energidataservice_import, metric_stromligning_import_today/_tomorrow, rates_import - and this PR adds no metric_spot_* branch to it (nor to the export side, compare.py:269).

Failure scenario: a spot-rate user (the Czech/Nord Pool or Frank Energie case this PR is written for) follows docs/compare.md, which lists the sources it supports, and writes a compare_list entry with metric_spot_import: 'sensor.current_buy_electricity_price_raw'. The key matches no branch, so the chain falls through to self.log("Using existing rate import data") and the row is costed against the live tariff while being labelled as the compared tariff - a silently wrong comparison result rather than an error, which is the worst of the three outcomes for a feature whose whole purpose is a what-if cost. The deprecated metric_octopus_import name still works in compare_list, so this is a gap rather than a regression, but it is the one consumer of the rate settings the PR leaves on the old name while docs/compare.md continues to advertise it. I traced both chains and confirmed there is no metric_spot match and no warning on the fall-through.

Comment thread apps/predbat/fetch.py Resolved
Comment thread apps/predbat/octopus.py Resolved
Comment thread apps/predbat/octopus.py Resolved
Comment thread apps/predbat/fetch.py Resolved
Comment thread apps/predbat/fetch.py
scale = 100.0
if rate_key not in data_all[0]:
rate_key = "value"
if rate_key not in data_all[0]:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated comment from the triage bot.

When data_all is non-empty but the layout matches none of the four documented keys, this falls through to rate_key = "price", from_key = "from", to_key = "till" and resets from_key/to_key even if the earlier block had already chosen start/end. minute_data() then finds no from key on any slot, skips every item and returns {}, and the only thing the user sees is fetch_sensor_data()'s generic Error: metric_spot_import is not set correctly in apps.yaml, or no energy rates can be read.

Failure scenario: a sensor whose slots are start / end / price - a plausible Nord Pool variant, and not in the table this PR adds to energy-rates.md - or any sensor with a mistyped key. Block 2 selects start/end with scale = 100, block 3 leaves rate_key as value (absent), and block 4 overrides from_key/to_key back to from/till (also absent), so nothing parses. The user is told their sensor is "not set correctly" with no indication that the key names are the problem, while the docs present the four layouts as the complete contract.

The information needed is in hand at this point: logging sorted(data_all[0]) once (via spot_rates_warn_once(), so it does not repeat every cycle) when rate_data comes back empty would name the mistake directly, and would also catch the unit confusion the docs call the most common error.

@springfall2008 springfall2008 removed the BOT_REVIEW Trigger an autotriage label Nov 17, 2026
@gcoan

gcoan commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

All six findings were fair. I've addressed them in the follow-up commit, and also guarded Kraken, which had the same silent out-ranking as the Octopus component.

  • One ordered list of rate sources: RATE_SOURCES in const.py, with rate_source() and rate_sources_below_octopus() in utils.py. fetch_sensor_data(), Compare and the auto-config guards all use it, so the order lives in one place.
  • Selection moved into fetch_import_rates() / fetch_export_rates() so it can be unit tested. The code is moved, not rewritten: I compared the old inline chains with the new methods across every combination of configured keys and fetcher results (1,024 scenarios, no differences).
  • Compare now supports metric_spot_import / metric_spot_export.
  • ComponentBase.set_arg_unless_user_set() replaces the direct self.base reads in OctopusAPI.automatic_config(), and is also used by Kraken.
  • rate_slot_keys() in utils.py replaces the duplicated slot key detection, and an unsupported layout now gives a one-off warning naming the keys it found.
  • New tests: rate_sources, fetch_rate_sources and auto_config_precedence (the real automatic_config() followed by the real fetch, for spot, Energi Data Service and Strømligning), extra cases in fetch_spot_rates and octopus_auto_rates, a Compare test, and Kraken wiring tests. The shared Kraken test fixture now gives its mock base an empty args_from_apps_yaml, as a real PredBat has.

Behaviour change to be aware of: EDF and E.ON (Kraken) users who set metric_spot_, metric_energidataservice_ or metric_stromligning_* themselves will now get their own source used, as Octopus users already do. Nothing changes for anyone who has not set one of those.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants