Skip to content

Nine fixes the specification asks for, in usage sharing, data updates, Evidence keys and the JavaScript Builder - #170

Merged
jwrosewell merged 21 commits into
rcfrom
remote/java-14
Nov 21, 2026
Merged

jwrosewell merged 21 commits into
rcfrom
remote/java-14

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The Java packages differed from the specification in three places that rc has not yet fixed.

  1. An unexpected error during a check against the update URL was logged as "Exception thrown" and gave AUTO_UPDATE_REFRESH_FAILED. The data update Errors table gives that case its own message and AUTO_UPDATE_UNKNOWN_ERROR.
  2. The JavaScript Builder took any Evidence key that began with query as a query parameter, so keys such as querystring or queryx.field were passed to the script too.
  3. The IP Intelligence getting started examples printed each value without its weighting.

This draft used to carry more. Its usage sharing, Evidence key filter, minifier and builder changes reached rc through other pull requests, and the handling of a failed polling or watcher check (DU-13 and DU-14) reached rc in #146. Those are no longer part of this change.

What the specification says

All at 51Degrees/specifications d79bbe3b, the specifications rc.

  1. data-updates.md lines 488 to 489, with the Errors table row for any other error at line 522, which gives Error, AUTO_UPDATE_UNKNOWN_ERROR and "An unhandled error occurred while checking for automatic updates for engine '<Engine type name>'"

    These messages MUST also be consistent across languages to make life easier for 51Degrees support.

  2. evidence.md line 10

    Keys MUST be case-insensitive and will usually be split into <prefix>.<field>.

  3. ip-intelligence-specification/required-examples.md line 107

    Examples MUST demonstrate accessing both the value and its associated weighting:

What changed

Links are to the head, a19de29. Against rc fc6f28a the change is 9 files, 400 lines added and 30 removed.

  1. The catch-all in checkForUpdateFromUrl follows the Errors table. In DataUpdateServiceDefault lines 1347 to 1348, an error nothing else caught now goes to rc's logUnhandledError, which logs the table's message at Error with the error attached, and the method returns AUTO_UPDATE_UNKNOWN_ERROR. Before, it logged "Exception thrown" and returned AUTO_UPDATE_REFRESH_FAILED. #146 did not change this path. These are the only two lines of DataUpdateServiceDefault this draft changes.
  2. Only query. marks a query parameter. JavaScriptBuilderElement lines 498 to 505 checks for the prefix together with its separator, in any case. A key like querystring is no longer read as a query parameter.
  3. The IP Intelligence getting started examples show each weighting. PropertyHelper gains weightedValues, which gives a single value a weighting of 1, and two asWeightedString methods. A value is printed through rc's toDisplayString, so an address still prints in its literal form. GettingStartedOnPrem and GettingStartedCloud read each value and its weighting in their own code, through weightedValues, getValue() and getWeighting(), and print, for example, "gb (weighting 1.00)".

No public name is removed or renamed.

Before you upgrade

Compared with the released packages on main:

  1. An unexpected error in a check against the update URL gives a different status and message. It now returns AUTO_UPDATE_UNKNOWN_ERROR and logs "An unhandled error occurred while checking for automatic updates for engine '<Engine type name>'". It returned AUTO_UPDATE_REFRESH_FAILED and logged "Exception thrown". Update any code or log alert that looks for the old status or text. The AUTO_UPDATE_UNKNOWN_ERROR value itself comes from rc, through Log and report a failed data update check by polling or the file watcher, and go on checking #146, and a switch over AutoUpdateStatus with no default may need a case for it.
  2. The JavaScript Builder leaves out keys that only start with query. Evidence such as querystring or queryx.field is no longer handed to the script as a query parameter. Keys starting with query. work as before.
  3. The IP Intelligence getting started examples print a weighting beside each value, so their console output changes. PropertyHelper is example code, not a package, and gains three public methods.

Tests

  • DataUpdateServiceLoggingTests.UnexpectedError_LogsTheLastRow breaks the URL formatter so the check fails with an error nothing expects. It asserts AUTO_UPDATE_UNKNOWN_ERROR and an Error entry with the table's message and the error attached.
  • JavaScriptUrlTests.Parameters_PrefixWithoutSeparator_IsLeftOut checks that a key starting query without the dot does not reach the script.
  • PropertyHelperWeightingTests (4 tests) and WeightingShownTest (2 tests) check the helper and that both getting started examples read and print a weighting.

No released test is changed or deleted. The two test files that exist on main each gain one test and lose no line.

At a19de29 this draft's own DataUpdateServiceCheckFailureTests.java is replaced by rc's copy, so its three tests (polling_CallbackThrows_IsCaughtAndLogged, polling_CheckThrows_IsLoggedReportedAndRescheduled and watcher_CallbackThrows_LaterFileStillApplied) are gone. They tested the DU-13 and DU-14 handling that rc's tests from #146 now cover.

Red, run 43772345068, from before the merges with rc. Each job checked out one test commit and ran its new tests, and each failed. Two of those jobs still apply to this change: the one for fd93b77, where UnexpectedError_LogsTheLastRow first came in, and the two for 4fe3b94, the weighting tests. Parameters_PrefixWithoutSeparator_IsLeftOut arrived with the prefix change in the merge of rc at d59e036, so no red run exists for it.

Build

Run 44675391723 built the head on nine configurations: Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21. Every Build and Test job passed, and the run concluded with success. Its commit, 31e1c11, is a19de29 with only a temporary workflow file added.

macOS was left out of this build. The rc head is built with macOS before it merges into main.

Merged with rc

5b2a7cf merges rc at fc6f28a, which holds #146 and #214. In PropertyHelper both sides' imports stay, and the draft's asWeightedString now prints through rc's toDisplayString. After that merge, DataUpdateServiceDefault.java still held this draft's own handling of failed checks beside rc's.

a19de29 then removes this draft's own DU-13 and DU-14 handling, since rc's reviewed handling from #146 covers the same rules. The removed code is runPollingCheck, reportComplete, handleAutomaticFailure, logUnhandled, CallbackFailure, and the catches that passed a callback's error on. The catch-all in item 1 of What changed is kept, because #146 did not touch it.

Since then rc has moved to 4dc7a39 with #212, which changes only context7.json. A trial merge of the head with it is clean.

What remains

  1. #150 item 3. checkForUpdateFromUrl still catches every error for a programmatic check too, and logs it and returns AUTO_UPDATE_UNKNOWN_ERROR. data-updates.md lines 484 to 486 says a programmatic check passes the error to its caller. Changing that would change what a programmatic caller gets, an exception where it now gets a status, so it is left for Log the rest of the data update Errors table as the specification gives it #150.

Merging

Several commit messages on this branch describe changes it no longer makes, among them the share all query string parameters default, the minifier and the DU-13 and DU-14 handling. Please squash with a hand-written subject and body that describe only the three changes above, and leave out the merges of rc.

Head a19de29.

…ry string parameters is on by default (US-5)
… string parameter is shared unless a user turns the option off (US-5)
…D_, the session cookie among them, is still not shared (US-35, US-36)
… cookie among them, even when all Evidence is shared (US-35, US-36)
@jwrosewell jwrosewell added the remote-conformance Specification conformance work from the remote packs label Nov 15, 2026
…eported, and does not stop later checks (DU-13, DU-14)
… unexpected one as AUTO_UPDATE_UNKNOWN_ERROR, and keep later checks running (DU-13, DU-14)
…e for MD5, temporary path, other errors and writes from memory (DU-15)
…on when Content-MD5 is missing, add AUTO_UPDATE_TEMP_PATH_NOT_SET and AUTO_UPDATE_UNKNOWN_ERROR, and warn on a failed write from memory (DU-15)
… whitelist and the JavaScript builder's query parameters (EV-5)
…t, and read query Evidence in the JavaScript builder without regard to case (EV-5)
…ify option, using a cut-down single pass minifier that keeps line breaks (JB-7)
…t is read the same whatever white space is around the commas (CF-8)
…stored under device is found as Device (PR-3)
…th its weighting, read through a new weightedValues helper (IP-32)
@jwrosewell jwrosewell changed the title Specification conformance, 51degrees-java, section 14 Nine fixes the specification asks for, in usage sharing, data updates, Evidence keys and the JavaScript Builder Nov 16, 2026
jwrosewell and others added 3 commits November 19, 2026 08:02
rc wins for every item it already carries. Usage sharing (US-5, US-35,
US-36) takes rc's ShareUsageBuilderBase, EvidenceKeyFilterShareUsage and
their tests, so the tests that expected every query string parameter to
be shared by default are gone. The evidence whitelist (EV-5), the aspect
engine builder (CF-8), the typed key map (PR-3), the data update Errors
table rows (DU-15) and the JavaScript minifier (JB-7) are rc's. The
hand-written minifier, its tests and its timing tests are removed, and
copies of tests rc already has are dropped.

Kept from this branch and fitted to rc:

1. An unhandled error during an automatic check is logged with the
   Errors table's last message and gives AUTO_UPDATE_UNKNOWN_ERROR, which
   is added at the end of AutoUpdateStatus. A polling check that throws
   is logged and the next check is still scheduled, and an error raised
   by a callback is told apart from an error in the check (DU-13, DU-14).
2. The file watcher catches a failure and resets the watch key in a
   finally block, so later changes to the file are still applied.
3. The IP Intelligence getting started examples read each value and its
   weighting through PropertyHelper.weightedValues, and every weighting
   the examples print is written by PropertyHelper.asWeightedString
   (IP-32).
4. The JavaScript Builder takes a key as a query parameter only when its
   prefix is "query" followed by the separator, in any case, with a test.
Conflicts resolved:
- DataUpdateService.java: rc's comment for AUTO_UPDATE_UNKNOWN_ERROR.
- DataUpdateServiceDefault.java: rc's checkForUpdateOnTimer for both
  polling timers, and rc's report of the check in place of this draft's
  reportComplete calls.
- DataUpdateServiceCheckFailureTests.java: rc's file.
- PropertyHelper.java: both imports. unwrapValue uses asWeightedString,
  which now writes the value through toDisplayString, so an address is
  shown in its literal form.
…e of this draft's

rc's checkForUpdateOnTimer, report, reportOnTimer and the watcher's own
catch now handle a failed check, so this draft's runPollingCheck,
reportComplete, handleAutomaticFailure, logUnhandled and CallbackFailure,
the rethrow of a callback's error in checkForUpdate and on start-up, and
the try and finally around the watcher's events are removed.

What rc lacks is kept: an unexpected error in a check against the update
URL is logged with the Errors table's last message, through rc's
logUnhandledError, and gives AUTO_UPDATE_UNKNOWN_ERROR in place of
AUTO_UPDATE_REFRESH_FAILED.
@jwrosewell
jwrosewell marked this pull request as ready for review November 21, 2026 16:53
@jwrosewell
jwrosewell merged commit fe37e1b into rc Nov 21, 2026
8 checks passed
@jwrosewell
jwrosewell deleted the remote/java-14 branch November 21, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

remote-conformance Specification conformance work from the remote packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants