Skip to content

Change case in the root locale everywhere in the main code - #135

Merged
jwrosewell merged 6 commits into
rcfrom
fix/root-locale
Nov 19, 2026
Merged

jwrosewell merged 6 commits into
rcfrom
fix/root-locale

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Java's String.toLowerCase() and toUpperCase() with no argument use the default locale of the JVM. Under a Turkish or Azeri default locale a capital I lowers to a dotless i (ı), and a small i uppers to a dotted capital İ, so a name that holds one no longer matches its lower case form. On rc at ceab0888, 74 lines under src/main call one of them with no argument (76 calls in 21 files), among them:

  • JsonBuilderElement.java lowers every element and property name it writes, so IsMobile became ısmobile, a key the client-side script never reads.
  • DeviceDetectionCloudEngine.java line 381 and its five neighbours, the IP Intelligence and Location cloud engines and CloudAspectEngineBase lower the Property name to find its value in the cloud's lower case response, so DeviceId looked for deviceıd and found no value.
  • PipelineBuilder.java lowers builder and method names from a configuration file to match them, and EvidenceKeyFilterShareUsage.java lowers Evidence keys before it compares them with the keys that are never shared.

The specification asks for lower case keys compared without regard to case. json-builder.md lines 59 to 61 at d79bbe3b says "These keys MUST be written in lower case. The key of a top-level entry is the Element Data's Data Key, and the key of a sub-entry is the Property name", and access-to-results.md lines 5 to 11 says "Where string keys are used ... Lower-case SHOULD be used. String comparisons MUST be case insensitive." Lowering in the root locale gives the same key on every JVM, whatever its default locale.

#83 fixed the four calls in EvidenceKeyFilterShareUsage.java, and this draft does the same for the rest of the main code.

What changed and why

  • Every case change under src/main passes Locale.ROOT, 76 calls in 21 files (d3990945), with import java.util.Locale where a file did not import it. Nothing else in those lines changed. EvidenceKeyFilterShareUsage.java gets the same four changes and the same import as Compare usage sharing keys in the root locale #83, line for line. Compare usage sharing keys in the root locale #83 has since merged into rc (b76596b), so that file merges as no change, and what this draft adds to rc 49bc431 is 73 calls in 20 files, one of them in the getNamedKey that rc added to the Device Detection examples' KeyHelper (see Base branch).
  • A check that keeps it so. ci/check-root-locale.ps1 reads every Java file git tracks under a src/main folder and fails on any toLowerCase() or toUpperCase() with no argument, naming each line. .github/workflows/root-locale.yml runs its Pester tests and then the check on every pull request and on each push to main and rc, as the environment variable names check does. Tests under src/test are left out, because a test may set a locale on purpose.
  • The numbers the examples write for a machine to read use the root locale too (d6263a26). CompareOnPrem.java line 424 at b33daa0f wrote a distance into a row of compare-output.csv with String.format("%.3f", ...), and MetricsOnPrem.java line 739 ended a comma separated row with String.format("%.2f", ...). Under a default locale with a decimal comma, such as German, 12.346 became 12,346 and the row gained a column. ShareUsageExample made up client IP addresses with String.format("%d.%d.%d.%d", ...), so under a locale whose digits are not 0 to 9, such as Arabic in Saudi Arabia, the address was Evidence no engine can read. Each now formats in Locale.ROOT, and the two CSV numbers move into package-private methods, kilometres and twoDecimalPlaces, so a test can reach them. The examples are not published packages, so this releases nothing.

Every published package whose main code changed is released by the version per package change (#63).

Tests

Tests first, in 31153475.

  • ci/check-root-locale.Tests.ps1 has 18 Pester tests. They check that the pattern finds a bare call and leaves Locale.ROOT, another locale, equalsIgnoreCase and Character.toLowerCase alone, that each line number is given, that only src/main Java files are read, and that the script run on a throwaway repository passes with a named locale and fails, naming the line, without one.
  • JsonBuilderTests.JsonBuilder_TurkishLocale_NamesLowerCasedInRootLocale sets the default locale to tr-TR and builds the JSON for an element IpData with a Property IsMobile, and expects ipdata and ismobile.
  • MissingPropertyHandlingTests.PropertyWithCapitalI_TurkishLocale_HasValue sets the default locale to tr-TR and has the Device Detection cloud engine read DeviceId from a response that gives deviceid, and expects the value.

Each ran on ubuntu-latest on a throwaway branch holding the commit and a temporary workflow that ran the Pester tests, the check, and mvn -Pdev -pl pipeline/pipeline.jsonbuilder,device-detection/device-detection.cloud -am test with only the two Java test classes, on Java 17 as the full build's Ubuntu_Java_17 entry does.

  • Red, run 43582636250 on 3115347 (tmp/java-locale-red). The check named 74 lines and failed with "74 case changes name no locale". JsonBuilder_TurkishLocale_NamesLowerCasedInRootLocale failed with {"ıpdata": {"ısmobile": "True"}} ==> expected: <true> but was: <false>, and PropertyWithCapitalI_TurkishLocale_HasValue failed, 1 of the 3 in its class. The 18 Pester tests passed.
  • Green, run 43582645800 on d399094 (tmp/java-locale-green). The check printed "Every case change in 525 main source files names its locale", the 18 Pester tests passed, JsonBuilderTests passed 16 of 16 and MissingPropertyHandlingTests 3 of 3.
  • An earlier red, run 43582121683, ran on the runner's default JDK, 25.0.5, where the build stopped in PMD before any Java test ran, so the throwaway workflow was moved to Java 17.
  • The example numbers. Tests first, in 576615ee. CsvNumbersTest.csvNumbers_GermanDefaultLocale_FullStop sets the default locale to German and expects kilometres(12.3456) as 12.346 and twoDecimalPlaces with a full stop too. RandomIpTest.randomIp_ArabicDefaultLocale_AsciiDigits sets the default locale to ar-SA and expects 20 made-up addresses written with the digits 0 to 9. Both ran on ubuntu-latest on Java 17 through a temporary workflow, building each examples folder against the packages on Maven Central. The red, run 43666180249 (tmp/java-exlocale-red2), ran the tests against the fix's methods with the default locale, as the methods do not exist before the fix. CsvNumbersTest failed with "expected:<12[.]346> but was:<12[,]346>" and RandomIpTest with the address ٢٣٦.١٦٦.٢٢٤.١٥٠. The green, run 43666180345 (tmp/java-exlocale-green2), ran d6263a2 and both passed.
  • The check also runs on this pull request, run 43583321941, and passed, and the CI script tests ran its Pester file on Ubuntu and Windows (run 43583321945).

Changes at this head

The previous head, bb1d8da, had been built only against rc 32b3784, and by rc 49bc431, rc had changed 16 of the 30 files this draft changes, so the draft no longer merged. One commit follows it.

  • 1442b1d3 merges rc 49bc431 in, as Base branch says. It adds no test, so it has no red. Its green is the tests of what the draft touches, below, and the locale check on the merged code.

Tests of what it touches

  • Run 44452081004 built 1442b1d, with one temporary workflow commit on top of it (ced187f) and nothing merged, on nine configurations, Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21, and every job passed. In its Ubuntu Java 17 job, JsonBuilderTests passed 18 of 18, the Device Detection cloud MissingPropertyHandlingTests 3 of 3, EvidenceKeyFilterShareUsageTests 15 of 15, PipelineBuilderTests 26 of 26, and CsvNumbersTest and RandomIpTest 1 of 1 each. macOS was left out of this build. The rc head is built with macOS before it merges into main.
  • On this pull request, "Every case change in the main code names its locale" (run 44452070268) passed its 18 Pester tests and printed "Every case change in 533 main source files names its locale.", and every other check on 1442b1d passed.

An earlier full build, run 43666245300, built bb1d8da with rc 32b3784 on the same nine configurations, and every job passed.

Released tests

The released tests on main that exercise what this draft touches include JsonBuilderTests (8 on main), the Device Detection cloud MissingPropertyHandlingTests (2 on main), EvidenceKeyFilterShareUsageTests and PipelineBuilderTests. The draft changes none of them. It adds one test to JsonBuilderTests and one to MissingPropertyHandlingTests, and the run above passed every test in each class.

Base branch

rc 49bc431 is merged into the head with a merge commit, 1442b1d3. Two places needed resolving, both by keeping rc's code and naming the root locale in it, as this draft does everywhere else in the main code.

  • IPIntelligenceCloudEngine.java conflicted. rc moved the processing into processCloudEngine, which the shared Cloud Aspect Engine logic calls, and skips a Property the response leaves out. rc's version is kept, with Locale.ROOT in the weighted Property name check, as this draft had it.
  • The Device Detection examples' KeyHelper merged without a conflict, but rc had replaced the line this draft changed with getNamedKey, whose keyName.toUpperCase() named no locale. The locale check found it, and it now passes Locale.ROOT.

rc has since moved to 3bb6e71 with #158, #173 and #179. None of them adds a case change with no locale under src/main, and the draft merges with 3bb6e71 with no conflict.

Before you upgrade

Under a default locale such as Turkish or Azeri, the JSON the JSON Builder writes now has ismobile where it had ısmobile, and the cloud engines now find a value for a Property whose name holds a capital I, such as DeviceId, where they found none. The JSON Builder's keys follow "These keys MUST be written in lower case" in json-builder.md (above). Code that read one of the dotless keys under such a locale reads the lower case key instead. Under any other default locale nothing changes.

What remains

  • The new check makes a pull request fail when it adds a case change with no locale under src/main, so any draft that adds one fails that check once this merges.
  • Formatting numbers and dates with the default locale was checked in the package code at rc b33daa0f. It needs no change here, and the one case whose output a machine reads is fixed in #139.
    • 22 String.format calls name no locale. 21 of them build log, warning and exception messages for a person to read. The 22nd, "%04x" in ShareUsageElement.java line 648, writes an escape into the usage sharing record, and the Formatter documentation says of x that "No localization is applied".
    • DateSent in the usage sharing record is written by a SimpleDateFormat made with no locale (ShareUsageBase.java line 66). #139 writes DateSent with a DateTimeFormatter in Locale.ROOT and in UTC, so it is left to that draft.
    • The data update's If-Modified-Since goes through HttpURLConnection.setIfModifiedSince, which takes a number and leaves the date to the JDK (DataUpdateServiceDefault.java line 833).
    • No NumberFormat, DecimalFormat, MessageFormat or DateTimeFormatter in the package code is made without a locale.
    • In the examples, the two numbers written into CSV files and the made-up client IP addresses are for a machine, and are now written in the root locale (above). The examples' other numbers and dates are printed for the person running them, so they keep that person's locale.

A check, run on every pull request, fails on any toLowerCase() or
toUpperCase() with no argument in a Java file under src/main, and two
tests set a Turkish default locale, where a capital I lowers to a
dotless i. The JSON builder must still write ipdata and ismobile, and the
Device Detection cloud engine must still read DeviceId from the cloud's
deviceid.
Every toLowerCase() and toUpperCase() under src/main now passes
Locale.ROOT, 76 calls in 21 files. Under a Turkish or Azeri default
locale a capital I lowered with the default locale becomes a dotless i,
so the JSON builder wrote keys the client script never reads, the cloud
engines looked for keys the response does not have, and names in the
Pipeline builder and the usage sharing filter stopped matching.
EvidenceKeyFilterShareUsage gets the same four changes as the usage
sharing options change, line for line.
CsvNumbersTest expects the distance CompareOnPrem writes into its CSV
and the average MetricsOnPrem writes into its CSV with a full stop under
a German default locale. RandomIpTest expects the client IP addresses
ShareUsageExample makes up to use the digits 0 to 9 under a Saudi
Arabian default locale.
CompareOnPrem and MetricsOnPrem wrote a distance and an average into
their CSV rows with String.format and the default locale, so a locale
with a decimal comma split one column into two. ShareUsageExample made
up client IP addresses the same way, so a locale whose digits are not 0
to 9 gave Evidence no engine can read. Each now formats in Locale.ROOT,
and the two CSV numbers are package-private methods so a test can reach
them.
Two places needed resolving, both by keeping rc's code and naming the root
locale in it, as this branch does everywhere else in the main code.

- IPIntelligenceCloudEngine.java conflicted. rc moved the processing into
  processCloudEngine, which the shared Cloud Aspect Engine logic calls, and
  skips a property the response leaves out. rc's version is kept, with
  Locale.ROOT in the weighted property name check, as this branch had it.
- The Device Detection examples' KeyHelper merged without a conflict, but
  rc had replaced the line this branch changed with getNamedKey, whose
  keyName.toUpperCase() named no locale. The root locale check found it, and
  it now passes Locale.ROOT.

ci/check-root-locale.ps1 passes on the merged tree, with every case change
in 533 main source files naming its locale.
@jwrosewell
jwrosewell marked this pull request as ready for review November 19, 2026 05:24
@jwrosewell
jwrosewell merged commit ebe5997 into rc Nov 19, 2026
11 checks passed
@jwrosewell
jwrosewell deleted the fix/root-locale branch November 19, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant