Repository navigation
Specification conformance, 51degrees-java, section 13 - #171
Merged
Merged
Conversation
…evel fodid caching.md line 144 at 706de1f0: "A cache MUST NOT store a result that holds such a value, and MUST NOT serve one that it finds." (CO-21)
The engine stores the fresh result it gets in its place. caching.md line 144 at 706de1f0 (CO-21).
…ache AspectEngineBase asks isCacheable before it stores a result and before it serves one it finds. The Cloud Request Engine answers false for a response whose JSON has a top-level member named fodid, in any case. caching.md line 144 at 706de1f0: "A cache MUST NOT store a result that holds such a value, and MUST NOT serve one that it finds." (CO-21)
exception-handling.md line 42 at 706de1f0: "This property MUST be exposed as a public property as part of the `Pipeline` public interface as `IsSuppressProcessExceptions` - for Pipeline wrappers such as `WebPipeline` to be able to use it as well to make a decision on whether the exceptions should be suppressed and logged during processing." (CO-26)
PipelineDefault gives the setting the builder made. The interface's default method gives the builder's default, so an implementation of Pipeline written before this still compiles. exception-handling.md line 42 at 706de1f0: "This property MUST be exposed as a public property as part of the `Pipeline` public interface as `IsSuppressProcessExceptions`" (CO-26)
…ults Covers getValues, the property getters, the match metrics and the device id, each started while another read is inside the native results. thread-safety.md line 57 at 706de1f0: "Reading an Element Data instance once the Flow Element that writes it has finished MUST therefore be safe from several threads at once." (CO-48) thread-safety.md line 61 at 706de1f0: "Such Element Data MUST let only one thread at a time read from its results, for example by holding a lock for each read" (CO-49)
…ive results DeviceDataBaseOnPremise gives subclasses its lock as getResultsLock and holds it while it asks whether a property is available. DeviceDataHashDefault holds it in propertyIsAvailable, in the public getValues and while it reads the match metrics and the device id. The lock is reentrant, so a read that calls another read is unchanged. thread-safety.md line 57 at 706de1f0: "Reading an Element Data instance once the Flow Element that writes it has finished MUST therefore be safe from several threads at once." (CO-48) thread-safety.md line 61 at 706de1f0: "Such Element Data MUST let only one thread at a time read from its results, for example by holding a lock for each read" (CO-49)
Reads the weighted doubles, the weighted floats and the single number. The free Lite file has neither property, so the test needs a paid IP Intelligence data file and is skipped without one. value-types.md line 99 at 706de1f0: "A value converted from Azimuth or Declination MUST be given to the caller rounded to 3 decimal places, whether it is given as a number, as a string or as a point in a WKT shape." (CO-58)
From 7992af70. 51degrees-cxx #177 (a1c0504d) gives a number stored as an angle rounded to the decimal places of its text, 3 by default, so the weighted Latitude and Longitude lists come rounded. The headers the wrappers use lose only private members in this range. cxx no longer has its device-detection-data and ip-intelligence-data folders, as the README expected. value-types.md line 99 at 706de1f0: "A value converted from Azimuth or Declination MUST be given to the caller rounded to 3 decimal places, whether it is given as a number, as a string or as a point in a WKT shape." (CO-58)
… builder conceptual-overview.md line 118 at 706de1f0: "A Pipeline instance MUST be immutable. In other words, once created, it cannot be changed by adding new elements, removing old ones, etc." (CO-12)
PipelineDefault kept the builder's list, so an element added to the builder after build joined the Pipeline already built. It now keeps an unmodifiable copy. conceptual-overview.md line 118 at 706de1f0: "A Pipeline instance MUST be immutable. In other words, once created, it cannot be changed by adding new elements, removing old ones, etc." (CO-12)
… interface For Device Detection and IP Intelligence. Passes as the code is, so no fix follows. README.md line 95 at 706de1f0: "Where both Cloud and On-Premise variants of an Engine are available, they MUST arrange that the Properties and values produced are compatible with each other so that the Engines can be substituted in the Pipeline without alteration to the consuming application." (CO-4)
…iguration Passes as the code is, so no fix follows. multi-pipeline-elements.md line 17 at 706de1f0: "If this feature is enabled, then all Flow Elements that maintain any such state information MUST ensure that they have the means to keep track of isolated instances of this state for each Pipeline to which they are added and use the correct state information when processing." (CO-6)
…n it must Two threads adding Element Data under one key create it once, errors added from eight threads are all kept, and without parallel elements a write takes no lock. Passes as the code is, so no fix follows. parallel-processing.md line 22 at 706de1f0: "Flow Data MUST be capable of enforcing thread-safe writes to data structures while also maintaining performance when it is not needed." (CO-7)
…Data Key Passes as the code is, so no fix follows. conceptual-overview.md line 73 at 706de1f0: "Each Element MUST:" (CO-11)
…red by case Both pass as the code is, so no fix follows. caching.md line 59 at 706de1f0: "All Flow Elements MUST advertize the Evidence keys that they make use of." (CO-14) caching.md line 82 at 706de1f0: "Comparison of Evidence values MUST be case-sensitive." (CO-17)
The Cloud Request Engine with a cache serves the second request from the cache when only the case of the key differs, and goes to the service when only the case of the value differs. Passes as the code is, so no fix follows. caching.md line 71 at 706de1f0: "Comparison of Evidence keys MUST be case-insensitive." (CO-16)
…tions are suppressed exception-handling.md line 73 at 706de1f0: "The implementor MUST ensure that exceptions/errors that happen in these areas are handled correctly in order to avoid the API taking down the customer's production website." (CO-28)
…t to When the Pipeline suppresses process exceptions, an error in PipelineFilter's own work (the not modified check, processing, the UA-CH headers or serving the JavaScript and JSON) is logged and the request goes on down the filter chain without the results. Otherwise it is thrown, as before. The FlowData is closed either way, and only if processing stored one. exception-handling.md line 73 at 706de1f0: "The implementor MUST ensure that exceptions/errors that happen in these areas are handled correctly in order to avoid the API taking down the customer's production website." (CO-28)
…d kept in the task exception-handling.md line 83 at 706de1f0: "When errors do occur, they will be logged and MUST NOT cause the process to crash." (CO-29)
The scheduled update checks and the file system watch run through logErrors, which logs an error the task did not expect and keeps it in the task. Before, the executor kept it in a future nobody read, so it was never logged. An error of the virtual machine itself is logged and thrown on. Rescheduling after such an error is left as it is. exception-handling.md line 83 at 706de1f0: "When errors do occur, they will be logged and MUST NOT cause the process to crash." (CO-29)
…t open Passes as the code is, so no fix follows. resource-cleanup.md line 28 at 706de1f0: "However, if the multi Pipeline Elements feature is implemented then there MUST be an option to disable this behavior." (CO-39)
Passes as the code is, so no fix follows. resource-cleanup.md line 84 at 706de1f0: "Element Data instances that do not require cleanup MUST be left alone in order for result caching to work correctly." (CO-45)
…only those exception-handling.md line 17 at 706de1f0: "In all cases, standard language features SHOULD be used to let IDEs and users know what exceptions/errors a particular function can throw." (CO-24)
process, get, getFromElement, getAs and the getAs typed getters name in their throws clauses and @throws tags the unchecked exceptions they throw for a mistake by the caller. get(String) before processing now throws IllegalStateException, as get(TypedKey) does, rather than a plain RuntimeException. A caller that catches RuntimeException is unchanged. exception-handling.md line 17 at 706de1f0: "In all cases, standard language features SHOULD be used to let IDEs and users know what exceptions/errors a particular function can throw." (CO-24)
An element that did not run, or a property with no value, is left out of getWhere's map. Evidence that is missing or of another type gives no value from tryGetEvidence. exception-handling.md line 57 at 706de1f0: "The general guidance is that these functions and accessors SHOULD only throw errors/exceptions if the failure is likely due to something that the API user has done wrong." (CO-27)
…et throws getWhere left a NullPointerException or NoSuchElementException to the caller when an element had not run or a property had no value, which is not the caller's mistake. It now leaves both out, and still throws IllegalStateException before processing. The Javadoc of get no longer promises null for missing data: get by key throws NoSuchElementException and get by type throws PipelineDataException, as FlowDataTests and the TypedKeyMap tests expect, and both are declared. exception-handling.md line 57 at 706de1f0: "The general guidance is that these functions and accessors SHOULD only throw errors/exceptions if the failure is likely due to something that the API user has done wrong." (CO-27)
… value For the Pipeline builder, the engine builders and the data file configuration builder, the Cloud Request Engine builder, the usage sharing and 51Degrees Pipeline builders, and the translation engine builder. The JavaScript Builder's builder already gives every default. reference-implementation-notes.md line 79 at 706de1f0: "In all cases, there SHOULD be some mechanism to specify the default values alongside configuration property setters." (CO-65)
…ne packages @DefaultValue on the 41 setters that had none, taken from the constant or field the builder or its configuration falls back to, or saying who sets it where it is not a value here. The randomisation setters in milliseconds gave the default in seconds, and three Cloud Request Engine endpoint defaults left out the slash after the base URL, so those are corrected. reference-implementation-notes.md line 79 at 706de1f0: "In all cases, there SHOULD be some mechanism to specify the default values alongside configuration property setters." (CO-65)
The same check as for Device Detection, for getValues and the property getters of IPIntelligenceDataHashDefault. thread-safety.md line 61 at 706de1f0: "Such Element Data MUST let only one thread at a time read from its results, for example by holding a lock for each read" (CO-49, and CO-48 at line 57)
…ve results IPIntelligenceDataBaseOnPremise gives subclasses its lock as getResultsLock and holds it while it asks whether a property is available. IPIntelligenceDataHashDefault holds it in propertyIsAvailable and in the public getValues, whose body moves to readValues unchanged. thread-safety.md line 61 at 706de1f0: "Such Element Data MUST let only one thread at a time read from its results, for example by holding a lock for each read" (CO-49, and CO-48 at line 57)
… its default For the on-premise Pipeline and engine builders of both products, and the country code translation engine builder. reference-implementation-notes.md line 79 at 706de1f0: "In all cases, there SHOULD be some mechanism to specify the default values alongside configuration property setters." (CO-65)
…ce builder setter @DefaultValue on the 42 setters that had none, from the constants the engine builders fall back to. Both on-premise Pipeline builders said update on startup defaults to enabled, but an unset value falls back to Constants.DEFAULT_UPDATE_ON_STARTUP, which is false, so their Javadoc now says disabled. reference-implementation-notes.md line 79 at 706de1f0: "In all cases, there SHOULD be some mechanism to specify the default values alongside configuration property setters." (CO-65)
…d response Lazy loading means process returns at once and the work is done on another thread. The check that keeps a 51Did out of the cache (caching.md line 144 at 706de1f0: "A cache MUST NOT store a result that holds such a value, and MUST NOT serve one that it finds.") must not make process wait for the answer. The test holds the answer back and reads the processing thread's stack, so no result depends on timing. Two more tests check that a lazy response is still cached, and that one with a fodid is not.
isCacheable read the cloud response through the getter that waits for lazy loading, so a cached lazy Cloud Request Engine waited in process for the cloud service. It now reads the stored response without waiting (AspectDataBase.getWithoutWaiting). With lazy loading the result is offered to the cache by the processing task once processing has finished, as the response is not there to check before then. Without lazy loading nothing changes. The check still answers caching.md line 144 at 706de1f0: "A cache MUST NOT store a result that holds such a value, and MUST NOT serve one that it finds."
This was referenced Nov 16, 2026
Resolves three conflicts with rc. FlowData.java: get(TypedKey) and get(String) throw NoSuchElementException when no element added data under the key, as the typed key map does, so this branch's javadoc stays. process() keeps rc's AggregateException, and the getAs methods keep rc's missing property case, which getMetaDataForProperty throws. DeviceDataHashDefault and IPIntelligenceDataHashDefault: rc's shortcut for a property the engine has an index for stays, and the native name lookup after it holds the results lock, as this branch does. The IP Intelligence reads use the base class's results lock in place of rc's second lock object, so every read of the native results holds the same lock.
rc now holds the parallel operation changes (Element Data added with one compare and swap and no lock, usage sharing last, the web Flow Data used by one thread, members' errors added under a lock) and the evidence key case, cloud response warning and example fixes. One conflict, in PipelineBuilderBase.java, where this branch imports DefaultValue and rc imports PipelineConfigurationException at the same line. Both are used, so both imports are kept. Every other file merged without a conflict.
The specification says an Engine, its results instances and the Element Data that reads from them must not take a lock to make those reads safe for several threads, and that the caller is responsible for thread-safe access. So the results lock this branch added to the Device Detection and IP Intelligence on-premise Element Data goes, with its getResultsLock() accessor and the two test classes that required it. The four files are as they are on rc, and the two base classes now say that an instance is not safe to use from several threads at once and that the caller is responsible. https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/thread-safety.md#L196-L221
rc adds Element Data with no lock, by one compare and swap, so the test that needed a second writer to wait for the first fails once rc is merged. The specification says a Flow Element adds its Element Data with one compare and swap and readers take no lock, and two Flow Elements that write one Element Data instance must not share a parallel group. So the same key test now checks what rc does. Both writers reach the factory while neither has finished, and both get the one instance kept. The class comment quotes the current thread safety page. https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/thread-safety.md#L88-L115
The specification says a Flow Element that hands data to another thread must copy it during processing, and must not read the Flow Data from that thread afterwards. Making the cache key reads the Flow Data's Evidence, so the cache write belongs on the thread that called process. This test records the thread of each cache write. It fails until the processing task stops writing to the cache. https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/thread-safety.md#L32-L40
With the deprecated lazy loading option the processing task stored the result in the cache once it was processed, and making the cache key read the Flow Data's Evidence from that task after process had returned. The specification says a Flow Element must not read the Flow Data from another thread afterwards. So the lazy path stores the result on the processing thread, as it did before this branch, and the task only processes. The result is not yet processed when it is stored, so the check before serving is what keeps a response with a 51Did from being served. The lazy 51Did test now says served, and the comment on isCacheable says which check applies when. https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/thread-safety.md#L32-L40 https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/caching.md#L151-L156
Serving the 51Degrees JavaScript or JSON can fail with an IOException, for example when the client has gone away. The specification says the web integration disposes of the Flow Data when the request ends, which releases its Element Data and native results, and that errors in the web integration must not take down the website. This test makes serving the JavaScript throw an IOException and checks that the failure is thrown and the Flow Data is still closed. It fails until the filter closes the Flow Data on that path.
The web filter caught only runtime errors from processing and serving, so an IOException from writing the 51Degrees JavaScript or JSON left the filter without closing the request's Flow Data. The specification says the web integration disposes of the Flow Data when the request ends. The filter now closes the Flow Data before throwing the IOException, and a failure to close is added to it as a suppressed exception. https://github.com/51Degrees/specifications/blob/d79bbe3b2aef86273570742f6b660f29bfe72816/pipeline-specification/features/web-integration.md#L27-L28
Two conflicts. In FlowDataDefault.getWhere this branch's version is kept, which skips an element with no data and a property with no value, and the property name is lowered in the root locale as rc now does. In PipelineBuilderBase setAutoCloseElements keeps this branch's DefaultValue and gains rc's AlternateName, and both imports stay.
…etters and this work's default value on setCache in the Hash engine builder
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Java packages differed from the core features of the specification in ten places, and nine more rules had no test.
Pipelineinterface did not expose whether process exceptions are suppressed, so a wrapper such as the web filter could not read it.doFiltereven when the Pipeline suppresses process exceptions.FlowDatadeclared none of the exceptions its methods throw.getWherethrew when an element had not run or a Property had no value, neither of which is the caller's mistake.What the specification says
Item 3 and the thread safety lines below are at 51Degrees/specifications d79bbe3b, the current specifications
rc. The rest are at 706de1f0, which is older than the thread safety rules.caching.md lines 144 to 150
exception-handling.md lines 42 to 44
thread-safety.md lines 205 to 211 and 219 to 221
value-types.md lines 99 to 101
conceptual-overview.md lines 118 to 119
exception-handling.md lines 73 to 75
exception-handling.md lines 83 to 84
exception-handling.md lines 17 to 18
exception-handling.md lines 57 to 59
reference-implementation-notes.md lines 79 to 80
The rules that gain a test here, where the code already met them, are README.md line 95 (cloud and on-premise engines can be substituted), multi-pipeline-elements.md lines 16 to 19 (state kept for each Pipeline), thread-safety.md lines 88 to 115 at d79bbe3b (Element Data added with one compare and swap and no lock, errors from a parallel group added under a lock), conceptual-overview.md line 73 (what each element MUST give), caching.md lines 59, 71 and 82 (cache keys from the advertised Evidence, keys compared ignoring case and values with case), resource-cleanup.md lines 27 to 29 (an option not to close shared elements) and resource-cleanup.md lines 84 to 85 (Element Data that needs no cleanup is left alone).
What changed
Links are to the head, b39a1e2.
AspectEngineBaseasks a new protectedisCacheablebefore it stores a result and before it serves one it finds. The default answers true.CloudRequestEngineDefaultanswers false for a response with a top-levelfodidmember in any case, so that request goes to the cloud service. With the deprecated lazy loading option, the result is stored on the processing thread before it has been processed, as before this draft, and the check before serving keeps out one that may not be cached.isCacheablereads the stored response without waiting, through a new protectedAspectDataBase.getWithoutWaiting, so a cached engine with lazy loading returns fromprocesswithout waiting for the cloud service. Without lazy loading nothing changes.Pipeline.isSuppressProcessExceptions().Pipelinegains it as a default method that gives the builder's default, so a class that already implementsPipelinestill compiles, andPipelineDefaultgives the value the builder set.DeviceDataBaseOnPremiseandIPIntelligenceDataBaseOnPremisesay that an instance is not safe to use from several threads at once and that the caller is responsible for thread-safe access. This draft takes no lock on the native results. The four Element Data files are otherwise as they are onrc.cxxsubmodule moves from 7992af70 to 178371ac, which is 167 commits and 71 merged pull requests of 51degrees-cxx (comparison). Among them, #177 rounds a value converted from Azimuth or Declination to 3 decimal places. Between the two commits the headers the SWIG wrappers use lose only private members, and this draft does not regenerate the wrappers committed here. The native library move, below, says what else a Java user can see and why the rest changes nothing for Java.PipelineDefaultkeeps an unmodifiable copy of the builder's list.PipelineFiltermeets an error in its own work andisSuppressProcessExceptions()is true, it logs the error and passes the request on down the filter chain without the results. Otherwise it throws as before. The Flow Data is closed either way, including when serving the JavaScript or JSON fails with anIOException, which is then thrown as before.DataUpdateServiceDefaultrun throughlogErrors, which logs an unexpected error and keeps it inside the task. An error of the virtual machine itself is logged and thrown on.FlowDatadeclares what it throws.FlowDatagainsthrowsclauses and@throwstags onprocess,get,getFromElement,getAsand the typed getters.FlowDataDefault.get(String)before processing throwsIllegalStateException, whichrcalready does.getWhereleaves out what is not there. An element that has not run and a Property with no value are left out of the map rather than throwing.getWherenow throwsIllegalStateExceptionbefore processing, where it threw a plainRuntimeExceptiononly when a Property matched and gave an empty map otherwise.getby key and by type still throw for data no element added, as asking for it is the caller's mistake, and their Javadoc now says so.@DefaultValueis added to 83 public setters, 41 in the pipeline packages, 23 in Device Detection, 18 in IP Intelligence and 1 in the country code translation builder. Four defaults that were given wrongly are corrected in their annotations.setUpdateRandomisationMaxMillisgave 600, a number of seconds, three Cloud Request Engine endpoint defaults left out the/after the base URL, and both on-premise Pipeline builders said update on start-up defaults to enabled, where the value they fall back to,Constants.DEFAULT_UPDATE_ON_STARTUP, is false.No public name is removed or renamed.
The native library move
Java's on-premise engines run the C and C++ code of the
cxxsubmodule through the SWIG wrappers committed in this repository. These changes among the 71 pull requests alter what a Java user can see, and Before you upgrade lists them.1.2.3.4/24or1.2.3.4 x, or one with a part above 255, was read as far as it went and is now skipped, and a value that could not be read at all no longer stops the whole lookup.51D_GetHighEntropyValuesand51D_StructuredUserAgentis used before the cookie value, whatever order the evidence is given in.Header.User-Agentis read where it was dropped.INSUFFICIENT_HANDLES. A Device Detection value read that fails part way through now reports the failure.The rest change nothing a Java user can see, for these reasons.
JavascriptHardwareProfile), #129 (type names, which that pull request says Java reads unchanged), #130 (IP Intelligence value text, which that pull request lists as unchanged for Java), #131 and #133 (property, component and profile metadata).srconly so that the sources build (CMake packaging, a macro name, a file name, and aconstand three includes for GCC 15), with no change to what the code does.Before you upgrade
Pipeline.isSuppressProcessExceptions()is new on the publicPipelineinterface, as a default method.AspectEngineBase.isCacheable(TData)is a new protected method that an engine can override to keep a result out of its cache.AspectDataBase.getWithoutWaiting(String)is a new protected method that reads a stored value without waiting for lazy loading.getWherethrowsIllegalStateExceptionbefore processing. It threw a plainRuntimeExceptiononly when a Property matched, and gave an empty map otherwise.IllegalStateExceptionis a subclass ofRuntimeException.getWhereno longer throws for missing data. A caller that relied onNoSuchElementExceptionorNullPointerExceptionfrom it gets a map without those entries instead.setSuppressProcessException(true), an error in the filter's own work is now logged and the request continues without results, where it ended the request.build()no longer joins the Pipeline already built.Tests
Each item has a test commit followed by its fix, and each test commit was run on a runner before its fix. The two results lock tests that were here,
DeviceDataHashThreadSafetyTestsandIPIntelligenceDataHashThreadSafetyTests, are gone with the locks, as Changes at this head below says.Red, run 43773302656. The run's commit is 55661b7 with only a temporary workflow added. Each of its eleven jobs checks out one test commit, builds the modules it tests with the modules they need on Java 17, runs only that commit's new tests, and fails.
CloudRequestEngineFodIdCacheTestsfodid,FodIdandFODID, and a stale one found therePipelineSuppressProcessExceptionsTestsNoSuchMethodExceptionPipelineImmutableTestsPipelineFilterErrorTestsIllegalStateExceptionleftdoFilterDataUpdateServiceBackgroundErrorTestsFlowDataDeclaredExceptionsTestsFlowDataAccessorErrorTestsNoSuchElementExceptionandNullPointerExceptionfromgetWhereBuilderDefaultValueTestsin five pipeline modulesBuilderDefaultValueTestsin Device Detection and IP IntelligenceRed for lazy loading, run 43775828622, on 089dfbb with only a temporary workflow added. Its job checks out 8ad48e8 and runs
CloudRequestEngineLazyCacheTests, and 1 of 3 fails, "process waited for the cloud service's answer".The rules the code already met, run 43774794645 and run 43774949637. Each job checks out the commit that adds the test, breaks the rule in the code on purpose with a
sedcommand its log shows, and runs the test, which fails.CloudOnPremiseSubstitutionTestsdevice-cloudJsonBuilderMultiPipelineTestsFlowDataConcurrentWriteTestsrc's add with no lock instead, as Changes at this head below saysFlowElementContractTestsgetElementDataKeygiven a defaultCacheKeyTestsCloudRequestEngineCacheKeyCaseTestsPipelineAutoCloseTestsFlowDataCleanupTestsRed and green with the Enterprise IP Intelligence data file, run 43777891646, on 089dfbb with only a temporary workflow added.
AngleRoundingTestsneeds that file, the only one that holds Latitude and Longitude, so the run fetches it as the full build does and runs the test at two commits. At the test commit 7802440, still oncxx7992af70, 2 of 3 fail, for example "Latitude 51.34891811883907 is not rounded to 3 decimal places. expected:<51.349>". At the fix 1a4236f, oncxx178371ac, 3 of 3 pass.Green, the builds below, in which these tests ran.
Build
Run 43775786230 built 089dfbb merged with
rc(e2472ee) on nine configurations, Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21, and every job passed. Every test class named in the tables above passed in all nine jobs with none skipped,AngleRoundingTestsamong them with the Enterprise IP Intelligence data file the build fetches.Run 44419992206 built 9f27d7a, the merge of
rcat ce10eaf, on the same nine configurations, and every job passed. Its workflow commit is 8c41ab4, on 9f27d7a.Run 44451937148 built the head b39a1e2 on the same nine configurations. Its workflow commit is 7a8252d, on b39a1e2. Eight jobs passed. Windows with Java 21 passed the tests of every pipeline module, then the JVM crashed in
rc'sDataRefreshTests.Refresh_FromSeveralThreads_EngineStillWorksin Device Detection on-premise, with "Insufficient memory allocated for the operation" and then an access violation, so the modules after it did not run their tests in that job. They ran and passed in the other eight. The same test crashed the JVM the same way on Windows with Java 21 in two builds that keeprc'scxx7992af70, run 43952784482 and run 43971336799, so the crash comes fromrcand not from the move to 178371ac. #198 tracks it.macOS was left out of these builds. The
rchead is built with macOS before it merges into main.Merged with rc
9f27d7a merged
rcat ce10eaf, after #175 and #177 had merged, keeping theNoSuchElementExceptionthe typed key map throws in theFlowDataJavadoc besiderc's cases. 8a51631 mergedrcat 49bc431, which holds the parallel operation changes. Its one conflict was two imports added at the same line ofPipelineBuilderBase.java,DefaultValuehere andPipelineConfigurationExceptiononrc, and both are kept.What remains
#181. The location (geo-location) builders and the developer examples' builders still have setters with no
@DefaultValue.After an unexpected error in a scheduled update check, this draft logs it but does not schedule the next polling check. #170 schedules it, and a trial merge of the two heads is clean.
With the deprecated lazy loading option a result is stored before it has been processed, so a response with a 51Did is kept from being served but not from being stored. The specification says the lazy loading option has no effect, and once it does nothing every result passes the check before it is stored.
The locks
rcalready takes on the native results, inDeviceDataBaseOnPremise,IPIntelligenceDataBaseOnPremiseandIPIntelligenceDataHashDefault, are not removed here. The specification's MUST NOT applies to them too, and removing them is separate work.The SWIG wrappers committed here are not regenerated from 178371ac. #122 records that the Java wrappers mark the results of
ConfigIpi'sgetComponents,getProperties,getValuesandgetProfilesas owned, which its change to the SWIG interface fixes once the wrappers are regenerated. Regenerating them is separate work.Changes at this head
Since 9f27d7a, each fix after its test, with the red of each test on a temporary branch holding one workflow commit on the test commit and building every configuration but macOS.
rcat 49bc431, as Merged with rc says.getResultsLock()and the two test classes that required them, because thread-safety.md lines 205 to 207 forbids them. Both base classes now say the caller is responsible.FlowDataConcurrentWriteTeststo checkrc's add by compare and swap, in which neither writer waits and both get the one instance kept (thread-safety.md lines 92 to 108). Before it, at 8a51631,getOrAdd_SameKeyFromTwoThreads_CreatesOncefailed on all nine configurations, run 44451912350 (workflow commit 1e14e28 on 8a51631).process_LazyResultIsStoredOnTheProcessingThreadtoCloudRequestEngineLazyCacheTests, and 94e4c75 stores a lazily loaded result on the processing thread, so the task only processes (thread-safety.md lines 32 to 40). The new test failed on all nine configurations at 4be153c, run 44451919998 (workflow commit 8f99474 on 4be153c).servingJavaScriptFails_IOException_FlowDataClosedtoPipelineFilterErrorTests, and b39a1e2 makes the filter close the Flow Data and throw theIOExceptionwhen serving the JavaScript or JSON fails (web-integration.md lines 27 to 28). The new test failed on all nine configurations at 8bd7e35, run 44451928425 (workflow commit 645074d on 8bd7e35).The green is run 44451937148, in which eight of the nine jobs passed, as Build says.
Run locally on Windows with Java 21 at b39a1e2 before pushing:
FlowDataConcurrentWriteTests3 of 3,ParallelElementDataAddTests1 of 1,ParallelErrorsLockTests,CloudRequestEngineLazyCacheTests4 of 4,CloudRequestEngineFodIdCacheTests5 of 5,CloudRequestEngineCacheKeyCaseTests2 of 2,AspectEngineCachedTests2 of 2,AspectEngineLazyLoadTests5 of 5,PipelineFilterErrorTests4 of 4 andPipelineFilterTests19 of 19 pass. The new lazy cache test and the new filter test each failed locally at their test commits.Merging
When this squashes, the messages of the lock commits and of the same key test that later commits replace no longer describe the change, so the squash message should describe the final change only.
Head b39a1e2.