Repository navigation
Let a cloud result replace one the cache will never serve - #219
Draft
jwrosewell wants to merge 2 commits into
Draft
jwrosewell wants to merge 2 commits into
jwrosewell wants to merge 2 commits into
Conversation
The Cloud Request Engine's result is stored while its request is still running. If that request then fails, or answers with a 51Did, the entry is never served. The cache the engine is given does not replace an existing entry by default, so the next result for the same Evidence was dropped, and every later request for that Evidence went to the cloud service while the dead entry stayed in the cache. The Cloud Request Engine's builder now sets a cache built by a PutCacheBuilder to replace an existing entry, which covers setCacheSize and the cloud pipeline builders' CacheConfiguration made with a size. Two tests make three requests with the same Evidence, the first failing or answering with a 51Did, and expect the third to be served from the cache.
With updateExisting set, LruCacheBase.add made the replacement on a list chosen at random and spliced it into the old item's place, which is in the old item's list. With more than one list, the default on any machine with more than one processor, the old list kept the replaced item as its first or last, the new list's first or last pointed into the old list, and the replacement named a list it was not in. Later moves and evictions walked these crossed chains and threw NullPointerException. No main code set updateExisting before the Cloud Request Engine's builder did. The replacement is now made on the old item's list. A new test replaces forty items in a cache with four lists, evicts past its size, and checks that each list runs from its first item to its last through items that belong to it. The Cloud Request Engine's builder javadoc now says the builder is set, not copied, and a further test fills a small cache after a replaced entry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Since #208 the Cloud Request Engine's response is a lazy task, and a result is offered to the cache while its request is usually still running. Such a result is stored, and is served only once its request has finished, without failing, with a response that has no top-level
fodidmember.If the stored request then fails, or answers with a
fodid, the entry is never served. The cache the engine is given did not replace an existing entry (LruPutCache.BuilderhasupdateExistingfalse, andCacheConfiguration(int)uses it). SoLruCacheBase.adddropped the next result for the same Evidence, and every later request for that Evidence went to the cloud service while the dead entry stayed in the cache. Every answer was still right. The cost was cloud requests, and afodidresponse left in the cache.Turning on
updateExistingshowed a second fault.LruCacheBase.addmade the replacement on a list chosen at random and spliced it into the old item's place, which is in the old item's list. With more than one list, the default on any machine with more than one processor, the lists were crossed after the first replacement, and later moves and evictions threwNullPointerException. No main code setupdateExistingbefore this change, so the path had never been live.The change
LruCacheBase.addmakes the replacement on the old item's list, so it takes the old item's place in the list it belongs to.CloudRequestEngineBuilder.configureEnginesets any cache built by aPutCacheBuilderto replace an existing entry, before the base class builds the engine's cache. That coverssetCacheSize, the cloud pipeline builders (DeviceDetectionCloudPipelineBuilder,IPIntelligenceCloudPipelineBuilder,GeoLocationCloudPipelineBuilder, which passnew CacheConfiguration(resultsCacheSize)tosetCache), and aCacheConfigurationmade with your ownLruPutCache.Builder. The builder in the configuration is set, not copied. A cache built by another kind ofCacheBuilderis left as it is.LruPutCacheTests.LruPutCache_ReplaceWithSeveralListsreplaces forty items in a cache with four lists, evicts past its size, and checks that each list runs from its first item to its last through items that belong to it.CloudRequestEngineLazyCacheTests.process_ResultAfterAFailureIsCachedandprocess_ResultAfterAFodIdIsCachedmake three requests with the same Evidence. The first is held back until itsprocesshas returned, so its result is stored while it is running, and then fails or answers with afodid. Each expects two data requests and the third answer served from the cache.process_ReplacedEntryThenEvictionsreplaces a failed entry and then fills a cache of four with forty other requests, each expected to get its own response.Red and green
Run locally on Windows with JDK 21 and Maven 3.9.16, with a clean build each time.
LruPutCache_ReplaceWithSeveralListson rc'sLruCacheBase: fails, "The list's last item is not the end of its chain".process_ResultAfterAFailureIsCachedandprocess_ResultAfterAFodIdIsCachedon rc'sCloudRequestEngineBuilder: both fail, the third request goes to the cloud service again (expected [failed, {...}, {...}] but was [failed, {...}, failed]).process_ReplacedEntryThenEvictionspasses on rc'sLruCacheBasetoo, on a 12 processor machine, so it is an end-to-end check and not the red for the list fault.LruPutCacheTests8 of 8,CloudRequestEngineLazyCacheTests9 of 9,CloudRequestEngineFodIdCacheTests5 of 5,CloudRequestEngineCacheKeyCaseTests2 of 2,CloudRequestEngineTests48 of 48. Pipeline Caching, Engines, Cloud Request Engine and FiftyOne, and the Device Detection, IP Intelligence and Location cloud modules, with everything they build on, pass, with a resource key set andValueCloudTestsleft out (that clone had no data files).ci/check-environment-variable-names.ps1passes.A full-matrix runner build of rc with this change is run 44958153864.
Before you upgrade
PutCacheBuildernow replaces an entry when a new result is put for its key, including where you passed your ownLruPutCache.BuilderwithsetUpdateExisting(false). That builder object is changed, not copied.LruPutCachebuilt withsetUpdateExisting(true)now keeps its lists whole after a replacement.LruLoadingCachenever replaces, so it is not affected.What remains
fodidresponse is still stored while its request runs, because what it holds is not known until it finishes. It is never served, and the next result for the key now replaces it. Storing only finished results would need the cache key made on the processing thread and the put made when the task ends, which needs a new method on the cache interfaces.