Skip to content

Results instances are not thread-safe, so reads take no lock - #241

Merged
jwrosewell merged 5 commits into
rcfrom
remote/results-not-thread-safe
Nov 21, 2026
Merged

jwrosewell merged 5 commits into
rcfrom
remote/results-not-thread-safe

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Problem

#80 made every read of an on-premise results instance, and closing one, take a lock, so that several threads could read one Flow Data's results at once. Every property read of every request pays for that lock, to protect the few callers that share a results instance between threads. On an Intel Mac, in 10 paired rounds against its parent, it cost 1.69 percent on the console benchmark and 1.86 percent on the web benchmark.

#80 merged into rc on 14 November, the day after the last release (#114, main 612aa28). No released version has taken this lock.

What the specification says

specifications#190 rewrote the thread safety page and is merged into the specification's rc as 52e9b047. The quotes below are from rc at d79bbe3b, which holds it.

Lines 198 to 212:

A results instance is the object an on-premise Engine's native code fills for one request, which references unmanaged memory, such as the Device Detection and IP Intelligence results. Multi-threaded access to a results instance is not supported, for performance reasons. Reading a value from it writes into it, so it is not safe to use from several threads at once, and neither is Element Data that reads its values from one.

An Engine, its results instances and the Element Data that reads from them MUST NOT take a lock to make those reads safe for several threads. The caller can almost always keep its use of a results instance on one thread, and a lock would cost every request to protect the few that share one. The caller is responsible for thread-safe access. A caller that needs to use a results instance, or Element Data that reads from one, from several threads at once MUST coordinate that itself, for example by wrapping its calls in a lock of its own.

The same page also has these:

  • Lines 73 to 75: in every language, data held in unmanaged memory doesn't support multi-threaded access.
  • Lines 219 to 221: code comments and documentation must say so plainly, and say the caller is responsible.
  • Lines 162 to 164: they must make clear that Element Data is written only by the Flow Element that provides it, or the ones that share it, and only while they process.
  • Lines 225 to 232: data that isn't written, apart from lazy properties, is safe to share between threads "except where it reads from a results instance".

What changed

Five commits on rc 2373d14, the merge base:

  1. c6883fe3 removes the lock and rewrites the thread tests.
  2. 7dc2615f adds the rule to the ElementData docstring in pipeline core.
  3. 03cc1f24 adds it to both SwigData docstrings.
  4. 6e801247 brings those docstrings into line with the specification's wording.
  5. def9267a says in the no lock test's docstring that it only looks at the instance.

Since the merge base, rc has changed two of the six files: both on-premise swig_data.py files now define OnPremiseMissingPropertyService. Those edits are in other parts of the files, and the head still merges cleanly into today's rc (b64b480).

In the code:

  • SwigData in device-detection/.../swig_data.py and ip-intelligence/.../swig_data.py takes no lock. The threading import, _read_lock and the with blocks in get() and close() are gone, so those lines are back to how they were before Let one thread at a time read a native results instance #80. get() still raises RESULTS_CLOSED once the results are closed.
  • Both SwigData docstrings now say an instance reads from a results instance that references unmanaged memory, that multi-threaded access isn't supported, and that reading a value writes into it. They say nothing takes a lock for those reads, and that the caller either gives each thread its own Flow Data or wraps every read and close() in a lock of its own.
  • The ElementData docstring in pipeline/.../elementdata.py says who writes Element Data and when, that once written it's safe to read from several threads because its values are Python's own data, and that Element Data reading from a results instance is the exception.
  • test_swig_data_threads.py in both on-premise packages used to assert that reads never overlap and that close() waits for a read in progress. Both now check the docstring and that an instance holds no threading lock. The second test is only a smoke test: its docstring says a lock kept at module level, on the engine or in AspectData would get past it. test_elementdata_documentation.py checks the ElementData wording.
  • Reading after close is still covered with real data by the released test_flowdata_close_releases_results in both packages, which this draft doesn't touch.
  • Neither on-premise readme mentions threads, so no readme changed.

Locks removed and kept

I searched every package's source on today's rc for threading.Lock, RLock, Condition, Semaphore, asyncio.Lock and with blocks on a lock. Only the two locks on a results instance go.

Lock Where What it guards Removed or kept
SwigData._read_lock Device Detection on-premise swig_data.py Reads of a results instance, and close() Removed
SwigData._read_lock IP Intelligence on-premise swig_data.py Reads of a results instance, and close() Removed
_properties_lock Cloud request engine cloudengine.py A cloud engine's property list Kept
_lock Usage sharing share_usage.py The usage sharing queue, sending flag and resume timer Kept
_invariants_lock Usage sharing share_usage.py Deciding whether a record carries the values that don't change, and building them Kept
_lock Usage sharing share_usage_backoff.py The pause after failed sends Kept
_lock and each data file's update lock Engines datafile_update_service.py A data file's update and the engine refresh Kept
_loop_lock() 51Did did_client.py The 51Did client's key cache Kept

None of these guards a results instance. No Python code here locks Flow Data, Evidence or a parallel group, and flowdata.py on rc takes no lock.

Released tests

The last release is main 612aa28. None of the three test files this draft changes shipped there, because #80 and #84 added them on rc on 14 November. So no released test is altered.

The released tests that exercise this code are the tests folders of the three packages at 612aa28 (23 files for pipeline core, 8 for Device Detection on-premise, 7 for IP Intelligence on-premise). They were copied unaltered and run locally on Windows with Python 3.14.5 against 6e80124 and against rc 2373d14, with both native modules built from cxx 178371ac and the free Lite data files. Since 6e80124 only a test docstring has changed.

Package 6e80124 rc 2373d14
Pipeline core 480 passed, 1 failed 480 passed, 1 failed
Device Detection on-premise 67 passed, 6 failed 67 passed, 6 failed
IP Intelligence on-premise 52 passed, 1 failed 52 passed, 1 failed

The same eight tests fail on rc without this draft, from other rc changes: test_setheader.py test_get_response_header_name, test_devicedetection.py test_excluded_property, test_missing_property_service_not_found_anywhere, test_missing_property_service_not_found_in_current, test_validate_data_false and test_validate_data_true, test_update_matched_useragent.py test_turned_off_leaves_user_agents_empty, and test_ipintelligence.py test_missing_licence_key.

Red and green

All three runs are on Ubuntu and Windows with Python 3.10 to 3.14, 10 Build and Test jobs each, no macOS. Each built a temporary branch with one workflow file added.

  • Red: run 44615015319 built 15aa629, which is rc 3abcb1c with this draft's three test files copied in unchanged. It failed in all 10 jobs. The Ubuntu Python 3.12 log shows the two test_elementdata_documentation.py tests failing in pipeline core, and both test_swig_data_threads.py tests failing in each on-premise package, with test_an_instance_holds_no_lock reporting Lists differ: [] != ['_read_lock']. No other test failed in that job.
  • Green: run 44615030191 built 14f5836, the head def9267 plus the workflow file. It passed in all 10 jobs.
  • Green: run 44465760633 built 817d939, also the head plus the workflow file. It passed in all 10 jobs, with pipeline core at 561 passed, Device Detection on-premise at 103 and IP Intelligence on-premise at 78 in each. Locally, with the native modules built, 6e80124 gave the same counts.

The red still holds for today's rc: no commit from 3abcb1c to b64b480 touches any of the six files, and both swig_data.py files there still have import threading and _read_lock.

Timing

c6883fe was timed against rc 2373d14 on an Intel Mac, with the same compiler, Python 3.13.18, the Lite data file and cxx 178371ac, in 10 paired rounds with the order alternating. All 20 runs passed. The later commits change only docstrings and tests.

Benchmark Head, ms per detection (range) rc, ms per detection (range) Paired difference, median (range) Head faster
Console, the figure CI publishes 0.061063 (0.060758 to 0.061882) 0.061563 (0.061208 to 0.062081) -0.56 percent (-1.49 to +0.68) 8 of 10
Web 0.212228 (0.209590 to 0.214197) 0.216838 (0.214832 to 0.219498) -2.69 percent (-3.94 to -0.49) 10 of 10

The web benchmark was faster in every round, so that gain is clear of the noise and wins back all of #80's 1.86 percent. The console ranges overlap, so that gain isn't.

What remains

  • Nobody has looked into why the console benchmark gets back less than the web one.
  • The ElementData docstring says Element Data isn't written once processed "apart from lazy properties". No Python Element Data has a lazy property yet, so that clause describes what the specification allows, not anything this package does today.
  • Neither on-premise readme says a results instance is the caller's to coordinate. The docstrings, which are the generated API documentation, do. A sentence in each readme can come in a later change.

Before you upgrade

Locking doesn't change for anyone upgrading from the last release. The lock only ever existed on rc, from #80, and this draft removes it before it ships.

What a released user does see is the documentation. The SwigData docstrings in both on-premise packages, and the ElementData docstring in pipeline core, now say that a results instance, and Element Data that reads from one, isn't safe to use from several threads at once, and that the caller is responsible for thread-safe access. Lines 219 to 221 require that statement, and lines 205 and 206 forbid the lock.

If you use each Flow Data on one thread, change nothing. If you share one between threads, give each thread its own Flow Data, or wrap every read and the call to close() in a lock of your own. The released packages didn't lock these reads either.

SwigData in the Device Detection and IP Intelligence on-premise packages
no longer holds a lock for each read of its native results or for
close(). A results instance is not thread-safe, the caller is responsible
for thread-safe access, and the class docstrings say so, with what to do
instead. specifications#190 adds this as the "Results instances" section
of the thread safety page.

The thread tests asserted that reads never overlap and that close() waits
for a read, which is no longer promised. They now check that the
docstring says the caller coordinates threads and that an instance holds
no lock. Closing results and reading afterwards is still tested with real
data by test_flowdata_close_releases_results in both packages.
The ElementData docstring says a finished instance is safe to read from
several threads at once, as its values are Python's own managed data and
do not change once written. It now also says that Element Data reading
from a results instance, which holds unmanaged memory, is the exception.
Multi-threaded access to a results instance is not supported, for
performance reasons, so the caller gives each thread its own Flow Data or
coordinates its own access. The documentation test checks both, as lines
3 to 16 of the thread safety page in specifications#190 frame it.
Both SwigData docstrings now say a results instance holds unmanaged
memory, and that multi-threaded access to it is not supported, for
performance reasons, as lines 3 to 16 of the thread safety page in
specifications#190 frame it. The caller either gives each thread its own
Flow Data or coordinates its own access. That wording replaces the
quotation of the rule against locking. The documentation tests check the
new wording.
jwrosewell and others added 2 commits November 17, 2026 18:56
The ElementData docstring now says an instance is written only by the Flow
Element that provides it, or the Flow Elements that share it, and only
while they process, and that the Flow Data holding it is used by one
thread at a time. Element Data that reads from a results instance takes no
lock to make those reads safe.

Both SwigData docstrings now follow the results instance section of the
thread safety page in specifications#190 at a90ba516. Reading a value
writes into the results instance, and the engine, its results instances
and the instance take no lock to make those reads safe for several
threads. The documentation tests check the new words.
test_an_instance_holds_no_lock in both on-premise packages looks only at
the attributes of one SwigData instance. A lock at module level, on the
engine or in AspectData would still pass it. Its docstring now says it is
a smoke test, so no one reads more into it than that.

The rule it checks is in the results instance section of the thread
safety page, as merged in specifications 52e9b047.
@jwrosewell
jwrosewell marked this pull request as ready for review November 21, 2026 16:43
@jwrosewell
jwrosewell merged commit 0dd02ca into rc Nov 21, 2026
3 checks passed
@jwrosewell
jwrosewell deleted the remote/results-not-thread-safe branch November 21, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants