[emschwartz]

Addresses #133. Stacked on #385: only the last commit is new, and the numbers below are measured against that PR.

When one array container is more than 64 times larger than the other, &, &= and intersection_len now binary search the larger one instead of merging. This follows CRoaring's intersect_skewed_uint16, by way of saik0's 2022 port linked in the issue. It searches for four values at once so their loads overlap. A galloping search was tried first and was about 2 to 5 times slower at these sizes.

Change in time over 10 interleaved runs of each binary. census1881 is the one dataset with many skewed pairs. and_skewed is a new bench of 10 or 4000 values against 4000.

	Apple M2 Max	EPYC 7713 (Zen 3)	EPYC 9845 (Zen 5)
pairwise_and/ref_ref census1881	-40%	-29%	-25%
pairwise_and/assign_ref census1881	-20%	-12%	-10%
pairwise_and/len census1881	-65%	-68%	-70%
and_skewed/and 10	-96%	-95%	-94%
and_skewed/and_assign 10	-89%	-86%	-90%
and_skewed/intersection_len 10	-97%	-97%	-96%

The other seven datasets pooled are 1 to 7% faster for ref_ref, 0.5 to 3% for assign_ref and 11 to 13% for len. Rows that only take the merge path, such as and_skewed 4000, move between -5% and +3%; the same binary against itself reaches 4% on a single row. Same machines and toolchains as #385.

Things to look at:

• 64 is CRoaring's threshold and was not tuned. It also applies with simd, where a 3-run check put the break-even against the vector merge at a ratio of 10 to 20.
• lower_bounds uses get_unchecked. Checked indexing made the skewed rows 10 to 18% slower in an earlier 3-run check.
• and_skewed is #[inline(never)]. With #[inline], and_skewed/and 4000 was 12% [+8, +17] slower on Zen 3 and 10% [+3, +18] on Zen 5, although the merge loop is the only code that row runs.
• Without simd, a skewed &= now builds a new Vec instead of filtering in place.
• is_disjoint and is_subset still merge.

Tests compare the skewed path with the existing merge on edge cases and a proptest, in both argument orders, for all three operations.

Authored with Claude (claude-opus-5-5).

[Kerollmops]

Hey @emschwartz 👋

Thank you for your contribution. It looks very interresting but can you rebase on top of #385 once you rebased this one on top of main, please?

Have a nice day 🌵