[futureweb]

• The Jira issue number for this PR is: MDEV-37370

Description

fts_phrase_or_proximity_search() scanned the match list of every other query word from the start again for each document of the first word (see the removed FIXME). As in the fix for MySQL Bug #37682648 (MySQL 8.0.43), without its boilerplate, each word keeps a cursor into its match list, so matching sorted lists is linear in their lengths for a fixed number of query words. As suggested in the Jira issue, ib_vector_t is kept and there is no Vector_wrapper. The template parameter for the query mode is not part of this change.

The match lists are not always sorted by document ID: cache nodes and overlapping posting lists on disk can produce unsorted lists. With such lists the old scan could already miss documents. For example, after two transactions committed out of document ID order, alpha has the list 10 and bravo the list 30, 10; the scan for bravo stops at 30, and '"alpha bravo"' misses document 10. So each list is checked once and sorted by document ID with std::sort() if it is not sorted, before the cursors are used.

Benchmark (release build of main with all our fixes, the ngram parser proposed in MDEV-10267 with n=2, 20,000 Latin documents, boolean mode queries in which each query word becomes a phrase of n-grams): 37.2 s without this commit, 10.1 s with it, with the same number of hits (1,076,673). MySQL 8.4.13 takes 9.5 s on the same data. A CJK corpus, which has short match lists, is unchanged.

Note for reviewers: the Jira issue has Fix Version 11.8 and is assigned to Thirunarayanan. This PR is based on 11.4 because the sort also fixes wrong results there. If you prefer 11.8, I will rebase it.

Release Notes

InnoDB fulltext phrase and proximity searches are faster for words that occur in many rows. They also no longer miss rows in some cases after transactions were committed out of document ID order.

How can this PR be tested?

New test innodb_fts.phrase_proximity_order: phrase and proximity searches with gaps between document IDs, repeated words, lists that end early, data in the cache and on disk, and two out-of-order commits that give an unsorted first and an unsorted second match list. Without the fix, the phrase and proximity queries miss document 10 after the out-of-order commit.

The innodb_fts suite and the main fulltext tests pass on 11.4 (debug build). On main with all our fixes and the ngram parser of MDEV-10267, the innodb_fts, maria and plugins suites and the main fulltext, plugin and alter tests pass.

Basing the PR against the correct MariaDB version

• This is a new feature and the PR is based against the latest MariaDB development branch.
• This is a bug fix and the PR is based against the earliest maintained branch in which the bug can be reproduced. (Jira also lists 10.11, whose first GA release is more than three years old. For the Fix Version 11.8, see the note above.)

PR quality check

• I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

🤖 Generated with Claude Code