[futureweb]

• The Jira issue number for this PR is: MDEV-41754

Description

Since MDEV-28730 (81c3836), InnoDB reads the FTS tables through QueryExecutor::read_by_index(). The callbacks of these reads return STOP when the scan has passed the records it needs (the next FTS_DOC_ID, or the next word of an auxiliary table), but row_search_mvcc() did not return the resulting DB_SUCCESS_LOCKED_REC: one callback path turned it into DB_SUCCESS, the other went on to the next record. So the scan went on to the end of the index. Each document fetch of phrase search, proximity search and query expansion (fts_query_fetch_document()) read from that document to the end of FTS_DOC_ID_INDEX, and each word lookup read to the end of the auxiliary table.

'"gamma delta"' on a table where every row matches, debug builds (from the Jira issue):

rows	11.4	main
1000	0.17 s	10.1 s
2000	0.41 s	41.1 s
4000	1.18 s	161.6 s

Fix:

• Keep DB_SUCCESS_LOCKED_REC in both callback paths of row_search_mvcc(), so that read_by_index() stops. This also restores the OPTIMIZE TABLE limit of innodb_ft_num_word_optimize words per pass, as in 11.4.
• Now that the scan stops, the prefix (LIKE) comparison in AuxRecordReader::compare_record() must not stop too early. It compares the whole word with the prefix in the collation's strnncoll() prefix mode, as IB_LIKE_PREFIX did in 11.4. It stops above the range only where the matches are adjacent in the index. It keeps scanning for collations with several levels (levels_for_order > 1), where secondary or tertiary weights can place nonmatching words between matches, and for non-UCA collations with a complex mapping (MY_CS_NON1TO1), such as tis620_thai_ci, which reorders Thai vowels before comparing. EQUAL mode keeps the full collation comparison.
• select_for_update() of a CONFIG key uses ROW_SEL_EXACT. A missing key now takes a gap lock on the next record instead of scanning all following CONFIG rows with next-key X locks up to the supremum.
• New debug-only status variable Innodb_row_search_callback_records, counted with SET debug_dbug='+d,fts_count_callback_records', for the test. innodb.innodb_status_variables excludes it like the other debug-only counters.

Notes for reviewers:

• Prefix matching follows collation weights, as in 11.4 and MyISAM: in a Czech collation, where ch is a contraction, 'kac*' does not match 'kachna'.
• The FTS cache path, fts_cache_find_wildcard(), has the same assumption that prefix matches are next to each other, which does not hold for multi-level collations. This is older (11.4 has it too) and not changed here: MDEV-41770.
• With the built-in parser, Thai letters are not word characters, so the tis620_thai_ci case is reached only through parser plugins; there is no mtr test for it.
• When document processing continues, for example during query expansion or an unsuccessful phrase verification, the next candidate can need a clustered index lookup before its document ID is rejected. This could be optimized separately.

Release Notes

InnoDB fulltext searches no longer read to the end of the index for each matching row or word, which made phrase searches, proximity searches and searches WITH QUERY EXPANSION very slow on large tables.

How can this PR be tested?

New test innodb_fts.read_to_end (debug build). It counts the internal record visits of phrase search, proximity search, query expansion, and word and prefix lookups. Except for the lookups of the last word and of a word after it, the limits cannot be met by reading to the end (for the 32 documents of the phrase search that would be 32*33/2 = 528 visits; the limit is 96). It also checks the rows returned by prefix searches with accent and case variants in utf8mb4_uca1400_ai_ci, utf8mb4_bin, utf8mb4_uca1400_as_ci and utf8mb4_uca1400_as_cs. Without the multi-level part of the fix, 'aab*' in utf8mb4_uca1400_as_ci returns only 2 of the 4 matching words. innodb.innodb_status_variables is updated for the new variable.

The innodb_fts suite, innodb.innodb_status_variables and innodb.index_lock_upgrade pass on 13.1 (debug build). On main with all our fixes and the ngram parser of MDEV-10267, the innodb_fts, maria and plugins suites and the main fulltext, plugin and alter tests pass.

Basing the PR against the correct MariaDB version

• This is a new feature and the PR is based against the latest MariaDB development branch.
• This is a bug fix and the PR is based against the earliest maintained branch in which the bug can be reproduced. (81c3836 is in 13.1, but not in 13.0, 12.3, 11.8 or 11.4.)

PR quality check

• I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

🤖 Generated with Claude Code