[futureweb]

• The Jira issue number for this PR is: MDEV-41757

Description

When a FULLTEXT index is built over existing rows, row_merge_fts_doc_tokenize() moves the start position of the next field by the length of the field plus one byte, also for a field that holds an empty string. fts_init_recover_doc(), which adds the rows that were not synced to the cache again after a crash, does the same. Rows added to an existing index (fts_fetch_doc_from_rec(), fts_fetch_doc_from_tuple()) and phrase search (fts_query_fetch_document()) skip an empty field, as they skip a NULL. So the words after an empty field were stored one byte too far, and phrase search, which starts to match at the stored position, did not find the row.

Before:

CREATE TABLE t (id INT PRIMARY KEY, a VARCHAR(20), c TEXT) ENGINE=InnoDB;
INSERT INTO t VALUES (1, '', 'hello world'), (2, NULL, 'hello world'),
                     (3, 'x', 'hello world');
ALTER TABLE t ADD FULLTEXT(a, c);
INSERT INTO t VALUES (4, '', 'hello world');
SELECT id FROM t WHERE MATCH(a, c) AGAINST('"hello world"' IN BOOLEAN MODE);
-- 2, 3, 4

After: 1, 2, 3, 4.

Fix: an empty field no longer moves the position, in both functions.

Notes for reviewers:

• Indexes built before this fix keep the wrong positions until they are rebuilt, for example with ALTER TABLE ... FORCE.
• Merging this into 13.1 and main needs one more change there: since MDEV-28730, fts_init_recover_all_docs() replaces fts_init_recover_doc(), and it also moves the position by the length of the locally stored part of an externally stored field (field_len from rec_get_nth_field()) instead of the length of its text (see the Jira issue). We have that change ready (on main; 13.1 has the same function) and can add it to the merge or open a separate PR against 13.1.

Found with a random test for the ngram parser proposed in MDEV-10267.

Release Notes

Phrase searches in an InnoDB FULLTEXT index on several columns no longer miss rows in which an earlier indexed column holds an empty string, when the index was built over existing rows or the rows were added to the index again after a crash. Indexes built before the fix must be rebuilt, for example with ALTER TABLE ... FORCE.

How can this PR be tested?

New cases in innodb_fts.phrase (an index built over rows with NULL, '' and ' ' in the first or second column, with the positions from INNODB_FT_INDEX_TABLE) and innodb_fts.crash_recovery (rows that are only in the FTS cache when the server is killed). Without the fix, the phrase search in innodb_fts.phrase misses rows 3 and 4 (empty title), and the one in innodb_fts.crash_recovery misses row 1 after the restart.

The innodb_fts suite and the main fulltext tests pass on 11.4 (debug build). On main with all our fixes and the ngram parser of MDEV-10267, the innodb_fts, maria and plugins suites and the main fulltext, plugin and alter tests pass.

Basing the PR against the correct MariaDB version

• This is a new feature and the PR is based against the latest MariaDB development branch.
• This is a bug fix and the PR is based against the earliest maintained branch in which the bug can be reproduced. (10.11 has the same code, but its first GA release is more than three years old. We can rebase onto 10.11 if wanted.)

PR quality check

• I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

🤖 Generated with Claude Code