[LuciferYang]

Closes #10367.

A narrow-key dictionary column (Int8, UInt8, Int16, UInt16 keys) could not be written or compacted once the batches that go into one data file referenced more distinct values than the key type addresses, even though each batch fit. The 2.1+ encoder merges the dictionaries of the batches it buffers into one page and, since #9958, rejects the page with InvalidInput.

The fragment writer now tracks, per data file, the distinct values each such column holds at any struct or list depth (DictionaryValueBudget). It finishes the file before a batch would push a column past its key capacity and continues in a new one, through the same close path as max_bytes_per_file. When compaction has planned row counts, the remaining rows are rebalanced the same way they are after a byte-limit close. Each data file is decoded on its own, so readers are unchanged. Files written this way read back with pylance 13.0.0 and 2.0.0. A file closed this way still ends at max_rows_per_file rows, since the writer takes no more rows than the current file has room for.

Values are counted with encoded_dictionary_values in lance-encoding, which follows the encoder's null handling. The encoder drops key validity, so the in-range key under a null slot also selects a value. Only value types the encoder can merge exactly are tracked: strings, binary, integers and floats. Columns of other value types keep the encoder's error.

Not covered, all keeping today's behavior:

• The 2.0 writer, which has its own key-type check.
• Writes that must produce exactly one file, such as the mem_wal flush and add/update columns.
• Dictionaries inside a Map.
• Binary-copy compaction (opt-in).

Compaction plans by row count, so it rewrites the split fragments into the same number of fragments on every run. Before this change it failed on every run.

Tests

• write_splits_files_before_dictionary_outgrows_key_type: three batches of 100 values starting at 0, 28 and 128, written in one go and through compaction, on 2.1 and 2.2, with and without stable row IDs. It checks that the first file holds exactly 128 values ([2000, 1000] rows), that compaction removes 3 fragments and adds 2, and that the values read back.
• write_keeps_max_rows_per_file_after_dictionary_close: three 150-row batches with max_rows_per_file = 200, where the first file closes early for its dictionary. It checks that the files hold [150, 200, 100] rows, so the next file still stops at the row limit.
• test_flush_rejects_dictionary_overflow_instead_of_splitting: a MemWAL flush whose Int8 dictionary column holds 200 distinct values fails with the encoder's key-type error instead of writing a second fragment.
• dictionary_budget unit tests: top-level, struct-child and list-item dictionaries, a sliced list, the max_rows cap, a value reached only through a null slot, an unreferenced first value, empty batches, view-typed values, and untracked value types.
• test_encoded_dictionary_values: in-range and out-of-range keys under null slots, and null values.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The fixed-size digest tracker addresses the retained-memory finding. File splitting preserves the declared dictionary key types and released-reader compatibility. No blocking issues remain.

[LuciferYang]

Thank you @Xuanwo