[chiliec]

What

Closes #10024

FLUX.1 diffusers-format LoRAs that also train the text encoders fail to load (#10024):

File "invokeai/backend/patches/lora_conversions/flux_diffusers_lora_conversion_utils.py", line 288, in lora_layers_from_flux_diffusers_grouped_state_dict
    assert len(grouped_state_dict) == 0
AssertionError

Diffusers keeps the CLIP layers under text_encoder. and (for e.g. OneTrainer) T5 layers under text_encoder_2., next to the transformer. layers. lora_layers_from_flux_diffusers_grouped_state_dict only converted the transformer keys, so any text-encoder key was left over and tripped the closing assert len(grouped_state_dict) == 0. The whole LoRA was rejected, transformer layers included.

Fix

Before stripping the transformer. prefix, split the text-encoder layers out and route them to the existing internal prefixes: text_encoder.* → FLUX_LORA_CLIP_PREFIX, text_encoder_2.* → FLUX_LORA_T5_PREFIX. This mirrors what the kohya (lora_te1_ / lora_te2_) and OneTrainer paths already do, and produces the same internal key shapes (text_model.encoder.layers.* for CLIP, encoder.block.* for T5). Only the transformer layers continue through the diffusers conversion, so the closing assert still guards against genuinely unhandled transformer keys.

Tests

Added test_lora_model_from_flux_diffusers_state_dict_with_text_encoder_layers to the existing diffusers conversion test, asserting that a state dict with transformer.*, text_encoder.*, and text_encoder_2.* keys loads and yields layers under all three prefixes.

Validation

Ran the conversion test module (full file: 3 existing load tests, detector true/false tests, the extra-keys error test, and the new text-encoder test):

ALL 10 TESTS PASSED

RED→GREEN confirmed: the new test raises AssertionError on the unpatched source and passes after the fix. ruff check is clean on both files.

Verified locally on Linux/CPU (torch 2.14.1+cpu). I could not run the project's full pytest suite on this machine — the repo's conftest.py imports the whole app, which pulls in packages with no Linux wheels here (patchmatch); the conversion tests were run directly against the installed package in app-import order.

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[Pfannkuchensack]

Thanks, this fixes #10024. Checked:

• Repro from #10024. A diffusers FLUX.1 LoRA with transformer. and text_encoder. layers now loads instead of tripping the closing assert.
• Patching real module trees. I converted a state dict with CLIP (text_encoder.text_model.encoder.layers.0.{self_attn.q_proj,mlp.fc1}) and T5 (text_encoder_2.encoder.block.0.layer.{0.SelfAttention.q,1.DenseReluDense.wi_0}) layers and applied it with LayerPatcher and the CLIP / T5 prefixes to a CLIP-L-shaped CLIPTextModel and a T5-XXL-shaped T5EncoderModel (one block each, transformers 5.10). Exactly those four weights change, each by B @ A * alpha / rank (relative error ~5e-8), and they are restored on exit. The text-encoder layers get the same alpha as the transformer layers because they go through the same get_lora_layer_values.
• Tests. The new test passes and fails with main's converter. tests/backend/patches: 279 passed, 2 skipped.

Two small things, neither blocking:

• The description mentions #10024 but has no closing keyword, so the issue won't close on merge. Could you add Closes #10024?
• #10045 also appends a test at the end of test_flux_diffusers_lora_conversion_utils.py, so whichever of the two merges second will need a trivial rebase there.

[chiliec]

Thanks for the thorough check! Added Closes #10024 to the description so the issue closes on merge.

#10045 had already landed, so I rebased on main (9427b32) and kept both tests at the end of test_flux_diffusers_lora_conversion_utils.py. The module passes (11 tests), and the new text-encoder test still fails against the old converter.