Repository navigation
[models] Speed up pre/post-processing, fuse attention and cache MASTER/PARSeq decoding, fix a crash on 1-px word crops - #2228
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2228 +/- ##
==========================================
+ Coverage 98.21% 98.24% +0.03%
==========================================
Files 181 181
Lines 12578 12624 +46
==========================================
+ Hits 12354 12403 +49
+ Misses 224 221 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes and speed-ups from an end-to-end sweep of the library. It ran 5 predictor configurations (default, layout + tables + formulas, rotated/straightened, rotated + layout, KIE) over ~120 real images and 10 PDFs, and exercised every exporter on each result. Predictions are unchanged throughout (see Verification).
Fixes
assume_straight_pages=Falseon a degenerate word crop.np.rot90of a 1-pixel-high crop has a negative stride on an axis of size 1. NumPy still flags it as C-contiguous, sonp.ascontiguousarrayreturns it unchanged, andtorch.from_numpyrejects it. The whole document failed.PreProcessornow copies such arrays.doctr.io.tensor_from_numpyfailed on any flipped or rotated view (e.g.img[::-1]), for every dtype. It now copies them first.SVT/IC03: the XML element text is checked before use. An entry without a file name now raises the existingFileNotFoundErrorinstead of crashing inos.path.join. In SVT's recognition mode, a word without text is skipped instead of raising aTypeErroronlen(None). This also fixes 19 mypy errors that only show up whentypes-tqdmis installed, as it is in some dev environments: thetqdm(...)loop then keeps theElementtype instead ofAny.tests/common/test_models_layout.pyandtests/pytorch/test_models_layout.pyshared a basename, so collecting both folders in one session raised an "import file mismatch" error. Renamed the latter totest_models_layout_pt.py.Performance
No thread pool in
PreProcessorandDetectionPostProcessor. The per-item work (word crop resizing, per-box contour processing) is bound by Python overhead, so threads only competed for the GIL and oversubscribed the cores on top of OpenCV/PyTorch's own threads:modules/transformer: attention usesF.scaled_dot_product_attention. Recognition gets 5–16% faster for PARSeq / ViTSTR / MASTER.scaled_dot_product_attentionkeeps its signature but returnsNonein place of the attention weights, which nothing used.Incremental, KV-cached autoregressive decoding for MASTER and PARSeq. The image features are projected once per batch, and each step only processes the newest token, with the previous ones cached. Before, every step re-ran the decoder over the whole sequence. MASTER did so over all 50 positions, re-projecting the encoder memory of its 3 layers each time. A token only attends to the previous ones, so the result is the same.
MultiHeadAttention.project_kv/attendand an optionalcachein the transformerDecoder;PARSeqDecoderaccepts precomputedcontent_kv/memory_kv.MASTER's ONNX export also takes 8 s instead of 123 s.
MASTER notes:
(N, max_length, vocab_size + 1).End-to-end latency per page from items 1 and 2 (RTX 4080,
db_mobilenet_v3_large+parseq, alternating runs of main vs this branch):crnn_mobilenet_v3_smallTests
tests/conftest.py, pytest-xdist only:cpu_count // num_workersPyTorch/OpenCV threads. Before, every worker started one thread per core:test_add_whitelist*[master]took up to 1064 s with-n 8versus ~2 s alone.-n 4the suite had 29 CUDA out-of-memory failures before; now it is green.-n 8: 41 min → 1 min 54 s, 1366 passed. CI runs serially and is unaffected.test_resize(174 lines) → 7 focused, parametrized tests.test_documentbuilder_tables(139 lines) → 6 tests, one per scenario.LayoutDatasetinvalid-label cases write their labels totmp_pathinstead of overwriting the session-wide fixture._moved_awaycontext manager, so the shared folder is restored even if a check fails.test_preprocessor_viewsandtest_tensor_from_numpy_views. Both fail without the fixes.Verification
add_whiteliston the fullocr_predictor: 17 mostly multilingual documents, withmask(German, Polish + German),nearest(anyascii andmapping="weights"), the context manager andremove(). Same words as main for PARSeq and MASTER, and no character outside the whitelist.make qualityis clean.