Repository navigation
Conversation
|
Benchmarks on Zen 3 (5950X), Inversion alone, 2000 independent random inputs per modulus, cycles per inversion, min of interleaved runs, against the previous version of this PR (d510b79):
Nothing in evmone inverts 384-bit values at runtime; the tests do. libsecp256k1 was not measured again; the previous version was on par with its Two details of the divsteps loop are worth their lines: keeping −g apart through the swap takes the negation off the critical path (3–6% per inversion), and capping the cancelled bits with Precompiles, safegcd / master, cycles (min of interleaved runs) and instructions:
The machine had a load average around 30, so the cycle ratios are good to a few percent. These benchmarks repeat a single input, which lets the branch predictor learn the binary GCD's branches, so they understate the gain for real inputs. With the inversion compiled in every unit, GCC 16 ecpairing had +3.3% instructions (+4% cycles) in an earlier version: the pairing.cpp unit hits GCC's |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1806 +/- ##
========================================
Coverage 98.23% 98.23%
========================================
Files 186 187 +1
Lines 17192 17313 +121
Branches 3963 3975 +12
========================================
+ Hits 16888 17008 +120
- Misses 235 236 +1
Partials 69 69
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
ce29f45 to
80e0f1f
Compare
bc3b258 to
d510b79
Compare
The modular inversion is tested through ModArith::inv() on a few values for each modulus, and through the curve precompiles, whose inversions see random-looking Montgomery forms. Test inv() on random values also for the orders of secp256k1 and P-256 and the P-256 prime, telling the non-invertible values of 2^256 - 1 by its prime factors. Test modinv_scaled() on small values, on values near powers of 2, and on the inputs needing the most divsteps of the safegcd algorithm found for 256 and 384 bits. Add bn254 addition, ecrecover and p256verify vectors whose inversions start from 2^211 or need the most divsteps found for their modulus. The tests come from the safegcd inversion (#1806) and pass with the current binary GCD.
The extended binary GCD in modinv_scaled() removes one bit per step, and a step takes up to two full-width subtractions and a modular halving. For random inputs its branches mispredict about once per step. The inversion takes most of the ecadd time and a few percent of ecrecover and p256verify. Replace it with the variable-time safegcd algorithm of Bernstein and Yang, implemented from the paper with the variable-time shortcuts of libsecp256k1's modinv64_var. Batches of 62 divsteps run on the low bits and produce a transition matrix, which is then applied to the full values. The values are signed integers in limbs of 62 bits, so applying the matrix is a series of multiply-accumulates of 64-bit signed numbers into a 128-bit signed sum shifted right by 62. Most iterations of the divsteps loop swap f and g; there the 5-bit inverse f⁻¹ ≡ (3⋅f) ⊕ 2 (mod 2⁵) lets w = −g⋅f⁻¹ take one multiplication instead of the three of libsecp256k1's 6-bit formula. The signed 128-bit integer and the widening multiplication smul() are added to the intx namespace as the prototype of their intx versions: __int128 where the compiler has it, two's complement words otherwise. modinv_scaled() stays constexpr, but at runtime it calls one copy compiled in modarith.cpp for uint256 and uint384. Compiled in every translation unit, the code used up the GCC inlining budget of the pairing code and left a Montgomery multiplication there out of line, which made ecpairing a few percent slower. ecadd gets 1.5-1.7x faster, and ecrecover and p256verify about 5% faster. The results match the binary GCD on a million random inputs, also with the two's complement 128-bit integer.
d510b79 to
58d933e
Compare
Pin intx to its main branch commit cd35df7 for the signed integer types until an intx release has them. Adapt modexp to the division functions of intx main: udivrem_by1() and udivrem_by2() take a quotient output, and Knuth's division of normalized values is udivrem_knuth_normalized().
The extended binary GCD in modinv_scaled() removes one bit per step,
and a step takes up to two full-width subtractions and a modular
halving. For random inputs its branches mispredict about once per
step. The inversion takes most of the ecadd time and a few percent of
ecrecover and p256verify.
Replace it with the variable-time safegcd algorithm of Bernstein and
Yang, implemented from the paper with the variable-time shortcuts of
libsecp256k1's modinv64_var. Batches of 62 divsteps run on the low
bits and produce a transition matrix, which is then applied to the
full values. The values are signed integers in limbs of 62 bits, so
applying the matrix is a series of multiply-accumulates of 64-bit
signed numbers into a 128-bit signed sum shifted right by 62. Most
iterations of the divsteps loop swap f and g; there the 5-bit inverse
f⁻¹ ≡ (3⋅f) ⊕ 2 (mod 2⁵) lets w = −g⋅f⁻¹ take one multiplication
instead of the three of libsecp256k1's 6-bit formula.
The signed 128-bit integer and the widening multiplication smul() are
added to the intx namespace as the prototype of their intx versions:
__int128 where the compiler has it, two's complement words otherwise.
modinv_scaled() stays constexpr, but at runtime it calls one copy
compiled in modarith.cpp for uint256 and uint384. Compiled in every
translation unit, the code used up the GCC inlining budget of the
pairing code and left a Montgomery multiplication there out of line,
which made ecpairing a few percent slower.
ecadd gets 1.5-1.7x faster, and ecrecover and p256verify about 5%
faster. The results match the binary GCD on a million random inputs,
also with the two's complement 128-bit integer.