Skip to content

crypto: Compute modular inverses with safegcd - #1806

Open
chfast wants to merge 6 commits into
masterfrom
crypto/safegcd
Open

chfast wants to merge 6 commits into
masterfrom
crypto/safegcd

Conversation

@chfast

@chfast chfast commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

The extended binary GCD in modinv_scaled() removes one bit per step,
and a step takes up to two full-width subtractions and a modular
halving. For random inputs its branches mispredict about once per
step. The inversion takes most of the ecadd time and a few percent of
ecrecover and p256verify.

Replace it with the variable-time safegcd algorithm of Bernstein and
Yang, implemented from the paper with the variable-time shortcuts of
libsecp256k1's modinv64_var. Batches of 62 divsteps run on the low
bits and produce a transition matrix, which is then applied to the
full values. The values are signed integers in limbs of 62 bits, so
applying the matrix is a series of multiply-accumulates of 64-bit
signed numbers into a 128-bit signed sum shifted right by 62. Most
iterations of the divsteps loop swap f and g; there the 5-bit inverse
f⁻¹ ≡ (3⋅f) ⊕ 2 (mod 2⁵) lets w = −g⋅f⁻¹ take one multiplication
instead of the three of libsecp256k1's 6-bit formula.

The signed 128-bit integer and the widening multiplication smul() are
added to the intx namespace as the prototype of their intx versions:
__int128 where the compiler has it, two's complement words otherwise.

modinv_scaled() stays constexpr, but at runtime it calls one copy
compiled in modarith.cpp for uint256 and uint384. Compiled in every
translation unit, the code used up the GCC inlining budget of the
pairing code and left a Montgomery multiplication there out of line,
which made ecpairing a few percent slower.

ecadd gets 1.5-1.7x faster, and ecrecover and p256verify about 5%
faster. The results match the binary GCD on a million random inputs,
also with the two's complement 128-bit integer.

@chfast

chfast commented Nov 6, 2026 •

Copy link
Copy Markdown
Member Author

Benchmarks on Zen 3 (5950X), -O3 -march=x86-64-v2. Updated for the current commit (58d933e: signed 62-bit limbs and 128-bit signed sums).

Inversion alone, 2000 independent random inputs per modulus, cycles per inversion, min of interleaved runs, against the previous version of this PR (d510b79):

previous current
GCC 16, bn254 p 4216 4216 +0.0%
GCC 16, secp256k1 p 4240 4239 −0.0%
GCC 16, secp256k1 n 4251 4254 +0.1%
GCC 16, P-256 n 4249 4250 +0.0%
GCC 16, BLS12-381 p (384-bit) 7057 7235 +2.5%
Clang 23, bn254 p 3918 3846 −1.8%
Clang 23, secp256k1 p 3943 3871 −1.8%
Clang 23, secp256k1 n 3954 3879 −1.9%
Clang 23, P-256 n 3951 3880 −1.8%
Clang 23, BLS12-381 p (384-bit) 6479 6269 −3.2%

Nothing in evmone inverts 384-bit values at runtime; the tests do. libsecp256k1 was not measured again; the previous version was on par with its modinv64_var with Clang 23 and up to 8% slower with GCC 16.

Two details of the divsteps loop are worth their lines: keeping −g apart through the swap takes the negation off the critical path (3–6% per inversion), and capping the cancelled bits with & 31 after min(1 − δ, left) was the fastest of four forms with both compilers. Clang 23 keeps update_vr() out of line; [[gnu::always_inline]] changes the 256-bit inversions by −0.3% (GCC) and +0.3% (Clang), so the code has no such attributes.

Precompiles, safegcd / master, cycles (min of interleaved runs) and instructions:

GCC 16 cycles GCC 16 instructions Clang 23 cycles Clang 23 instructions
ecadd 0.66 0.75 0.59 0.61
ecrecover 0.95 0.99 0.96 0.98
p256verify 0.95 0.99 0.96 0.98
ecmul 0.99 0.99 0.97 0.98
ecpairing 0.99 1.00 1.00 1.00

The machine had a load average around 30, so the cycle ratios are good to a few percent. These benchmarks repeat a single input, which lets the branch predictor learn the binary GCD's branches, so they understate the gain for real inputs.

With the inversion compiled in every unit, GCC 16 ecpairing had +3.3% instructions (+4% cycles) in an earlier version: the pairing.cpp unit hits GCC's inline-unit-growth limit, and the extra safegcd body left ModArith::mul out of line in the Fq2 multiplication. The runtime calls go to the copy in modarith.cpp, so this is gone.

@codspeed

codspeed Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 20 benchmarks

2 warnings

⚠️ 297 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 96 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

⚡ 277 improved benchmarks
❌ 20 regressed benchmarks
✅ 602 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_comparison[fork_Osaka-blockchain_test-opcode_SGT--benchmark-gas-value_1M] 7.1 ms 8.8 ms -20.09%
❌ test_comparison[fork_Osaka-blockchain_test-opcode_SLT--benchmark-gas-value_1M] 7.1 ms 8.8 ms -20.01%
❌ baseline/execute/micro/signextend/zero 480.3 µs 498 µs -3.56%
❌ baseline/execute/micro/signextend/one 480.4 µs 498.1 µs -3.56%
❌ test_arithmetic[fork_Osaka-blockchain_test-opcode_SIGNEXTEND--benchmark-gas-value_1M] 7.3 ms 7.6 ms -2.81%
❌ test_mcopy[fork_Osaka-blockchain_test-fixed_src_dst_True-copy_size_0-mem_size_1024-benchmark-gas-value_1M] 7.9 ms 8.1 ms -2.44%
❌ test_mcopy[fork_Osaka-blockchain_test-fixed_src_dst_True-copy_size_0-mem_size_32-benchmark-gas-value_1M] 7.9 ms 8.1 ms -2.44%
❌ test_mod_arithmetic[fork_Osaka-blockchain_test-opcode_MULMOD-mod_bits_127-benchmark-gas-value_1M] 43 ms 44 ms -2.43%
❌ test_mcopy[fork_Osaka-blockchain_test-fixed_src_dst_True-copy_size_0-mem_size_0-benchmark-gas-value_1M] 7.9 ms 8.1 ms -2.42%
❌ test_mcopy[fork_Osaka-blockchain_test-fixed_src_dst_True-copy_size_0-mem_size_10240-benchmark-gas-value_1M] 7.9 ms 8.1 ms -2.41%
❌ test_comparison[fork_Osaka-blockchain_test-opcode_GT--benchmark-gas-value_1M] 8.4 ms 8.6 ms -2.29%
❌ test_comparison[fork_Osaka-blockchain_test-opcode_LT--benchmark-gas-value_1M] 8.4 ms 8.6 ms -2.22%
❌ test_mcopy[fork_Osaka-blockchain_test-fixed_src_dst_True-copy_size_0-mem_size_256-benchmark-gas-value_1M] 7.9 ms 8.1 ms -2.19%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP12-benchmark-gas-value_1M] 9.4 ms 9.5 ms -1.71%
❌ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_zeros_data-opcode_LOG4-benchmark-gas-value_1M] 2.3 ms 2.3 ms -1.46%
❌ test_log[fork_Osaka-blockchain_test-fixed_offset_True-zeros_topic-1_MiB_zeros_data-opcode_LOG3-benchmark-gas-value_1M] 2.3 ms 2.3 ms -1.16%
❌ test_push[fork_Osaka-blockchain_test-opcode_PUSH1-benchmark-gas-value_1M] 9.5 ms 9.6 ms -1.13%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP3-benchmark-gas-value_1M] 9.4 ms 9.5 ms -1.05%
❌ baseline/analyse/main/structarray_alloc 5.3 µs 5.4 µs -1.04%
❌ test_bitwise[fork_Osaka-blockchain_test-opcode_SHR--benchmark-gas-value_1M] 12.3 ms 12.5 ms -1%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing crypto/safegcd (e4b95f0) with master (0a50c91)

Open in CodSpeed

@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.27007% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.23%. Comparing base (6fb712f) to head (e4b95f0).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
lib/evmone_precompiles/modarith.hpp 99.24% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master    #1806    +/-   ##
========================================
  Coverage   98.23%   98.23%            
========================================
  Files         186      187     +1     
  Lines       17192    17313   +121     
  Branches     3963     3975    +12     
========================================
+ Hits        16888    17008   +120     
- Misses        235      236     +1     
  Partials       69       69            
Flag Coverage Δ
eest-develop-gmp 27.19% <94.16%> (+0.43%) ⬆️
eest-legacy 17.10% <0.00%> (-0.11%) ⬇️
eest-libsecp256k1 29.15% <97.08%> (+0.43%) ⬆️
eest-stable 83.94% <97.08%> (+0.20%) ⬆️
evmone-unittests 93.95% <97.08%> (+0.01%) ⬆️
riscv64 94.84% <99.27%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.65% <99.27%> (+0.04%) ⬆️
tooling 94.89% <ø> (+0.02%) ⬆️
tests 99.80% <ø> (ø)
Files with missing lines Coverage Δ
lib/evmone_precompiles/modarith.cpp 100.00% <100.00%> (ø)
lib/evmone_precompiles/modexp.cpp 99.69% <100.00%> (+<0.01%) ⬆️
lib/evmone_precompiles/modarith.hpp 99.47% <99.24%> (-0.53%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chfast
chfast force-pushed the crypto/safegcd branch 4 times, most recently from bc3b258 to d510b79 Compare November 8, 2026 00:06
chfast added a commit that referenced this pull request Nov 9, 2026
The modular inversion is tested through ModArith::inv() on a few values
for each modulus, and through the curve precompiles, whose inversions
see random-looking Montgomery forms.

Test inv() on random values also for the orders of secp256k1 and P-256
and the P-256 prime, telling the non-invertible values of 2^256 - 1 by
its prime factors. Test modinv_scaled() on small values, on values near
powers of 2, and on the inputs needing the most divsteps of the safegcd
algorithm found for 256 and 384 bits. Add bn254 addition, ecrecover
and p256verify vectors whose inversions start from 2^211 or need the
most divsteps found for their modulus.

The tests come from the safegcd inversion (#1806) and pass with the
current binary GCD.
The extended binary GCD in modinv_scaled() removes one bit per step,
and a step takes up to two full-width subtractions and a modular
halving. For random inputs its branches mispredict about once per
step. The inversion takes most of the ecadd time and a few percent of
ecrecover and p256verify.

Replace it with the variable-time safegcd algorithm of Bernstein and
Yang, implemented from the paper with the variable-time shortcuts of
libsecp256k1's modinv64_var. Batches of 62 divsteps run on the low
bits and produce a transition matrix, which is then applied to the
full values. The values are signed integers in limbs of 62 bits, so
applying the matrix is a series of multiply-accumulates of 64-bit
signed numbers into a 128-bit signed sum shifted right by 62. Most
iterations of the divsteps loop swap f and g; there the 5-bit inverse
f⁻¹ ≡ (3⋅f) ⊕ 2 (mod 2⁵) lets w = −g⋅f⁻¹ take one multiplication
instead of the three of libsecp256k1's 6-bit formula.

The signed 128-bit integer and the widening multiplication smul() are
added to the intx namespace as the prototype of their intx versions:
__int128 where the compiler has it, two's complement words otherwise.

modinv_scaled() stays constexpr, but at runtime it calls one copy
compiled in modarith.cpp for uint256 and uint384. Compiled in every
translation unit, the code used up the GCC inlining budget of the
pairing code and left a Montgomery multiplication there out of line,
which made ecpairing a few percent slower.

ecadd gets 1.5-1.7x faster, and ecrecover and p256verify about 5%
faster. The results match the binary GCD on a million random inputs,
also with the two's complement 128-bit integer.
Pin intx to its main branch commit cd35df7 for the signed integer
types until an intx release has them. Adapt modexp to the division
functions of intx main: udivrem_by1() and udivrem_by2() take
a quotient output, and Knuth's division of normalized values is
udivrem_knuth_normalized().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant