Skip to content

test: Avoid copying JSON strings in the test loaders - #1817

Closed
chfast wants to merge 1 commit into
masterfrom
test/json-string-ref
Closed

chfast wants to merge 1 commit into
masterfrom
test/json-string-ref

Conversation

@chfast

@chfast chfast commented Nov 16, 2026

Copy link
Copy Markdown
Member

The state and blockchain test loaders read JSON strings with getstd::string(), which copies each string only to parse or compare it. Use get_ref<const std::string&>() to reference the string stored in the JSON document instead. A non-string value still throws json::type_error, only with a different message.

This removes an allocation and a copy from loading most test values and makes the state tests execute about 0.1% fewer instructions. The copies were found with LLVM's CopyProf (-fcopyprof).

The state and blockchain test loaders read JSON strings with
get<std::string>(), which copies each string only to parse or compare
it. Use get_ref<const std::string&>() to reference the string stored
in the JSON document instead. A non-string value still throws
json::type_error, only with a different message.

This removes an allocation and a copy from loading most test values and
makes the state tests execute about 0.1% fewer instructions. The copies
were found with LLVM's CopyProf (-fcopyprof).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues requiring changes.

No open issues 🎉

What changed in this PR

Reduces unnecessary string allocations and copies in the state and blockchain test loaders by reading strings directly from the JSON document.

Changes:

  • Replaces get<std::string>() with get_ref<const std::string&>() for parsing, comparisons, and expected-exception mapping.
  • Preserves json::type_error for non-string inputs, with different diagnostic wording.
File Description
test/​utils/​statetest_loader.cpp Borrows JSON strings for shared value conversions, destination checks, and exception mapping.
test/​utils/​blockchaintest_loader.cpp Borrows expected-exception strings before mapping them.

Review effort: Balanced

@codspeed

codspeed Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 0.52%

2 warnings

⚠️ 23 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 5 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

⚡ 7 improved benchmarks
❌ 16 regressed benchmarks
✅ 876 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP13-benchmark-gas-value_1M] 9.3 ms 9.5 ms -2.29%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP6-benchmark-gas-value_1M] 9.3 ms 9.5 ms -2.1%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP16-benchmark-gas-value_1M] 9.3 ms 9.5 ms -2.05%
❌ baseline/analyse/main/structarray_alloc 5.2 µs 5.3 µs -2.04%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP10-benchmark-gas-value_1M] 9.3 ms 9.5 ms -1.91%
❌ test_calldatacopy_from_call[fork_Osaka-blockchain_test-calldata_size_1024-non_zero_data_False-fixed_src_dst_True-benchmark-gas-value_1M] 5.2 ms 5.2 ms -1.77%
❌ test_calldatasize[fork_Osaka-blockchain_test-zero_data_True-calldata_size_32-benchmark-gas-value_1M] 13.9 ms 14.1 ms -1.53%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP15-benchmark-gas-value_1M] 9.4 ms 9.5 ms -1.18%
❌ test_block_context_ops[fork_Osaka-blockchain_test-opcode_NUMBER-benchmark-gas-value_1M] 14.1 ms 14.2 ms -1.16%
❌ test_msize[fork_Osaka-blockchain_test-mem_size_1000-benchmark-gas-value_1M] 12.9 ms 13 ms -1.15%
❌ baseline/analyse/main/swap_math 14.1 µs 14.2 µs -1.14%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP14-benchmark-gas-value_1M] 9.4 ms 9.5 ms -1.08%
❌ test_calldatasize[fork_Osaka-blockchain_test-zero_data_True-calldata_size_1024-benchmark-gas-value_1M] 14.1 ms 14.2 ms -1.07%
❌ test_dup[fork_Osaka-blockchain_test-opcode_DUP5-benchmark-gas-value_1M] 9.4 ms 9.5 ms -1.04%
❌ test_calldatasize[fork_Osaka-blockchain_test-zero_data_False-calldata_size_32-benchmark-gas-value_1M] 14 ms 14.2 ms -1.02%
❌ test_returndatasize_zero[fork_Osaka-blockchain_test-benchmark-gas-value_1M] 14.9 ms 15.1 ms -1.02%
⚡ test_codecopy[fork_Osaka-blockchain_test-fixed_src_dst_True-0.75x max code size-benchmark-gas-value_1M] 42.9 ms 41.8 ms +2.72%
⚡ baseline/analyse/main/sha1_shifts 11 µs 10.8 µs +2.06%
⚡ test_calldatasize[fork_Osaka-blockchain_test-zero_data_False-calldata_size_0-benchmark-gas-value_1M] 14.3 ms 14 ms +1.83%
⚡ test_calldatasize[fork_Osaka-blockchain_test-zero_data_False-calldata_size_256-benchmark-gas-value_1M] 14.1 ms 13.9 ms +1.77%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing test/json-string-ref (8edb50a) with master (ba92871)

Open in CodSpeed

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.23%. Comparing base (ba92871) to head (8edb50a).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1817   +/-   ##
=======================================
  Coverage   98.23%   98.23%           
=======================================
  Files         186      186           
  Lines       17192    17192           
  Branches     3963     3963           
=======================================
  Hits        16888    16888           
  Misses        235      235           
  Partials       69       69           
Flag Coverage Δ
eest-develop-gmp 26.74% <81.81%> (-0.01%) ⬇️
eest-legacy 17.20% <81.81%> (-0.01%) ⬇️
eest-libsecp256k1 28.71% <81.81%> (-0.01%) ⬇️
eest-stable 83.73% <90.90%> (-0.02%) ⬇️
evmone-unittests 93.94% <100.00%> (ø)
riscv64 94.81% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.60% <ø> (ø)
tooling 94.87% <100.00%> (ø)
tests 99.80% <ø> (ø)
Files with missing lines Coverage Δ
test/utils/blockchaintest_loader.cpp 97.14% <100.00%> (ø)
test/utils/statetest_loader.cpp 92.60% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chfast

chfast commented Nov 17, 2026

Copy link
Copy Markdown
Member Author

Not worth it, we plan to migrate JSON library.

@chfast chfast closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants