Skip to content

Page gateway trie endpoints with checkpoint iterator state - #1663

Open
stefangutica wants to merge 1 commit into
developmentfrom
feat/gateway-trie-pagination
Open

stefangutica wants to merge 1 commit into
developmentfrom
feat/gateway-trie-pagination

Conversation

@stefangutica

@stefangutica stefangutica commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Reasoning

  • With the new gateway limits, the node trie endpoints no longer return the whole trie. Large tries used to fail with trie collection limit exceeded or trie operations timeout. Now they answer in pages that are resumed with an iteratorState checkpoint.
  • The API reads only the first page, so token lists of large accounts and the fungible token list come back incomplete.

Proposed Changes

  • GatewayService.getPaged walks every page. It sends iteratorState as base64url without padding and merges the items of every page. It retries 429 system_busy and restarts the walk once on iterator node unavailable.
  • address/:address/esdt, network/esdt/fungible-tokens and address/:address/esdts/roles use getPaged.
  • The proxy routes /address/:address/keys and /address/:address/esdt forward numKeys and iteratorState to the gateway.

How to test

  • npx jest src/test/unit/services/gateway.paged.spec.ts src/test/unit/services/esdt.address.spec.ts
  • Point the gateway URL at a metachain observer. Then GET /address/erd1qqqqqqqqqqqqqqqpqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqllls0lczs7/keys?numKeys=10 returns iteratorState. Send it back as iteratorState=<base64url> until it is absent. Every page together equals the response without numKeys.
  • GET /accounts/:address/tokens for an account with more than 10,000 trie entries returns all its tokens.

@github-actions

Copy link
Copy Markdown

k6 load testing comparison.
Base Commit Hash: 0d3c8ce
Target Commit Hash: 0a7c02e

Metric Base Target Diff
AvgMax9095AvgMax9095AvgMax9095
Accounts45.89652.7573.59102.0844.921474.3058.9863.57-2.12% ✅+125.86% 🔴-19.86% ✅-37.73% ✅
Mex25.50744.5453.0866.9340.56919.5554.4559.21+59.05% 🔴+23.51% 🔴+2.57% 🔴-11.54% ✅
Nodes3749.2717201.6613053.3717201.5965.0416985.0658.1661.19-98.27% ✅-1.26% ✅-99.55% ✅-99.64% ✅
Pool25.49646.9153.1167.2440.68873.1554.5159.31+59.58% 🔴+34.97% 🔴+2.63% 🔴-11.79% ✅
Transactions114.659809.43264.10541.3067.9512756.4861.7471.50-40.73% ✅+30.04% 🔴-76.62% ✅-86.79% ✅
Blocks83.15933.83151.64528.5951.97875.4159.1164.95-37.50% ✅-6.26% ✅-61.02% ✅-87.71% ✅
Tokens140.695348.75423.81589.7040.15254.4554.4359.33-71.46% ✅-95.24% ✅-87.16% ✅-89.94% ✅
Test Run Duration60005.7560003.35

Legend: Avg - Average Response Time, Max - Maximum Response Time, 90 - 90th Percentile, 95 - 95th Percentile
All times are in milliseconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant