Repository navigation
Stop caching every mempool tx for bitcoind - #1216
benthecarman wants to merge 1 commit into
Conversation
With a bitcoind chain source, every mempool transaction was fetched into `mempool_txs_cache` and kept there until it left the mempool, so the node held a decoded copy of every transaction in bitcoind's mempool, which can reach hundreds of MB when the mempool is full. Only the onchain wallet consumes these transactions, and it keeps just the relevant ones. The cache only needs the transactions we have to emit again, i.e., those that entered the mempool above our best processed height. Restrict it to those. As a result, if a poll fails partway through, the retry fetches the transactions it already fetched again, rather than taking them from the cache. This only lowers steady-state memory use. `mempool_entries_cache` still holds one small entry per mempool transaction, and the first poll after startup still collects the whole mempool before handing it to the wallet. Also stop pre-sizing the transaction cache and the emit buffer to the whole mempool on every poll. This change was made with the help of Claude Code.
|
👋 Thanks for assigning @tnull as a reviewer! |
| match self.get_raw_transaction(&entry.txid).await { | ||
| Ok(Some(tx)) => { | ||
| mempool_txs_cache.insert(entry.txid, (tx.clone(), entry.time)); | ||
| if !ancestor_within_height { |
There was a problem hiding this comment.
One concern with caching only !ancestor_within_height txs: if a getrawtransaction fails halfway through the first big poll, we return Err below (before latest_mempool_timestamp is stored), so the retry has no cache to resume from and refetches every ancestor_within_height tx. At startup latest_mempool_timestamp is 0, so that is nearly the whole mempool. The old code resumed from the cache.
In a quick model (20k-entry mempool at startup, transient failure per RPC), a 1-in-10,000 failure rate went from ~2.9 polls / ~20k RPCs with the old code to ~6.7 polls / ~58k RPCs with this PR, and the cost grows quickly with mempool size and error rate.
What we need to keep is exactly the set that may still be emitted, so the insert could be:
if !ancestor_within_height || entry.time > prev_mempool_time {
mempool_txs_cache.insert(entry.txid, (tx.clone(), entry.time));
}Steady-state memory stays small, since a successful poll advances latest_mempool_timestamp, so the next retain drops these entries.
| // Only transactions we may have to emit again are cached, i.e., those that entered the | ||
| // mempool above our best processed height. Caching every mempool transaction would keep a | ||
| // full copy of the mempool in memory. | ||
| mempool_txs_cache.retain(|txid, _| { |
There was a problem hiding this comment.
To go with the insert condition suggested above, the retain could keep an entry while it may still need emitting, i.e. the same condition as the skip check further down:
mempool_txs_cache.retain(|txid, _| {
mempool_entries_cache.get(txid).is_some_and(|entry| {
entry.height > best_processed_height || entry.time > prev_mempool_time
})
});prev_mempool_time is already loaded at the top of this function. After a successful poll latest_mempool_timestamp advances, so the next retain drops everything that was only kept for the retry, and the steady-state cache stays as small as with the current PR. I haven't compiled this against your branch, so please double-check it.
Ldk-server was taking a lot of memory on my server. Doing this reduced memory by 100MB for me
With a bitcoind chain source, every mempool transaction was fetched into
mempool_txs_cacheand kept there until it left the mempool, so the node held a full decoded copy of bitcoind's mempool (roughly 0.9 KB per transaction, i.e. hundreds of MB when the mempool is full). Only the onchain wallet consumes these transactions, and it keeps just the relevant ones.The cache only serves transactions we may have to emit again, i.e. those that entered the mempool above our best processed height. Restrict it to those. Also stop shrinking the entries map and pre-sizing the emit buffer to the full mempool on every two-second poll, which churned multi-megabyte allocations and fragmented the heap.