Optimize ChannelMonitor persistence on block connections.
authorG8XSU <3442979+G8XSU@users.noreply.github.com>
Fri, 14 Jun 2024 23:56:36 +0000 (16:56 -0700)
committerG8XSU <3442979+G8XSU@users.noreply.github.com>
Wed, 19 Jun 2024 07:04:23 +0000 (00:04 -0700)
commitbf28957f6d488d1258a1e6f21af29344b18e2b75
tree9473656bc431c7d708fe10d221730b03da3357c8
parent1912d8df01997c27ebe73323473ec6b1d45c1f1c
Optimize ChannelMonitor persistence on block connections.

Currently, every block connection triggers the persistence of all
ChannelMonitors with an updated best_block. This approach poses
challenges for large node operators managing thousands of channels.
Furthermore, it leads to a thundering herd problem
(https://en.wikipedia.org/wiki/Thundering_herd_problem), overwhelming
the storage with simultaneous requests.

To address this issue, we now persist ChannelMonitors at a
regular cadence, spreading their persistence across blocks to
mitigate spikes in write operations.

Outcome: After doing this, Ldk's IO footprint should be reduced
by ~50 times. The processing time required to sync each block
will be significantly reduced, particularly for nodes with 1000s
of channels, as write latency plays a significant role in this process.
As a result, the Node/ChainMonitor will be blocked for a shorter
duration, leading to further efficiency gains.
lightning/src/chain/chainmonitor.rs
lightning/src/chain/channelmonitor.rs
lightning/src/chain/onchaintx.rs