]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
2 weeks agoAvoid startup `PeerState` entries for peers with unfunded channels 2024-10-mon-ids-after-close
Matt Corallo [Thu, 10 Oct 2024 19:42:16 +0000 (19:42 +0000)]
Avoid startup `PeerState` entries for peers with unfunded channels

If a peer creates a channel with us which never reaches the funding
stage (or never gets any commitment updates after creation), we'll
avoid inserting the `update_id` into
`closed_channel_monitor_update_ids` at runtime to avoid keeping a
`PeerState` entry around for no reason. However, on startup we
still create a `ChannelMonitorUpdate` with a `ChannelForceClosed`
update step to ensure the `ChannelMonitor` is locked and shut down.

This is pretty redundant, and results in a bunch of on-startup
`ChannelMonitorUpdate`s for any old but non-archived
`ChannelMonitor`s. Instead, here, we check if a `ChannelMonitor`
already saw a `ChannelForceClosed` update step before we generate
the on-startup `ChannelMonitorUpdate`.

This also allows us to skip the `closed_channel_monitor_update_ids`
insertion as we can be confident we'll never have a
`ChannelMonitorUpdate` for this channel at all.

2 weeks agoStop using a constant for monitor `update_id`s after closure
Matt Corallo [Sun, 6 Oct 2024 19:58:29 +0000 (19:58 +0000)]
Stop using a constant for monitor `update_id`s after closure

Because `ChannelManager` doesn't have a corresponding `Channel`
after the channels are closed, we'd always used an `update_id` of
`u64::MAX` for any `ChannelMonitorUpdate`s we need to build after
the channel is closed.

This completely breaks the abstraction of `update_id`s and leaks
into persistence logic - because we might have more than one
`ChannelMonitorUpdate` with the same (`u64::MAX`) value, suddenly
instead of being able to safely use `update_id` as IDs, the
`MonitorUpdatingPersister` has to have special logic to handle
this.

Worse, because we don't have a unique ID with which to refer to
post-close `ChannelMonitorUpdate`s we cannot track when they
complete async persistence. This means we cannot properly support
async persist for forwarded payments where the inbound edge has hit
the chain prior to the preimage coming to us.

Here we rectify this by using consistent `update_id`s even after a
channel has closed. In order to do so we have to keep some state
for all channels for which the `ChannelMonitor` has not been
archived (after which point we can be confident we will not need to
update them). While this violates our long-standing policy of
having no state at all in `ChannelManager`s for closed channels,
its only a `(ChannelId, u64)` pair per channel, so shouldn't be
problematic for any of our users (as they already store a whole
honkin `ChannelMonitor` for these channels anyway).

While limited changes are made to the connection-count-limiting
logic, reviewers should carefully analyze the interactions the new
map created here has with that logic.

2 weeks agoPrefer to use `MonitorUpdateRegeneratedOnStartup` where possible
Matt Corallo [Sun, 6 Oct 2024 19:54:32 +0000 (19:54 +0000)]
Prefer to use `MonitorUpdateRegeneratedOnStartup` where possible

In the next commit we'll drop the magic `u64::MAX`
`ChannelMonitorUpdate::update_id` value used when we don't know the
`ChannelMonitor`'s `latest_update_id` (i.e. when the channel is
closed). In order to do so, we will store further information about
`ChannelMonitor`s in the per-peer structure, keyed by the
counterparty's node ID, which will be used when applying
`ChannelMonitorUpdate`s to closed channels.

By taking advantage of the change in the previous commit, that
information is now reliably available when we generate the
`ChannelMonitorUpdate` (when claiming HTLCs), but in order to
ensure it is available when applying the `ChannelMonitorUpdate` we
need to use `BackgroundEvent::MonitorUpdateRegeneratedOnStartup`
instead of
`BackgroundEvent::ClosedMonitorUpdateRegeneratedOnStartup` where
possible.

Here we do this, leaving `ClosedMonitorUpdateRegeneratedOnStartup`
only used to ensure very old channels (created in 0.0.118 or
earlier) which are not in the `ChannelManager` are force-closed on
startup.

2 weeks agoReq the counterparty node id when claiming against a closed chan
Matt Corallo [Wed, 9 Oct 2024 19:05:18 +0000 (19:05 +0000)]
Req the counterparty node id when claiming against a closed chan

Currently we store in-flight `ChannelMonitorUpdate`s in the
per-peer structure in `ChannelManager`. This is nice and simple as
we're generally updating it when we're updating other per-peer
data, so we already have the relevant lock(s) and map entries.

Sadly, when we're claiming an HTLC against a closed channel, we
didn't have the `counterparty_node_id` available until it was
added in 0.0.124 (and now we only have it for HTLCs which were
forwarded in 0.0.124). This means we can't look up the per-peer
structure when claiming old HTLCs, making it difficult to track the
new `ChannelMonitorUpdate` as in-flight.

While we could transition the in-flight `ChannelMonitorUpdate`
tracking to a new global map indexed by `OutPoint`, doing so would
result in a major lock which would be highly contended across
channels with different peers.

Instead, as we move towards tracking in-flight
`ChannelMonitorUpdate`s for closed channels we'll keep our existing
storage, leaving only the `counterparty_node_id` issue to contend
with.

Here we simply accept the issue, requiring that
`counterparty_node_id` be available when claiming HTLCs against a
closed channel. On startup, we explicitly check for any forwarded
HTLCs which came from a closed channel where the forward happened
prior to 0.0.124, failing to deserialize, or logging an warning if
the channel is still open (implying things may work out, but panics
may occur if the channel closes prior to HTLC resolution).

While this is a somewhat dissapointing resolution, LDK nodes which
forward HTLCs are generally fairly well-upgraded, so it is not
anticipated to be an issue in practice.

3 weeks agoBuild `per_peer_state` immediately in `ChannelManager` deser
Matt Corallo [Fri, 4 Oct 2024 17:54:00 +0000 (17:54 +0000)]
Build `per_peer_state` immediately in `ChannelManager` deser

Instead of first building a map from peers to a list of channels
then pulling out of that to build the `per_peer_state`, we build
`per_peer_state` immediately and store channels in it immediately.

This avoids an unnecessary map indirection but also gives us
access to the new fields in `per_peer_state` when reading
`Channel`s which we'll need in a coming commit.

3 weeks agoAvoid a `short_to_chan_info` read lock in `claim_funds_from_hop`
Matt Corallo [Sat, 21 Sep 2024 04:23:09 +0000 (04:23 +0000)]
Avoid a `short_to_chan_info` read lock in `claim_funds_from_hop`

In 453ed11f80b40f28b6e95a74b1f7ed2cd7f012ad we started tracking the
counterparty's `node_id` in `HTLCPreviousHopData`, however we were
still trying to look it up using `prev_short_channel_id` in
`claim_funds_from_hop`.

Because we now usually have the counterparty's `node_id` directly
accessible, we should skip the `prev_short_channel_id` lookup.

This will also be more important in the next commit where we need
to look up state for our counterparty to generate
`ChannelMonitorUpdate`s whether we have a live channel or not.

3 weeks agoAdd missing `update_maps_on_chan_removal` call in signer restore
Matt Corallo [Sun, 29 Sep 2024 19:30:48 +0000 (19:30 +0000)]
Add missing `update_maps_on_chan_removal` call in signer restore

When a channel is closed, we have to call
`update_maps_on_chan_removal` in the same per-peer-state lock as
the removal of the `ChannelPhase` object. We forgot to do so in
`ChannelManager::signer_unblocked` leaving dangling references to
the channel.

We also take this opportunity to include more context in the
channel-closure log in `ChannelManager::signer_unblocked` and add
documentation to `update_maps_on_chan_removal` and
`finish_close_channel` to hopefully avoid this issue in the future.

3 weeks agoPass the `peer_state` lock through to `update_maps_on_chan_removal`
Matt Corallo [Sun, 29 Sep 2024 15:22:29 +0000 (15:22 +0000)]
Pass the `peer_state` lock through to `update_maps_on_chan_removal`

`update_maps_on_chan_removal` is used to perform `ChannelManager`
state updates when a channel is being removed, prior to dropping
the `peer_state` lock. In a future commit we'll use it to update
fields in the `per_peer_state`, but in order to do so we'll need to
have access to that state in the macro.

Here we get set up for this by passing the per-peer state to
`update_maps_on_chan_removal`, which is sadly a fairly large patch.

5 weeks agoMerge pull request #3322 from TheBlueMatt/2024-06-mpp-claim-without-man
Matt Corallo [Mon, 28 Oct 2024 14:08:32 +0000 (14:08 +0000)]
Merge pull request #3322 from TheBlueMatt/2024-06-mpp-claim-without-man

Stop relying on ChannelMonitor persistence after manager read

5 weeks agoDoc the on-upgrade `ChannelMonitor` startup persistence semantics 2024-06-mpp-claim-without-man
Matt Corallo [Sun, 15 Sep 2024 17:24:19 +0000 (17:24 +0000)]
Doc the on-upgrade `ChannelMonitor` startup persistence semantics

Because the new startup `ChannelMonitor` persistence semantics rely
on new information stored in `ChannelMonitor` only for claims made
in the upgraded code, users upgrading from previous version of LDK
must apply the old `ChannelMonitor` persistence semantics at least
once (as the old code will be used to handle partial claims).

5 weeks agoStop relying on `ChannelMonitor` persistence after manager read
Matt Corallo [Thu, 20 Jun 2024 15:17:10 +0000 (15:17 +0000)]
Stop relying on `ChannelMonitor` persistence after manager read

When we discover we've only partially claimed an MPP HTLC during
`ChannelManager` reading, we need to add the payment preimage to
all other `ChannelMonitor`s that were a part of the payment.

We previously did this with a direct call on the `ChannelMonitor`,
requiring users write the full `ChannelMonitor` to disk to ensure
that updated information made it.

This adds quite a bit of delay during initial startup - fully
resilvering each `ChannelMonitor` just to handle this one case is
incredibly excessive.

Over the past few commits we dropped the need to pass HTLCs
directly to the `ChannelMonitor`s using the background events to
provide `ChannelMonitorUpdate`s insetad.

Thus, here we finally drop the requirement to resilver
`ChannelMonitor`s on startup.

5 weeks agoReplay MPP claims via background events using new CM metadata
Matt Corallo [Mon, 30 Sep 2024 20:09:01 +0000 (20:09 +0000)]
Replay MPP claims via background events using new CM metadata

When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.

In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.

Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.

Here we finally implement claiming using the new MPP part list and
metadata stored in `ChannelMonitor`s. In doing so, we use much more
of the existing HTLC-claiming pipeline in `ChannelManager`,
utilizing the on-startup background events flow as well as properly
re-applying the RAA-blockers to ensure preimages cannot be lost.

5 weeks agoHandle duplicate payment claims during initialization
Matt Corallo [Sun, 15 Sep 2024 23:27:35 +0000 (23:27 +0000)]
Handle duplicate payment claims during initialization

In the next commit we'll start using (much of) the normal HTLC
claim pipeline to replay payment claims on startup. In order to do
so, however, we have to properly handle cases where we get a
`DuplicateClaim` back from the channel for an inbound-payment HTLC.

Here we do so, handling the `MonitorUpdateCompletionAction` and
allowing an already-completed RAA blocker.

5 weeks agoMove payment claim initialization to an fn on `ClaimablePayments`
Matt Corallo [Mon, 16 Sep 2024 00:16:51 +0000 (00:16 +0000)]
Move payment claim initialization to an fn on `ClaimablePayments`

Here we wrap the logic which moves claimable payments from
`claimable_payments` to `pending_claiming_payments` to a new
utility function on `ClaimablePayments`. This will allow us to call
this new logic during `ChannelManager` deserialization in a few
commits.

5 weeks agoMove `ChannelManager`-read preimage relay to after struct build
Matt Corallo [Mon, 30 Sep 2024 19:42:51 +0000 (19:42 +0000)]
Move `ChannelManager`-read preimage relay to after struct build

In a coming commit we'll use the existing `ChannelManager` claim
flow to claim HTLCs which we found partially claimed on startup,
necessitating having a full `ChannelManager` when we go to do so.

Here we move the re-claim logic down in the `ChannelManager`-read
logic so that we have that.

5 weeks agoStore info about claimed payments, incl HTLCs in `ChannelMonitor`s
Matt Corallo [Mon, 16 Sep 2024 00:07:48 +0000 (00:07 +0000)]
Store info about claimed payments, incl HTLCs in `ChannelMonitor`s

When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.

In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.

Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.

Here we store the required MPP parts and metadata in
`ChannelMonitor`s and make them available to `ChannelManager` on
load.

5 weeks agoPass info about claimed payments, incl HTLCs to `ChannelMonitor`s
Matt Corallo [Sun, 15 Sep 2024 23:50:31 +0000 (23:50 +0000)]
Pass info about claimed payments, incl HTLCs to `ChannelMonitor`s

When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.

In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.

Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.

Here we take the first step, building a list of MPP parts and
metadata in `ChannelManager` and passing it through to
`ChannelMonitor` in the `ChannelMonitorUpdate`s.

5 weeks agoUse a struct to track MPP parts pending claiming
Matt Corallo [Fri, 14 Jun 2024 14:10:38 +0000 (14:10 +0000)]
Use a struct to track MPP parts pending claiming

When we started tracking which channels had MPP parts claimed
durably on-disk in their `ChannelMonitor`, we did so with a tuple.
This was fine in that it was only ever accessed in two places, but
as we will start tracking it through to the `ChannelMonitor`s
themselves in the coming commit(s), it is useful to have it in a
struct instead.

5 weeks agoMerge pull request #3007 from arik-so/arik/trampoline/2024-04-trampoline-blinded...
Matt Corallo [Wed, 23 Oct 2024 21:52:10 +0000 (21:52 +0000)]
Merge pull request #3007 from arik-so/arik/trampoline/2024-04-trampoline-blinded-hop-serialization

Serialize blinded Trampoline hops

5 weeks agoTest non-legacy blinded Trampoline payloads.
Arik Sosman [Sun, 22 Sep 2024 14:06:24 +0000 (23:06 +0900)]
Test non-legacy blinded Trampoline payloads.

5 weeks agoAdd non-legacy blinded Trampoline payloads.
Arik Sosman [Sun, 22 Sep 2024 14:06:05 +0000 (23:06 +0900)]
Add non-legacy blinded Trampoline payloads.

5 weeks agoTest blinded Trampoline payload serialization.
Arik Sosman [Thu, 27 Jun 2024 14:30:21 +0000 (10:30 -0400)]
Test blinded Trampoline payload serialization.

5 weeks agoSerialize blinded path forwards in Trampoline onions.
Arik Sosman [Thu, 18 Apr 2024 21:52:29 +0000 (14:52 -0700)]
Serialize blinded path forwards in Trampoline onions.

5 weeks agoAdd missing `inbound_payment_id_secret` write in `ChannelManager`
Matt Corallo [Mon, 30 Sep 2024 21:02:53 +0000 (21:02 +0000)]
Add missing `inbound_payment_id_secret` write in `ChannelManager`

In aa09c33a1719944769ba98624bfe18ea33083f44 we added a new secret
in `ChannelManager` with which to derive inbound `PaymentId`s. We
added read support for the new field, but forgot to add writing
support for it. Here we fix this oversight.

5 weeks agoAllow `clippy::unwrap-or-default` because its usually wrong
Matt Corallo [Wed, 23 Oct 2024 15:59:23 +0000 (15:59 +0000)]
Allow `clippy::unwrap-or-default` because its usually wrong

`or_default` is generally less readable than writing out the thing
we're writing, as `Default` is opaque but explicit constructors
generally are not. Thus, we ignore the clippy lint (ideally we
could invert it and ban the use of `Default` in the crate entirely
but alas).

5 weeks agoMerge pull request #3379 from arik-so/exclude-windows-msrv
Matt Corallo [Wed, 23 Oct 2024 13:53:18 +0000 (13:53 +0000)]
Merge pull request #3379 from arik-so/exclude-windows-msrv

Skip MSRV on Windows in CI

5 weeks agoMerge pull request #3142 from TheBlueMatt/2024-06-robust-updates
Matt Corallo [Wed, 23 Oct 2024 00:12:03 +0000 (00:12 +0000)]
Merge pull request #3142 from TheBlueMatt/2024-06-robust-updates

Reliably deliver gossip messages from our `ChannelMessageHandler`

5 weeks agoSkip MSRV on Windows in CI
Arik Sosman [Tue, 22 Oct 2024 20:09:24 +0000 (13:09 -0700)]
Skip MSRV on Windows in CI

5 weeks agoMerge pull request #3376 from yellowred/yellowred/monitor_update_name_pub
Matt Corallo [Tue, 22 Oct 2024 17:13:05 +0000 (17:13 +0000)]
Merge pull request #3376 from yellowred/yellowred/monitor_update_name_pub

Make monitor update name public

5 weeks agoMake monitor update name public
olegkubrakov [Thu, 17 Oct 2024 21:28:12 +0000 (14:28 -0700)]
Make monitor update name public

These structs are meant for MonitoringUpdatingPersister implementation, but some
external implementations may still reuse them, so going to make them public.

6 weeks agoMerge pull request #3377 from G8XSU/hex-impls
Matt Corallo [Tue, 22 Oct 2024 13:55:34 +0000 (13:55 +0000)]
Merge pull request #3377 from G8XSU/hex-impls

Implement to_lower_hex() Display/Debug for PaymentId & OfferId

6 weeks agoMerge pull request #3267 from arik-so/rgs_metrics_2024-08
Gursharan Singh [Mon, 21 Oct 2024 23:15:43 +0000 (16:15 -0700)]
Merge pull request #3267 from arik-so/rgs_metrics_2024-08

Update RGS stats for August 2024.

6 weeks agoImplement to_lower_hex() Display/Debug for OfferId.
G8XSU [Mon, 21 Oct 2024 22:36:11 +0000 (15:36 -0700)]
Implement to_lower_hex() Display/Debug for OfferId.

6 weeks agoImplement to_lower_hex() Display/Debug for PaymentId.
G8XSU [Mon, 21 Oct 2024 22:32:30 +0000 (15:32 -0700)]
Implement to_lower_hex() Display/Debug for PaymentId.

6 weeks agoUpdate processing times for iPhone.
Arik Sosman [Wed, 11 Sep 2024 19:01:34 +0000 (12:01 -0700)]
Update processing times for iPhone.

6 weeks agoUpdate RGS stats for August 2024.
Arik Sosman [Thu, 22 Aug 2024 16:55:20 +0000 (09:55 -0700)]
Update RGS stats for August 2024.

6 weeks agoMerge pull request #3370 from dunxen/2024-10-dry-funding-created-signed
Matt Corallo [Mon, 21 Oct 2024 19:25:18 +0000 (19:25 +0000)]
Merge pull request #3370 from dunxen/2024-10-dry-funding-created-signed

DRY `funding_created()` and `funding_signed()` for V1 channels

6 weeks agoDRY `funding_created()` and `funding_signed()` for V1 channels
Duncan Dean [Fri, 4 Oct 2024 14:35:43 +0000 (16:35 +0200)]
DRY `funding_created()` and `funding_signed()` for V1 channels

There is a decent amount of shared code in these two methods so we make
an attempt to share that code here by introducing the
`InitialRemoteCommitmentReceiver` trait. This trait will also come in
handy when we need similar commitment_signed handling behaviour for
dual-funded channels.

6 weeks agoMerge pull request #3356 from jkczyz/2024-10-inflight-scoring
Matt Corallo [Sat, 19 Oct 2024 13:41:11 +0000 (13:41 +0000)]
Merge pull request #3356 from jkczyz/2024-10-inflight-scoring

Don't over-penalize channels with inflight HTLCs

6 weeks agoUse total_inflight_amount_msat for probability fns
Jeffrey Czyz [Fri, 18 Oct 2024 22:42:18 +0000 (17:42 -0500)]
Use total_inflight_amount_msat for probability fns

Rename parameters used when calculating success probability to make it
clear that the total mount in-flight should be used rather than the
payment amount.

6 weeks agoCorrect comments in avoids_saturating_channels
Jeffrey Czyz [Wed, 16 Oct 2024 20:18:17 +0000 (15:18 -0500)]
Correct comments in avoids_saturating_channels

6 weeks agoCorrect base_penalty_amount_multiplier_msat docs
Jeffrey Czyz [Thu, 10 Oct 2024 23:20:25 +0000 (18:20 -0500)]
Correct base_penalty_amount_multiplier_msat docs

Commit df52da7b31494c7ec77a705cca4c44bc840f8a95 modified
ProbabilisticScorer to apply some penalty amount multipliers to the
total amount flowing over the channel. However, the commit updated the
docs for base_penalty_amount_multiplier_msat even though that behavior
didn't change. This commit reverts those docs.

6 weeks agoDon't over-penalize channels with inflight HTLCs
Jeffrey Czyz [Thu, 10 Oct 2024 23:01:23 +0000 (18:01 -0500)]
Don't over-penalize channels with inflight HTLCs

Commit df52da7b31494c7ec77a705cca4c44bc840f8a95 modified
ProbabilisticScorer to apply some penalty amount multipliers (e.g.,
liquidity_penalty_amount_multiplier_msat) to the total amount flowing
over the channel (i.e., including inflight HTLCs), not just the payment
in question. This led to over-penalizing in-use channels. Instead, only
apply the total amount when calculating success probability.

6 weeks agoMerge pull request #3359 from TheBlueMatt/2024-10-less-types-re-export
Gursharan Singh [Fri, 18 Oct 2024 21:35:49 +0000 (14:35 -0700)]
Merge pull request #3359 from TheBlueMatt/2024-10-less-types-re-export

Remove deprecated re-exports

6 weeks agoMerge pull request #3297 from TheBlueMatt/2024-09-cleanup-package
Matt Corallo [Fri, 18 Oct 2024 17:44:29 +0000 (17:44 +0000)]
Merge pull request #3297 from TheBlueMatt/2024-09-cleanup-package

Cleanup `PackageTemplate`a bit

6 weeks agoAdd a test for the fee-bump rate of timeout HTLC claims on cp txn 2024-09-cleanup-package
Matt Corallo [Fri, 18 Oct 2024 15:57:25 +0000 (15:57 +0000)]
Add a test for the fee-bump rate of timeout HTLC claims on cp txn

In a previous commit we updated the fee-bump-rate of claims against
HTLC timeouts on counterparty commitment transactions so that
instead of immediately attempting to bump every block we consider
the fact that we actually have at least `MIN_CLTV_EXPIRY_DELTA`
blocks to do so, and bumping at the appropriate rate given that.

Here we test that by adding an extra check to an existing test
that we do not bump in the very next block after the HTLC timeout
claim was initially broadcasted.

6 weeks agoSet correct `counterparty_spendable_height` for outb local HTLCs
Matt Corallo [Wed, 18 Sep 2024 18:20:46 +0000 (18:20 +0000)]
Set correct `counterparty_spendable_height` for outb local HTLCs

For outbound HTLCs, the counterparty can spend the output
immediately. This fixes the `counterparty_spendable_height` in the
`PackageTemplate` claiming outbound HTLCs on local commitment
transactions, which was previously spuriously set to the HTLC
timeout (at which point *we* can claim the HTLC).

6 weeks agoStop exporting `lightning::ln::features` 2024-10-less-types-re-export
Matt Corallo [Thu, 17 Oct 2024 19:38:19 +0000 (19:38 +0000)]
Stop exporting `lightning::ln::features`

Now that the module only contains some implementations of
serialization for the `lightning_types::features` structs, there's
no reason for it to be public.

6 weeks agoAdd a test of gossip message buffer limiting in `PeerManager` 2024-06-robust-updates
Matt Corallo [Tue, 20 Aug 2024 02:22:22 +0000 (02:22 +0000)]
Add a test of gossip message buffer limiting in `PeerManager`

This adds a simple test that the gossip message buffer in
`PeerManager` is limited, including the new behavior of bypassing
the limit when the broadcast comes from the
`ChannelMessageHandler`.

6 weeks agoAdd a constructor for the test `SocketDescriptor` and `hang_writes`
Matt Corallo [Tue, 20 Aug 2024 01:57:06 +0000 (01:57 +0000)]
Add a constructor for the test `SocketDescriptor` and `hang_writes`

In testing, its useful to be able to tell the `SocketDescriptor` to
pretend the system network buffer is full, which we add here by
creating a new `hang_writes` flag. In order to simplify
constructing, we also add a new constructor which existing tests
are moved to.

6 weeks agoFix the `establish_connection` utility method for a 2nd connection
Matt Corallo [Tue, 20 Aug 2024 02:13:55 +0000 (02:13 +0000)]
Fix the `establish_connection` utility method for a 2nd connection

The `establish_connection` method should work for more than one
connection per `PeerManager`, which we fix here.

6 weeks agoReliably deliver gossip messages from our `ChannelMessageHandler`
Matt Corallo [Mon, 24 Jun 2024 20:24:36 +0000 (20:24 +0000)]
Reliably deliver gossip messages from our `ChannelMessageHandler`

When our `ChannelMessageHandler` creates gossip broadcast
`MessageSendEvent`s, we generally want these to be reliably
delivered to all our peers, even if there's not much buffer space
available.

Here we do this by passing an extra flag to `forward_broadcast_msg`
which indicates where the message came from, then ignoring the
buffer-full criteria when the flag is set.

6 weeks agoRename `soonest_conf_deadline` to `counterparty_spendable_height`
Matt Corallo [Wed, 18 Sep 2024 16:48:24 +0000 (16:48 +0000)]
Rename `soonest_conf_deadline` to `counterparty_spendable_height`

This renames the field in `PackageTemplate` which describes the
height at which a counterparty can make a claim to an output to
match its actual use.

Previously it had been set based on when a counterparty can claim
an output but also used for other purposes. In the previous commit
we cleaned up its use for fee-bumping-rate, so here we can rename
it as it is now only used as the `counteraprty_spendable_height`.

6 weeks agoClean up `PackageTemplate::get_height_timer` to consider type
Matt Corallo [Wed, 18 Sep 2024 16:00:20 +0000 (16:00 +0000)]
Clean up `PackageTemplate::get_height_timer` to consider type

`PackageTemplate::get_height_timer` is used to decide when to next
bump our feerate on claims which need to make it on chain within
some window. It does so by comparing the current height with some
deadline and increasing the bump rate as the deadline approaches.

However, the deadline used is the `counterparty_spendable_height`,
which is the height at which the counterparty might be able to
spend the same output, irrespective of why. This doesn't make sense
for all output types, for example outbound HTLCs are spendable by
our counteraprty immediately (by revealing the preimage), but we
don't need to get our HTLC timeout claims confirmed immedaitely,
as we actually have `MIN_CLTV_EXPIRY` blocks before the inbound
edge of a forwarded HTLC becomes claimable by our (other)
counterparty.

Thus, here, we adapt `get_height_timer` to look at the type of
output being claimed, and adjust the rate at which we bump the fee
according to the real deadline.

6 weeks agoStop passing current height to `PackageTemplate::build_package`
Matt Corallo [Fri, 6 Sep 2024 00:33:45 +0000 (00:33 +0000)]
Stop passing current height to `PackageTemplate::build_package`

Now that we don't store the confirmation height of the inputs
being spent, passing the current height to
`PackageTemplate::build_package` is useless - we only use it to set
the height at which we should next bump the fee, but we just want
it to be "next block", so we might as well use `0` and avoid the
extra argument. Further, in one case we were already passing `0`,
so passing the argument is just confusing as we can't rely on it
being set.

Note that this does remove an assertion that we never merge
packages that were crated at different heights, and in the future
we may wish to do that (as there's no specific reason not to), but
we do not currently change the behavior.

6 weeks agoDrop unused `PackageTemplate::height_original`
Matt Corallo [Thu, 5 Sep 2024 23:48:02 +0000 (23:48 +0000)]
Drop unused `PackageTemplate::height_original`

This has never been used, and its set to a fixed value of zero for
HTLCs on local commitment transactions making it impossible to rely
on so might as well remove it.

6 weeks agoMerge pull request #3330 from tnull/2024-09-add-macros-crate
Matt Corallo [Thu, 17 Oct 2024 15:34:18 +0000 (15:34 +0000)]
Merge pull request #3330 from tnull/2024-09-add-macros-crate

Add `lightning-macros` crate and drop `bdk_macros` dependency

6 weeks agoMerge pull request #3371 from tnull/2024-10-allow-to-set-payment-and-description...
Matt Corallo [Thu, 17 Oct 2024 15:33:22 +0000 (15:33 +0000)]
Merge pull request #3371 from tnull/2024-10-allow-to-set-payment-and-description-hash

Invoice utils: allow to set both description hash and payment hash

6 weeks agoUpdate `lightning-transaction-sync` to use macros crate
Elias Rohrer [Sat, 21 Sep 2024 05:21:39 +0000 (14:21 +0900)]
Update `lightning-transaction-sync` to use macros crate

6 weeks agoAdd `lightning-macros` crate
Elias Rohrer [Sat, 21 Sep 2024 04:51:21 +0000 (13:51 +0900)]
Add `lightning-macros` crate

Previously, we used the `bdk_macros` dependency for some simple proc
macros in `lightning-transaction-sync`. However, post-1.0 BDK doesn't
further maintain this crate and will at some point probably yank it
together with the old `bdk` crate that was split up.

Here, we create a new crate for utility proc macros and ~~steal~~ add
what we currently use (slightly modified for the latest `syn` version's
API though). In the future we may want to expand this crate, e.g., for
some `maybe_async` macros in the context of an `async KVStore`
implementation.

6 weeks agoRename `PackageTemplate::timelock` `counteraprty_spendable_height`
Matt Corallo [Fri, 6 Sep 2024 00:25:00 +0000 (00:25 +0000)]
Rename `PackageTemplate::timelock` `counteraprty_spendable_height`

This function was very confusing - its used to determine by when
we have to stop aggregating this claim with others as it starts to
be at risk of pinning due to the counterparty's ability to spend
the output.

It is not ever used as a timelock for a transaction, and thus its
name is very confusing.

Instead we rename it `counterparty_spendable_height`.

6 weeks agoRename claim cleaning match bool for accuracy
Matt Corallo [Thu, 5 Sep 2024 21:06:16 +0000 (21:06 +0000)]
Rename claim cleaning match bool for accuracy

We don't actually care if a confirmed transaction claimed other
outputs, only that it claimed a superset of the outputs in the
pending claim we're looking at. Thus, the variable to detect that
is renamed `is_claim_subset_of_tx` instead of `are_sets_equal`.

6 weeks agoMerge pull request #3316 from optout21/signer_extend
Matt Corallo [Wed, 16 Oct 2024 18:43:09 +0000 (18:43 +0000)]
Merge pull request #3316 from optout21/signer_extend

[Splicing] Signer extended with method to sign prev funding transaction input

6 weeks agoInvoice utils: allow to set both description hash and payment hash
Elias Rohrer [Wed, 16 Oct 2024 15:04:38 +0000 (17:04 +0200)]
Invoice utils: allow to set both description hash and payment hash

6 weeks agoSigner extended with method to sign prev funding transaction input
optout [Tue, 15 Oct 2024 21:12:43 +0000 (23:12 +0200)]
Signer extended with method to sign prev funding transaction input

6 weeks agoMerge pull request #3329 from arik-so/monitor_archive_docs_followup
Gursharan Singh [Tue, 15 Oct 2024 20:54:46 +0000 (13:54 -0700)]
Merge pull request #3329 from arik-so/monitor_archive_docs_followup

Document monitor archival idempotency requirement (#3276 followup)

7 weeks agoDeprecate the `lightning::util::string` re-export from types
Matt Corallo [Sat, 12 Oct 2024 22:06:57 +0000 (22:06 +0000)]
Deprecate the `lightning::util::string` re-export from types

Like we've done for `features` and `payment`, here we mark the
`lightning::util::string` re-export as deprecated.

7 weeks agoDrop the `Payment{Hash,Preimage,Secret}` re-exports in `lightning`
Matt Corallo [Sat, 12 Oct 2024 21:59:54 +0000 (21:59 +0000)]
Drop the `Payment{Hash,Preimage,Secret}` re-exports in `lightning`

These re-exports were deprecated in 0.0.124 in favor of the
`lightning::types::payment::*` paths, which we use here.

7 weeks agoMerge pull request #3363 from G8XSU/event-logging-2
Elias Rohrer [Tue, 15 Oct 2024 07:32:37 +0000 (09:32 +0200)]
Merge pull request #3363 from G8XSU/event-logging-2

Revert "Add logging around event processing."

7 weeks agoMerge pull request #3366 from TheBlueMatt/2024-10-0.0.125-relnotes
Elias Rohrer [Tue, 15 Oct 2024 07:31:13 +0000 (09:31 +0200)]
Merge pull request #3366 from TheBlueMatt/2024-10-0.0.125-relnotes

Add CHANGELOG entry for 0.0.125

7 weeks agoAdd CHANGELOG entry for 0.0.125 2024-10-0.0.125-relnotes
Matt Corallo [Mon, 14 Oct 2024 17:28:25 +0000 (17:28 +0000)]
Add CHANGELOG entry for 0.0.125

7 weeks agoRevert "Add logging around event processing."
G8XSU [Mon, 16 Sep 2024 22:50:24 +0000 (15:50 -0700)]
Revert "Add logging around event processing."

This reverts commit 85eb8145fba1dbf3b9348d9142cc105ee13db33b.
Logging here can be overly verbose and moreover in case of event
handling failure, we loop back without any added delay.

7 weeks agoMerge pull request #3354 from tnull/2024-10-fix-block-init-synchronize
Matt Corallo [Mon, 14 Oct 2024 17:03:09 +0000 (17:03 +0000)]
Merge pull request #3354 from tnull/2024-10-fix-block-init-synchronize

`lightning-block-sync`: Fix `synchronize_listeners` always calling default implementation

7 weeks agoDrop `lightning::ln::features::*` type aliases
Matt Corallo [Sat, 12 Oct 2024 21:34:36 +0000 (21:34 +0000)]
Drop `lightning::ln::features::*` type aliases

These were deprecated in 0.0.124, and we drop them here in favor of
`lightning::types::features::*`.

7 weeks agoDocument monitor archival idempotency requirement.
Arik Sosman [Thu, 19 Sep 2024 16:30:49 +0000 (01:30 +0900)]
Document monitor archival idempotency requirement.

7 weeks agoMerge pull request #3349 from TheBlueMatt/2024-10-3270-followups
Matt Corallo [Fri, 11 Oct 2024 13:47:39 +0000 (13:47 +0000)]
Merge pull request #3349 from TheBlueMatt/2024-10-3270-followups

Minor #3270 Followups

7 weeks agoMerge pull request #3353 from tnull/2024-10-simplify-block-sync
Matt Corallo [Fri, 11 Oct 2024 13:45:53 +0000 (13:45 +0000)]
Merge pull request #3353 from tnull/2024-10-simplify-block-sync

Drop unnecessary `Result` in `RpcClient::new`

7 weeks agoFix `synchronize_listeners` calling default implementation
Elias Rohrer [Thu, 10 Oct 2024 15:16:18 +0000 (17:16 +0200)]
Fix `synchronize_listeners` calling default implementation

Previously, the `ChainListenerSet` `Listen` implementation wouldn't
forward to the listeners `block_connected` implementation outside of
tests. This would result in the default implementation of
`Listen::block_connected` being used and the listeners implementation
never being called.

7 weeks agoDrop unnecessary `Result` in `RestClient::new`
Elias Rohrer [Wed, 9 Oct 2024 17:35:42 +0000 (19:35 +0200)]
Drop unnecessary `Result` in `RestClient::new`

.. as it's infallible

7 weeks agoDrop unnecessary `Result` in `RpcClient::new`
Elias Rohrer [Wed, 9 Oct 2024 10:07:44 +0000 (12:07 +0200)]
Drop unnecessary `Result` in `RpcClient::new`

.. as it's infallible

7 weeks agoMerge pull request #3339 from arik-so/lint-script-file
Matt Corallo [Wed, 9 Oct 2024 14:09:13 +0000 (14:09 +0000)]
Merge pull request #3339 from arik-so/lint-script-file

Create script file for easy local linting

7 weeks agoCreate script file for easy local linting
Arik Sosman [Wed, 25 Sep 2024 02:21:19 +0000 (11:21 +0900)]
Create script file for easy local linting

7 weeks agoMerge pull request #3346 from TheBlueMatt/2024-10-dns-feature-flag
Matt Corallo [Tue, 8 Oct 2024 19:46:11 +0000 (19:46 +0000)]
Merge pull request #3346 from TheBlueMatt/2024-10-dns-feature-flag

Add support for parsing the dns_resolver feature bit

7 weeks agoMerge pull request #3235 from Mirebella/add-local-balance-msats
Matt Corallo [Tue, 8 Oct 2024 17:56:46 +0000 (17:56 +0000)]
Merge pull request #3235 from Mirebella/add-local-balance-msats

Add `last_local_balance_msats` field

8 weeks agoMerge pull request #3351 from carlaKC/arch-update-keys-interface
Matt Corallo [Tue, 8 Oct 2024 15:30:56 +0000 (15:30 +0000)]
Merge pull request #3351 from carlaKC/arch-update-keys-interface

docs/trivial: update diagram to reflect changes to KeysInterface

8 weeks agoAdd `Event::ChannelClosed::last_local_balance_msats`
Mirebella [Sun, 11 Aug 2024 15:50:45 +0000 (17:50 +0200)]
Add `Event::ChannelClosed::last_local_balance_msats`

Users commonly want to know what their balance was when a channel
was closed, which this provides in a somewhat simplified manner.

It does not consider pending HTLCs and will always overstate our
balance by transaction fees.

8 weeks agoarch: update diagram to reflect changes to KeysInterface
Carla Kirk-Cohen [Tue, 8 Oct 2024 13:17:47 +0000 (09:17 -0400)]
arch: update diagram to reflect changes to KeysInterface

KeysInterface was split into EntropySource, SignerProvider and
NodeSigner in #1930.

8 weeks agoHold a reference to byte arrays when serializing to bech32 2024-10-3270-followups
Matt Corallo [Thu, 3 Oct 2024 16:54:20 +0000 (16:54 +0000)]
Hold a reference to byte arrays when serializing to bech32

When we serialize from a byte array to bech32 in
`lightning-invoice`, we can either copy the array itself into the
iterator or hold a reference to the array and iterate through that.

In aa2f6b47df312f026213d0ceaaff20ffe955c377 we opted to copy the
array into the iterator, which is fine for the current array sizes
we're working with, but does result in additional memory on the
stack if, in the future, we end up writing large arrays.

Instead, here, we switch to using the slice serialization code when
writing arrays, (very marginally) reducing code size and reducing
stack usage.

2 months agoMarginally reduce allocations in `lightning-invoice`
Matt Corallo [Thu, 3 Oct 2024 16:54:14 +0000 (16:54 +0000)]
Marginally reduce allocations in `lightning-invoice`

In aa2f6b47df312f026213d0ceaaff20ffe955c377 we refactored
`lightning-invoice` de/serialization to use the new version of
`bech32`, but in order to keep the public API the same we
introduced one allocation we could have skipped.

Instead, here, we replace the public `Utf8Error` with
`FromUtf8Error` which contains the original data which failed
conversion, removing an allocation in the process.

2 months agoCheck that the HRPs generated in BOLT 11 `RawHrp` are always valid
Matt Corallo [Thu, 3 Oct 2024 16:54:10 +0000 (16:54 +0000)]
Check that the HRPs generated in BOLT 11 `RawHrp` are always valid

...in `debug_assertions`.

2 months agoDrop one unnecessary allocation added in aa2f6b47df312f026213d0ceaa
Matt Corallo [Thu, 3 Oct 2024 16:54:03 +0000 (16:54 +0000)]
Drop one unnecessary allocation added in aa2f6b47df312f026213d0ceaa

In aa2f6b47df312f026213d0ceaaff20ffe955c377 we refactored
`lightning-invoice` de/serialization to use the new version of
`bech32`, but ended up adding one unnecessary allocation in our
offers logic, which we drop here.

2 months agoMarginally reduce allocations in `lightning-invoice`
Matt Corallo [Thu, 3 Oct 2024 16:53:56 +0000 (16:53 +0000)]
Marginally reduce allocations in `lightning-invoice`

In aa2f6b47df312f026213d0ceaaff20ffe955c377 we refactored
`lightning-invoice` de/serialization to use the new version of
`bech32`, also reducing some trivial unnecessary allocations when
we did so.

Here we drop a few additional allocations which came up in review.

2 months agoMerge pull request #3270 from optout21/bech32-iterser
Matt Corallo [Thu, 3 Oct 2024 17:02:55 +0000 (17:02 +0000)]
Merge pull request #3270 from optout21/bech32-iterser

Upgrade bech32 dependency (iterative)

2 months agoUpgrade bech32 dependency, bech32 serialization improvements
optout [Wed, 2 Oct 2024 19:21:07 +0000 (21:21 +0200)]
Upgrade bech32 dependency, bech32 serialization improvements

2 months agoAllow a `DNSResolverMessageHandler` to set `dns_resolver` feature 2024-10-dns-feature-flag
Matt Corallo [Wed, 2 Oct 2024 18:21:33 +0000 (18:21 +0000)]
Allow a `DNSResolverMessageHandler` to set `dns_resolver` feature

A `DNSResolverMessageHandler` which handles resolution requests
should want the `NodeFeatures` included in the node's
`node_announcement` to include `dns_resolver` to indicate to the
world that it provides that service. Here we enable this by
requesting extra feature flags from the `DNSResolverMessageHandler`
in the features `OnionMessenger`, in turn, provides to
`PeerManager` (which builds the `node_announcement`).

2 months agoAdd support for parsing the `dns_resolver` feature bit
Matt Corallo [Wed, 2 Oct 2024 18:12:38 +0000 (18:12 +0000)]
Add support for parsing the `dns_resolver` feature bit

This feature bit is used to indicate that a node will make DNS
queries on behalf of onion message senders, returning DNSSEC TXT
proofs for the requested names.

It is used to signal support for bLIP 32 resolution and can be used
to find nodes from which we can try to resolve BIP 32 HRNs.

2 months agoMerge pull request #3179 from TheBlueMatt/2024-07-human-readable-names-resolution-1
Arik [Wed, 2 Oct 2024 17:00:13 +0000 (10:00 -0700)]
Merge pull request #3179 from TheBlueMatt/2024-07-human-readable-names-resolution-1

Add the core functionality required to resolve Human Readable Names

2 months agoMerge pull request #3301 from dunxen/2024-9-fixnevertypefallback
Matt Corallo [Wed, 2 Oct 2024 14:29:09 +0000 (14:29 +0000)]
Merge pull request #3301 from dunxen/2024-9-fixnevertypefallback

Add an explicit_type TLV syntax for avoiding certain cases of type inference

2 months agoAdd Bolt11InvoiceFeatures serialization tests
optout [Wed, 11 Sep 2024 20:32:27 +0000 (22:32 +0200)]
Add Bolt11InvoiceFeatures serialization tests

2 months agoAdd an `explicit_type` TLV syntax for avoiding certain cases of type inference
Duncan Dean [Fri, 6 Sep 2024 10:26:19 +0000 (12:26 +0200)]
Add an `explicit_type` TLV syntax for avoiding certain cases of type inference

This new syntax is used to fix "dependency on fallback of ! -> ()".
This avoids cases where code compiles with a fallback of the
never type leading to the unit type. The behaviour in Rust edition 2024
would make this a compile error.

See: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/builtin/static.DEPENDENCY_ON_UNIT_NEVER_TYPE_FALLBACK.html#