]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
20 months agoFix typos in lightning-transaction-sync
benthecarman [Fri, 3 Mar 2023 05:36:54 +0000 (23:36 -0600)]
Fix typos in lightning-transaction-sync

20 months agoMerge pull request #2055 from TheBlueMatt/2023-02-fning-msrv
Matt Corallo [Mon, 27 Feb 2023 22:30:14 +0000 (22:30 +0000)]
Merge pull request #2055 from TheBlueMatt/2023-02-fning-msrv

Remove the lightning-transaction-sync MSRV

20 months agoMove `lightning-transaction-sync` out of the main workspace 2023-02-fning-msrv
Matt Corallo [Mon, 27 Feb 2023 17:32:07 +0000 (17:32 +0000)]
Move `lightning-transaction-sync` out of the main workspace

Because `lightning-transaction-sync` does not have an MSRV (and
because its dev-dependencies are huge), we can't build it by
default when devs run `cargo test`, so it is moved out of the
top-level workspace.

20 months agoRemove the `lightning-transaction-sync` MSRV
Matt Corallo [Mon, 27 Feb 2023 17:31:15 +0000 (17:31 +0000)]
Remove the `lightning-transaction-sync` MSRV

Apparently it has dependencies that don't track an MSRV at all, so
we can't practically enforce one in CI.

20 months agoFix silent rebase conflict in previous PR
Matt Corallo [Mon, 27 Feb 2023 18:28:26 +0000 (18:28 +0000)]
Fix silent rebase conflict in previous PR

20 months agoMerge pull request #2043 from valentinewallace/2023-02-initial-send-path-fails
Matt Corallo [Mon, 27 Feb 2023 18:10:27 +0000 (18:10 +0000)]
Merge pull request #2043 from valentinewallace/2023-02-initial-send-path-fails

`PaymentPathFailed`: add initial send error details

20 months agoMerge pull request #2033 from tnull/2023-02-make-esplora-sync-test-more-robust
Matt Corallo [Mon, 27 Feb 2023 17:55:43 +0000 (17:55 +0000)]
Merge pull request #2033 from tnull/2023-02-make-esplora-sync-test-more-robust

Make `test_esplora_syncs` more robust

20 months agoMerge pull request #2025 from TheBlueMatt/2023-02-no-pub-genesis-hashes
Matt Corallo [Mon, 27 Feb 2023 17:50:22 +0000 (17:50 +0000)]
Merge pull request #2025 from TheBlueMatt/2023-02-no-pub-genesis-hashes

Remove genesis block hash from public API

20 months agoFix PaymentPathFailed generation and scid on initial send
Valentine Wallace [Fri, 24 Feb 2023 03:46:45 +0000 (22:46 -0500)]
Fix PaymentPathFailed generation and scid on initial send

20 months agoChange PaymentPathFailed's optional network update to a Failure enum
Valentine Wallace [Mon, 13 Feb 2023 22:55:42 +0000 (17:55 -0500)]
Change PaymentPathFailed's optional network update to a Failure enum

This let us capture the errors when we fail without committing to an HTLC vs
failing via update_fail.

20 months agoImplement writeable for APIError
Valentine Wallace [Sun, 19 Feb 2023 23:32:29 +0000 (18:32 -0500)]
Implement writeable for APIError

20 months agoser_macros: Document behavior of upgradable_* variants
Valentine Wallace [Sat, 25 Feb 2023 21:03:43 +0000 (16:03 -0500)]
ser_macros: Document behavior of upgradable_* variants

20 months agoFix upgradable_required fields to actually be required in lower level macros
Valentine Wallace [Sun, 19 Feb 2023 21:52:22 +0000 (16:52 -0500)]
Fix upgradable_required fields to actually be required in lower level macros

When using lower level macros such as read_tlv_stream, upgradable_required
fields have been treated as regular options. This is incorrect, they should
either be upgradable_options or treated as required fields.

20 months agoMerge pull request #1977 from jkczyz/2023-01-offers-fuzz
Matt Corallo [Sat, 25 Feb 2023 18:03:51 +0000 (18:03 +0000)]
Merge pull request #1977 from jkczyz/2023-01-offers-fuzz

BOLT 12 deserialization fuzzers

20 months agoRename OptionDeserWrapper->RequiredWrapper
Valentine Wallace [Sat, 25 Feb 2023 00:50:24 +0000 (19:50 -0500)]
Rename OptionDeserWrapper->RequiredWrapper

Makes more sense and sets us up for the UpgradableRequired version of it

20 months agoDisambiguate ignorable and ignorable_option
Valentine Wallace [Mon, 20 Feb 2023 17:42:45 +0000 (12:42 -0500)]
Disambiguate ignorable and ignorable_option

Would rather not rename ignorable to ignorable_required, so rename both of them
to upgradable_*

20 months agoSupport deserializing an Option-al MaybeReadable
Valentine Wallace [Mon, 13 Feb 2023 21:49:22 +0000 (16:49 -0500)]
Support deserializing an Option-al MaybeReadable

Prior to this change, our impl_writeable_tlv_based macros only supported
deserializing a MaybeReadable if it's non-Optional.

20 months agoRemove all_paths_failed from PaymentPathFailed
Valentine Wallace [Sun, 12 Feb 2023 00:38:48 +0000 (19:38 -0500)]
Remove all_paths_failed from PaymentPathFailed

This field was previous useful in manual retries for users to know when all
paths of a payment have failed and it is safe to retry. Now that we support
automatic retries in ChannelManager and no longer support manual retries, the
field is no longer useful.

For backwards compat, we now always write false for this field. If we didn't do
this, previous versions would default this field's value to true, which can be
problematic because some clients have relied on the field to indicate when a
full payment retry is safe.

20 months agoMerge pull request #2050 from douglaz/import-lightning-io-futures
wpaulino [Fri, 24 Feb 2023 19:03:32 +0000 (11:03 -0800)]
Merge pull request #2050 from douglaz/import-lightning-io-futures

Only import lightning::io if futures are enabled

20 months agoOnly import lightning::io if futures are enabled
Allan Douglas R. de Oliveira [Fri, 24 Feb 2023 02:56:11 +0000 (02:56 +0000)]
Only import lightning::io if futures are enabled

20 months agoMake `test_esplora_syncs` more robust
Elias Rohrer [Wed, 15 Feb 2023 00:16:23 +0000 (18:16 -0600)]
Make `test_esplora_syncs` more robust

The test generally works and the parts in question *should* never fail.
However, they did, so we give the test server instances some leeway.

20 months agoFix amount overflow in Invoice building
Jeffrey Czyz [Thu, 16 Feb 2023 02:17:18 +0000 (20:17 -0600)]
Fix amount overflow in Invoice building

An overflow can occur when multiplying the offer amount by the requested
quantity when no amount is given in the request. Return an error instead
of overflowing.

20 months agoFix amount overflow in Offer parsing and building
Jeffrey Czyz [Wed, 15 Feb 2023 22:43:41 +0000 (16:43 -0600)]
Fix amount overflow in Offer parsing and building

An overflow can occur when multiplying the offer amount by the requested
quantity when checking if the given amount is enough. Return an error
instead of overflowing.

20 months agoFuzz test for bech32 decoding
Jeffrey Czyz [Thu, 9 Feb 2023 17:09:23 +0000 (11:09 -0600)]
Fuzz test for bech32 decoding

Fuzz testing bech32 decoding along with deserializing the underlying
message can result in overly exhaustive searches. Instead, the message
deserializations are now fuzzed separately. Add fuzzing for bech32
decoding.

20 months agoExpose Bech32Encode trait for fuzzing
Jeffrey Czyz [Thu, 9 Feb 2023 16:59:11 +0000 (10:59 -0600)]
Expose Bech32Encode trait for fuzzing

In order to fuzz test Bech32Encode parsing independent of the underlying
message deserialization, the trait needs to be exposed. Conditionally
expose it only for fuzzing.

20 months agoFuzz test for parsing Invoice
Jeffrey Czyz [Fri, 20 Jan 2023 22:30:45 +0000 (16:30 -0600)]
Fuzz test for parsing Invoice

An invoice is serialized as a TLV stream and encoded as bytes. Add a
fuzz test that parses the TLV stream and deserializes the underlying
Invoice. Then compare the original bytes with those obtained by
re-serializing the Invoice.

20 months agoFuzz test for parsing InvoiceRequest
Jeffrey Czyz [Fri, 20 Jan 2023 19:34:34 +0000 (13:34 -0600)]
Fuzz test for parsing InvoiceRequest

An invoice request is serialized as a TLV stream and encoded as bytes.
Add a fuzz test that parses the TLV stream and deserializes the
underlying InvoiceRequest. Then compare the original bytes with those
obtained by re-serializing the InvoiceRequest.

20 months agoRename `BestBlock::from_genesis` to `from_network` for clarity 2023-02-no-pub-genesis-hashes
Matt Corallo [Wed, 15 Feb 2023 06:09:00 +0000 (06:09 +0000)]
Rename `BestBlock::from_genesis` to `from_network` for clarity

20 months agoRemove genesis block hash from public API
Matt Corallo [Thu, 9 Feb 2023 19:20:22 +0000 (19:20 +0000)]
Remove genesis block hash from public API

Forcing users to pass a genesis block hash has ended up being
error-prone largely due to byte-swapping questions for bindings
users. Further, our API is currently inconsistent - in
`ChannelManager` we take a `Bitcoin::Network` but in `NetworkGraph`
we take the genesis block hash.

Luckily `NetworkGraph` is the only remaining place where we require
users pass the genesis block hash, so swapping it for a `Network`
is a simple change.

20 months agoAdd missing import path in ser macro
Valentine Wallace [Sun, 19 Feb 2023 23:32:06 +0000 (18:32 -0500)]
Add missing import path in ser macro

20 months agoMerge pull request #2014 from valentinewallace/2023-02-rework-partial-pmt-fail
Matt Corallo [Thu, 23 Feb 2023 21:54:16 +0000 (21:54 +0000)]
Merge pull request #2014 from valentinewallace/2023-02-rework-partial-pmt-fail

Rework auto-retry send errors

20 months agoClarify Retry::Timeout vs PaymentParams::expiry_time in docs
Valentine Wallace [Thu, 23 Feb 2023 20:49:44 +0000 (15:49 -0500)]
Clarify Retry::Timeout vs PaymentParams::expiry_time in docs

20 months agoFix outdated PendingOutboundPayment::Abandoned docs
Valentine Wallace [Sun, 12 Feb 2023 00:35:48 +0000 (19:35 -0500)]
Fix outdated PendingOutboundPayment::Abandoned docs

20 months agoOn initial send retries, avoid previously failed scids
Valentine Wallace [Thu, 16 Feb 2023 21:40:51 +0000 (16:40 -0500)]
On initial send retries, avoid previously failed scids

Previously, we could have tried the same failed channels over and over until
retries are exhausted.

20 months agoIn-line retry_with_route method
Valentine Wallace [Sun, 19 Feb 2023 21:12:24 +0000 (16:12 -0500)]
In-line retry_with_route method

Since it's only used one place now

20 months agoRework auto retry send errors
Valentine Wallace [Fri, 10 Feb 2023 21:09:01 +0000 (15:09 -0600)]
Rework auto retry send errors

Prior to this, we returned PaymentSendFailure from auto retry send payment
methods. This implied that we might return a PartialFailure from them, which
has never been the case. So it makes sense to rework the errors to be a better
fit for the methods.

We're taking error handling in a totally different direction now to make it
more asynchronous, see send_payment_internal for more information.

20 months agoFix InvalidRoute error to be ChannelUnavailable
Valentine Wallace [Wed, 22 Feb 2023 19:09:58 +0000 (14:09 -0500)]
Fix InvalidRoute error to be ChannelUnavailable

InvalidRoute is reserved for malformed routes, not routes where a channel or
its peer is unavailable

20 months agoMerge pull request #1897 from TheBlueMatt/2022-11-monitor-updates-always-async
Matt Corallo [Wed, 22 Feb 2023 19:12:31 +0000 (19:12 +0000)]
Merge pull request #1897 from TheBlueMatt/2022-11-monitor-updates-always-async

Always process `ChannelMonitorUpdate`s asynchronously

20 months agoDon't generate a `ChannelMonitorUpdate` for closed chans on shutdown 2022-11-monitor-updates-always-async
Matt Corallo [Sun, 19 Feb 2023 00:13:51 +0000 (00:13 +0000)]
Don't generate a `ChannelMonitorUpdate` for closed chans on shutdown

The `Channel::get_shutdown` docs are very clear - if the channel
jumps to `Shutdown` as a result of not being funded when we go to
initiate shutdown we should not generate a `ChannelMonitorUpdate`
as there's no need to bother with the shutdown script - we're
force-closing anyway.

However, this wasn't actually implemented, potentially causing a
spurious monitor update for no reason.

20 months agoUse the new monitor persistence flow for `funding_created` handling
Matt Corallo [Sat, 3 Dec 2022 03:15:04 +0000 (03:15 +0000)]
Use the new monitor persistence flow for `funding_created` handling

Building on the previous commits, this finishes our transition to
doing all message-sending in the monitor update completion
pipeline, unifying our immediate- and async- `ChannelMonitor`
update and persistence flows.

20 months agoUse new monitor persistence flow in funding_signed handling
Matt Corallo [Mon, 6 Feb 2023 23:03:38 +0000 (23:03 +0000)]
Use new monitor persistence flow in funding_signed handling

In the previous commit, we moved all our `ChannelMonitorUpdate`
pipelines to use a new async path via the
`handle_new_monitor_update` macro. This avoids having two message
sending pathways and simply sends messages in the "monitor update
completed" flow, which is shared between sync and async monitor
updates.

Here we reuse the new macro for handling `funding_signed` messages
when doing an initial `ChannelMonitor` persistence. This provides
a similar benefit, simplifying the code a trivial amount, but
importantly allows us to fully remove the original
`handle_monitor_update_res` macro.

20 months agoAlways process `ChannelMonitorUpdate`s asynchronously
Matt Corallo [Wed, 11 Jan 2023 21:37:57 +0000 (21:37 +0000)]
Always process `ChannelMonitorUpdate`s asynchronously

We currently have two codepaths on most channel update functions -
most methods return a set of messages to send a peer iff the
`ChannelMonitorUpdate` succeeds, but if it does not we push the
messages back into the `Channel` and then pull them back out when
the `ChannelMonitorUpdate` completes and send them then. This adds
a substantial amount of complexity in very critical codepaths.

Instead, here we swap all our channel update codepaths to
immediately set the channel-update-required flag and only return a
`ChannelMonitorUpdate` to the `ChannelManager`. Internally in the
`Channel` we store a queue of `ChannelMonitorUpdate`s, which will
become critical in future work to surface pending
`ChannelMonitorUpdate`s to users at startup so they can complete.

This leaves some redundant work in `Channel` to be cleaned up
later. Specifically, we still generate the messages which we will
now ignore and regenerate later.

This commit updates the `ChannelMonitorUpdate` pipeline across all
the places we generate them.

20 months agoMove TODO from `handle_monitor_update_res` into `Channel`
Matt Corallo [Sat, 3 Dec 2022 05:38:24 +0000 (05:38 +0000)]
Move TODO from `handle_monitor_update_res` into `Channel`

The TODO mentioned in `handle_monitor_update_res` about how we
might forget about HTLCs in case of permanent monitor update
failure still applies in spite of all our changes. If a channel is
drop'd in general, monitor-pending updates may be lost if the
monitor update failed to persist.

This was always the case, and is ultimately the general form of the
the specific TODO, so we simply leave comments there

20 months agoHandle `MonitorUpdateCompletionAction`s after monitor update sync
Matt Corallo [Fri, 27 Jan 2023 06:14:18 +0000 (06:14 +0000)]
Handle `MonitorUpdateCompletionAction`s after monitor update sync

In a previous PR, we added a `MonitorUpdateCompletionAction` enum
which described actions to take after a `ChannelMonitorUpdate`
persistence completes. At the time, it was only used to execute
actions in-line, however in the next commit we'll start (correctly)
leaving the existing actions until after monitor updates complete.

20 months agoMerge pull request #1988 from TheBlueMatt/2023-01-limited-chans
Matt Corallo [Wed, 22 Feb 2023 00:39:13 +0000 (00:39 +0000)]
Merge pull request #1988 from TheBlueMatt/2023-01-limited-chans

Limit the number of pending un-funded inbound channel

20 months agoLimit the number of pending un-funded inbound channel 2023-01-limited-chans
Matt Corallo [Thu, 26 Jan 2023 04:47:25 +0000 (04:47 +0000)]
Limit the number of pending un-funded inbound channel

Because we store some (not large, but not zero) state per-peer,
it's useful to limit the number of peers we have connected, at
least with some buffer.

Much more importantly, each channel has a relatively large cost,
especially around the `ChannelMonitor`s we have to build for each.

Thus, here, we limit the number of channels per-peer which aren't
(yet) on-chain, as well as limit the number of (inbound) peers
which don't have a (funded-on-chain) channel.

Fixes #1889

20 months agoAdd an `inbound` flag to the `peer_connected` message handlers
Matt Corallo [Thu, 26 Jan 2023 04:45:58 +0000 (04:45 +0000)]
Add an `inbound` flag to the `peer_connected` message handlers

Its useful for the message handlers to know if a peer is inbound
for DoS decision-making reasons.

20 months agoMerge pull request #2035 from TheBlueMatt/2023-02-fix-no-con-discon
Matt Corallo [Tue, 21 Feb 2023 21:28:05 +0000 (21:28 +0000)]
Merge pull request #2035 from TheBlueMatt/2023-02-fix-no-con-discon

Fix (and DRY) the conditionals before calling peer_disconnected

20 months agoDerive traits for InvoiceRequest and Invoice
Jeffrey Czyz [Wed, 8 Feb 2023 00:29:08 +0000 (18:29 -0600)]
Derive traits for InvoiceRequest and Invoice

Offer and Refund derive Debug, Clone, and PartialEq. For consistency,
derive these traits for InvoiceRequest and Invoice as well.

20 months agoMerge pull request #2040 from alecchendev/2023-02-indexed-map-btreeset-to-vec
Matt Corallo [Tue, 21 Feb 2023 19:57:51 +0000 (19:57 +0000)]
Merge pull request #2040 from alecchendev/2023-02-indexed-map-btreeset-to-vec

Replace `BTreeSet` in `IndexedMap` with sorted `Vec`

20 months agoRemove the `peer_disconnected` `no_connection_possible` flag 2023-02-fix-no-con-discon
Matt Corallo [Tue, 21 Feb 2023 19:10:43 +0000 (19:10 +0000)]
Remove the `peer_disconnected` `no_connection_possible` flag

Long ago, we used the `no_connection_possible` to signal that a
peer has some unknown feature set or some other condition prevents
us from ever connecting to the given peer. In that case we'd
automatically force-close all channels with the given peer. This
was somewhat surprising to users so we removed the automatic
force-close, leaving the flag serving no LDK-internal purpose.

Distilling the concept of "can we connect to this peer again in the
future" to a simple flag turns out to be ripe with edge cases, so
users actually using the flag to force-close channels would likely
cause surprising behavior.

Thus, there's really not a lot of reason to keep the flag,
especially given its untested and likely to be broken in subtle
ways anyway.

20 months agoAdd a further debug_assert that disconnecting peers are connected
Matt Corallo [Wed, 15 Feb 2023 06:29:29 +0000 (06:29 +0000)]
Add a further debug_assert that disconnecting peers are connected

20 months agoCorrect `funding_transaction_generated` err msg and fix fuzz check
Matt Corallo [Wed, 15 Feb 2023 01:23:20 +0000 (01:23 +0000)]
Correct `funding_transaction_generated` err msg and fix fuzz check

This fixes new errors in `full_stack_target` pointed out by
Chaincode's generous fuzzing infrastructure. Specifically, there's
no reason to check the error message in the
`funding_transaction_generated` return value - it can only return
a failure if the channel has closed since the funding transaction
was generated (which is fine) or if the signer refuses to sign
(which can't happen in fuzzing).

20 months agoCorrect the "is peer live" checks in `PeerManager`
Matt Corallo [Wed, 15 Feb 2023 01:20:38 +0000 (01:20 +0000)]
Correct the "is peer live" checks in `PeerManager`

In general, we should be checking if a `Peer` has `their_features`
set as the "is this peer connected and have they finished the
handshake" flag as it indicates an `Init` message was received.

While none of these appear to be reachable bugs, there were a
number of places where we checked other flags for this purpose,
which may lead to sending messages before `Init` in the future.

Here we clean these cases up to always use the correct check (via
the new util method).

20 months agoAdd test of an initial message other than `Init`
Matt Corallo [Wed, 15 Feb 2023 01:54:21 +0000 (01:54 +0000)]
Add test of an initial message other than `Init`

This test fails without the previous commit.

20 months agoFix (and DRY) the conditionals before calling `peer_disconnected`
Matt Corallo [Wed, 15 Feb 2023 01:13:57 +0000 (01:13 +0000)]
Fix (and DRY) the conditionals before calling `peer_disconnected`

If we have a peer that sends a non-`Init` first message, we'll call
`peer_disconnected` without ever having called `peer_connected`
(which has to wait until we have an `Init` message). This is a
violation of our API guarantees, though should generally not be an
issue.

Because this bug was repeated in a few places, we also take this
opportunity to DRY up the logic which checks the peer state before
calling `peer_disconnected`.

Found by the new `ChannelManager` assertions and the
`full_stack_target` fuzzer.

20 months agoReplace `BTreeSet` in `IndexedMap` with sorted `Vec`
Alec Chen [Thu, 16 Feb 2023 22:34:06 +0000 (16:34 -0600)]
Replace `BTreeSet` in `IndexedMap` with sorted `Vec`

The `Vec` is sorted not on `IndexedMap::insert`, but on
`IndexedMap::range` to avoid unnecessary work while reading a network
graph.

20 months agoPass pending_events into pay_internal
Valentine Wallace [Thu, 9 Feb 2023 19:34:37 +0000 (13:34 -0600)]
Pass pending_events into pay_internal

Useful for generating Payment(Path)Failed events in this method

20 months agoPass payment hash into pay_internal
Valentine Wallace [Thu, 9 Feb 2023 19:21:52 +0000 (13:21 -0600)]
Pass payment hash into pay_internal

Useful for generating Payment(Path)Failed events in this method

20 months agoMerge pull request #2026 from valentinewallace/2023-02-dedup-pending-forwardable-evs
Matt Corallo [Sun, 19 Feb 2023 18:25:03 +0000 (18:25 +0000)]
Merge pull request #2026 from valentinewallace/2023-02-dedup-pending-forwardable-evs

Deduplicate `PendingHTLCsForwardable` events on generation

20 months agoDeduplicate PendingHTLCsForwardable events when queueing
Valentine Wallace [Fri, 17 Feb 2023 22:41:21 +0000 (17:41 -0500)]
Deduplicate PendingHTLCsForwardable events when queueing

20 months agoOn retryable update_fail, don't queue redundant PendingHTLCsForwardable
Valentine Wallace [Fri, 17 Feb 2023 22:35:09 +0000 (17:35 -0500)]
On retryable update_fail, don't queue redundant PendingHTLCsForwardable

20 months agoCheck for abandon-able payments on startup
Valentine Wallace [Fri, 17 Feb 2023 22:14:43 +0000 (17:14 -0500)]
Check for abandon-able payments on startup

20 months agoAdd a new monitor update result handling macro
Matt Corallo [Sat, 3 Dec 2022 04:25:37 +0000 (04:25 +0000)]
Add a new monitor update result handling macro

Over the next few commits, this macro will replace the
`handle_monitor_update_res` macro. It takes a different approach -
instead of receiving the message(s) that need to be re-sent after
the monitor update completes and pushing them back into the
channel, we'll not get the messages from the channel at all until
we're ready for them.

This will unify our message sending into only actually fetching +
sending messages in the common monitor-update-completed code,
rather than both there *and* in the functions that call `Channel`
when new messages are originated.

20 months agoAdd storage for `ChannelMonitorUpdate`s in `Channel`s
Matt Corallo [Mon, 19 Dec 2022 20:41:42 +0000 (20:41 +0000)]
Add storage for `ChannelMonitorUpdate`s in `Channel`s

In order to support fully async `ChannelMonitor` updating, we need
to ensure that we can replay `ChannelMonitorUpdate`s if we shut
down after persisting a `ChannelManager` but without completing a
`ChannelMonitorUpdate` persistence. In order to support that we
(obviously) have to store the `ChannelMonitorUpdate`s in the
`ChannelManager`, which we do here inside the `Channel`.

We do so now because in the coming commits we will start using the
async persistence flow for all updates, and while we won't yet
support fully async monitor updating it's nice to get some of the
foundational structures in place now.

20 months agoTrack actions to execute after a `ChannelMonitor` is updated
Matt Corallo [Wed, 30 Nov 2022 18:49:44 +0000 (18:49 +0000)]
Track actions to execute after a `ChannelMonitor` is updated

When a `ChannelMonitor` update completes, we may need to take some
further action, such as exposing an `Event` to the user initiating
another `ChannelMonitor` update, etc. This commit adds the basic
structure to track such actions and serialize them as required.

Note that while this does introduce a new map which is written as
an even value which users cannot opt out of, the map is only filled
in when users use the asynchronous `ChannelMonitor` updates. As
these are still considered beta, breaking downgrades for such users
is considered acceptable here.

20 months agoAdd an infallible no-sign version of send_commitment_no_status_check
Matt Corallo [Thu, 1 Dec 2022 00:25:32 +0000 (00:25 +0000)]
Add an infallible no-sign version of send_commitment_no_status_check

In the coming commits we'll move to async `ChannelMonitorUpdate`
application, which means we'll want to generate a
`ChannelMonitorUpdate` (including a new counterparty commitment
transaction) before we actually send it to our counterparty. To do
that today we'd have to actually sign the commitment transaction
by calling the signer, then drop it, apply the
`ChannelMonitorUpdate`, then re-sign the commitment transaction to
send it to our peer.

In this commit we instead split `send_commitment_no_status_check`
and `send_commitment_no_state_update` into `build_` and `send_`
variants, allowing us to generate new counterparty commitment
transactions without actually signing, then build them for sending,
with signatures, later.

20 months agoFix the err msg provided when a send fails due to peer disconnected
Matt Corallo [Wed, 1 Feb 2023 20:54:10 +0000 (20:54 +0000)]
Fix the err msg provided when a send fails due to peer disconnected

We haven't had a `MonitorUpdateInProgress` check in
`Channel::is_live` for some time.

20 months agoMerge pull request #2009 from TheBlueMatt/2023-02-no-racey-retries
Matt Corallo [Thu, 16 Feb 2023 23:41:09 +0000 (23:41 +0000)]
Merge pull request #2009 from TheBlueMatt/2023-02-no-racey-retries

Fix (and test) threaded payment retries

20 months agoFix (and test) threaded payment retries 2023-02-no-racey-retries
Matt Corallo [Fri, 3 Feb 2023 23:05:58 +0000 (23:05 +0000)]
Fix (and test) threaded payment retries

The new in-`ChannelManager` retries logic does retries as two
separate steps, under two separate locks - first it calculates
the amount that needs to be retried, then it actually sends it.
Because the first step doesn't udpate the amount, a second thread
may come along and calculate the same amount and end up retrying
duplicatively.

Because we generally shouldn't ever be processing retries at the
same time, the fix is trivial - simply take a lock at the top of
the retry loop and hold it until we're done.

20 months agoAlways check `next_route` in `TestRouter` is fully consumed
Matt Corallo [Fri, 3 Feb 2023 02:17:41 +0000 (02:17 +0000)]
Always check `next_route` in `TestRouter` is fully consumed

...rather than only in std.

20 months agoTest if a given mutex is locked by the current thread in tests
Matt Corallo [Tue, 7 Feb 2023 19:46:08 +0000 (19:46 +0000)]
Test if a given mutex is locked by the current thread in tests

In anticipation of the next commit(s) adding threaded tests, we
need to ensure our lockorder checks work fine with multiple
threads. Sadly, currently we have tests in the form
`assert!(mutex.try_lock().is_ok())` to assert that a given mutex is
not locked by the caller to a function.

The fix is rather simple given we already track mutexes locked by a
thread in our `debug_sync` logic - simply replace the check with a
new extension trait which (for test builds) checks the locked state
by only looking at what was locked by the current thread.

20 months agoMerge pull request #2037 from valentinewallace/2023-02-fix-probe-leak
Matt Corallo [Thu, 16 Feb 2023 21:29:26 +0000 (21:29 +0000)]
Merge pull request #2037 from valentinewallace/2023-02-fix-probe-leak

Fix outbound payments probe leak

20 months agoMerge pull request #1921 from arik-so/2022-12-prohibit-old-rgs-updates
Matt Corallo [Thu, 16 Feb 2023 21:20:28 +0000 (21:20 +0000)]
Merge pull request #1921 from arik-so/2022-12-prohibit-old-rgs-updates

Throw error for RGS data that's more than two weeks old

20 months agoThrow error for RGS data that's more than two weeks old, fixing #1785
Arik Sosman [Fri, 16 Dec 2022 18:52:56 +0000 (10:52 -0800)]
Throw error for RGS data that's more than two weeks old, fixing #1785

20 months agoRemove pending probes on update_fail
Valentine Wallace [Wed, 15 Feb 2023 22:30:47 +0000 (17:30 -0500)]
Remove pending probes on update_fail

Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail

20 months agoMerge pull request #2008 from valentinewallace/2023-02-remove-manual-retries
Matt Corallo [Thu, 16 Feb 2023 00:49:04 +0000 (00:49 +0000)]
Merge pull request #2008 from valentinewallace/2023-02-remove-manual-retries

Abandon payments on behalf of the user and remove manual retries

20 months agoReword and fix grammar in PartialFailure docs
Valentine Wallace [Mon, 13 Feb 2023 16:36:46 +0000 (11:36 -0500)]
Reword and fix grammar in PartialFailure docs

20 months agoRemove retry_payments method
Valentine Wallace [Sun, 5 Feb 2023 22:05:12 +0000 (17:05 -0500)]
Remove retry_payments method

We're no longer supporting manual retries since
ChannelManager::send_payment_with_retry can be parameterized by a retry
strategy

This commit also updates all docs related to retry_payment and abandon_payment.
Since these docs frequently overlap with changes in preceding commits where we
start abandoning payments on behalf of the user, all the docs are updated in
one go.

20 months agoWhen processing pending htlcs, abandon outbounds that are not retryable
Valentine Wallace [Sat, 4 Feb 2023 02:44:08 +0000 (21:44 -0500)]
When processing pending htlcs, abandon outbounds that are not retryable

20 months agoAbandon payment on behalf of the user on payment path failure
Valentine Wallace [Fri, 3 Feb 2023 17:53:01 +0000 (12:53 -0500)]
Abandon payment on behalf of the user on payment path failure

Removed retry_single_path_payment, it's replaced by automatic_retries with
AutoRetry::Success

20 months agoAbandon payment if retry fails in process_pending_htlcs
Valentine Wallace [Fri, 3 Feb 2023 17:49:07 +0000 (12:49 -0500)]
Abandon payment if retry fails in process_pending_htlcs

20 months agoMerge pull request #1832 from jkczyz/2022-11-composite-handler
Matt Corallo [Wed, 15 Feb 2023 17:45:39 +0000 (17:45 +0000)]
Merge pull request #1832 from jkczyz/2022-11-composite-handler

Macro for composing custom message handlers

20 months agoPass pending events to outbound_payments::abandon_payment
Valentine Wallace [Fri, 3 Feb 2023 17:44:14 +0000 (12:44 -0500)]
Pass pending events to outbound_payments::abandon_payment

This makes it uniform with the outbound payment methods that generate events
and set us up for abandoning payments on behalf of the user.

20 months agoFix indentation in outbound payment mark_abandoned
Valentine Wallace [Fri, 3 Feb 2023 16:53:44 +0000 (11:53 -0500)]
Fix indentation in outbound payment mark_abandoned

20 months agoMerge pull request #2020 from valentinewallace/2023-02-test-inflight-scoring
Matt Corallo [Wed, 15 Feb 2023 01:25:09 +0000 (01:25 +0000)]
Merge pull request #2020 from valentinewallace/2023-02-test-inflight-scoring

Fix and test in-flight HTLC scoring in between retries

20 months agoAdd CI testing for lightning-custom-message crate
Jeffrey Czyz [Thu, 2 Feb 2023 03:35:38 +0000 (21:35 -0600)]
Add CI testing for lightning-custom-message crate

20 months agoRe-write CustomMessageHandler documentation
Jeffrey Czyz [Tue, 31 Jan 2023 16:44:19 +0000 (10:44 -0600)]
Re-write CustomMessageHandler documentation

Documentation for CustomMessageHandler wasn't clear how it is related to
PeerManager and contained some grammatical and factual errors. Re-write
the docs and link to the lightning_custom_message crate.

20 months agoFix compilation errors in some doc tests
Jeffrey Czyz [Tue, 3 Jan 2023 19:31:15 +0000 (13:31 -0600)]
Fix compilation errors in some doc tests

These are seen in newer versions of rustc.

20 months agoMacro for composing custom message handlers
Jeffrey Czyz [Tue, 3 Jan 2023 17:24:30 +0000 (11:24 -0600)]
Macro for composing custom message handlers

BOLT 1 specifies a custom message type range for use with experimental
or application-specific messages. While a `CustomMessageHandler` can be
defined to support more than one message type, defining such a handler
requires a significant amount of boilerplate and can be error prone.

Add a crate exporting a `composite_custom_message_handler` macro for
easily composing pre-defined custom message handlers. The resulting
handler can be further composed with other custom message handlers using
the same macro.

This requires a separate crate since the macro needs to support "or"
patterns in macro_rules, which is only available in edition 2021.

https://doc.rust-lang.org/edition-guide/rust-2021/or-patterns-macro-rules.html

Otherwise, a crate defining a handler for a set of custom messages could
not easily be reused with another custom message handler. Doing so would
require explicitly duplicating the reused handlers type ids, but those
may change when the crate is updated.

20 months agoMove `fairrwlock` to the `sync` module
Matt Corallo [Tue, 7 Feb 2023 19:39:24 +0000 (19:39 +0000)]
Move `fairrwlock` to the `sync` module

20 months agoMerge pull request #2029 from wpaulino/update-pgp-key
Matt Corallo [Tue, 14 Feb 2023 22:20:46 +0000 (22:20 +0000)]
Merge pull request #2029 from wpaulino/update-pgp-key

Update Wilmer Paulino's PGP key

20 months agoRemove unnecessary scoring methods from Router trait
Valentine Wallace [Thu, 9 Feb 2023 20:36:35 +0000 (14:36 -0600)]
Remove unnecessary scoring methods from Router trait

20 months agoFix computing in-flight HTLCs in between retries + test
Valentine Wallace [Tue, 7 Feb 2023 19:10:41 +0000 (14:10 -0500)]
Fix computing in-flight HTLCs in between retries + test

20 months agotest_utils: parameterize TestRouter by TestScorer
Valentine Wallace [Tue, 7 Feb 2023 19:04:55 +0000 (14:04 -0500)]
test_utils: parameterize TestRouter by TestScorer

This allows us set scoring expectations and ensure in-flight htlcs are factored
into scoring

20 months agoMerge pull request #1966 from ViktorTigerstrom/2023-01-store-channels-per-peer-followups
Matt Corallo [Tue, 14 Feb 2023 18:44:35 +0000 (18:44 +0000)]
Merge pull request #1966 from ViktorTigerstrom/2023-01-store-channels-per-peer-followups

1507 followups

20 months agoDon't serialize nodes which we have no channels to
Viktor Tigerström [Sat, 4 Feb 2023 18:14:42 +0000 (20:14 +0200)]
Don't serialize nodes which we have no channels to

20 months agoInitialize `list_channels_with_filter` result vec with capacity
Viktor Tigerström [Wed, 18 Jan 2023 01:30:36 +0000 (02:30 +0100)]
Initialize `list_channels_with_filter` result vec with capacity

20 months agoUpdate unknown peer passed to msg and api functions handling
Viktor Tigerström [Wed, 18 Jan 2023 01:20:09 +0000 (02:20 +0100)]
Update unknown peer passed to msg and api functions handling

20 months agoAlways remove disconnected peers with no channels
Viktor Tigerström [Tue, 17 Jan 2023 23:28:00 +0000 (00:28 +0100)]
Always remove disconnected peers with no channels

When a peer disconnects but still has channels, the peer's `peer_state`
entry in the `per_peer_state` is not removed by the `peer_disconnected`
function. If the channels with that peer is later closed while still
being disconnected (i.e. force closed), we therefore need to remove the
peer from `peer_state` separately.

To remove the peers separately, we push such peers to a separate HashSet
that holds peers awaiting removal, and remove the peers on a timer to
limit the negative effects on parallelism as much as possible.