Matt Corallo [Tue, 9 Nov 2021 21:22:47 +0000 (21:22 +0000)]
Correct Channel type serialization logic
Currently, we write out the Channel's `ChannelTypeFeatures` as an
odd type, implying clients which don't understand the
`ChannelTypeFeatures` field can simply ignore it. This is obviously
nonsense if the channel type is some future version - the client
needs to fail to deserialize as it doesn't understand the channel's
type.
We adapt the serialization logic here to only write out the
`ChannelTypeFeatures` field if it is something other than
only-static-remote-key, and simply consider that "default" (as it
is the only supported type today). Then, we write out the channel
type as an even TLV, implying clients which do not understand it
must fail to read the `Channel`.
Note that we do not need to bother reserving the TLV type no longer
written as it never appeared in a release (merged post-0.0.103).
Matt Corallo [Tue, 16 Nov 2021 16:18:20 +0000 (16:18 +0000)]
Merge pull request #1119 from TheBlueMatt/2021-10-less-aggressive-htlc-timeouts
Be less aggressive in outbound HTLC CLTV timeout checks
Matt Corallo [Wed, 13 Oct 2021 04:19:13 +0000 (04:19 +0000)]
Be less aggressive in outbound HTLC CLTV timeout checks
We currently assume our counterparty is naive and misconfigured and
may force-close a channel to get an HTLC we just forwarded them.
There shouldn't be any reason to do this - we don't have any such
bug, and we shouldn't start by assuming our counterparties are
buggy. Worse, this results in refusing to forward payments today,
failing HTLCs for largely no reason.
Instead, we keep a fairly conservative check, but not one which
will fail HTLC forwarding spuriously - testing only that the HTLC
doesn't expire for a few blocks from now.
Fixes #1114.
Matt Corallo [Sat, 13 Nov 2021 00:23:28 +0000 (00:23 +0000)]
Merge pull request #1131 from TheBlueMatt/2021-10-upstream-dust
Use upstream rust-bitcoin's dust calculation instead of our own
Matt Corallo [Mon, 11 Oct 2021 17:22:08 +0000 (17:22 +0000)]
Use upstream rust-bitcoin's dust calculation instead of our own
Not only does this move to common code, but it fixes handling of
all output types except for a few trivial cases.
Matt Corallo [Fri, 12 Nov 2021 17:59:18 +0000 (17:59 +0000)]
Merge pull request #1143 from TheBlueMatt/2021-10-no-payment-id-leaks
Fix a minor memory leak on PermanentFailure mon errs when sending
Matt Corallo [Fri, 12 Nov 2021 15:22:41 +0000 (15:22 +0000)]
Merge pull request #1165 from lightning-signer/2021-11-fix-anchors
Fix countersignatory (to_remote) output redeemscript when anchors enabled
Ken Sedgwick [Thu, 11 Nov 2021 20:01:44 +0000 (12:01 -0800)]
Renamed script_for_p2wpkh to get_p2wpkh_redeemscript to match convention
Ken Sedgwick [Wed, 10 Nov 2021 08:09:24 +0000 (00:09 -0800)]
Fix to_remote output redeemscript when anchors enabled
Matt Corallo [Tue, 26 Oct 2021 21:40:14 +0000 (21:40 +0000)]
Fix a minor memory leak on PermanentFailure mon errs when sending
If we send a payment and fail to update the first-hop channel state
with a `PermanentFailure` ChannelMonitorUpdateErr, we would have an
entry in our pending payments map, but possibly not return the
PaymentId back to the user to retry the payment, leading to a (rare
and relatively minor) memory leak.
Matt Corallo [Thu, 11 Nov 2021 15:47:37 +0000 (15:47 +0000)]
Merge pull request #1105 from TheBlueMatt/2021-10-log-persist-time
Log before+after ChannelMonitor/Manager updates for visibility
Matt Corallo [Mon, 4 Oct 2021 03:11:36 +0000 (03:11 +0000)]
Log before+after ChannelMonitor/Manager updates for visibility
I realized on my own node that I don't have any visibility into how
long a monitor or manager persistence call takes, potentially
blocking other operations. This makes it much more clear by adding
a relevant log_trace!() print immediately before and immediately
after persistence.
Matt Corallo [Mon, 8 Nov 2021 23:56:47 +0000 (23:56 +0000)]
Merge pull request #1158 from jkczyz/2021-11-scorer-tests
Scorer unit tests
Jeffrey Czyz [Thu, 4 Nov 2021 21:55:01 +0000 (16:55 -0500)]
Add unit tests for Scorer
Test basic and channel failure penalties, including after a
(de-)serialization round trip.
Jeffrey Czyz [Thu, 4 Nov 2021 18:58:11 +0000 (13:58 -0500)]
Add SinceEpoch time to test Scorer hermetically
In order to test Scorer hermetically, sleeps must be avoided. Add a
SinceEpoch abstraction for manually advancing time. Implement the Time
trait for SinceEpoch so that it can be used with ScorerUsingTime in
tests.
Matt Corallo [Fri, 5 Nov 2021 15:45:21 +0000 (15:45 +0000)]
Merge pull request #1154 from TheBlueMatt/2021-11-103-doc-tweaks
0.0.103 CHANGELOG tweaks from Jeff
Matt Corallo [Wed, 3 Nov 2021 16:51:56 +0000 (16:51 +0000)]
Add note about PaymentId fields to 0.0.103 changelog
Matt Corallo [Wed, 3 Nov 2021 16:05:02 +0000 (16:05 +0000)]
0.0.103 CHANGELOG tweaks from Jeff
Matt Corallo [Wed, 3 Nov 2021 16:58:33 +0000 (16:58 +0000)]
Merge pull request #1078 from TheBlueMatt/2021-09-chan-types
Implement channel_type negotiation
Matt Corallo [Wed, 3 Nov 2021 03:30:39 +0000 (03:30 +0000)]
Merge pull request #1153 from TheBlueMatt/2021-11-0.0.103
One final fix + cut 0.0.103
Matt Corallo [Wed, 3 Nov 2021 01:14:53 +0000 (01:14 +0000)]
Bump crate versions to 0.0.103/invoice 0.11
Matt Corallo [Wed, 3 Nov 2021 01:05:05 +0000 (01:05 +0000)]
Update CHANGELOG for 0.0.103
Matt Corallo [Wed, 3 Nov 2021 01:11:25 +0000 (01:11 +0000)]
Tweak serialization of ScorerUsingTime for better forward compat
Matt Corallo [Tue, 2 Nov 2021 21:56:00 +0000 (21:56 +0000)]
Merge pull request #1150 from TheBlueMatt/2021-11-103-bindings
Pre-0.0.103 Bindings Tweaks
Matt Corallo [Mon, 1 Nov 2021 22:01:57 +0000 (22:01 +0000)]
Add `(C-not exported)` tags as required in tuple types
This prepares us for C bindings auto-exporting tuple type fields.
Matt Corallo [Mon, 1 Nov 2021 04:23:30 +0000 (04:23 +0000)]
Add `(C-not exported)` tag to a `Payee` modifier with move semantics
This matches the other `Payee` move-modifier functions.
Matt Corallo [Sun, 31 Oct 2021 00:26:54 +0000 (00:26 +0000)]
Remove now-unused import in routing/mod.rs
Matt Corallo [Sun, 31 Oct 2021 18:21:46 +0000 (18:21 +0000)]
Make payment_path_failed path type bindings-mappable
The bindings don't currently support passing `Vec`s of objects
which it mappes as "opaque types". This is because it will require
clones to convert its own list of references to Rust's list of
objects.
In the near future we should resolve this limitation, allowing us
to revert this (and make `find_route`'s method signature similarly
cleaner), but for now we must avoid `Vec<OpaqueType>`.
Matt Corallo [Sun, 31 Oct 2021 18:20:27 +0000 (18:20 +0000)]
Rename Payee::new to Payee::from_node_id to clarify it somewhat
This also differentiates it from the bindings default-constructed
`new` method which is constructed when all fields are exposed and
of mappable types.
Matt Corallo [Sun, 31 Oct 2021 18:19:39 +0000 (18:19 +0000)]
Remove trailing ;s from macro calls to silence new rustc warnings
Matt Corallo [Tue, 2 Nov 2021 20:36:22 +0000 (20:36 +0000)]
Merge pull request #1146 from jkczyz/2021-10-score-serialization
Scorer serialization
Jeffrey Czyz [Fri, 29 Oct 2021 04:44:26 +0000 (23:44 -0500)]
Implement (de)serialization for Scorer
Scorer should be serialized to retain penalty data between restarts.
Implement (de)serialization for Scorer by serializing last failure times
as duration since the UNIX epoch. For no-std, the zero-Duration is used.
Jeffrey Czyz [Fri, 29 Oct 2021 13:52:27 +0000 (08:52 -0500)]
Parameterize Scorer by a Time trait
Scorer uses time to determine how much to penalize a channel after a
failure occurs. Parameterizing it by time cleans up the code such that
no-std support is in a single AlwaysPresent struct, which implements the
Time trait. Time is implemented for std::time::Instant when std is
available.
This parameterization also allows for deterministic testing since a
clock could be devised to advance forward as needed.
Jeffrey Czyz [Fri, 29 Oct 2021 04:23:45 +0000 (23:23 -0500)]
Refactor channel failure penalty logic
Move channel failure penalty logic into a ChannelFailure abstraction.
This encapsulates the logic for accumulating penalties and decaying them
over time. It also is responsible for the no-std behavior. This cleans
up Scorer and will make it easier to serialize it.
Jeffrey Czyz [Mon, 1 Nov 2021 21:35:39 +0000 (16:35 -0500)]
Clarify Scorer docs around penalizing channels
Matt Corallo [Mon, 1 Nov 2021 22:19:08 +0000 (22:19 +0000)]
Merge pull request #1149 from jkczyz/2021-11-network-graph
Shared ownership of NetworkGraph
Jeffrey Czyz [Mon, 1 Nov 2021 20:01:39 +0000 (15:01 -0500)]
Make NetGraphMsgHandler::network_graph private
Since NetworkGraph has shared ownership, NetGraphMsgHandler does not
need to expose its field.
Jeffrey Czyz [Mon, 1 Nov 2021 18:14:14 +0000 (13:14 -0500)]
Parameterize NetGraphMsgHandler with NetworkGraph
NetworkGraph is owned by NetGraphMsgHandler, but DefaultRouter requires
a reference to it. Introduce shared ownership to NetGraphMsgHandler so
that both can use the same NetworkGraph.
Matt Corallo [Sun, 31 Oct 2021 18:33:16 +0000 (18:33 +0000)]
Merge pull request #1141 from TheBlueMatt/2021-10-payment-id-on-partial-failure
Rewrite InvoicePayer retry to correctly handle MPP partial failures
Matt Corallo [Thu, 28 Oct 2021 18:46:02 +0000 (18:46 +0000)]
Check for invoice expiry in InvoicePayer before we send any HTLCs
Matt Corallo [Wed, 27 Oct 2021 22:22:48 +0000 (22:22 +0000)]
Add integration test for InvoicePayerretry on an immediate failure
Matt Corallo [Wed, 27 Oct 2021 22:15:11 +0000 (22:15 +0000)]
Add an integration test for InvoicePayer paying when one part fails
This tests the multi-part-single-failure-immediately fixes in the
previous commit.
Matt Corallo [Tue, 26 Oct 2021 22:52:06 +0000 (22:52 +0000)]
Rewrite InvoicePayer retry to correctly handle MPP partial failures
This rewrites a good chunk of the retry logic in `InvoicePayer` to
address two issues:
* it was not considering the return value of `send_payment` (and
`retry_payment`) may indicate a failure on some paths but not
others,
* it was not considering that more failures may still come later
when removing elements from the retry count map. This could
result in us seeing an MPP-partial-failure, failing to retry,
removing the retries count entry, and then retrying other parts,
potentially forever.
Matt Corallo [Wed, 27 Oct 2021 22:12:07 +0000 (22:12 +0000)]
Dont unwrap `RouteParameter::expiry_time` as users can set it
Users can provide anything they want as `RouteParameters` so we
shouldn't assume any fields are set any particular way, including
`expiry_time` set at all.
Matt Corallo [Sat, 30 Oct 2021 01:52:43 +0000 (01:52 +0000)]
Expand `InvoicePayer` documentation somewhat to clarify edge-cases
Matt Corallo [Tue, 26 Oct 2021 21:39:31 +0000 (21:39 +0000)]
Provide payment retry data when an MPP payment failed partially
This will allow `InvoicePayer` to properly retry payments that only
partially failed to send.
Matt Corallo [Tue, 26 Oct 2021 21:38:46 +0000 (21:38 +0000)]
Move PaymentId to a [u8; 32] in bindings as for other hash objects
This should allow us to fix
https://github.com/lightningdevkit/ldk-garbagecollected/issues/52
Matt Corallo [Fri, 29 Oct 2021 20:16:36 +0000 (20:16 +0000)]
Merge pull request #1144 from jkczyz/2021-10-invoice-payer-scoring
Penalize failed channels
Jeffrey Czyz [Tue, 24 Aug 2021 04:56:59 +0000 (23:56 -0500)]
Test InvoicePayer in BackgroundProcessor
Proof of concept showing InvoicePayer can be used with an
Arc<ChannelManager> passed to BackgroundProcessor. Likely do not need to
merge this commit.
Jeffrey Czyz [Wed, 27 Oct 2021 15:39:22 +0000 (10:39 -0500)]
Penalize failed channels in Scorer
As payments fail, the channel responsible for the failure may be
penalized. Implement Scorer::payment_path_failed to penalize the failed
channel using a configured penalty. As time passes, the penalty is
reduced using exponential decay, though penalties will accumulate if the
channel continues to fail. The decay interval is also configurable.
Jeffrey Czyz [Thu, 14 Oct 2021 18:04:39 +0000 (13:04 -0500)]
Notify scorer of failing payment path and channel
Upon receiving a PaymentPathFailed event, the failing payment may be
retried on a different path. To avoid using the channel responsible for
the failure, a scorer should be notified of the failure before being
used to find a new route.
Add a payment_path_failed method to routing::Score and call it in
InvoicePayer's event handler. Introduce a LockableScore parameterization
to InvoicePayer so the scorer is locked only once before calling
find_route.
Matt Corallo [Thu, 28 Oct 2021 20:57:21 +0000 (20:57 +0000)]
Merge pull request #1137 from TheBlueMatt/2021-10-ping-fixes
Give peers which are sending us messages/receiving messages from us longer to respond to ping
Matt Corallo [Wed, 27 Oct 2021 20:06:13 +0000 (20:06 +0000)]
Log peer public key more thoroughly when logging in peer_handler
Matt Corallo [Thu, 21 Oct 2021 22:33:42 +0000 (22:33 +0000)]
Give peers one timer tick to finish handshake before disconnecting
This ensures we don't let a hung connection stick around forever if
the peer never completes the initial handshake.
This also resolves a race where, on receiving a second connection
from a peer, we may reset their_node_id to None to prevent sending
messages even though the `channel_encryptor`
`is_ready_for_encryption()`. Sending pings only checks the
`channel_encryptor` status, not `their_node_id` resulting in an
`unwrap` on `None` in `enqueue_message`.
Matt Corallo [Mon, 11 Oct 2021 04:24:08 +0000 (04:24 +0000)]
Give peers which are sending us messages longer to respond to ping
See comment for rationale.
Matt Corallo [Thu, 28 Oct 2021 17:43:58 +0000 (17:43 +0000)]
Util-ify enqueueing an encoded message in peer_handler
This marginally simplifies coming commits.
Matt Corallo [Mon, 25 Oct 2021 17:31:34 +0000 (17:31 +0000)]
Constify the ratio in buf limits between forward and init sync msgs
Matt Corallo [Wed, 27 Oct 2021 22:13:36 +0000 (22:13 +0000)]
Merge pull request #1142 from TheBlueMatt/2021-10-on-sent-fees
Track the amount spent on fees as payments are retried
Matt Corallo [Wed, 27 Oct 2021 19:04:22 +0000 (19:04 +0000)]
Track the amount spent on fees as payments are retried
Especially once we merge the `InvoicePayer` logic soon, we'll want
to expose the total fee paid in the `PaymentSent` event.
Matt Corallo [Wed, 27 Oct 2021 18:57:16 +0000 (18:57 +0000)]
Pass the failing/succeeding `Path` to PendingOutboundPayment meths
This will make the next commit much simpler
Matt Corallo [Wed, 27 Oct 2021 18:24:44 +0000 (18:24 +0000)]
Add a utility trait in `router` to get the fees along a given path
Matt Corallo [Wed, 27 Oct 2021 16:58:19 +0000 (16:58 +0000)]
Merge pull request #1059 from jkczyz/2021-08-payment-retry
Payment Retries
Jeffrey Czyz [Tue, 24 Aug 2021 05:14:10 +0000 (00:14 -0500)]
Implement Payer and Router for lightning crate
Implements Payer for ChannelManager and Rotuer for find_route, which can
be used to parameterize InvoicePayer when needing payment retries.
Jeffrey Czyz [Thu, 30 Sep 2021 18:30:24 +0000 (11:30 -0700)]
Fail payment retry if Invoice is expired
According to BOLT 11:
- after the `timestamp` plus `expiry` has passed
- SHOULD NOT attempt a payment
Add a convenience method for checking if an Invoice has expired, and use
it to short-circuit payment retries.
Jeffrey Czyz [Mon, 4 Oct 2021 16:39:59 +0000 (11:39 -0500)]
Support paying zero-value invoices
Jeffrey Czyz [Tue, 24 Aug 2021 05:08:15 +0000 (00:08 -0500)]
Add InvoicePayer for retrying failed payments
When a payment fails, it's useful to retry the payment once the network
graph and channel scores are updated. InvoicePayer is a utility for
making payments which will retry any failed payment paths for a payment
up to a configured number of total attempts. It is parameterized by a
Payer and Router for ease of customization and testing.
Implement EventHandler for InvoicePayer as a decorator that intercepts
PaymentPathFailed events and retries that payment using the parameters
from the event. It delegates to the decorated EventHandler after retries
have been exhausted and for other events.
Jeffrey Czyz [Mon, 25 Oct 2021 23:48:52 +0000 (18:48 -0500)]
Unify route finding methods
An upcoming Router interface will be used for finding a Route both when
initially sending a payment and also when retrying failed payment paths.
Unify the three varieties of get_route so the interface can consist of a
single method implemented by the new `find_route` method. Give get_route
pub(crate) visibility so it can still be used in tests.
Jeffrey Czyz [Mon, 4 Oct 2021 14:20:49 +0000 (09:20 -0500)]
Rewrite Invoice's interface in terms of msats
InvoiceBuilder's interface was changed recently to work in terms of
msats. Update Invoice's interface to return the amount in msats, too,
and make amount_pico_btc private.
Jeffrey Czyz [Thu, 30 Sep 2021 22:29:44 +0000 (15:29 -0700)]
Add PaymentId to PaymentPathFailed event
The PaymentId is needed when retrying payments. Include it in the
PaymentPathFailed event so it can be used in that manner.
Jeffrey Czyz [Sun, 17 Oct 2021 22:21:01 +0000 (17:21 -0500)]
Add PaymentId to PaymentSent event
The payment_hash may not uniquely identify the payment if it has been
reused. Include the payment_id in PaymentSent events so it can
correlated with the send_payment call.
Jeffrey Czyz [Tue, 24 Aug 2021 04:55:28 +0000 (23:55 -0500)]
Expose log_bytes! macro for use in other crates
Needed to log PaymentHash in the lightning-invoice crate when retrying
payments.
Matt Corallo [Tue, 26 Oct 2021 02:03:02 +0000 (02:03 +0000)]
Add `PeerManager::disconnect_all_peers` to avoid complexity in BP
In the coming commits simply calling `timer_tick_occurred` will no
longer disconnect all peers, so its helpful to have a utility
method.
Matt Corallo [Mon, 25 Oct 2021 21:51:08 +0000 (21:51 +0000)]
Correct send-bounding logic in `TestRoutingMessageHandler`
The `cmp::min` appeared to confused `end` for a count.
Matt Corallo [Mon, 25 Oct 2021 20:32:27 +0000 (20:32 +0000)]
Merge pull request #1138 from TheBlueMatt/2021-10-payee-in-monitors
Store `Payee` info with HTLCs
Matt Corallo [Mon, 25 Oct 2021 04:46:26 +0000 (04:46 +0000)]
Store `Payee` information in `HTLCSource::OutboundRoute`.
This stores and tracks HTLC payee information with HTLCSource info,
allowing us to provide it back to the user if the HTLC fails and
ensuring persistence by keeping it with the HTLC itself as it
passes between Channel and ChannelMonitor.
Matt Corallo [Mon, 25 Oct 2021 04:42:29 +0000 (04:42 +0000)]
Copy `Payee` into `Route`s to provide them to `ChannelManager`
Matt Corallo [Mon, 25 Oct 2021 17:52:30 +0000 (17:52 +0000)]
Make `Payee::pubkey` pub.
`Payee` is expected to be used by users to get routes for payment
retries, potentially with their own router. Thus, its helpful if it
is pub, even if it is redundant with the last hop in the `path`
field in `Events::PaymentPathFailed`.
Matt Corallo [Mon, 25 Oct 2021 16:09:06 +0000 (16:09 +0000)]
Merge pull request #1134 from jkczyz/2021-10-payee-arg
Payee abstraction for use in get_route and PaymentPathFailed
Jeffrey Czyz [Fri, 22 Oct 2021 05:27:58 +0000 (00:27 -0500)]
Use option TLV decoding for short_channel_id
Using ignorable TLV decoding is only applicable for an Option containing
an enum, but short_channel_id is an Option<u64>. Use option TLV encoding
instead.
Jeffrey Czyz [Thu, 21 Oct 2021 22:52:53 +0000 (17:52 -0500)]
Include PaymentPathRetry data in PaymentPathFailed
When a payment path fails, it may be retried. Typically, this means
re-computing the route after updating the NetworkGraph and channel
scores in order to avoid the failing hop. The last hop in
PaymentPathFailed's path field contains the pubkey, amount, and CLTV
values needed to pass to get_route. However, it does not contain the
payee's features and route hints from the invoice.
Include the entire set of parameters in PaymentPathRetry and add it to
the PaymentPathFailed event. Add a get_retry_route wrapper around
get_route that takes PaymentPathRetry. This allows an EventHandler to
retry failed payment paths using the payee's route hints and features.
Jeffrey Czyz [Wed, 20 Oct 2021 14:22:38 +0000 (09:22 -0500)]
Remove outdated line from get_route docs
Jeffrey Czyz [Wed, 20 Oct 2021 14:15:31 +0000 (09:15 -0500)]
Define Payee abstraction for use in get_route
A payee can be identified by a pubkey and optionally have an associated
set of invoice features and route hints. Use this in get_route instead
of three separate parameters. This may be included in PaymentPathFailed
later to use when finding a new route.
Matt Corallo [Fri, 17 Sep 2021 17:32:36 +0000 (17:32 +0000)]
Support send/recv'ing the new channel_type field in open_channel
This implements the channel type negotiation, though as we currently
only support channels with only static_remotekey set, it doesn't
implement the negotiation explicitly.
Matt Corallo [Fri, 17 Sep 2021 17:32:11 +0000 (17:32 +0000)]
Support de/ser of the new channel_type field in open_channel
Matt Corallo [Fri, 17 Sep 2021 16:59:09 +0000 (16:59 +0000)]
Add a ChannelTypeFeatures features object for the new channel_type
Its semantics are somewhat different from existing features,
however not enough to merit a different struct entirely.
Specifically, it only supports required features (if you send a
channel_type, the counterparty has to accept it wholesale or try
again, it cannot select only a subset of the flags) and it is
serialized differently (only appearing in TLVs).
Matt Corallo [Fri, 17 Sep 2021 16:00:24 +0000 (16:00 +0000)]
Fix `cargo doc` on older rustc
Apparently at least rustc 1.48 doesn't support `Self` in doc links,
so we make it explicit.
Matt Corallo [Fri, 22 Oct 2021 19:33:33 +0000 (19:33 +0000)]
Merge pull request #1104 from TheBlueMatt/2021-10-payment-id-in-monitors
Reload pending outbound payments from ChannelMonitors on startup
Matt Corallo [Wed, 20 Oct 2021 23:40:09 +0000 (23:40 +0000)]
Add a test of an HTLC being fulfilled and then later failed
Peers probably shouldn't do this, but if they want to give us free
money, we should take it and not generate any spurious events.
Matt Corallo [Sun, 10 Oct 2021 23:42:03 +0000 (23:42 +0000)]
Move test_dup_htlc_onchain_fails_on_reload to payment_tests
test_dup_htlc_onchain_fails_on_reload is now more of a
payment_test than a functional_test, testing for handling of
pending payments.
Matt Corallo [Sun, 10 Oct 2021 23:36:57 +0000 (23:36 +0000)]
Add some basic test coverage of monitor payment data reloading
Matt Corallo [Sun, 10 Oct 2021 23:36:44 +0000 (23:36 +0000)]
Reload pending payments from ChannelMonitor HTLC data on reload
If we go to send a payment, add the HTLC(s) to the channel(s),
commit the ChannelMonitor updates to disk, and then crash, we'll
come back up with no pending payments but HTLC(s) ready to be
claim/failed.
This makes it rather impractical to write a payment sender/retryer,
as you cannot guarantee atomicity - you cannot guarantee you'll
have retry data persisted even if the HTLC(s) are actually pending.
Because ChannelMonitors are *the* atomically-persisted data in LDK,
we lean on their current HTLC data to figure out what HTLC(s) are a
part of an outbound payment, rebuilding the pending payments list
on reload.
Matt Corallo [Mon, 4 Oct 2021 04:04:46 +0000 (04:04 +0000)]
Add PaymentSecrets to HTLCSource::OutboundRoute objects
Matt Corallo [Thu, 21 Oct 2021 19:05:48 +0000 (19:05 +0000)]
Rename payment object vars to refer to payments and not session IDs
Matt Corallo [Sun, 3 Oct 2021 22:33:12 +0000 (22:33 +0000)]
Track payments after they resolve until all HTLCs are finalized
In the next commit, we will reload lost pending payments from
ChannelMonitors during restart. However, in order to avoid
re-adding pending payments which have already been fulfilled, we
must ensure that we do not fully remove pending payments until all
HTLCs for the payment have been fully removed from their
ChannelMonitors.
We do so here, introducing a new PendingOutboundPayment variant
called `Completed` which only tracks the set of pending HTLCs.
Matt Corallo [Sat, 2 Oct 2021 22:35:07 +0000 (22:35 +0000)]
Inform ChannelManager when fulfilled HTLCs are finalized
When an HTLC has been failed, we track it up until the point there
exists no broadcastable commitment transaction which has the HTLC
present, at which point Channel returns the HTLCSource back to the
ChannelManager, which fails the HTLC backwards appropriately.
When an HTLC is fulfilled, however, we fulfill on the backwards path
immediately. This is great for claiming upstream HTLCs, but when we
want to track pending payments, we need to ensure we can check with
ChannelMonitor data to rebuild pending payments. In order to do so,
we need an event similar to the HTLC failure event, but for
fulfills instead.
Specifically, if we force-close a channel, we remove its off-chain
`Channel` object entirely, at which point, on reload, we may notice
HTLC(s) which are not present in our pending payments map (as they
may have received a payment preimage, but not fully committed to
it). Thus, we'd conclude we still have a retryable payment, which
is untrue.
This commit does so, informing the ChannelManager via a new return
element where appropriate of the HTLCSource corresponding to the
failed HTLC.
Matt Corallo [Sun, 3 Oct 2021 00:46:10 +0000 (00:46 +0000)]
Implement `HashMap` read for `MaybeReadable` values
This allows us to read a `HashMap` that has values which may be
skipped if they are some backwards-compatibility type.
We also take this opportunity to fail deserialization if keys are
duplicated.
Matt Corallo [Sun, 10 Oct 2021 23:56:11 +0000 (23:56 +0000)]
Make `Channel::monitor_updating_restored`'s return tuple a struct
This improves readability at the callsite and in the function.
Matt Corallo [Sat, 2 Oct 2021 01:59:38 +0000 (01:59 +0000)]
Make `Channel::revoke_and_ack`'s return tuple a struct
This substantially improves readability at the callsite and in the
function.
Matt Corallo [Wed, 20 Oct 2021 00:53:26 +0000 (00:53 +0000)]
Merge pull request #1108 from TheBlueMatt/2021-10-persist-mon-blocks
Persist ChannelMonitors after new blocks are connected
Matt Corallo [Tue, 19 Oct 2021 21:49:03 +0000 (21:49 +0000)]
Clarify the contexts in which persist_new_channel may be called
Its somewhat confusing that `persist_new_channel` is called on
startup for an existing channel in common deployments, so we call
it out explicitly.