Matt Corallo [Sun, 21 Nov 2021 22:29:48 +0000 (22:29 +0000)]
Support `logger::Record` in C by String-ing the fmt::Arguments
This adds a new (non-feature) cfg argument `c_bindings` which will
be set when building C bindings. With this, we can (slightly) tweak
behavior and API based on whether we are being built for Rust or C
users.
Ideally we'd never need this, but as long as we can keep the API
consistent-enough to avoid material code drift, this gives us a
cheap way of doing the "right" thing for both C and Rust when the
two are in tension.
We also move lightning-background-processor to support the same
MSRV as the main lightning crate, instead of only
lightning-net-tokio's MSRV.
In upcoming commits, we'll be making the payment secret and payment hash/preimage
derivable from info about the payment + a node secret. This means we don't
need to store any info about incoming payments and can eventually get rid of the
channelmanager::pending_inbound_payments map.
Matt Corallo [Fri, 12 Nov 2021 15:52:59 +0000 (15:52 +0000)]
Move `Score` into a `scoring` module instead of a top-level module
Traits in top-level modules is somewhat confusing - generally
top-level modules are just organizational modules and don't contain
things themselves, instead placing traits and structs in
sub-modules. Further, its incredibly awkward to have a `scorer`
sub-module, but only have a single struct in it, with the relevant
trait it is the only implementation of somewhere else. Not having
`Score` in the `scorer` sub-module is further confusing because
it's the only module anywhere that references scoring at all.
Matt Corallo [Fri, 12 Nov 2021 04:16:23 +0000 (04:16 +0000)]
Penalize large HTLCs relative to channels in default `Scorer`
Sending HTLCs which are any greater than a very small fraction of the
channel size tend to fail at a much higher rate. Thus, by default
we start applying a penalty at only 1/8th the channel size and
increase it linearly as the amount reaches the channel's capacity,
20 msat per 1024th of the channel capacity.
Matt Corallo [Fri, 12 Nov 2021 03:52:58 +0000 (03:52 +0000)]
Provide `Score` the HTLC amount and channel capacity
This should allow `Score` implementations to make substantially
better decisions, including of the form "willing to pay X to avoid
routing over this channel which may have a high failure rate".
Jeffrey Czyz [Fri, 12 Nov 2021 03:47:59 +0000 (21:47 -0600)]
Add PaymentHash parameter to Router::find_route
Implementations of Router may need the payment hash in order to look up
pre-computed routes from a probe for a given payment. Add a PaymentHash
parameter to Router::find_route to allow for this.
Jeffrey Czyz [Wed, 10 Nov 2021 00:07:23 +0000 (18:07 -0600)]
Replace expect_value_msat with expect_send
Modify all InvoicePayer unit tests to use expect_send instead of
expect_value_msat, since the former can discern whether the send was for
an invoice, spontaneous payment, or a retry. Updates tests to set payer
expectations if they weren't already and assert these before returning a
failure.
Jeffrey Czyz [Tue, 9 Nov 2021 23:32:37 +0000 (17:32 -0600)]
Support spontaneous payments in InvoicePayer
InvoicePayer handles retries not only when handling PaymentPathFailed
events but also for some types of PaymentSendFailure on the initial
send. Expand InvoicePayer's interface with a pay_pubkey function for
spontaneous (keysend) payments. Add a send_spontaneous_payment function
to the Payer trait to support this and implement it for ChannelManager.
Jeffrey Czyz [Tue, 9 Nov 2021 15:37:34 +0000 (09:37 -0600)]
Refactor InvoicePayer for spontaneous payments
To support spontaneous payments, InvoicePayer's sending logic must be
invoice-agnostic. Refactor InvoicePayer::pay_invoice_internal such that
invoice-specific code is in pay_invoice_using_amount and the remaining
logic is in pay_internal.
Further refactor the code's payment_cache locking such that it is
accessed consistently when needed, and tidy up the code a bit.
Matt Corallo [Tue, 9 Nov 2021 21:22:47 +0000 (21:22 +0000)]
Correct Channel type serialization logic
Currently, we write out the Channel's `ChannelTypeFeatures` as an
odd type, implying clients which don't understand the
`ChannelTypeFeatures` field can simply ignore it. This is obviously
nonsense if the channel type is some future version - the client
needs to fail to deserialize as it doesn't understand the channel's
type.
We adapt the serialization logic here to only write out the
`ChannelTypeFeatures` field if it is something other than
only-static-remote-key, and simply consider that "default" (as it
is the only supported type today). Then, we write out the channel
type as an even TLV, implying clients which do not understand it
must fail to read the `Channel`.
Note that we do not need to bother reserving the TLV type no longer
written as it never appeared in a release (merged post-0.0.103).
Matt Corallo [Wed, 13 Oct 2021 04:19:13 +0000 (04:19 +0000)]
Be less aggressive in outbound HTLC CLTV timeout checks
We currently assume our counterparty is naive and misconfigured and
may force-close a channel to get an HTLC we just forwarded them.
There shouldn't be any reason to do this - we don't have any such
bug, and we shouldn't start by assuming our counterparties are
buggy. Worse, this results in refusing to forward payments today,
failing HTLCs for largely no reason.
Instead, we keep a fairly conservative check, but not one which
will fail HTLC forwarding spuriously - testing only that the HTLC
doesn't expire for a few blocks from now.
Matt Corallo [Tue, 26 Oct 2021 21:40:14 +0000 (21:40 +0000)]
Fix a minor memory leak on PermanentFailure mon errs when sending
If we send a payment and fail to update the first-hop channel state
with a `PermanentFailure` ChannelMonitorUpdateErr, we would have an
entry in our pending payments map, but possibly not return the
PaymentId back to the user to retry the payment, leading to a (rare
and relatively minor) memory leak.
Matt Corallo [Mon, 4 Oct 2021 03:11:36 +0000 (03:11 +0000)]
Log before+after ChannelMonitor/Manager updates for visibility
I realized on my own node that I don't have any visibility into how
long a monitor or manager persistence call takes, potentially
blocking other operations. This makes it much more clear by adding
a relevant log_trace!() print immediately before and immediately
after persistence.
Jeffrey Czyz [Thu, 4 Nov 2021 18:58:11 +0000 (13:58 -0500)]
Add SinceEpoch time to test Scorer hermetically
In order to test Scorer hermetically, sleeps must be avoided. Add a
SinceEpoch abstraction for manually advancing time. Implement the Time
trait for SinceEpoch so that it can be used with ScorerUsingTime in
tests.
Matt Corallo [Sun, 31 Oct 2021 18:21:46 +0000 (18:21 +0000)]
Make payment_path_failed path type bindings-mappable
The bindings don't currently support passing `Vec`s of objects
which it mappes as "opaque types". This is because it will require
clones to convert its own list of references to Rust's list of
objects.
In the near future we should resolve this limitation, allowing us
to revert this (and make `find_route`'s method signature similarly
cleaner), but for now we must avoid `Vec<OpaqueType>`.
Jeffrey Czyz [Fri, 29 Oct 2021 04:44:26 +0000 (23:44 -0500)]
Implement (de)serialization for Scorer
Scorer should be serialized to retain penalty data between restarts.
Implement (de)serialization for Scorer by serializing last failure times
as duration since the UNIX epoch. For no-std, the zero-Duration is used.
Jeffrey Czyz [Fri, 29 Oct 2021 13:52:27 +0000 (08:52 -0500)]
Parameterize Scorer by a Time trait
Scorer uses time to determine how much to penalize a channel after a
failure occurs. Parameterizing it by time cleans up the code such that
no-std support is in a single AlwaysPresent struct, which implements the
Time trait. Time is implemented for std::time::Instant when std is
available.
This parameterization also allows for deterministic testing since a
clock could be devised to advance forward as needed.
Jeffrey Czyz [Fri, 29 Oct 2021 04:23:45 +0000 (23:23 -0500)]
Refactor channel failure penalty logic
Move channel failure penalty logic into a ChannelFailure abstraction.
This encapsulates the logic for accumulating penalties and decaying them
over time. It also is responsible for the no-std behavior. This cleans
up Scorer and will make it easier to serialize it.
Jeffrey Czyz [Mon, 1 Nov 2021 18:14:14 +0000 (13:14 -0500)]
Parameterize NetGraphMsgHandler with NetworkGraph
NetworkGraph is owned by NetGraphMsgHandler, but DefaultRouter requires
a reference to it. Introduce shared ownership to NetGraphMsgHandler so
that both can use the same NetworkGraph.
Matt Corallo [Tue, 26 Oct 2021 22:52:06 +0000 (22:52 +0000)]
Rewrite InvoicePayer retry to correctly handle MPP partial failures
This rewrites a good chunk of the retry logic in `InvoicePayer` to
address two issues:
* it was not considering the return value of `send_payment` (and
`retry_payment`) may indicate a failure on some paths but not
others,
* it was not considering that more failures may still come later
when removing elements from the retry count map. This could
result in us seeing an MPP-partial-failure, failing to retry,
removing the retries count entry, and then retrying other parts,
potentially forever.
Matt Corallo [Wed, 27 Oct 2021 22:12:07 +0000 (22:12 +0000)]
Dont unwrap `RouteParameter::expiry_time` as users can set it
Users can provide anything they want as `RouteParameters` so we
shouldn't assume any fields are set any particular way, including
`expiry_time` set at all.
Jeffrey Czyz [Wed, 27 Oct 2021 15:39:22 +0000 (10:39 -0500)]
Penalize failed channels in Scorer
As payments fail, the channel responsible for the failure may be
penalized. Implement Scorer::payment_path_failed to penalize the failed
channel using a configured penalty. As time passes, the penalty is
reduced using exponential decay, though penalties will accumulate if the
channel continues to fail. The decay interval is also configurable.
Jeffrey Czyz [Thu, 14 Oct 2021 18:04:39 +0000 (13:04 -0500)]
Notify scorer of failing payment path and channel
Upon receiving a PaymentPathFailed event, the failing payment may be
retried on a different path. To avoid using the channel responsible for
the failure, a scorer should be notified of the failure before being
used to find a new route.
Add a payment_path_failed method to routing::Score and call it in
InvoicePayer's event handler. Introduce a LockableScore parameterization
to InvoicePayer so the scorer is locked only once before calling
find_route.
Matt Corallo [Thu, 21 Oct 2021 22:33:42 +0000 (22:33 +0000)]
Give peers one timer tick to finish handshake before disconnecting
This ensures we don't let a hung connection stick around forever if
the peer never completes the initial handshake.
This also resolves a race where, on receiving a second connection
from a peer, we may reset their_node_id to None to prevent sending
messages even though the `channel_encryptor`
`is_ready_for_encryption()`. Sending pings only checks the
`channel_encryptor` status, not `their_node_id` resulting in an
`unwrap` on `None` in `enqueue_message`.