Wilmer Paulino [Mon, 25 Sep 2023 23:57:04 +0000 (16:57 -0700)]
Use correct input sequence for HTLC claims from counterparty commitments
HTLC outputs, like the `to_remote` output, in commitment transactions
with anchor outputs also have an additional `1 CSV` constraint on the
counterparty. When spending such outputs, their corresponding input
needs to have their sequence set to 1. This was done for HTLC claims
from holder commitments, but unfortunately not for counterparty
commitments as we were lacking test coverage.
Valentine Wallace [Wed, 27 Sep 2023 03:44:27 +0000 (23:44 -0400)]
get_route: fix path value contribution to include min htlc overpay
Previously, the fuzzer hit a debug panic because we wouldn't include the amount
overpaid to meet a last hop's min_htlc in the total collected paths value. We
now include this value and also penalize hops along the overpaying path to
ensure that it gets deprioritized in path selection.
Valentine Wallace [Wed, 27 Sep 2023 02:21:15 +0000 (22:21 -0400)]
get_route: fix path_min for first_hop<>network_node candidates
Previously, we would add a first_hop<>network_node channel that did not have
enough contribution amount to cover the next channel's min htlc plus fees,
because we were storing the next hop as having a path_min that did not include
fees, and would add a connecting first_hop node that did not have enough
contribution amount, leading to a debug panic upon invalid path construction.
Valentine Wallace [Tue, 26 Sep 2023 22:57:02 +0000 (18:57 -0400)]
get_route: fix path_min for first_hop<>blinded_hint candidates
See previous commit, but the bug where we would underestimate how much a first
hop candidate needed to be able to relay was also present in blinded paths.
Valentine Wallace [Wed, 27 Sep 2023 01:19:33 +0000 (21:19 -0400)]
get_route: fix path_min when adding first_hop<>route_hint candidates
Previously, we would add a candidate hop to the list of potential hops even
though its available contribution wasn't sufficient to meet the next hop's
min_htlc. We'd subsequently build an invalid path using this hop and hit a
debug assertion.
Valentine Wallace [Fri, 22 Sep 2023 20:13:36 +0000 (16:13 -0400)]
get_route: fix outdated var name
Previously this variable was a bool, but has since been updated to be an
Option, so rename accordingly.
Valentine Wallace [Tue, 12 Sep 2023 18:57:59 +0000 (14:57 -0400)]
Pathfinding: ignore blinded route hints where we are the intro node
See tests, but the fuzzer found several panics from not fully ignoring these
hints.
We should support these route hints eventually, but it will involve some
reworking of the Path/BlindedTail structs.
Valentine Wallace [Tue, 12 Sep 2023 19:01:53 +0000 (15:01 -0400)]
Remove trailing whitespace in get_route
Because my text editor loves to do that.
Valentine Wallace [Tue, 12 Sep 2023 18:56:54 +0000 (14:56 -0400)]
Refuse to pathfind when provided our_node_id matches internal dummy pk
The fuzzer managed to hit this and it causes some invalid paths to be generated
internally.
Valentine Wallace [Tue, 12 Sep 2023 18:55:36 +0000 (14:55 -0400)]
Pathfinding: log when we ignore one hop blinded route hints
Matt Corallo [Wed, 27 Sep 2023 03:51:12 +0000 (03:51 +0000)]
Merge pull request #2601 from TheBlueMatt/2023-09-117-alpha2
Bump versions to 0.0.117-alpha2/invoice 0.25.0-alpha2
Matt Corallo [Wed, 27 Sep 2023 03:47:16 +0000 (03:47 +0000)]
Merge pull request #2603 from TheBlueMatt/2023-09-default-route-limit
Set a default max_total_routing_fee_msat of 1% + 50sats
Matt Corallo [Wed, 27 Sep 2023 03:47:09 +0000 (03:47 +0000)]
Merge pull request #2602 from TheBlueMatt/2023-09-descriptor-hash
Derive `Hash` for `SpendableOutputDescriptor`
Matt Corallo [Wed, 27 Sep 2023 03:46:39 +0000 (03:46 +0000)]
Merge pull request #2359 from domZippilli/2023-06-monitor-updating-persister
Monitor updating persister
Dom Zippilli [Wed, 30 Aug 2023 00:42:07 +0000 (17:42 -0700)]
Add MonitorUpdatingPersister
MonitorUpdatingPersister is an implementation of Persister that stores
ChannelMonitorUpdates separately from ChannelMonitors. Its RFC is
in #2545, at https://github.com/orgs/lightningdevkit/discussions/2545.
Co-Authored-By: Elias Rohrer <dev@tnull.de>
Matt Corallo [Tue, 26 Sep 2023 22:54:12 +0000 (22:54 +0000)]
Merge pull request #2486 from wvanlint/batch_channel_opens
Batch funding for v1 channel establishments
Matt Corallo [Tue, 26 Sep 2023 21:28:29 +0000 (21:28 +0000)]
Set a default `max_total_routing_fee_msat` of 1% + 50sats
When using the normal default constructors, we should have some
fee maximum to ensure our default behavior is safe. Here we pick
1% + 50 sats to ensure we're always willing to pay
reasonabl(y high) fees, but not anything too wild.
Matt Corallo [Tue, 26 Sep 2023 21:12:46 +0000 (21:12 +0000)]
Derive `Debug` and `Display` for `Route`
Matt Corallo [Tue, 26 Sep 2023 20:41:49 +0000 (20:41 +0000)]
Derive `Hash` for `SpendableOutputDescriptor`
This allows for easy descriptor de-duplication before building a
claiming transaction.
Willem Van Lint [Sat, 29 Jul 2023 00:21:47 +0000 (17:21 -0700)]
Batch funding for v1 channel establishments
Willem Van Lint [Tue, 26 Sep 2023 05:39:26 +0000 (22:39 -0700)]
Rename finish_force_close_channel to finish_close_channel
Willem Van Lint [Tue, 26 Sep 2023 05:36:26 +0000 (22:36 -0700)]
Always call finish_force_close_channel on closure
This is a step towards more unified closing of channels, and provides a
place where the per_peer_state lock is not held.
Matt Corallo [Tue, 26 Sep 2023 20:21:08 +0000 (20:21 +0000)]
Bump versions to 0.0.117-alpha2/invoice 0.25.0-alpha2
Matt Corallo [Tue, 26 Sep 2023 20:07:52 +0000 (20:07 +0000)]
Merge pull request #2417 from tnull/2023-07-max-total-fee
Add config option to set maximum total routing fee
Elias Rohrer [Tue, 26 Sep 2023 13:30:39 +0000 (15:30 +0200)]
Test `max_total_routing_fee_msat` handling when retrying overpaid paths
We setup an MPP scenario with two paths in which we need to overpay to
reach `htlc_minimum_msat`. We then fail the overpaid path and check that
on retry our `max_total_routing_fee_msat` only accounts for the path
fees, but not for the fees overpaid in the first attempt.
Elias Rohrer [Tue, 26 Sep 2023 08:48:33 +0000 (10:48 +0200)]
Check `max_total_routing_fee` is accounted for in `test_threaded_payment_retries`
Elias Rohrer [Wed, 20 Sep 2023 13:32:37 +0000 (15:32 +0200)]
Check `max_total_routing_fee` is reduced in `mpp_retry` test
We check that the `RouteParameters::max_total_routing_fee` field is reduced accordingly
to our previously used fees.
Elias Rohrer [Mon, 4 Sep 2023 12:51:42 +0000 (14:51 +0200)]
Account for leftover fee budget when retrying via `check_retry_payment`
Matt Corallo [Tue, 26 Sep 2023 16:37:40 +0000 (16:37 +0000)]
Correct comment in `shutdown_on_unfunded_channel`
which described the script type incorrectly.
Matt Corallo [Thu, 21 Sep 2023 20:21:13 +0000 (20:21 +0000)]
Refer to an "outage" over a "timeout", for when failure happens
Timeouts may be worth retrying, but an outage is a more general
term which obviously cannot be retried.
Matt Corallo [Thu, 21 Sep 2023 20:19:08 +0000 (20:19 +0000)]
Correct ChannelUnavailable error docs on `send_payment_with_route`
Monitor update failure can no longer lead to a `ChannelUnavailable`
error, but more common cases such as the peer being disconnected
always could, so those should be documented clearer.
Matt Corallo [Thu, 21 Sep 2023 20:03:13 +0000 (20:03 +0000)]
Matt Corallo [Tue, 26 Sep 2023 16:36:16 +0000 (16:36 +0000)]
Merge pull request #2597 from TheBlueMatt/2023-09-finish-force-close-deadlocks
Fix potential peer_state deadlocks in `finish_force_close_channel`
Elias Rohrer [Tue, 18 Jul 2023 13:46:06 +0000 (15:46 +0200)]
Account for leftover fee budget when retrying `PartialFailure`s
Elias Rohrer [Fri, 14 Jul 2023 14:46:52 +0000 (16:46 +0200)]
Test we adhere to `max_total_routing_fee_msat`
Elias Rohrer [Fri, 14 Jul 2023 11:25:33 +0000 (13:25 +0200)]
Consider `RouteParameters::max_total_routing_fee_msat` in `get_route`
We exclude any candidate hops if we find that using them would let the
aggregated path routing fees exceed `max_total_routing_fee_msat`.
Moreover, we return an error if the aggregated fees over all paths of
the selected route would surpass `max_total_routing_fee_msat`.
Elias Rohrer [Fri, 14 Jul 2023 09:47:22 +0000 (11:47 +0200)]
Introduce `RouteParameters::max_total_routing_fee_msat`
Currently, users have no means to upper-bound the total fees accruing
when finding a route. Here, we add a corresponding field to
`RouteParameters` which will be used to limit the candidate set during
path finding in the following commits.
Matt Corallo [Mon, 25 Sep 2023 19:49:15 +0000 (19:49 +0000)]
Provide some test coverage of `shutdown` msgs for unfunded chans
We have code to handle receiving `shutdown` messages on unfudned
channels. However, it had no test coverage, which we add here.
Matt Corallo [Mon, 25 Sep 2023 18:05:53 +0000 (18:05 +0000)]
Fix potential peer_state deadlocks in `finish_force_close_channel`
`ChannelManager::finish_force_close_channel` exists to do cleanups
which must happen without the `per_peer_state` mutex held. However,
because it lacked lock assertions, several changes snuck in
recently which resulted in it running with peer-state locks held,
risking a deadlock if some HTLCs need to be failed.
Matt Corallo [Mon, 25 Sep 2023 17:08:41 +0000 (17:08 +0000)]
Merge pull request #2583 from Evanfeenstra/pub-make-onion
Pub make onion
Matt Corallo [Mon, 25 Sep 2023 16:56:03 +0000 (16:56 +0000)]
Merge pull request #2576 from valentinewallace/2023-09-fix-outbound-bp-fail-ev
Fix `PaymentPathFailed::payment_failed_permanently` on blinded path fail
Matt Corallo [Mon, 25 Sep 2023 16:00:04 +0000 (16:00 +0000)]
Merge pull request #2594 from benthecarman/debug-monitor-update-id
Implement Debug for MonitorUpdateId
benthecarman [Sun, 24 Sep 2023 05:34:27 +0000 (00:34 -0500)]
Implement Debug for MonitorUpdateId
Valentine Wallace [Fri, 15 Sep 2023 20:55:12 +0000 (16:55 -0400)]
Blame outbound channel on UPDATE onion failure with 0-len update
We've run into this several times in the wild, likely due to
https://github.com/ElementsProject/lightning/issues/6200 wherein a node on the
path will error with 0x1000 but not provide a channel update (a spec
violation).
Previously, we would blame the inbound edge even though the buggy peer wanted
us to blame the outbound edge. Since this issue seems to be recurring and our
blaming the inbound edge is causing us to punish innocent channels, trust the
peer that the outbound edge is the one to blame.
Valentine Wallace [Thu, 14 Sep 2023 15:46:02 +0000 (11:46 -0400)]
Fix PaymentPathFailed::payment_failed_permanently on blinded path fail
Previously this value would be incorrectly set to true because we wouldn't
account for blinded hops when determining if we were processing the last hop's
failure packet.
Valentine Wallace [Thu, 14 Sep 2023 15:41:35 +0000 (11:41 -0400)]
Correct DecodedOnionFailure when processing we-are-intro-node path
We don't support sending to paths where we are the intro node yet, but may as
well set the failure correctly now.
Valentine Wallace [Thu, 14 Sep 2023 15:33:01 +0000 (11:33 -0400)]
DecodedOnionFailure::payment_retryable -> ::payment_failed_permanently
Our ultimate goal with this field is to set
PaymentPathFailed::payment_failed_permanently, so use this name rather than
flipping a bool back and forth across methods.
Valentine Wallace [Wed, 20 Sep 2023 18:49:58 +0000 (14:49 -0400)]
Struct-ify onion util internal result type
Improves readability.
Valentine Wallace [Thu, 14 Sep 2023 03:52:11 +0000 (23:52 -0400)]
Rename onion util internal var
This variable is ultimately for setting
PaymentPathFailed::payment_failed_permanently, so use this name rather than
flipping a bool back and forth.
Elias Rohrer [Fri, 22 Sep 2023 07:09:41 +0000 (09:09 +0200)]
Merge pull request #2589 from ErikDeSmedt/reexport_route_hint_hop
Reexport RouteHintHop
Matt Corallo [Thu, 21 Sep 2023 23:20:17 +0000 (23:20 +0000)]
Merge pull request #2592 from TheBlueMatt/2023-09-117-alpha
Update crate version numbers to 0.0.117-alpha1/invoice 0.25-alpha1
Matt Corallo [Thu, 21 Sep 2023 20:40:13 +0000 (20:40 +0000)]
Merge pull request #2590 from TheBlueMatt/2023-09-default-score-params
Use `Default::default()` to construct `()` as a test scoring param
Matt Corallo [Thu, 21 Sep 2023 20:27:12 +0000 (20:27 +0000)]
Update crate version numbers to 0.0.117-alpha1/invoice 0.25-alpha1
Matt Corallo [Thu, 21 Sep 2023 20:22:16 +0000 (20:22 +0000)]
Merge pull request #2562 from TheBlueMatt/2023-08-no-perm-fail
Drop the ChannelMonitorUpdateStatus::PermanentFailure variant
Matt Corallo [Thu, 14 Sep 2023 20:02:46 +0000 (20:02 +0000)]
Add an `UnrecoverableError` variant to `ChannelMonitorUpdateStatus`
While there is no great way to handle a true failure to persist a
`ChannelMonitorUpdate`, it is confusing for users for there to be
no error variant at all on an I/O operation.
Thus, here we re-add the error variant removed over the past
handful of commits, but rather than handle it in a truly unsafe
way, we simply panic, optimizing for maximum mutex poisoning to
ensure any future operations fail and return immediately.
In the future, we may consider changing the handling of this to
instead set some "disconnect all peers and fail all operations"
bool to give the user a better chance to shutdown in a semi-orderly
fashion, but there's only so much that can be done in lightning if
we truly cannot persist new updates.
Matt Corallo [Sun, 10 Sep 2023 20:21:50 +0000 (20:21 +0000)]
Drop doc comments on `ChainMonitor` trait impl methods
In general, doc comments on trait impl blocks are not very visible
in rustdoc output, and unless they provide useful information they
should be elided.
Here we drop useless doc comments on `ChainMonitor`'s `Watch` impl
methods.
Matt Corallo [Sun, 10 Sep 2023 20:05:43 +0000 (20:05 +0000)]
Drop error handling in `handle_new_monitor_update`
Now that `handle_new_monitor_update` can no longer return an error,
it similarly no longer needs any code to handle errors. Here we
remove that code, substantially reducing macro variants.
Matt Corallo [Sun, 10 Sep 2023 17:48:49 +0000 (17:48 +0000)]
Clean up code flow in `ChannelManager`
In the previous commit various dead code was removed. Here we
finish that cleanup by removing uneccessary indentation and syntax.
Matt Corallo [Sun, 10 Sep 2023 17:39:24 +0000 (17:39 +0000)]
Drop `PermamentFailure` persistence handling in ChannelManager
Matt Corallo [Fri, 8 Sep 2023 00:05:56 +0000 (00:05 +0000)]
Update `ChannelMonitorUpdateStatus` documentation with async support
Since we now (almost) support async monitor update persistence, the
documentation on `ChannelMonitorUpdateStatus` can be updated to no
longer suggest users must keep a local copy that persists before
returning. However, because there are still a few remaining issues,
we note that async support is currently beta and explicily warn of
potential for funds-loss.
Fixes #1684
Matt Corallo [Wed, 30 Aug 2023 18:22:33 +0000 (18:22 +0000)]
Rename `MonitorEvent::CommitmentTxConfirmed` to `HolderForceClosed`
The `MonitorEvent::CommitmentTxConfirmed` has always been a result
of us force-closing the channel, not the counterparty doing so.
Thus, it was always a bit of a misnomer. Worse, it carried over
into the channel's `ClosureReason` in the event API.
Here we simply rename it and use the proper `ClosureReason`.
Matt Corallo [Wed, 30 Aug 2023 18:19:35 +0000 (18:19 +0000)]
Drop `ChannelMonitorUpdate::UpdateFailed` as its now unused
Matt Corallo [Wed, 30 Aug 2023 18:16:03 +0000 (18:16 +0000)]
Drop `channel_perm_failed` tracking in `ChainMonitor`
Now that `PermanentFailure` is not a possible return value, we can
simply remove handling of it in `ChannelMonitor`.
Matt Corallo [Sun, 10 Sep 2023 17:14:32 +0000 (17:14 +0000)]
Drop the `ChannelMonitorUpdateStatus::PermanentFailure` variant
When a `ChannelMonitorUpdate` fails to apply, it generally means
we cannot reach our storage backend. This, in general, is a
critical issue, but is often only a transient issue.
Sadly, users see the failure variant and return it on any I/O
error, resulting in channel force-closures due to transient issues.
Users don't generally expect force-closes in most cases, and
luckily with async `ChannelMonitorUpdate`s supported we don't take
any risk by "delaying" the `ChannelMonitorUpdate` indefinitely.
Thus, here we drop the `PermanentFailure` variant entirely, making
all failures instead be "the update is in progress, but won't ever
complete", which is equivalent if we do not close the channel
automatically.
Matt Corallo [Sun, 10 Sep 2023 22:11:56 +0000 (22:11 +0000)]
Rewrite failure payment retry tests to avoid perm-fail storage
Two tests in the payment tests currently rely on failing to persist
ChannelMonitorUpdates as their method of failing payments before
they even get out the door.
In the coming commits we'll drop the persist failure error codes,
so here rewrite these tests to rely on trying to send more than is
available in a channel.
Erik De Smedt [Tue, 19 Sep 2023 18:57:40 +0000 (20:57 +0200)]
Reexport RouteHintHop
Earlier @benthecarman re-exported `RouteHint` to make life-easier
for developpers that use `lightning-invoice` and don't use the
`lightning`-crate.
This only solved part of the issue. To create a `RouteHint` the
developer must also have access to `RouteHintHop`.
See also:
PR https://github.com/lightningdevkit/rust-lightning/pull/2572
commit
79b426f49b08a66e404669ce7d1332c3977c5d95
Matt Corallo [Thu, 21 Sep 2023 01:44:23 +0000 (01:44 +0000)]
Use `Default::default()` to construct `()` as a test scoring param
In bindings, we can't use unbounded generic types, and thus have to
rip out the `ScoreParams` and replace them with static
`ProbabilisticScoringFeeParams` universally. To make this easier,
using `Default::default()` everywhere allows the type to change out
from under the test without the test needing to change.
Matt Corallo [Wed, 20 Sep 2023 22:21:02 +0000 (22:21 +0000)]
Merge pull request #2547 from TheBlueMatt/2023-04-nonlinear-scoring
Add an option to make the success probability estimation nonlinear
Evan Feenstra [Fri, 15 Sep 2023 22:47:23 +0000 (15:47 -0700)]
public make_onion_message static method on OnionMessenger
Matt Corallo [Sat, 16 Sep 2023 17:04:11 +0000 (17:04 +0000)]
Avoid unnecessary newline in middle of log statement
Matt Corallo [Mon, 24 Apr 2023 01:52:41 +0000 (01:52 +0000)]
Add an option to make the success probability estimation nonlinear
Our "what is the success probability of paying over a channel with
the given liquidity bounds" calculation currently assumes the
probability of where the liquidity lies in a channel is constant
across the entire capacity of a channel. This is obviously a
somewhat dubious assumption given most nodes don't materially
rebalance and flows within the network often push liquidity
"towards the edges".
Here we add an option to consider this when scoring channels during
routefinding. Specifically, if a new `linear_success_probability`
flag is unset on `ProbabilisticScoringFeeParameters`, rather than
assuming a PDF of `1` (across the channel's capacity scaled from 0
to 1), we use `(x - 0.5)^2`.
This assumes liquidity is likely to be near the edges, which
matches experimental results. Further, calculating the CDF (i.e.
integral) between arbitrary points on the PDF is trivial, which we
do as our main scoring function.
While this (finally) introduces floats in our scoring, its not
practical to exponentiate using fixed-precision, and benchmarks
show this is a performance regression, but not a huge one, more
than made up for by the increase in payment success rates.
Matt Corallo [Sat, 16 Sep 2023 18:39:03 +0000 (18:39 +0000)]
Score in-flight amounts as amounts, not a capacity reduction
When we started considering the in-flight amounts when scoring, we
took the approach of considering the in-flight amount as an
effective reduction in the channel's total capacity. When we were
scoring using a flat success probability PDF, that was fine,
however in the next commit we'll move to a highly nonlinear one,
which makes this a pretty confusing heuristic.
Here, instead, we move to considering the in-flight amount as
simply an extension of the amount we're trying to send over the
channel, which is equivalent for the flat success probability PDF,
but makes much more sense in a nonlinear world.
Matt Corallo [Mon, 24 Apr 2023 22:53:28 +0000 (22:53 +0000)]
Scale the success probability of channels without info down by 75%
If we are examining a channel for which we have no information at
all, we traditionally assume the HTLC success probability is
proportional to the channel's capacity. While this may be the case,
it is not the case that a tiny payment over a huge channel is
guaranteed to succeed, as we assume. Rather, the probability of
such success is likely closer to 50% than 100%.
Here we try to capture this by simply scaling the success
probability for channels where we have no information down
linearly. We pick 75% as the upper bound rather arbitrarily - while
50% may be more accurate, its possible it would lead to an
over-reliance on channels which we have paid through in the past,
which aren't necessarily always the best candidates.
Note that we only do this scaling for the historical bucket
tracker, as there we can be confident we've never seen a successful
HTLC completion on the given channel. If we were to apply the same
scaling to the simple liquidity bounds based scoring we'd penalize
channels we've never tried over those we've only ever fails to pay
over, which is obviously not a good outcome.
Matt Corallo [Mon, 24 Apr 2023 05:16:51 +0000 (05:16 +0000)]
Split out success probability calculation to allow for changes
Our "what is the success probability of paying over a channel with
the given liquidity bounds" calculation is reused in a few places,
and is a key assumption across our main score calculation and the
historical bucket score calculations.
Here we break it out into a function to make it easier to
experiment with different success probability calculations.
Note that this drops the numerator +1 in the liquidity scorer,
which was added to compensate for the divisor + 1 (which exists to
avoid divide-by-zero), making the new math slightly less correct
but not by any material amount.
Matt Corallo [Tue, 19 Sep 2023 20:43:08 +0000 (20:43 +0000)]
Merge pull request #2587 from wpaulino/anchors-fee-fixes
Anchor outputs fee fixes
Wilmer Paulino [Mon, 18 Sep 2023 21:07:57 +0000 (14:07 -0700)]
Sanity check fees on transactions produced by the bump event handler
We add a few debug assertions to ensure we don't overpay fees by 5% more
than expected.
Wilmer Paulino [Mon, 18 Sep 2023 21:00:37 +0000 (14:00 -0700)]
Account for existing input amounts throughout coin selection
We'd previously ignore the existing amount transactions were already
attempting to spend when deciding whether we should add more inputs
throughout coin selection. This would result in us attaching more inputs
than necessary to satisfy our target amount. In the case of HTLC
transactions, we'd burn the HTLC amount completely, since the pre-signed
transaction has zero fee (input amount == output amount).
Along the way, we also fix the slight overpayment in anchor
transactions. We now properly account for the fees the transaction
already paid for, simply by pretending the fees are part of the anchor
input amount.
Wilmer Paulino [Fri, 15 Sep 2023 00:38:22 +0000 (17:38 -0700)]
Limit external claim feerate bumps
Since we don't know the total input amount of an external claim (those
which come anchor channels), we can't limit our feerate bumps by the
amount of funds we have available to use. Instead, we choose to limit it
by a margin of the new feerate estimate.
valentinewallace [Mon, 18 Sep 2023 19:56:08 +0000 (15:56 -0400)]
Merge pull request #2584 from TheBlueMatt/2023-09-msrv-try-2
Correct syn pinning on cargo 1.48
Matt Corallo [Mon, 18 Sep 2023 16:41:57 +0000 (16:41 +0000)]
Merge pull request #2534 from tnull/2023-08-upstream-preflight-probing
Upstream and fix preflight probing
Matt Corallo [Mon, 18 Sep 2023 16:16:46 +0000 (16:16 +0000)]
Merge pull request #2582 from TheBlueMatt/2023-09-one-less-clone
Avoid unnecessarily cloning unsigned Transaction when broadcasting
Elias Rohrer [Tue, 12 Sep 2023 11:20:31 +0000 (13:20 +0200)]
Expose `AChannelManager` trait and use it in `lightning-invoice`
Elias Rohrer [Mon, 11 Sep 2023 11:50:10 +0000 (13:50 +0200)]
Probe up to second-to-last hop if last was provided by route hint
If the last hop was provided by route hint we assume it's not an announced channel.
If furthermore only a single route hint is provided we refrain from probing through
all the way to the end and instead probe up to the second-to-last channel.
Optimally we'd do this not based on above mentioned assumption but
rather by checking inclusion in our network graph. However, we don't
have access to our graph in `ChannelManager`.
Elias Rohrer [Fri, 1 Sep 2023 14:39:04 +0000 (16:39 +0200)]
Test preflight probing sends and skips if necessary
Elias Rohrer [Mon, 28 Aug 2023 13:03:04 +0000 (15:03 +0200)]
Add preflight probing capabilities
We add a `ChannelManager::send_preflight_probes` method that can be used
to send pre-flight probes given some [`RouteParameters`]. Additionally,
we add convenience methods in for spontaneous probes and send pre-flight
probes for a given invoice.
As pre-flight probes might take up some of the available liquidity, we
here introduce that channels whose available liquidity is less than the
required amount times
`UserConfig::preflight_probing_liquidity_limit_multiplier` won't be used
to send pre-flight probes.
This commit is a more or less a carbon copy of the pre-flight
probing code recently added to LDK Node.
Elias Rohrer [Tue, 12 Sep 2023 13:51:37 +0000 (15:51 +0200)]
Include `maybe_announced` field in `RouteHop`
When sending preflight probes, we want to exclude last hops that are
possibly announced. To this end, we here include a new field in
`RouteHop` that will be `true` when we either def. know the hop to be
announced, or, if there exist public channels between the hop's
counterparties that this hop might refer to (i.e., be an alias for).
Matt Corallo [Sun, 17 Sep 2023 00:57:00 +0000 (00:57 +0000)]
Replace `cargo build` calls in CI with `cargo check`
We're not actually using the build output, so there's no reason to
do a build vs just running check.
Matt Corallo [Sun, 17 Sep 2023 00:10:29 +0000 (00:10 +0000)]
Move coverage generation to llvm-cov in the hopes its more stable
Matt Corallo [Fri, 15 Sep 2023 23:52:40 +0000 (23:52 +0000)]
Correct syn pinning on cargo 1.48
Sadly the pinning introduced in
050f5a90297678f2cad36feee9a1db2367e
was brittle in the face of any further syn updates, and has already
broken.
Here we fix it by looking up the actual version of syn to pin.
Note that this dependency is somewhat nonsense as its actually only
a `criterion` dependency, pulled in even though we haven't set the
bench flag (as we aren't yet using `resolver = 2`).
Matt Corallo [Fri, 15 Sep 2023 22:38:57 +0000 (22:38 +0000)]
Merge pull request #2176 from TheBlueMatt/2023-04-expose-success-prob
Move the historical bucket tracker to 32 unequal sized buckets
Matt Corallo [Fri, 15 Sep 2023 21:31:56 +0000 (21:31 +0000)]
Merge pull request #2579 from rmalonson/unsignedtx
Remove one unnecessary call for sign_holder_commitment_and_htlcs
Matt Corallo [Fri, 15 Sep 2023 19:17:34 +0000 (19:17 +0000)]
Avoid unnecessarily cloning unsigned Transaction when broadcasting
Our `Trusted*` wrappers in `chan_utils` expose additional inner
fields by reference. However, because they were not explicitly
marked as returning a reference with the wrapped struct's
lifetimes, rustc was considering them to return a reference with
the wrapper struct's lifetime.
This is unnecessarily restrictive, and resulted in the addition of
a clone in
9850c5814abdf69883b51a4afe73912ff5436fd9 which we remove
here.
Rachel Malonson [Fri, 15 Sep 2023 02:39:30 +0000 (19:39 -0700)]
Remove unnecessary signing call in ChannelMonitor
Elias Rohrer [Fri, 15 Sep 2023 19:18:57 +0000 (21:18 +0200)]
Merge pull request #2577 from TheBlueMatt/2023-09-msrv
Fix MSRV tests and drop internet-required test
Matt Corallo [Thu, 7 Sep 2023 22:34:30 +0000 (22:34 +0000)]
Move to a constant for "bucket one" in the scoring buckets
Scoring buckets are stored as fixed point ints, with a 5-bit
fractional part (i.e. a value of 1.0 is stored as "32"). Now that
we also have 32 buckets, this leads to the codebase having many
references to 32 which could reasonably be confused for each other.
Thus, we add a constant here for the value 1.0 in our fixed-point
scheme.
Matt Corallo [Tue, 6 Jun 2023 04:08:32 +0000 (04:08 +0000)]
Decay `historical_estimated_channel_liquidity_*` result to `None`
`historical_estimated_channel_liquidity_probabilities` previously
decayed to `Some(([0; 8], [0; 8]))`. This was thought to be useful
in that it allowed identification of cases where data was previously
available but is now decayed away vs cases where data was never
available. However, with the introduction of
`historical_estimated_payment_success_probability` (which uses the
existing scoring routines so will decay to `None`) this is
unnecessarily confusing.
Given data which has decayed to zero will also not be used anyway,
there's little reason to keep the old behavior, and we now decay to
`None`.
We also take this opportunity to split the overloaded
`get_decayed_buckets`, removing uneccessary code during scoring.
Matt Corallo [Sat, 19 Aug 2023 18:43:45 +0000 (18:43 +0000)]
Special-case the 0th minimum bucket in historical scoring
Points in the 0th minimum bucket either indicate we sent a payment
which is < 1/16,384th of the channel's capacity or, more likely,
we failed to send a payment. In either case, averaging the success
probability across the full range of upper-bounds doesn't make a
whole lot of sense - if we've never managed to send a "real"
payment over a channel, we should be considering it quite poor.
To address this, we special-case the 0th minimum bucket and only
look at the largest-offset max bucket when calculating the success
probability.
Matt Corallo [Sun, 16 Apr 2023 04:03:08 +0000 (04:03 +0000)]
Track "steady-state" channel balances in history buckets not live
The lower-bound of the scoring history buckets generally never get
used - if we try to send a payment and it fails, we don't learn
a new lower-bound for the liquidity of a channel, and if we
successfully send a payment we only learn a lower-bound that
applied *before* we sent the payment, not after it completed.
If we assume channels have some "steady-state" liquidity, then
tracking our liquidity estimates *after* a payment doesn't really
make sense - we're not super likely to make a second payment across
the same channel immediately (or, if we are, we can use our
un-decayed liquidity estimates for that). By the time we do go to
use the same channel again, we'd assume that its back at its
"steady-state" and the impacts of our payment have been lost.
To combat both of these effects, here we "subtract" the impact of
any just-successful payments from our liquidity estimates prior to
updating the historical buckets.
Matt Corallo [Fri, 8 Sep 2023 17:48:50 +0000 (17:48 +0000)]
Move the historical bucket tracker to 32 unequal sized buckets
Currently we store our historical estimates of channel liquidity in
eight evenly-sized buckets, each representing a full octile of the
channel's total capacity. This lacks precision, especially at the
edges of channels where liquidity is expected to lie.
To mitigate this, we'd originally checked if a payment lies within
a bucket by comparing it to a sliding scale of 64ths of the
channel's capacity. This allowed us to assign penalties to payments
that fall within any more than the bottom 64th or lower than the
top 64th of a channel.
However, this still lacks material precision - on a 1 BTC channel
we could only consider failures for HTLCs above 1.5 million sats.
With today's lightning usage often including 1-100 sat payments in
tips, this is a rather significant lack of precision.
Here we rip out the existing buckets and replace them with 32
*unequal* sized buckets. This allows us to focus our precision at
the edges of a channel (where the liquidity is likely to lie, and
where precision helps the most).
We set the size of the edge buckets to 1/16,384th of the channel,
with the size increasing exponentially until it approaches the
inner buckets. For backwards compatibility, the buckets divide
evenly into octets, allowing us to convert the existing buckets
into the new ones cleanly.
This allows us to consider HTLCs down to 6,000 sats for 1 BTC
channels. In order to avoid failing to penalize channels which have
always failed, we drop the sliding scale for comparisons and simply
check if the payment is above the minimum bucket we're analyzing and
below *or in* the maximum one. This generates somewhat more
pessimistic scores, but fixes the lower bound where we suddenly
assign a 0% failure probability.
While this does represent a regression in routing performance, in
some cases the impact of not having to examine as many nodes
dominates, leading to a performance increase.
On a Xeon E3-1220 v5, the `large_mpp_routes` benchmark shows a 15%
performance increase, while the more stable benchmarks show an 8%
and 15% performance regression.
Matt Corallo [Fri, 8 Sep 2023 17:47:43 +0000 (17:47 +0000)]
Implement serialization for `[u16; 32]`, DRYing it with `[u8; *]`
In the next commit we'll need serialization for `[u16; 32]`, which
we add here, unifying it with the `[u8; *]` serialization macro.