]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
2 years agoMerge pull request #1741 from TheBlueMatt/2022-09-invoice-bindings-nostd-111
valentinewallace [Mon, 26 Sep 2022 13:23:10 +0000 (09:23 -0400)]
Merge pull request #1741 from TheBlueMatt/2022-09-invoice-bindings-nostd-111

[0.0.111-bindings] Don't make references to `std` in `lightning-invoice` in bindings

2 years agoDon't make references to `std` in `lightning-invoice` in bindings 2022-09-invoice-bindings-nostd-111
Matt Corallo [Sun, 25 Sep 2022 14:24:13 +0000 (14:24 +0000)]
Don't make references to `std` in `lightning-invoice` in bindings

As we support `no-std` for `lightning-invoice` builds, we should
support them in `c_bindings` as well, which we add a test for in
CI here.

2 years agoMerge pull request #1736 from TheBlueMatt/2022-09-111-bindings-2
Matt Corallo [Sun, 25 Sep 2022 07:51:53 +0000 (07:51 +0000)]
Merge pull request #1736 from TheBlueMatt/2022-09-111-bindings-2

[0.0.111-bindings] Add a bindings-only version of Future::register_callback

2 years agoRename `MultiThreadedLockableScoreLock` `MultiThreadedLockedScore` 2022-09-111-bindings-2
Matt Corallo [Fri, 23 Sep 2022 08:17:01 +0000 (08:17 +0000)]
Rename `MultiThreadedLockableScoreLock` `MultiThreadedLockedScore`

as the first is quite a mouthful, and the second isn't materially
less specific.

2 years agoAdd a bindings-only version of `Future::register_callback`
Matt Corallo [Fri, 23 Sep 2022 21:08:26 +0000 (21:08 +0000)]
Add a bindings-only version of `Future::register_callback`

While we could, in theory, add support to the bindings logic to map
`Box<dyn Trait>`, there isn't a whole lot of use doing so when its
incredibly trivial to do directly.

This adds a trivial wrapper around `Future::register_callback` that
is only built in bindings and which is linked in the
`register_callback` docs for visibility.

2 years agoMerge pull request #1730 from TheBlueMatt/2022-09-111-bindings-and-backports
Matt Corallo [Fri, 23 Sep 2022 16:09:55 +0000 (16:09 +0000)]
Merge pull request #1730 from TheBlueMatt/2022-09-111-bindings-and-backports

[0.0.111-bindings] Bindings Branch Rebase + Backports

2 years agoExport Onion Message structs in their respective modules 2022-09-111-bindings-and-backports
Matt Corallo [Sun, 11 Sep 2022 18:38:02 +0000 (18:38 +0000)]
Export Onion Message structs in their respective modules

Re-exports in Rust make `use` statements a little shorter, but for
otherwise don't materially change a crate's API. Sadly, the C
bindings generator currently can't figure out re-exports, but it
also exports everything into one global namespace, so it doesn't
matter much anyway.

2 years agoTag `KVStore` `(C-not exported)` as `Writeable` isn't mapped
Matt Corallo [Sun, 26 Jun 2022 18:14:30 +0000 (18:14 +0000)]
Tag `KVStore` `(C-not exported)` as `Writeable` isn't mapped

Currently `Writeable` is mapped manually, making it impossible to
automatically map a trait method that is parameterized by
`Writeable` (as is true for the `write` method on `KVStore`).

Ultimately we'll want to move to automatically mapping `Writeable`
like any other trait (only manually mapping the std `Write` and
`Read` traits), so this is only a candidate for the bindings branch,
not upstream. That may take a few releases, however.

2 years agoRestrict ChannelInfo::as_directed_from visibility
Jeffrey Czyz [Tue, 29 Mar 2022 15:20:39 +0000 (10:20 -0500)]
Restrict ChannelInfo::as_directed_from visibility

Bindings can't handle references in return types, so reduce the
visibility to pub(crate).

2 years agoSimplify type aliasing somewhat around times
Matt Corallo [Tue, 1 Mar 2022 03:47:28 +0000 (03:47 +0000)]
Simplify type aliasing somewhat around times

.. as the current C bindings generator isn't capable of handling
type aliases in generics in type alias definition currently.

2 years agoMake `as_directed_to` non-public
Matt Corallo [Tue, 1 Mar 2022 03:46:52 +0000 (03:46 +0000)]
Make `as_directed_to` non-public

...as the bindings generation does not currently have the ability
to map a reference to a `NodeId` inside a tuple.

2 years agoAdd a `MutexGuard` wrapper for the bindings-only `LockableScore`
Matt Corallo [Mon, 19 Sep 2022 09:11:11 +0000 (09:11 +0000)]
Add a `MutexGuard` wrapper for the bindings-only `LockableScore`

In the bindings, we don't directly export any `std` types. Instead,
we provide trivial wrappers around them which expose only a
bindings-compatible API (and which is code in-crate, where the
bindings generator can see it).

We never quite finished this for `MultiThreadedLockableScore` - due
to some limitations in the bindings generator and the way the
scores are used in `lightning-invoice`, the scoring API ended up
being further concretized in patches for the bindings. Luckily the
scoring interface has been rewritten somewhat, and it so happens
that we can now generate bindings with the upstream code.

The final piece of the puzzle is done here, where we add a struct
which wraps `MutexGuard` so that we can expose the lock for
`MultiThreadedLockableScore`.

2 years agoDowngrade `hashbrown` to meet MSRV
Matt Corallo [Thu, 22 Sep 2022 14:07:25 +0000 (14:07 +0000)]
Downgrade `hashbrown` to meet MSRV

`hashbrown` depends on `ahash` which depends on `once_cell`. Sadly,
in https://github.com/matklad/once_cell/issues/201 the `once_cell`
maintainer decided they didn't want to do the work of having an
MSRV policy for `once_cell`, making `ahash`, and thus `hashbrown`
require the latest compiler. I've reached out to `ahash` to suggest
they drop the dependency (as they could trivially work around not
having it), but until then we simply downgrade `hashbrown`.

`rust-bitcoin` also requires an older `hashbrown` so we're actually
reducing our total `no-std` code here anyway.

2 years agoAvoid returning a reference to a u64.
Matt Corallo [Sun, 18 Sep 2022 13:55:08 +0000 (13:55 +0000)]
Avoid returning a reference to a u64.

In c353c3ed7c40e689a3b9fb6730c6dabbd3c92cc5 an accessor method was
added which returns an `Option<&u64>`. While this allows Rust to
return an 8-byte object, returning a reference to something
pointer-sized is a somewhat strange API.

Instead, we opt for a straight `Option<u64>`, which is sadly
somewhat larger on the stack, but is simpler and already supported
in the bindings generation.

2 years agoSwap `Vec<&RouteHop>` parameters for slices
Matt Corallo [Sun, 18 Sep 2022 13:34:39 +0000 (13:34 +0000)]
Swap `Vec<&RouteHop>` parameters for slices

In c353c3ed7c40e689a3b9fb6730c6dabbd3c92cc5, new scorer-updating
methods were added to the `Router` object, however they were
passed as a `Vec` of references. We use the list-of-references
pattern to make bindings simpler (by not requiring allocations per
entry), however there's no reason prefer to passing a `Vec` over
a slice, given the `Vec` doesn't hold ownership of the objects
anyway.

2 years agoMove Scorer requirement away from Router trait
Jurvis Tan [Thu, 15 Sep 2022 21:58:08 +0000 (14:58 -0700)]
Move Scorer requirement away from Router trait

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.

* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.

2 years agoMerge pull request #1712 from TheBlueMatt/2022-09-111 v0.0.111
Matt Corallo [Tue, 13 Sep 2022 00:27:54 +0000 (00:27 +0000)]
Merge pull request #1712 from TheBlueMatt/2022-09-111

Cut 0.0.111

2 years agoBump versions to lightning* 0.0.111 and lightning-invoice 0.19 2022-09-111
Matt Corallo [Mon, 12 Sep 2022 22:33:26 +0000 (22:33 +0000)]
Bump versions to lightning* 0.0.111 and lightning-invoice 0.19

2 years agoUpdate release notes for 0.0.111
Matt Corallo [Fri, 9 Sep 2022 18:39:15 +0000 (18:39 +0000)]
Update release notes for 0.0.111

2 years agoMerge pull request #1714 from TheBlueMatt/2022-09-111-bindings-discovered-cleanups
Matt Corallo [Mon, 12 Sep 2022 20:51:51 +0000 (20:51 +0000)]
Merge pull request #1714 from TheBlueMatt/2022-09-111-bindings-discovered-cleanups

Small Cleanups Discovered during Bindings for 0.0.111

2 years agoMerge pull request #1715 from TheBlueMatt/2022-09-fix-msg-send
Matt Corallo [Mon, 12 Sep 2022 20:01:32 +0000 (20:01 +0000)]
Merge pull request #1715 from TheBlueMatt/2022-09-fix-msg-send

Fix encryption of broadcasted gossip messages

2 years agoUpdate `Simple*PeerManager` type aliases to support Onion Messages 2022-09-111-bindings-discovered-cleanups
Matt Corallo [Mon, 12 Sep 2022 18:05:58 +0000 (18:05 +0000)]
Update `Simple*PeerManager` type aliases to support Onion Messages

Note that `SimpleArcPeerHandler` is also updated to not wrap
`IgnoringMessageHandler` in an `Arc`, as `IgnoringMessageHandler`
is already zero-sized.

2 years agoMerge pull request #1716 from TheBlueMatt/2022-09-log-unreadable-type
Matt Corallo [Mon, 12 Sep 2022 18:10:47 +0000 (18:10 +0000)]
Merge pull request #1716 from TheBlueMatt/2022-09-log-unreadable-type

Include the message type when we send unreadable gossip msg errors

2 years agoEncrypt+MAC most P2P messages in-place 2022-09-fix-msg-send
Matt Corallo [Mon, 12 Sep 2022 15:20:37 +0000 (15:20 +0000)]
Encrypt+MAC most P2P messages in-place

For non-gossip-broadcast messages, our current flow is to first
serialize the message into a `Vec`, and then allocate a new `Vec`
into which we write the encrypted+MAC'd message and header.

This is somewhat wasteful, and its rather simple to instead
allocate only one buffer and encrypt the message in-place.

2 years agoFix encryption of broadcasted gossip messages
Matt Corallo [Mon, 12 Sep 2022 15:16:41 +0000 (15:16 +0000)]
Fix encryption of broadcasted gossip messages

In 47e818f198abafba01b9ad278582886f9007dac2, forwarding broadcasted
gossip messages was split into a separate per-peer message buffer.
However, both it and the original regular-message queue are
encrypted immediately when the messages are enqueued. Because the
lightning P2P encryption algorithm is order-dependent, this causes
messages to fail their MAC checks as the messages from the two
queues may not be sent to peers in the order in which they were
encrypted.

The fix is to simply queue broadcast gossip messages unencrypted,
encrypting them when we add them to the regular outbound buffer.

2 years agoDrop unused type parameter on `BlindedRoute::new`
Matt Corallo [Sun, 11 Sep 2022 21:18:01 +0000 (21:18 +0000)]
Drop unused type parameter on `BlindedRoute::new`

I'm not sure why rustc didn't complain about the unused parameter
or why we're allowed to get away without explicitly bounding the
`Sign` in the `KeysInterface`, but the current code requires all
`BlindedPath` construction to explicitly turbofish an unused type.

2 years agoAdd relevant `(C-not exported)` tags on OnionMessenger aliases
Matt Corallo [Sun, 11 Sep 2022 21:12:16 +0000 (21:12 +0000)]
Add relevant `(C-not exported)` tags on OnionMessenger aliases

The "helpful" type aliases don't make sense for C bindings as all
generics are concretized anyway.

2 years agoInline generic bounds rather than using the `where` clause
Matt Corallo [Sat, 10 Sep 2022 20:31:52 +0000 (20:31 +0000)]
Inline generic bounds rather than using the `where` clause

The bindings generator is pretty naive in its generic resolution
and doesn't like `where` clauses for bounds that are simple traits.
This should eventually change, but for now its simplest to just
inline the relevant generic bounds.

2 years agoDo not use blanket impls when building for `c_bindings`
Matt Corallo [Sat, 10 Sep 2022 20:31:42 +0000 (20:31 +0000)]
Do not use blanket impls when building for `c_bindings`

The C bindings generator isn't capable of figuring out if a blanket
impl applies in a given context, and instead opts to always write
out any relevant impl's for a trait. Thus, blanket impls should be
disabled when building with `#[cfg(c_bindings)]`.

2 years agoRename `{Signed,}RawInvoice::hash` to avoid naming collisions
Matt Corallo [Sat, 10 Sep 2022 20:31:12 +0000 (20:31 +0000)]
Rename `{Signed,}RawInvoice::hash` to avoid naming collisions

Now that `{Signed,}RawInvoice` implement the std `Hash` trait,
having a method called `hash` is ambiguous.

Instead, we rename the `hash` methods `signed_hash` to make it
clear that the hash is the one used for the purpose of signing the
invoice.

2 years agoInclude the message type when we send unreadable gossip msg errors 2022-09-log-unreadable-type
Matt Corallo [Mon, 12 Sep 2022 15:35:56 +0000 (15:35 +0000)]
Include the message type when we send unreadable gossip msg errors

2 years agoMerge pull request #1710 from TheBlueMatt/2022-09-compile-warn
Matt Corallo [Sun, 11 Sep 2022 14:54:05 +0000 (14:54 +0000)]
Merge pull request #1710 from TheBlueMatt/2022-09-compile-warn

Fix several compile warnings added in some of my recent commits

2 years agoFix several compile warnings when testing in no-std mode 2022-09-compile-warn
Matt Corallo [Fri, 9 Sep 2022 19:41:58 +0000 (19:41 +0000)]
Fix several compile warnings when testing in no-std mode

2 years agoFix (really dumb) warning rustc introduced in latest beta
Matt Corallo [Fri, 9 Sep 2022 17:09:27 +0000 (17:09 +0000)]
Fix (really dumb) warning rustc introduced in latest beta

2 years agoFix several compile warnings added in some of my recent commits
Matt Corallo [Fri, 9 Sep 2022 16:01:54 +0000 (16:01 +0000)]
Fix several compile warnings added in some of my recent commits

2 years agoMerge pull request #1688 from valentinewallace/2022-08-flip-om-feature-bit
Matt Corallo [Fri, 9 Sep 2022 21:48:33 +0000 (21:48 +0000)]
Merge pull request #1688 from valentinewallace/2022-08-flip-om-feature-bit

Onion messages: flip feature bit ðŸŽ‰

2 years agoMerge pull request #1713 from TheBlueMatt/2022-09-bad-doc-versions
Matt Corallo [Fri, 9 Sep 2022 21:13:36 +0000 (21:13 +0000)]
Merge pull request #1713 from TheBlueMatt/2022-09-bad-doc-versions

Correct `get_claimable_balance` version info

2 years agoMerge pull request #1711 from TheBlueMatt/2022-08-0conf-panic
Matt Corallo [Fri, 9 Sep 2022 20:24:07 +0000 (20:24 +0000)]
Merge pull request #1711 from TheBlueMatt/2022-08-0conf-panic

Fix spurious panic on receipt of a block while awaiting funding

2 years agoUpdate ChannelMessageHandler::provided_node_features docs
Valentine Wallace [Fri, 9 Sep 2022 20:01:41 +0000 (16:01 -0400)]
Update ChannelMessageHandler::provided_node_features docs

To be uniform with the other handler provided_node_features docs

2 years agoDon't advertise onion messages in known channel features
Valentine Wallace [Fri, 9 Sep 2022 16:41:23 +0000 (12:41 -0400)]
Don't advertise onion messages in known channel features

2 years agoOR InitFeatures and NodeFeatures from onion message handler
Valentine Wallace [Fri, 9 Sep 2022 16:29:13 +0000 (12:29 -0400)]
OR InitFeatures and NodeFeatures from onion message handler

Similar to how we OR our InitFeaures and NodeFeatures across both our channel
and routing message handlers, we also want to OR the features of our onion
message handler.

2 years agoSupport forwarding onion messages in advertised features
Valentine Wallace [Fri, 26 Aug 2022 23:40:10 +0000 (19:40 -0400)]
Support forwarding onion messages in advertised features

In upcoming commit(s), onion message support will be advertised conditionally
based on the OnionMessageProvider provided to PeerManager.

2 years agoAdd missing wumbo feature bit docs
Valentine Wallace [Sat, 6 Aug 2022 22:23:30 +0000 (18:23 -0400)]
Add missing wumbo feature bit docs

2 years agoAdd a new NodeFeatures constructor to capture the types of flags
Valentine Wallace [Fri, 9 Sep 2022 16:17:33 +0000 (12:17 -0400)]
Add a new NodeFeatures constructor to capture the types of flags

When ChannelMessageHandler implementations wish to return a NodeFeatures which
contain all the known flags that are relevant to channel handling, but not
gossip  handling, they currently need to do so by manually constructing a
NodeFeatures with all known flags and then clearing the ones they don't want.

Instead of spreading this logic across the codebase, this consolidates such
construction into one place in features.rs.

2 years agoOR NodeFeatures from both Channel and Routing message handlers
Valentine Wallace [Fri, 9 Sep 2022 16:12:50 +0000 (12:12 -0400)]
OR NodeFeatures from both Channel and Routing message handlers

When we broadcast a node announcement, the features we support are really a
combination of all the various features our different handlers support. This
commit captures this concept by OR'ing our NodeFeatures across both our channel
and routing message handlers.

2 years agoCorrect `get_claimable_balance` version info 2022-09-bad-doc-versions
Matt Corallo [Fri, 9 Sep 2022 19:30:30 +0000 (19:30 +0000)]
Correct `get_claimable_balance` version info

5a8ede09fb3c8bbcd8694d94c12dac9ea7485537 updated the documentation
on `get_claimable_balance` to note that if a channel went on-chain
with an LDK version older than 0.0.108 some
counterparty-revoked-output claimable balances my be missing.
However, this failed to account for the fact that we rely on the
entirely-new-in-0.0.111
`confirmed_commitment_tx_counterparty_output` field for some
balances as well.

Thus, the comment should have been in terms of 0.0.111, not
0.0.108.

2 years agoMerge pull request #1709 from tnull/2022-09-make-access-error-debug
Matt Corallo [Fri, 9 Sep 2022 18:46:41 +0000 (18:46 +0000)]
Merge pull request #1709 from tnull/2022-09-make-access-error-debug

Derive `Debug` for `AccessError`

2 years agoFix spurious panic on receipt of a block while awaiting funding 2022-08-0conf-panic
Matt Corallo [Mon, 29 Aug 2022 18:42:34 +0000 (18:42 +0000)]
Fix spurious panic on receipt of a block while awaiting funding

When we receive a block we always test if we should send our
channel_ready via `check_get_channel_ready`. If the channel in
question requires confirmations, we quickly return if the funding
transaction has not yet confirmed (or even been defined), however
for 0conf channels the checks are necessarily more involved.

In any case, we wish to panic if the funding transaction has
confirmations prior to when it should have been broadcasted. This
is useful as it is easy for users to violate our broadcast-time
invariants without noticing and the panic gives us an opportunity
to catch it.

Sadly, in the case of 0conf channels, if we hadn't yet seen the
funding transaction at all but receive a block we would hit this
sanity check as we don't check whether there are actually funding
transaction confirmations prior to panicing.

2 years agoEnable all feature sets to OR with another set of the same type
Valentine Wallace [Fri, 9 Sep 2022 15:54:22 +0000 (11:54 -0400)]
Enable all feature sets to OR with another set of the same type

2 years agoMerge pull request #1701 from TheBlueMatt/2022-09-feature-or
Matt Corallo [Fri, 9 Sep 2022 16:47:25 +0000 (16:47 +0000)]
Merge pull request #1701 from TheBlueMatt/2022-09-feature-or

Fetch InitFeatures from both Channel and Routing Message Handlers

2 years agoMake clear_initial_routing_sync more consistent with other APIs 2022-09-feature-or
Matt Corallo [Wed, 7 Sep 2022 17:43:14 +0000 (17:43 +0000)]
Make clear_initial_routing_sync more consistent with other APIs

2 years agoAdd a new InitFeatures constructor to capture the types of flags
Matt Corallo [Wed, 7 Sep 2022 17:55:01 +0000 (17:55 +0000)]
Add a new InitFeatures constructor to capture the types of flags

When `ChannelMessageHandler` implementations wish to return an
`InitFeatures` which contain all the known flags that are relevant
to channel handling, but not gossip handling, they currently need
to do so by manually constructing an InitFeatures with all known
flags and then clearing the ones they dont want.

Instead of spreading this logic out across the codebase, this
consolidates such construction to one place in features.rs.

2 years agoOR InitFeatures from both Channel and Routing message handlers
Matt Corallo [Wed, 7 Sep 2022 17:35:50 +0000 (17:35 +0000)]
OR InitFeatures from both Channel and Routing message handlers

When we go to send an Init message to new peers, the features we
support are really a combination of all the various features our
different handlers support. This commit captures this concept by
OR'ing our InitFeatures across both our Channel and Routing
handlers.

Note that this also disables setting the `initial_routing_sync`
flag in init messages, as was intended in
e742894492c55802b241eebc585bbd28aa16481b, per the comment added on
`clear_initial_routing_sync`, though this should not be a behavior
change in practice as nodes which support gossip queries ignore the
initial routing sync flag.

2 years agoFetch our `InitFeatures` from `ChannelMessageHandler`
Matt Corallo [Wed, 7 Sep 2022 17:51:16 +0000 (17:51 +0000)]
Fetch our `InitFeatures` from `ChannelMessageHandler`

Like we now do for `NodeFeatures`, this converts to asking our
registered `ChannelMessageHandler` for our `InitFeatures` instead
of hard-coding them to the global LDK known set.

This allows handlers to set different feature bits based on what
our configuration actually supports rather than what LDK supports
in aggregate.

2 years agoDerive `Debug` for `AccessError`
Elias Rohrer [Fri, 9 Sep 2022 13:18:43 +0000 (15:18 +0200)]
Derive `Debug` for `AccessError`

2 years agoMerge pull request #1699 from TheBlueMatt/2022-08-announcement-rework
Matt Corallo [Fri, 9 Sep 2022 00:34:10 +0000 (00:34 +0000)]
Merge pull request #1699 from TheBlueMatt/2022-08-announcement-rework

2 years agoMerge pull request #1704 from TheBlueMatt/2022-09-always-probe-failed
Matt Corallo [Thu, 8 Sep 2022 20:35:15 +0000 (20:35 +0000)]
Merge pull request #1704 from TheBlueMatt/2022-09-always-probe-failed

Dont use PaymentPathFailed a probe fails without making it out

2 years agoAdd a folder to track CHANGELOG entries for the next release 2022-08-announcement-rework
Matt Corallo [Wed, 7 Sep 2022 00:01:12 +0000 (00:01 +0000)]
Add a folder to track CHANGELOG entries for the next release

2 years agoMove `broadcast_node_announcement` to `PeerManager`
Matt Corallo [Tue, 6 Sep 2022 22:34:29 +0000 (22:34 +0000)]
Move `broadcast_node_announcement` to `PeerManager`

Some `NodeFeatures` will, in the future, represent features which
are not enabled by the `ChannelManager`, but by other message
handlers handlers. Thus, it doesn't make sense to determine the
node feature bits in the `ChannelManager`.

The simplest fix for this is to change to generating the
node_announcement in `PeerManager`, asking all the connected
handlers which feature bits they support and simply OR'ing them
together. While this may not be sufficient in the future as it
doesn't consider feature bit dependencies, support for those could
be handled at the feature level in the future.

This commit moves the `broadcast_node_announcement` function to
`PeerHandler` but does not yet implement feature OR'ing.

2 years agoSend channel_{announcement,update} msgs on connection, not timer
Matt Corallo [Tue, 6 Sep 2022 21:30:33 +0000 (21:30 +0000)]
Send channel_{announcement,update} msgs on connection, not timer

When we connect to a new peer, immediately send them any
channel_announcement and channel_update messages for any public
channels we have with other peers. This allows us to stop sending
those messages on a timer when they have not changed and ensures
we are sending messages when we have peers connected, rather than
broadcasting at startup when we have no peers connected.

2 years agoImprove debuggability when tests fail due to excess events 2022-09-always-probe-failed
Matt Corallo [Wed, 7 Sep 2022 21:53:13 +0000 (21:53 +0000)]
Improve debuggability when tests fail due to excess events

2 years agoDont use PaymentPathFailed a probe fails without making it out
Matt Corallo [Wed, 7 Sep 2022 21:39:17 +0000 (21:39 +0000)]
Dont use PaymentPathFailed a probe fails without making it out

When we fail to forward a probe HTLC at all and immediately fail it
(e.g. due to the first hop channel closing) we'd previously
spuriously generate only a `PaymentPathFailed` event. This violates
the expected API, as users expect a `ProbeFailed` event instead.

This fixes the oversight by ensuring we generate the correct event.

Thanks to @jkczyz for pointing out this issue.

2 years agoDrop redundant code in `fail_holding_cell_htlcs`
Matt Corallo [Wed, 7 Sep 2022 21:09:50 +0000 (21:09 +0000)]
Drop redundant code in `fail_holding_cell_htlcs`

`fail_holding_cell_htlcs` calls through to
`fail_htlc_backwards_internal` for HTLCs that need to be
failed-backwards but opts to generate its own payment failure
events for `HTLCSource:;OutboundRoute` HTLCs. There is no reason
for that as `fail_htlc_backwards_internal` will also happily
generate (now-)equivalent events for `HTLCSource::OutboundRoute`
HTLCs.

Thus, we can drop the redundant code and always call
`fail_htlc_backwards_internal` for each HTLC in
`fail_holding_cell_htlcs`.

2 years agoMerge pull request #1702 from TheBlueMatt/2022-09-one-hop-retryable
Matt Corallo [Thu, 8 Sep 2022 17:34:04 +0000 (17:34 +0000)]
Merge pull request #1702 from TheBlueMatt/2022-09-one-hop-retryable

Mark failed one-hop HTLCs as retrably

2 years agoMerge pull request #1700 from TheBlueMatt/2022-09-missing-event-deser
Matt Corallo [Thu, 8 Sep 2022 15:41:58 +0000 (15:41 +0000)]
Merge pull request #1700 from TheBlueMatt/2022-09-missing-event-deser

Add missing deserialization of Event::HTLCHandlingFailed

2 years agoAdd missing deserialization of Event::HTLCHandlingFailed 2022-09-missing-event-deser
Matt Corallo [Tue, 6 Sep 2022 22:51:29 +0000 (22:51 +0000)]
Add missing deserialization of Event::HTLCHandlingFailed

17e6c374c513f2eca810fa4e931be65f0d4fc29f added the
`HTLCHandlingFailed` event, including serialization thereof,
however failed to add corresponding deserialization. This corrects
that oversight by adding said deserialization.

Thanks to @wpaulino for catching the oversight.

2 years agoRename `rejected_by_dest` -> `payment_failed_permanently` 2022-09-one-hop-retryable
Matt Corallo [Wed, 7 Sep 2022 20:58:05 +0000 (20:58 +0000)]
Rename `rejected_by_dest` -> `payment_failed_permanently`

The `rejected_by_dest` field of the `PaymentPathFailed` event has
always been a bit of a misnomer, as its really more about retry
than where a payment failed. Now is as good a time as any to
rename it.

2 years agoMark failed counterparty-is-destination HTLCs retryable
Matt Corallo [Wed, 7 Sep 2022 20:02:04 +0000 (20:02 +0000)]
Mark failed counterparty-is-destination HTLCs retryable

When our counterparty is the payment destination and we receive
an `HTLCFailReason::Reason` in `fail_htlc_backwards_internal` we
currently always set `rejected_by_dest` in the `PaymentPathFailed`
event, implying the HTLC should *not* be retried.

There are a number of cases where we use `HTLCFailReason::Reason`,
but most should reasonably be treated as retryable even if our
counterparty was the destination (i.e. `!rejected_by_dest`):
 * If an HTLC times out on-chain, this doesn't imply that the
   payment is no longer retryable, though the peer may well be
   offline so retrying may not be very useful,
 * If a commitment transaction "containing" a dust HTLC is
   confirmed on-chain, this definitely does not imply the payment
   is no longer retryable
 * If the channel we intended to relay over was closed (or
   force-closed) we should retry over another path,
 * If the channel we intended to relay over did not have enough
   capacity we should retry over another path,
 * If we received a update_fail_malformed_htlc message from our
   peer, we likely should *not* retry, however this should be
   exceedingly rare, and appears to nearly never appear in practice

Thus, this commit simply disables the behavior here, opting to
treat all `HTLCFailReason::Reason` errors as retryable.

Note that prior to 93e645daf46f85949ae0edf60d36bf21e9fde8af this
change would not have made sense as it would have resulted in us
retrying the payment over the same channel in some cases, however
we now "blame" our own channel and will avoid it when routing for
the same payment.

2 years agoMerge pull request #1697 from TheBlueMatt/2022-08-event-docs
valentinewallace [Wed, 7 Sep 2022 02:04:39 +0000 (22:04 -0400)]
Merge pull request #1697 from TheBlueMatt/2022-08-event-docs

Clarify and consolidate event handling requirements

2 years agoMerge pull request #1698 from TheBlueMatt/2022-09-release-names
valentinewallace [Wed, 7 Sep 2022 01:57:16 +0000 (21:57 -0400)]
Merge pull request #1698 from TheBlueMatt/2022-09-release-names

Add release names to CHANGELOG

2 years agoMerge pull request #1691 from TheBlueMatt/2022-08-dust-retry
valentinewallace [Wed, 7 Sep 2022 00:02:50 +0000 (20:02 -0400)]
Merge pull request #1691 from TheBlueMatt/2022-08-dust-retry

Correct payment resolution after on chain failure of dust HTLCs

2 years agoAdd release names to CHANGELOG 2022-09-release-names
Matt Corallo [Tue, 6 Sep 2022 21:17:15 +0000 (21:17 +0000)]
Add release names to CHANGELOG

There have been release names in Github for some time, though some
releases were missed. This adds them to CHANGELOG.

2 years agoClarify and consolidate event handling requirements 2022-08-event-docs
Matt Corallo [Tue, 6 Sep 2022 20:56:24 +0000 (20:56 +0000)]
Clarify and consolidate event handling requirements

We've seen a bit of user confusion about the requirements for event
handling, largely because the idempotency and consistency
requirements weren't super clearly phrased. While we're at it, we
also consolidate some documentation out of the event handling
function onto the trait itself.

Fixes #1675.

2 years agoCorrect payment resolution after on chain failure of dust HTLCs 2022-08-dust-retry
Matt Corallo [Fri, 2 Sep 2022 21:10:43 +0000 (21:10 +0000)]
Correct payment resolution after on chain failure of dust HTLCs

Previously, we wouldn't mark a dust HTLC as permanently resolved if
the commitment transaction went on chain. This resulted in us
always considering the HTLC as pending on restart, when we load the
pending payments set from the monitors.

Fixes #1653.

2 years agoMerge pull request #1657 from TheBlueMatt/2022-08-async-man-update
valentinewallace [Tue, 6 Sep 2022 20:06:06 +0000 (16:06 -0400)]
Merge pull request #1657 from TheBlueMatt/2022-08-async-man-update

Add a background processor which is async

2 years agoMerge pull request #1695 from TheBlueMatt/2022-08-log-chan_update
Matt Corallo [Tue, 6 Sep 2022 19:15:05 +0000 (19:15 +0000)]
Merge pull request #1695 from TheBlueMatt/2022-08-log-chan_update

Ensure we log private channel_updates at a non-GOSSIP log level

2 years agoEnsure we log private channel_updates at a non-GOSSIP log level 2022-08-log-chan_update
Matt Corallo [Mon, 5 Sep 2022 16:28:11 +0000 (16:28 +0000)]
Ensure we log private channel_updates at a non-GOSSIP log level

If we receive a channel_update for one of our private channels, we
will not log the message at the usual TRACE log level as the
message falls into the gossip range. However, for our own channels
they aren't *just* gossip, as we store that info and it changes
how we generate invoices. Thus, we add a log in `ChannelManager`
here at the DEBUG log level.

2 years agoAdd a background processing function that is async. 2022-08-async-man-update
Matt Corallo [Tue, 9 Aug 2022 06:01:10 +0000 (06:01 +0000)]
Add a background processing function that is async.

Adds a method which operates like BackgroundProcessor::start but
instead of functioning by spawning a background thread it is async.

2 years agoAdd a `Future` which can receive manager persistence events
Matt Corallo [Tue, 9 Aug 2022 04:15:21 +0000 (04:15 +0000)]
Add a `Future` which can receive manager persistence events

This allows users who don't wish to block a full thread to receive
persistence events.

The `Future` added here is really just a trivial list of callbacks,
but from that we can build a (somewhat ineffecient)
std::future::Future implementation and can (at least once a mapping
for Box<dyn Trait> is added) include the future in no-std bindings
as well.

Fixes #1595

2 years agoRemove internal references to `persistence` in waker.rs
Matt Corallo [Mon, 29 Aug 2022 21:13:58 +0000 (21:13 +0000)]
Remove internal references to `persistence` in waker.rs

2 years agoMerge pull request #1692 from TheBlueMatt/2022-08-time-goes-backwards
Matt Corallo [Sat, 3 Sep 2022 16:15:33 +0000 (16:15 +0000)]
Merge pull request #1692 from TheBlueMatt/2022-08-time-goes-backwards

Handle monotonic clock going backwards during runtime

2 years agoMerge pull request #1650 from valentinewallace/2022-08-take-onionmsgs-public
Matt Corallo [Fri, 2 Sep 2022 22:20:32 +0000 (22:20 +0000)]
Merge pull request #1650 from valentinewallace/2022-08-take-onionmsgs-public

2 years agoHandle monotonic clock going backwards during runtime 2022-08-time-goes-backwards
Matt Corallo [Fri, 2 Sep 2022 21:57:32 +0000 (21:57 +0000)]
Handle monotonic clock going backwards during runtime

We've had some users complain that `duration_since` is panic'ing
for them. This is possible if the machine being run on is buggy and
the "monotonic clock" goes backwards, which sadly some ancient
systems can do.

Rust addressed this issue in 1.60 by forcing
`Instant::duration_since` to not panic if the machine is buggy
(and time goes backwards), but for users on older rust versions we
do the same by hand here.

2 years agoMove open_zero_conf_channel utility to common test utils
Matt Corallo [Fri, 2 Sep 2022 21:02:13 +0000 (21:02 +0000)]
Move open_zero_conf_channel utility to common test utils

2 years agoLimit OnionMessenger outbound buffer size
Valentine Wallace [Thu, 1 Sep 2022 19:26:17 +0000 (15:26 -0400)]
Limit OnionMessenger outbound buffer size

Drop OMs if they push us over the max OnionMessenger outbound buffer size

2 years agoDon't construct OnionMessage while holding peer lock
Valentine Wallace [Fri, 2 Sep 2022 17:41:41 +0000 (13:41 -0400)]
Don't construct OnionMessage while holding peer lock

2 years agoRefuse to send and forward OMs to disconnected peers
Valentine Wallace [Thu, 4 Aug 2022 15:05:07 +0000 (11:05 -0400)]
Refuse to send and forward OMs to disconnected peers

We also refuse to connect to peers that don't advertise onion message
forwarding support.

2 years agoExpose onion message module as public
Valentine Wallace [Wed, 3 Aug 2022 15:42:54 +0000 (11:42 -0400)]
Expose onion message module as public

And fix warnings

2 years agoMerge pull request #1643 from jurvis/jurvis/2022-07-inflights-htlcs-across-payments
Jeffrey Czyz [Thu, 1 Sep 2022 20:01:30 +0000 (15:01 -0500)]
Merge pull request #1643 from jurvis/jurvis/2022-07-inflights-htlcs-across-payments

Track in-flight HTLCs across payments when routing

2 years agoMake payment tests more realistic
jurvis [Sun, 28 Aug 2022 06:07:50 +0000 (23:07 -0700)]
Make payment tests more realistic

Made sure that every hop has a unique receipient. When we simulate
calling `channel_penalty_msat` in `TestRouter`’s find route, use
actual previous node ids instead of just using the payer’s.

2 years agoKeep track of inflight HTLCs across payments
jurvis [Tue, 30 Aug 2022 05:50:44 +0000 (22:50 -0700)]
Keep track of inflight HTLCs across payments

Added two methods, `process_path_inflight_htlcs` and
`remove_path_inflight_htlcs`, that updates that `payment_cache` map with
path information that may have failed, succeeded, or have been given up
on.

Introduced `AccountForInflightHtlcs`, which will wrap our user-provided
scorer. We move the `S:Score` type parameterization from the `Router` to
`find_route`, so we can use our newly introduced
`AccountForInflightHtlcs`.

`AccountForInflightHtlcs` keeps track of a map of inflight HTLCs by
their short channel id, direction, and give us the value that is being
used up.

This map will in turn be populated prior to calling `find_route`, where
we’ll use `create_inflight_map`, to generate a current map of all
inflight HTLCs based on what was stored in `payment_cache`.

2 years agoMerge pull request #1687 from TheBlueMatt/2022-08-fuzz-msrv
Matt Corallo [Tue, 30 Aug 2022 17:12:47 +0000 (17:12 +0000)]
Merge pull request #1687 from TheBlueMatt/2022-08-fuzz-msrv

Drop honggfuzz `arbitrary` dependency to meet MSRV

2 years agoDrop honggfuzz `arbitrary` dependency to meet MSRV 2022-08-fuzz-msrv
Matt Corallo [Tue, 30 Aug 2022 04:03:47 +0000 (04:03 +0000)]
Drop honggfuzz `arbitrary` dependency to meet MSRV

2 years agoChange `payment_cache` to accept `PaymentInfo`
jurvis [Tue, 30 Aug 2022 05:49:24 +0000 (22:49 -0700)]
Change `payment_cache` to accept `PaymentInfo`

Introduces a new `PaymentInfo` struct that contains both the previous
`attempts` count that was tracked as well as the paths that are also
currently inflight.

2 years agoMerge pull request #1604 from valentinewallace/2022-07-OMs-followup
Jeffrey Czyz [Mon, 29 Aug 2022 17:17:36 +0000 (12:17 -0500)]
Merge pull request #1604 from valentinewallace/2022-07-OMs-followup

Onion messages: add some initial rate limiting

2 years agoImplement buffering onion messages for peers.
Valentine Wallace [Thu, 25 Aug 2022 18:30:29 +0000 (14:30 -0400)]
Implement buffering onion messages for peers.

In this commit, we check if a peer's outbound buffer has room for onion
messages, and if so pulls them from an implementer of a new trait,
OnionMessageProvider.

Makes sure channel messages are prioritized over OMs, and OMs are prioritized
over gossip.

The onion_message module remains private until further rate limiting is added.

2 years agoImplement OnionMessageProvider for OnionMessenger
Valentine Wallace [Mon, 22 Aug 2022 16:22:22 +0000 (12:22 -0400)]
Implement OnionMessageProvider for OnionMessenger

2 years agoPeerManager: bump the read pause limit
Valentine Wallace [Tue, 16 Aug 2022 18:33:10 +0000 (14:33 -0400)]
PeerManager: bump the read pause limit

...to make sure we can still get channel messages out after enqueuing some big gossip messages.

2 years agoAdd boilerplate for sending and receiving onion messages in PeerManager
Valentine Wallace [Sat, 6 Aug 2022 04:33:48 +0000 (00:33 -0400)]
Add boilerplate for sending and receiving onion messages in PeerManager

Adds the boilerplate needed for PeerManager and OnionMessenger to work
together, with some corresponding docs and misc updates mostly due to the
PeerManager public API changing.

2 years agoMerge pull request #1683 from valentinewallace/2022-08-multiqueue-peerman
Matt Corallo [Thu, 25 Aug 2022 21:30:15 +0000 (21:30 +0000)]
Merge pull request #1683 from valentinewallace/2022-08-multiqueue-peerman