Matt Corallo [Thu, 6 Oct 2022 15:44:03 +0000 (15:44 +0000)]
Test a full `no-std` build in the `no-std-check` crate in CI
Rust is incredibly forgiving in attempts to access `std`, making it
rather difficult to test `no-std` properly. In practice, the only
decent way to do so is to actually build for a platform that does
not have `std`, which we do here by building the `no-std-check`
crate for `arm-thumbv7m-none-eabi`.
Matt Corallo [Thu, 6 Oct 2022 15:42:41 +0000 (15:42 +0000)]
Correct `rapid-gossip-sync` `no-std` build and test
While `rapid-gossip-sync` recently gained a `no-std` feature, it
didn't actually work, as there were still dangling references to
`std` and prelude assumptions. This makes `rapid-gossip-sync`
build (and test) properly in `no-std`.
Matt Corallo [Tue, 16 Aug 2022 21:58:06 +0000 (21:58 +0000)]
Add a TODO for an important issue for making async mon updates safe
If we receive a monitor event from a forwarded-to channel which
contains a preimage for an HTLC, we have to propogate that preimage
back to the forwarded-from channel monitor. However, once we have
that update, we're running in a relatively unsafe state - we have
the preimage in memory, but if we were to crash the forwarded-to
channel monitor will not regenerate the update with the preimage
for us. If we haven't managed to write the monitor update to the
forwarded-from channel by that point, we've lost the preimage, and,
thus, money!
Matt Corallo [Mon, 18 Jul 2022 01:32:27 +0000 (01:32 +0000)]
Rework `chain::Watch` return types to make async updates less scary
When a `chain::Watch` `ChannelMonitor` update method is called, the
user has three options:
(a) persist the monitor update immediately and return success,
(b) fail to persist the monitor update immediately and return
failure,
(c) return a flag indicating the monitor update is in progress and
will complete in the future.
(c) is rather harmless, and in some deployments should be expected
to be the return value for all monitor update calls, but currently
requires returning `Err(ChannelMonitorUpdateErr::TemporaryFailure)`
which isn't very descriptive and sounds scarier than it is.
Instead, here, we change the return type used to be a single enum
(rather than a Result) and rename `TemporaryFailure`
`UpdateInProgress`.
Matt Corallo [Fri, 23 Sep 2022 21:08:26 +0000 (21:08 +0000)]
Add a bindings-only version of `Future::register_callback`
While we could, in theory, add support to the bindings logic to map
`Box<dyn Trait>`, there isn't a whole lot of use doing so when its
incredibly trivial to do directly.
This adds a trivial wrapper around `Future::register_callback` that
is only built in bindings and which is linked in the
`register_callback` docs for visibility.
Matt Corallo [Thu, 22 Sep 2022 14:07:25 +0000 (14:07 +0000)]
Downgrade `hashbrown` to meet MSRV
`hashbrown` depends on `ahash` which depends on `once_cell`. Sadly,
in https://github.com/matklad/once_cell/issues/201 the `once_cell`
maintainer decided they didn't want to do the work of having an
MSRV policy for `once_cell`, making `ahash`, and thus `hashbrown`
require the latest compiler. I've reached out to `ahash` to suggest
they drop the dependency (as they could trivially work around not
having it), but until then we simply downgrade `hashbrown`.
`rust-bitcoin` also requires an older `hashbrown` so we're actually
reducing our total `no-std` code here anyway.
Matt Corallo [Sun, 18 Sep 2022 13:55:08 +0000 (13:55 +0000)]
Avoid returning a reference to a u64.
In c353c3ed7c40e689a3b9fb6730c6dabbd3c92cc5 an accessor method was
added which returns an `Option<&u64>`. While this allows Rust to
return an 8-byte object, returning a reference to something
pointer-sized is a somewhat strange API.
Instead, we opt for a straight `Option<u64>`, which is sadly
somewhat larger on the stack, but is simpler and already supported
in the bindings generation.
Matt Corallo [Sun, 18 Sep 2022 13:34:39 +0000 (13:34 +0000)]
Swap `Vec<&RouteHop>` parameters for slices
In c353c3ed7c40e689a3b9fb6730c6dabbd3c92cc5, new scorer-updating
methods were added to the `Router` object, however they were
passed as a `Vec` of references. We use the list-of-references
pattern to make bindings simpler (by not requiring allocations per
entry), however there's no reason prefer to passing a `Vec` over
a slice, given the `Vec` doesn't hold ownership of the objects
anyway.
Matt Corallo [Mon, 19 Sep 2022 09:11:11 +0000 (09:11 +0000)]
Add a `MutexGuard` wrapper for the bindings-only `LockableScore`
In the bindings, we don't directly export any `std` types. Instead,
we provide trivial wrappers around them which expose only a
bindings-compatible API (and which is code in-crate, where the
bindings generator can see it).
We never quite finished this for `MultiThreadedLockableScore` - due
to some limitations in the bindings generator and the way the
scores are used in `lightning-invoice`, the scoring API ended up
being further concretized in patches for the bindings. Luckily the
scoring interface has been rewritten somewhat, and it so happens
that we can now generate bindings with the upstream code.
The final piece of the puzzle is done here, where we add a struct
which wraps `MutexGuard` so that we can expose the lock for
`MultiThreadedLockableScore`.
Remove `forward_htlc` after `channel_state` lock order
The `forward_htlc` was prior to this commit only held at the same time
as the `channel_state` lock during the write process of the
`ChannelManager`. This commit removes the lock order dependency, by
taking the `channel_state`lock temporarily during the write process.
As we are eventually removing the `channel_state` lock, this commit
moves the `forward_htlcs` map out of the `channel_state` lock, to ease
that process.
Jurvis Tan [Thu, 15 Sep 2022 21:58:08 +0000 (14:58 -0700)]
Move Scorer requirement away from Router trait
We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
Matt Corallo [Fri, 16 Sep 2022 14:40:32 +0000 (14:40 +0000)]
Stop building with lockorder debugging in benchmarks
`cargo bench` sets `#[cfg(test)]` so our current checks for
enabling our lockorder debugging end up matching when we're trying
to build performance benchmarks.
This adds explicit checks to our debug_lockorder logic to filter
out `feature = "_bench_unstable"` builds.
Matt Corallo [Tue, 5 Oct 2021 04:32:49 +0000 (04:32 +0000)]
Do not broadcast commitment txn on Permanent mon update failure
See doc updates for more info on the edge case this prevents, and
there isn't really a strong reason why we would need to broadcast
the latest state immediately. Specifically, in the case of HTLC
claims (the most important reason to ensure we have state on chain
if it cannot be persisted), we will still force-close if there are
HTLCs which need claiming and are going to expire.
Surprisingly, there were no tests which failed as a result of this
change, but a new one has been added.
Matt Corallo [Fri, 9 Sep 2022 04:38:14 +0000 (04:38 +0000)]
Assert that all defined features are in the known features set
Now that the features contexts track the full set of all known
features, rather than the set of supported features, all defined
features should be listed in the context definition macro.
This adds a compile-time assertion to check that all bits for known
features are set in the context known set.
Matt Corallo [Fri, 9 Sep 2022 04:31:18 +0000 (04:31 +0000)]
Stop tracking feature bits as known or required in features.rs
Now that the `*Features::known` constructor has been removed, there
is no reason to define feature bits as either optional required in
`features.rs` - that logic now belongs in the modules that are
responsible for the given features.
Instead, we only list all features in each context.
Matt Corallo [Wed, 14 Sep 2022 01:05:25 +0000 (01:05 +0000)]
Remove the `*Features::known` constructor
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we (finally) remove the `known` constructor entirely,
modifying tests in the `features` module as required.
Matt Corallo [Wed, 14 Sep 2022 01:05:11 +0000 (01:05 +0000)]
Remove all remaining references to `*Features::known`
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
In anticipation of removing the `known` constructor, this commit
removes all remaining references to it outside of features.rs.
Matt Corallo [Fri, 9 Sep 2022 04:15:40 +0000 (04:15 +0000)]
Stop relying on `*Features::known` in fuzzing tests
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in our fuzz tests.
Matt Corallo [Fri, 9 Sep 2022 04:00:33 +0000 (04:00 +0000)]
Stop relying on `*Features::known` in BP and persister tests
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the
`lightning-background-processor` and `lightning-persister` crate
tests.
Matt Corallo [Fri, 9 Sep 2022 02:34:27 +0000 (02:34 +0000)]
Stop relying on the `*Features::known` method in net-tokio
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the
`lightning-net-tokio` crate.
Matt Corallo [Mon, 12 Sep 2022 19:20:38 +0000 (19:20 +0000)]
Stop relying on the `*Features::known` method in lightning-invoice
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the
`lightning-invoice` crate.
Matt Corallo [Thu, 8 Sep 2022 21:24:34 +0000 (21:24 +0000)]
Stop relying on `*Features::known` in channel{,manager}.rs
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the channel modules.
Matt Corallo [Thu, 8 Sep 2022 21:18:07 +0000 (21:18 +0000)]
Stop relying on `*Features::known` in functional test utils
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the
functional_test_utils module.
Matt Corallo [Thu, 8 Sep 2022 21:04:11 +0000 (21:04 +0000)]
Stop relying on the `*Features::known` method in `routing` tests
As we move towards specify supported/required feature bits in the
module(s) where they are supported, the global `known` feature set
constructors no longer make sense.
Here we stop relying on the `known` method in the `routing` module,
which was only used in tests.
Matt Corallo [Mon, 12 Sep 2022 17:47:16 +0000 (17:47 +0000)]
List supported/required feature bits explicitly in ChannelManager
Historically, LDK has considered the "set of known/supported
feature bits" to be an LDK-level thing. Increasingly this doesn't
make sense - different message handlers may provide or require
different feature sets.
In a previous PR, we began the process of transitioning with
feature bits sent to peers being sourced from the attached message
handler.
This commit makes further progress by moving the concept of which
feature bits are supported by our ChannelManager into
channelmanager.rs itself, via the new `provided_*_features`
methods, rather than in features.rs via the `known_channel_features`
and `known` methods.
Wilmer Paulino [Fri, 26 Aug 2022 19:56:09 +0000 (12:56 -0700)]
Update anchors test vectors to zero HTLC transaction fee variant
Each test featuring HTLCs had a minimum and maximum feerate case. This
is no longer necessary for the zero HTLC transaction anchors variant as
the commitment feerate does not impact whether HTLCs can be trimmed or
not, only the dust limit does.
Wilmer Paulino [Mon, 29 Aug 2022 19:34:34 +0000 (12:34 -0700)]
Account for zero fee HTLC transaction within dust limit calculation
With the zero fee HTLC transaction anchors variant, HTLCs can no longer
be trimmed due to their amount being too low to have a mempool valid
HTLC transaction. Now they can only be trimmed based on the dust limit
of each party within the channel.
Wilmer Paulino [Thu, 25 Aug 2022 20:36:17 +0000 (13:36 -0700)]
Exclude HTLC transactions from broadcast on anchor channels
HTLC transactions from anchor channels are constrained by a CSV of 1
block, so broadcasting them along with the unconfirmed commitment
tranasction will result in them being immediately rejected as premature.
Wilmer Paulino [Thu, 25 Aug 2022 20:11:19 +0000 (13:11 -0700)]
Avoid commitment broadcast upon detected funding spend
There's no need to broadcast our local commitment transaction if we've
already seen a confirmed one as it'll be immediately rejected as a
duplicate/conflict.
This will also help prevent dispatching spurious events for bumping
commitment and HTLC transactions through anchor outputs (once
implemented in future work) and the dispatch for said events follows the
same flow as our usual commitment broadcast.
Expand the BlockSource trait to allow filtered blocks now that
chain::Listen supports them (d629a7edb7241eee7fde9f5ccdf1c481d2d6297b).
This makes it possible to use BIP 157/158 compact block filters with
lightning-block-sync.
Matt Corallo [Mon, 12 Sep 2022 19:06:17 +0000 (19:06 +0000)]
Move checking of specific require peer feature bits to handlers
As we remove the concept of a global "known/supported" feature set
in LDK, we should also remove the concept of a global "required"
feature set. This does so by moving the checks for specific
required features into handlers.
Specifically, it allows the handler `peer_connected` method to
return an `Err` if the peer should be disconnected. Only one such
required feature bit is currently set - `static_remote_key`, which
is required in `ChannelManager`.
Matt Corallo [Mon, 12 Sep 2022 19:34:59 +0000 (19:34 +0000)]
Swap some `peer_connected` features to `known` from `empty` in test
In the next commit we'll enforce counterparty `InitFeatures`
matching our required set in `ChannelManager`, implying they must
be set for many tests where they previously did not need to be (as
they were enforced in `PeerManager`, which is not used in
functional tests).
Matt Corallo [Wed, 7 Sep 2022 21:08:22 +0000 (21:08 +0000)]
Correctly handle BADONION onion errors
Currently we entirely ignore the BADONION bit when deciding how to
handle HTLC failures. This opens us up to an attack where a
malicious node always fails HTLCs backwards via
`update_fail_malformed_htlc` with an error code of
`BADONION|NODE|PERM|X`. In this case, we may decide to interpret
this as a permanent node failure for the node encrypting the onion,
i.e. the counterparty of the node who sent the
`update_fail_malformed_htlc` message and ultimately failed the
HTLC.
Thus, any node we route through could cause us to fully remove its
counterparty from our network graph. Luckily we do not do any
persistent tracking of removed nodes, and thus will re-add the
removed node once it is re-announced or on restart, however we are
likely to add such persistent tracking (at least in-memory) in the
future.
Matt Corallo [Mon, 12 Sep 2022 15:20:37 +0000 (15:20 +0000)]
Encrypt+MAC most P2P messages in-place
For non-gossip-broadcast messages, our current flow is to first
serialize the message into a `Vec`, and then allocate a new `Vec`
into which we write the encrypted+MAC'd message and header.
This is somewhat wasteful, and its rather simple to instead
allocate only one buffer and encrypt the message in-place.
Matt Corallo [Mon, 12 Sep 2022 15:16:41 +0000 (15:16 +0000)]
Fix encryption of broadcasted gossip messages
In 47e818f198abafba01b9ad278582886f9007dac2, forwarding broadcasted
gossip messages was split into a separate per-peer message buffer.
However, both it and the original regular-message queue are
encrypted immediately when the messages are enqueued. Because the
lightning P2P encryption algorithm is order-dependent, this causes
messages to fail their MAC checks as the messages from the two
queues may not be sent to peers in the order in which they were
encrypted.
The fix is to simply queue broadcast gossip messages unencrypted,
encrypting them when we add them to the regular outbound buffer.
Matt Corallo [Sun, 11 Sep 2022 21:18:01 +0000 (21:18 +0000)]
Drop unused type parameter on `BlindedRoute::new`
I'm not sure why rustc didn't complain about the unused parameter
or why we're allowed to get away without explicitly bounding the
`Sign` in the `KeysInterface`, but the current code requires all
`BlindedPath` construction to explicitly turbofish an unused type.
Matt Corallo [Sat, 10 Sep 2022 20:31:52 +0000 (20:31 +0000)]
Inline generic bounds rather than using the `where` clause
The bindings generator is pretty naive in its generic resolution
and doesn't like `where` clauses for bounds that are simple traits.
This should eventually change, but for now its simplest to just
inline the relevant generic bounds.
Matt Corallo [Sat, 10 Sep 2022 20:31:42 +0000 (20:31 +0000)]
Do not use blanket impls when building for `c_bindings`
The C bindings generator isn't capable of figuring out if a blanket
impl applies in a given context, and instead opts to always write
out any relevant impl's for a trait. Thus, blanket impls should be
disabled when building with `#[cfg(c_bindings)]`.