Add HMAC, and nonce to OffersContext::InboundPayment
Introduce HMAC and nonce calculation when sending Invoice with
reply path, so that if we receive InvoiceError back for the
corresponding Invoice we can verify the payment hash before logging it.
- The trait defines the public method one may define for creating and
verifying the HMAC.
- Using a pub trait to define these method allows the flexibility for
other `OffersMessageHandler` construct to construct the HMAC and
authenticate the message.
HMAC Construction and Verification for PaymentHash
When a InvoiceError is received for a sent BOLT12Invoice, the
corresponding PaymentHash is to be logged. Introduce hmac construction
and verification function for PaymentHash for this purpose.
Vincenzo Palazzo [Tue, 27 Aug 2024 08:18:28 +0000 (10:18 +0200)]
event: store the outpoint when is_manual_broadcast
With [1], it's possible to specify `manual_broadcast` for
the channel funding transaction. When `is_manual_broadcast` is
set to true, the transaction in the `DiscardFunding` event is
replaced with a dummy empty transaction.
This commit checks if `is_manual_broadcast` is true and
stores the funding OutPoint in the DiscardFunding event instead.
Matt Corallo [Mon, 26 Aug 2024 19:25:46 +0000 (19:25 +0000)]
Return owned `String`s for onion message message types
Returning a reference from a trait method is relatively difficult
to map in bindings and is currently handled by storing the object
in the trait instance, returning a reference to the local field.
This is fine when the object we're returning only needs to live as
long as the trait, but when it needs to be `'static` (as is the
case for onion message `msg_type`s), there's not really a good way
to map them at all.
Instead, here, condition on `#[cfg(c_bindings)]` we return a fully
owned `String`. This is obviously relatively less effecient, but
the extra allocation and `memcpy` isn't the end of the world,
especially given it should be released relatively quickly.
Note that this breaks doctests in with `c_bindings`.
Matt Corallo [Thu, 22 Aug 2024 01:31:07 +0000 (01:31 +0000)]
Add a `MessageSendInstructions::ForReply`
In order to allow onion message handlers to reply asynchronously
without introducing a circular dependency graph, the message
handlers need to be able to send replies described by
`MessageSendInstructions`. This allows them to send replies via the
normal message queuing (i.e. without making a function call to
`OnionMessenger`).
Here we enable that by adding a `MessageSendInstructions::ForReply`
variant which holds `ReplyInstruction`s.
Matt Corallo [Wed, 21 Aug 2024 19:10:46 +0000 (19:10 +0000)]
3/3 Use `MessageSendInstructions` instead of `PendingOnionMessage`
Now that the `MessageRouter` can `create_blinded_paths` forcing
callers of the `OnionMessenger` to provide it with a reply path up
front is unnecessary complexity, doubly so in message handlers.
Here we take the next step towards untangling that, moving from
`PendingOnionMessage` to `MessageSendInstructions` for the outbound
message queue in `AsyncPaymentsMessageHandler`. Better, we can also
drop the `c_bindings`-specific message queue variant, unifying the
APIs.
Matt Corallo [Thu, 22 Aug 2024 01:42:55 +0000 (01:42 +0000)]
2/3 Use `MessageSendInstructions` instead of `PendingOnionMessage`
Now that the `MessageRouter` can `create_blinded_paths` forcing
callers of the `OnionMessenger` to provide it with a reply path up
front is unnecessary complexity, doubly so in message handlers.
Here we take the next step towards untangling that, moving from
`PendingOnionMessage` to `MessageSendInstructions` for the outbound
message queue in `OffersMessageHandler`. Better, we can also drop
the `c_bindings`-specific message queue variant, unifying the APIs.
Because `ChannelManager` needs to actually control the reply path
set in individual messages, however, we have to halfway this patch,
adding a new `MessageSendInstructions` variant that allows
specifying the `reply_path` explicitly. Still, because other message
handlers are moving this way, its nice to be consistent.
Matt Corallo [Thu, 22 Aug 2024 01:41:27 +0000 (01:41 +0000)]
1/3 Use `MessageSendInstructions` instead of `PendingOnionMessage`
Now that the `MessageRouter` can `create_blinded_paths` forcing
callers of the `OnionMessenger` to provide it with a reply path up
front is unnecessary complexity, doubly so in message handlers.
Here we take the first step towards untangling that, moving from
`PendingOnionMessage` to `MessageSendInstructions` for the outbound
message queue in `CustomMessageHandler`. Better, we can also drop
the `c_bindings`-specific message queue variant, unifying the APIs.
Matt Corallo [Thu, 22 Aug 2024 18:43:33 +0000 (18:43 +0000)]
Give `MessageSendInstructions` any `Destination`, not only Blinded
In the next commit we'll use `MessageSendInstructions` for all
messages, so it needs the ability to take any `Destination`, not
only a `Destination::Blinded`.
Matt Corallo [Thu, 22 Aug 2024 01:35:18 +0000 (01:35 +0000)]
Pull the guts of `ResponseInstruction` into a new enum
In the coming commits we'll use the `ResponseInstruction` enum's
contents for all messages, allowing message senders to rely on the
in-`OnionMessegner` reply path selection logic.
In order to do so and avoid users confusing the new
`MessageSendInstructions` for `ResponseInstruction`, we leave
`ResponseInstruction` as a now-unconstructible struct which wraps a
`MessageSendInstructions`.
Matt Corallo [Wed, 21 Aug 2024 15:32:14 +0000 (15:32 +0000)]
Remove message type bound on `ResponseInstruction`
Our onion message handlers generally have one or more methods which
return a `ResponseInstruction` parameterized with the expected
message type (enum) of the message handler.
Sadly, that restriction is impossible to represent in our bindings -
our bindings concretize all LDK structs, enums, and traits into a
single concrete instance with generics set to our concrete trait
instances (which hold a jump table). This prevents us from having
multiple instances of `ResponseInstruction` structs for different
message types.
Our bindings do, however, support different parameterizations of
standard enums, including `Option`s and tuples.
In order to support bindings for the onion message handlers, we
are thus forced into std types bound by expected message types,
which we do here by making `ResponseInstruction` contain only the
instructions and generally using it as a two-tuple of
`(message, ResponseInstruction)`.
Matt Corallo [Wed, 21 Aug 2024 16:46:24 +0000 (16:46 +0000)]
Disambiguate blinded path `ForwardNode`s between payment + message
We currently have two structs with identical names in our public
API - `blinded_path::message::ForwardNode` and
`blinded_path::payment::ForwardNode`. This makes the API somewhat
awkward to use - users have to try (and fail) to import
`ForwardNode` twice only to then have to change their imports.
More importantly, however, this makes the API very hard to use in
some bindings languages where rename-imports or module imports
aren't available.
Matt Corallo [Sat, 17 Aug 2024 21:23:16 +0000 (21:23 +0000)]
Remove uneccessary `std` bounds on many tests
We never actually build with `#![no_std]` in tests as Rust does
not support it. Thus, many tests have spurious `std` feature gates
when they run just fine without them. Here we remove those gates,
though note that tests that depend on behavior elsewhere in the
codebase which is `std`-gated obviously need to retain their
feature gates.
Matt Corallo [Sat, 17 Aug 2024 15:59:34 +0000 (15:59 +0000)]
Drop `std::Error` impl for `PeerHandleError`
Not sure why we ever really had this, no one really ever bounds
anything on `std::Error` and its kinda a dead type, so there's no
need for us to `impl` it for our types.
Matt Corallo [Fri, 16 Aug 2024 22:36:30 +0000 (22:36 +0000)]
Simplify `Instant` mocking in outbound payments
To handle `std` and `no-std` concepts of time in scoring, we'd
originally written a generic `Time` trait which we could use to
fetch the current time, observe real (not wall-clock) elapsed time,
and serialize the time.
Eventually, scoring stopped using this `Time` trait but outbound
payment retry expiry started using it instead to mock time in
tests.
Since scoring no longer uses the full features which required the
`Time` trait, we can substantially simplify by just having the
mocking option.
Matt Corallo [Fri, 16 Aug 2024 20:46:37 +0000 (20:46 +0000)]
Remove note about `std`/`no-std` scorer serialization compat
In 81389dee306d960a8030bec5ffa304004148ce85 we removed a note about
mixing the `std` and `no-std` feature when de/serializing
`ProbabilisticScorer`s but forgot to note that there was a second
copy of that note in the module documentation.
Jeffrey Czyz [Fri, 16 Aug 2024 02:37:03 +0000 (21:37 -0500)]
Use a single iterator to construct a BlindedPath
Instead of using separate iterators for pubkeys and TLVs, use an
iterator that yields a tuple of them. This allows for holding a mutable
reference to the TLVs such that they can be padded. With two iterators,
the pubkey iterator would have a reference to the ForwardNode slice when
constructing a payment path. However, this would prevent holding mutable
references in the TLVs iterator.
Jeffrey Czyz [Fri, 16 Aug 2024 01:05:33 +0000 (20:05 -0500)]
Generalize build_keys_helper
When constructing a blinded path, two iterators are used: one for the
pubkeys and another for Writeable TLVs. The first iterator is used in
the build_keys_helper utility function while the second is used inside
of a callback. Update this utility to work on any type that can be
borrowed as a PublicKey. This allows for using a single iterator of
tuples, which is necessary for padding the hops without additional
allocations and clones.
Jeffrey Czyz [Thu, 15 Aug 2024 23:06:46 +0000 (18:06 -0500)]
Add a construct_keys_callback for blinded paths
When constructing a BlindedPath, an Option<Destination> parameter to
construct_keys_callback is never passed. Make a separate utility
function that doesn't take this parameter. This allows for using the
build_keys_helper macro with a Iterator yielding items other than
PublicKey.
Jeffrey Czyz [Thu, 15 Aug 2024 22:56:17 +0000 (17:56 -0500)]
Use PublicKey values in construct_keys_callback
Instead of accepting iterators yielding PublicKey by reference in
utils::construct_keys_callback, take iterators yielding values since
these implement Copy and need to be copied anyway. This will help avoid
a situation when padding where both a reference and mutable reference
are needed.
Jeffrey Czyz [Thu, 15 Aug 2024 22:36:55 +0000 (17:36 -0500)]
Refactor helper macro from construct_keys_callback
When constructing a BlindedPath, utils::construct_blinded_hops uses two
iterators. However, this makes it difficult to pad blinded hops to equal
sizes without allocating a vector or cloning data. Refactor the
construct_keys_callback utility function so that is can be used with an
Iterator with different Item types. This allows using a single Iterator
of tuples while still supporting its use only with pubkeys.
G8XSU [Tue, 20 Aug 2024 17:23:18 +0000 (10:23 -0700)]
Doc Clarity: Handling gaps in persisted ChannelMonitorUpdates
If there are any gaps in the persisted [`ChannelMonitorUpdate`]s,
implementer can safely ignore [`ChannelMonitorUpdate`]s after the gap and load without them.
Since, acc. to channel-manager, we have only made progress if all contiguos
ChannelMonitorUpdates have been persisted.
Duncan Dean [Fri, 16 Aug 2024 09:34:55 +0000 (11:34 +0200)]
Deprecate AvailableBalances::balance_msat
The ChannelMonitor::get_claimable_balances method provides a more
straightforward approach to the balance of a channel, which satisfies
most use cases. The computation of AvailableBalances::balance_msat is
complex and originally had a different purpose that is not applicable
anymore. We deprecate AvailableBalances::balance_msat now and will remove
it in a following release.
Co-authored-by: Willem Van Lint <noreply@wvanlint.dev>
This test was added some time ago in 0c034e9a82e4339fb32af9da63832ac2a64abb0b, but never made any sense.
`PeerManager::process_events` will go around its loop as many times
is required to ensure we've always processed all events which were
pending prior to a `process_events` call, so having a test that
checks that we never go around more than twice is obviously broken.
And, indeed, in CI this tests fails with some regularity.
Instead, the test here is changed to ensure that we detectably go
around the loop again at least once.
Support next_blinding_override in blinded payment paths.
This allow us to forward blinded payments where the blinded path that we are
forwarding within was concatenated to another blinded path that starts at the
next hop.
Also allows constructing blinded paths using this override.
Matt Corallo [Sat, 17 Aug 2024 21:13:08 +0000 (21:13 +0000)]
Remove the `std` feature implications from the `lightning` crate
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.
This takes one last step, removing the implications of the `std`
feature from the `lightning` crate.
Matt Corallo [Fri, 16 Aug 2024 20:32:58 +0000 (20:32 +0000)]
Drop the `no-std` feature from `lightning-rapid-gossip-sync`
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.
This takes another step by removing the `no-std` feature entirely
from the `lightning-rapid-gossip-sync` crate and removing all
feature implications on dependencies from the remaining `std`
feature.
Matt Corallo [Fri, 16 Aug 2024 19:17:42 +0000 (19:17 +0000)]
Drop the `no-std` feature from `lightning-invoice`
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.
This takes another step by removing the `no-std` feature entirely
from the `lightning-invoice` crate and removing all feature
implications on dependencies from the remaining `std` feature.
Matt Corallo [Fri, 16 Aug 2024 19:07:05 +0000 (19:07 +0000)]
Drop the `no-std` feature from BP and drop feature implications
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.
This takes the first step by removing the `no-std` feature entirely
from the `lightning-background-processor` crate and removing most
feature implications on dependencies from the remaining `std`
feature.
It also addresses a CI oversight where we were not testing
`lightning-background-processor` without the `std` feature in CI at
all.
Matt Corallo [Fri, 16 Aug 2024 20:13:18 +0000 (20:13 +0000)]
Ensure we always pass a `path` for in-workspace dependencies
In order to ensure our crates depend on the workspace copies of
each other in test builds we need to override the crates.io
dependency with a local `path`.
We can do this in one of two ways - either specify the `path` in
the dependency listing in each crate's `Cargo.toml` or use the
workspace `Cargo.toml` to `patch` all dependencies. The first is
tedious while the second lets us have it all in one place. However,
the second option does break `cargo *` in individual crate
directories (forcing the use of `cargo -p crate *` instead) and
makes it rather difficult to depend on local versions of workspace
crates.
Thus, here we drop the `patch.crates-io` from our top-level
`Cargo.toml` entirely.
Still, we do update the `ci/ci-tests.sh` script here to use
`cargo -p crate` instead of switching to each crate's directory as
it allows `cargo` to use a shared `target` and may speed up tests.
Matt Corallo [Fri, 16 Aug 2024 18:47:38 +0000 (18:47 +0000)]
Optimize `BufReader` somewhat for the only used case
`rust-bitcoin` doesn't ever actually *use* its `BufRead`
requirement when deserializing objects, and forcing it is somewhat
inefficient, so we optimize the only (actual) case here by passing
reads straight through to the backing stream.
Matt Corallo [Mon, 19 Aug 2024 14:37:03 +0000 (14:37 +0000)]
Re-export `lightning_types` in top-level `lightning` modules
Since we now have many types in one place, it makes sense to export
them in that place. Further, doing so finally somewhat starts to
reduce our `lightning::ln` module size, which historically is the
dumping ground for everything when most things really should be
top-level modules in `lightning`.
Here we take a step in the right direction by exporting
`lightning_types` as `lightning::types` and encouraging users to
use those paths directly rather than the ones in `lightning::ln`.
Matt Corallo [Mon, 11 Dec 2023 02:10:51 +0000 (02:10 +0000)]
Tweak alignment and memory order in the `ProbabilisticScorer`
During routing, the majority of our time is spent in the scorer.
Given the scorer isn't actually doing all that much computation,
this means we're quite sensitive to memory latency. Thus, the cache
lines our data sits on is incredibly important.
Here, we manually lay out the `ChannelLiquidity` and
`HistoricalLiquidityTracker` structs to ensure that we can do the
non-historical scoring and skip historical scoring for channels
with insufficient data by just looking at the same cache line the
channel's SCID is on.
Sadly, to do the full historical scoring we need to load a second
128-byte cache line pair, but we have some time to get there. We
might consider issuing a preload instruction in the future.
Matt Corallo [Sat, 9 Dec 2023 04:18:46 +0000 (04:18 +0000)]
Cache the total points tracked in our historical liquidity data
When we go to score a channel using the historical liquidity data,
the first thing we do is step through all the valid bucket
combinations, multiply the min and max bucket, and then add them
together to calculate the total number of points tracked. This
isn't a free operation, and for scorers without much data it
represents a large part of the total time spent scoring during
routefinding.
Thus, here we cache this value, updating it every time the buckets
are updated.
Matt Corallo [Sat, 9 Dec 2023 04:30:08 +0000 (04:30 +0000)]
Change the directed history tracker's storage of its direction
Rather than storing the two direction's buckets in
`HistoricalMinMaxBuckets` (renamed
`DirectedHistoricalLiquidityTracker`), we store a single reference
to the `HistoricalLiquidityTracker` as well as the direction bool.
This will allow us in the next commit to reference fields in the
`HistoricalLiquidityTracker` aside from the two directions.
Matt Corallo [Sat, 9 Dec 2023 04:29:12 +0000 (04:29 +0000)]
Make the historical bucket data private to `bucketed_history`
In a comming commit we'll cache some additional data in the
historical bucket tracker. In order to do so, here we isolate the
buckets themselves into the `bucketed_history` module, reducing
the possibility of accidentally updating them directly without
updating caches.
Matt Corallo [Sat, 9 Dec 2023 03:32:40 +0000 (03:32 +0000)]
Store both min and max historical buckets in one, new, struct
In the coming commits we'll isolate historical bucket logic slightly
further, allowing us to cache some state. This is the first step
towards that, storing the historical liquidity information in a new
`HistoricalLiquidityTracker` rather than in the general
`ChannelLiquidity`.
Jeffrey Czyz [Wed, 14 Aug 2024 22:39:13 +0000 (17:39 -0500)]
Add PaymentId authentication to public API
When receiving an InvoiceError message, it should be authenticated
before using it to abandon the payment. Add methods to PaymentId's
public API for constructing and verifying an HMAC for use in
OffersContext::OutboundPayment. This allows other implementations of
OffersMessageHandler to construct the HMAC and authenticate the message.
Arik Sosman [Thu, 15 Aug 2024 06:57:14 +0000 (23:57 -0700)]
Fix fuzz warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Arik Sosman [Thu, 15 Aug 2024 02:52:01 +0000 (19:52 -0700)]
Fix lightning-transaction sync warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Arik Sosman [Wed, 14 Aug 2024 20:31:40 +0000 (13:31 -0700)]
Fix lightning-block-sync warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.