Duncan Dean [Wed, 26 Jul 2023 09:20:08 +0000 (11:20 +0200)]
Introduce `ChannelPhase` enum
We introduce the `ChannelPhase` enum which will contain the different
channel structs wrapped by each of its variants so that we can place
these within a single `channel_by_id` map in `peer_state` in the
following commits. This will reduce the number of map lookup operations
we need to do in `ChannelManager`'s various methods. It will also make
certain channel counting logic easier to reason about with less risk of
forgetting to modify logic when new channels structs are introduced for
V2 channel establishment.
Replace a constant three retry attempts for BOLT 12 invoice payments
with a retry strategy specified when creating a pending outbound
payment. This is configured by users in a later commit when constructing
an InvoiceRequest or a Refund.
An upcoming commit requires serializing Retry, so use a type with a
fixed byte length. Otherwise, using eight bytes to serialize a usize
would fail to read on 32-bit machines.
Add a send_payment_for_bolt12_invoice method to OutboundPayments for
initiating payment of a BOLT 12 invoice. This will be called from an
OffersMessageHandler, after which any retries are handled using the
Retryable logic.
Consolidate the creation and insertion of onion_session_privs to the
PendingOutboundPayment::Retryable arm. In an upcoming commit, this
method will be reused for an initial BOLT 12 invoice payment. However,
onion_session_privs are created using a different helper.
Jeffrey Czyz [Wed, 30 Aug 2023 17:01:15 +0000 (12:01 -0500)]
Add PendingOutboundPayment::InvoiceReceived
When a BOLT 12 invoice has been received, a payment attempt is made and
any errors result in abandoning the PendingOutboundPayment. This results
in generating at PaymentFailed event, which has a PaymentHash. Thus,
when receiving an invoice, transition from AwaitingInvoice to a new
InvoiceReceived state, the latter of which contains a PaymentHash such
the abandon_payment helper can still be used.
When a BOLT 12 invoice has been requested, in order to guarantee
invoice payment idempotency the future payment needs to be tracked. Add
an AwaitingInvoice variant to PendingOutboundPayment such that only
requested invoices are paid only once. Timeout after a few timer ticks
if a request has not been received.
When an invoice is requested but either receives an error or never
receives a response, surface an event to indicate to the user that the
corresponding future payment has failed.
When receiving a BOLT 12 invoice originating from either an invoice
request or a refund, the invoice should only be paid once. To accomplish
this, require that the invoice includes an encrypted payment id in the
payer metadata. This allows ChannelManager to track a payment when
requesting but prior to receiving the invoice. Thus, it can determine if
the invoice has already been paid.
Jeffrey Czyz [Thu, 24 Aug 2023 20:16:53 +0000 (15:16 -0500)]
Add an encryption key to ExpandedKey for Offers
Metadata such as the PaymentId should be encrypted when included in an
InvoiceRequest or a Refund, as it is user data and is exposed to the
payment recipient. Add an encryption key to ExpandedKey for this purpose
instead of reusing offers_base_key.
InvoiceRequest::verify_and_respond_using_derived_keys takes a payment
hash. To avoid generating one for invoice requests that ultimately
cannot be verified, split the method into one for verifying and another
for responding.
Matt Corallo [Mon, 28 Aug 2023 18:39:04 +0000 (18:39 +0000)]
Drop dep `tokio`'s `io-util` feat as it broke MSRV and isn't useful
We use `tokio`'s `io-util` feature to provide the
`Async{Read,Write}Ext` traits, which allow us to simply launch a
read future or `poll_write` directly as well as `split` the
`TcpStream` into a read/write half. However, these traits aren't
actually doing much for us - they are really just wrapping the
`readable` future (which we can trivially use ourselves) and
`poll_write` isn't doing anything for us that `poll_write_ready`
can't.
Similarly, the split logic is actually just `Arc`ing the
`TcpStream` and busy-waiting when an operation is busy to prevent
concurrent reads/writes. However, there's no reason to prevent
concurrent access at the stream level - we aren't ever concurrently
writing or reading (though we may concurrently read and write,
which is fine).
Worse, the `io-util` feature broke MSRV (though they're likely to
fix this upstream) and carries two additional dependencies (only
one on the latest upstream tokio).
Arik Sosman [Sat, 26 Aug 2023 00:34:10 +0000 (17:34 -0700)]
Fix flaky aggregated HTLC revocation test.
Releasing write locks in between monitor updates
requires storing a set of cloned keys to iterate
over. For efficiency purposes, that set of keys
is an actual set, as opposed to array, which means
that the iteration order may not be consistent.
The test was relying on an event array index to
access the revocation transaction. We change that
to accessing a hash map keyed by the txid, fixing
the test.
Arik Sosman [Fri, 25 Aug 2023 19:31:33 +0000 (12:31 -0700)]
Release write lock between monitor update iterations.
Previously, updating block data on a chain monitor
would acquire a write lock on all of its associated
channel monitors and not release it until the loop
completed.
Now, we instead acquire it on each iteration,
fixing #2470.
jbesraa [Tue, 22 Aug 2023 15:57:06 +0000 (18:57 +0300)]
Split LockableScore responsibilities between read & write operations
- Split Score from LockableScore to ScoreLookUp to handle read
operations and ScoreUpdate to handle write operations
- Change all struct that implemented Score to implement ScoreLookUp
and/or ScoreUpdate
- Change Mutex's to RwLocks to allow multiple data readers
- Change LockableScore to Deref in ScorerAccountingForInFlightHtlcs
as we only need to read
- Add ScoreLookUp and ScoreUpdate docs
- Remove reference(&'a) and Sized from Score in ScorerAccountingForInFlightHtlcs
as Score implements Deref
- Split MultiThreadedScoreLock into MultiThreadedScoreLockWrite and MultiThreadedScoreLockRead.
After splitting LockableScore, we split MultiThreadedScoreLock following
the same way, splitting a single score into two srtucts, one for read and
other for write.
MultiThreadedScoreLock is used in c_bindings.
Router: account for blinded path fee, etc on first_hop<>intro hop add
This previously led to a debug panic in the router because we wouldn't account
for the blinded path fee when calculating first_hop<>intro_node hop's available
liquidity and construct an invalid path that forwarded more over said hop than
was actually available.
This also led to us hitting unreachable code, see direct_to_matching_intro_nodes
test description.
Matt Corallo [Mon, 5 Jun 2023 17:22:36 +0000 (17:22 +0000)]
Fail UTXO lookups if the block doesn't have five confirmations
The BOLT spec mandates that channels not be announced until they
have at least six confirmations. This is important to enforce not
because we particularly care about any specific DoS concerns, but
because if we do not we may have to handle reorgs of channel
funding transactions which change their SCID or have conflicting
SCIDs.
Matt Corallo [Sun, 30 Apr 2023 00:48:57 +0000 (00:48 +0000)]
Make the `P2PGossipSync` `UtxoLookup` exchangable without &mut self
Because a `UtxoLookup` implementation is likely to need a reference
to the `PeerManager` which contains a reference to the
`P2PGossipSync`, it is likely to be impossible to get a mutable
reference to the `P2PGossipSync` by the time we want to add a
`UtxoLookup` without a ton of boilerplate and trait wrapping.
Instead, we simply place the `UtxoLookup` in a `RwLock`, allowing
us to modify it without a mutable self reference.
The lifetime bounds updates in tests required in this commit are
entirely unclear to me, but do allow tests to continue building, so
somehow make rustc happier.
Matt Corallo [Sat, 29 Apr 2023 22:32:57 +0000 (22:32 +0000)]
Implement the `UtxoSource` interface for REST/RPC clients
In LDK, we expect users operating nodes on the public network to
implement the `UtxoSource` interface in order to validate the
gossip they receive from the network.
Sadly, because the DoS attack of flooding a node's gossip store
isn't a common issue, and because we do not provide an
implementation off-the-shelf to make doing so easily, many of our
downstream users do not have a `UtxoSource` implementation.
In order to change that, here we implement an async `UtxoSource`
in the `lightning-block-sync` crate, providing one for users who
sync the chain from Bitcoin Core's RPC or REST interfaces.
Matt Corallo [Sat, 20 May 2023 23:28:18 +0000 (23:28 +0000)]
Expose the historical success probability calculation itself
In 3f32f60ae7e75a4be96d3d5adc8d18b53445e5e5 we exposed the
historical success probability buckets directly, with a long method
doc explaining how to use it. While this is great for logging
exactly what the internal model thinks, its also helpful to let
users know what the internal model thinks the success probability
is directly, allowing them to compare route success probabilities.
Here we do so but only for the historical tracking buckets.
Matt Corallo [Mon, 10 Apr 2023 22:54:48 +0000 (22:54 +0000)]
Correctly apply penalty bounds on the per-amount penalties
When we attempt to score a channel which has a success probability
very low, we may have a log well above our cut-off of two. For the
liquidity penalties this works great, we bound it by
`NEGATIVE_LOG10_UPPER_BOUND` and `min` the two scores. For the
amount liquidity penalty we didn't do any `min`ing at all.
This fix is to min the log itself first and then reuse the min'd
log in both calculations.
Matt Corallo [Mon, 10 Apr 2023 07:05:31 +0000 (07:05 +0000)]
Don't rely on `calculate_success_probability*` to handle amt > cap
Currently we let an `htlc_amount >= channel_capacity` pass through
from `penalty_msat` to
`calculate_success_probability_times_billion`, but only if its only
marginally bigger (less than 65/64ths). This is fine as
`calculate_success_probability_times_billion` handles bogus values
just fine (it will always return a zero probability in such cases).
However, this is risky, and in fact breaks in the coming commits,
so instead check it before ever calling through to the historical
bucket probability calculations.
Here we implement `WatchtowerPersister`, which provides a test-only
sample implementation of `Persist` similar to how we might imagine a
user to build watchtower-like functionality in the persistence pipeline.
We test that the `WatchtowerPersister` is able to successfully build and
sign a valid justice transaction that sweeps a counterparty's funds if
they broadcast an old commitment.
Expose revokeable output index and building a justice tx from commitment
For watchtowers to be able to build justice transactions for our
counterparty's revoked commitments, they need to be able to find the
revokeable output for them to sweep. Here we cache `to_self_delay` in
`CommitmentTransaction` to allow for finding this output on the struct
directly. We also add a simple helper method to aid in building the
initial spending transaction.
This also adds a unit test for both of these helpers, and
refactors a bit of a previous `CommitmentTransaction` unit test to make
adding these easier.
Enable monitor to rebuild initial counterparty commitment tx
Upon creating a channel monitor, it is provided with the initial
counterparty commitment transaction info directly before the very first
time it is persisted. Because of this, the very first counterparty
commitment is not seen as an update in the persistence pipeline, and so
our previous changes to the monitor and updates cannot be used to
reconstruct this commitment.
To be able to expose the counterparty's transaction for the very first
commitment, we add a thin wrapper around
`provide_latest_counterparty_commitment_tx`, that stores the necessary
data needed to reconstruct the initial commitment transaction in the
monitor.
Matt Corallo [Wed, 23 Aug 2023 03:30:56 +0000 (03:30 +0000)]
Remove redundant payment preimag hashing in HTLC claim pipeline
Currently, when we receive an HTLC claim from a peer, we first hash
the preimage they gave us before removing the HTLC, then
immediately pass the preimage to the inbound channel and hash the
preimage again before removing the HTLC and sending our peer an
`update_fulfill_htlc`. This second hash is actually only asserted
on, never used in any meaningful way as we have the htlc data
present in the same code.
Here we simply drop this second hash and move it into a
`debug_assert`.
Matt Corallo [Wed, 23 Aug 2023 02:57:35 +0000 (02:57 +0000)]
Include payment hash in more early payment logs
If a user has issues with a payment, the most obvious thing they'll
do is check logs for the payment hash. Thus, we should ensure our
logs that show a payment's lifecycle include the payment hash and
are emitted (a) as soon as LDK learns of the payment, (b) once the
payment goes out to the peer (which is already reasonably covered
in the commitment transaction building logs) and (c) when the
payment ultimately is fulfilled or fails.
Alec Chen [Wed, 14 Jun 2023 20:14:14 +0000 (15:14 -0500)]
Add feerate and balances to `LatestCounterpartyCommitmentTXInfo`
This adds the feerate and local and remote output values to this channel
monitor update step so that a monitor can reconstruct the counterparty's
commitment transaction from an update. These commitment transactions
will be exposed to users in the following commits to support third-party
watchtowers in the persistence pipeline.
With only the HTLC outputs currently available in the monitor update, we
can tell how much of the channel balance is in-flight and towards which
side, however it doesn't tell us the amount that resides on either side.
Because of dust, we can't reliably derive the remote value from the
local value and visa versa. Thus, it seems these are the minimum fields
that need to be added.
projects / rust-lightning / log