]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
2 years agoAdd missing deserialization of Event::HTLCHandlingFailed 2022-09-missing-event-deser
Matt Corallo [Tue, 6 Sep 2022 22:51:29 +0000 (22:51 +0000)]
Add missing deserialization of Event::HTLCHandlingFailed

17e6c374c513f2eca810fa4e931be65f0d4fc29f added the
`HTLCHandlingFailed` event, including serialization thereof,
however failed to add corresponding deserialization. This corrects
that oversight by adding said deserialization.

Thanks to @wpaulino for catching the oversight.

2 years agoMerge pull request #1657 from TheBlueMatt/2022-08-async-man-update
valentinewallace [Tue, 6 Sep 2022 20:06:06 +0000 (16:06 -0400)]
Merge pull request #1657 from TheBlueMatt/2022-08-async-man-update

Add a background processor which is async

2 years agoMerge pull request #1695 from TheBlueMatt/2022-08-log-chan_update
Matt Corallo [Tue, 6 Sep 2022 19:15:05 +0000 (19:15 +0000)]
Merge pull request #1695 from TheBlueMatt/2022-08-log-chan_update

Ensure we log private channel_updates at a non-GOSSIP log level

2 years agoEnsure we log private channel_updates at a non-GOSSIP log level 2022-08-log-chan_update
Matt Corallo [Mon, 5 Sep 2022 16:28:11 +0000 (16:28 +0000)]
Ensure we log private channel_updates at a non-GOSSIP log level

If we receive a channel_update for one of our private channels, we
will not log the message at the usual TRACE log level as the
message falls into the gossip range. However, for our own channels
they aren't *just* gossip, as we store that info and it changes
how we generate invoices. Thus, we add a log in `ChannelManager`
here at the DEBUG log level.

2 years agoAdd a background processing function that is async. 2022-08-async-man-update
Matt Corallo [Tue, 9 Aug 2022 06:01:10 +0000 (06:01 +0000)]
Add a background processing function that is async.

Adds a method which operates like BackgroundProcessor::start but
instead of functioning by spawning a background thread it is async.

2 years agoAdd a `Future` which can receive manager persistence events
Matt Corallo [Tue, 9 Aug 2022 04:15:21 +0000 (04:15 +0000)]
Add a `Future` which can receive manager persistence events

This allows users who don't wish to block a full thread to receive
persistence events.

The `Future` added here is really just a trivial list of callbacks,
but from that we can build a (somewhat ineffecient)
std::future::Future implementation and can (at least once a mapping
for Box<dyn Trait> is added) include the future in no-std bindings
as well.

Fixes #1595

2 years agoRemove internal references to `persistence` in waker.rs
Matt Corallo [Mon, 29 Aug 2022 21:13:58 +0000 (21:13 +0000)]
Remove internal references to `persistence` in waker.rs

2 years agoMerge pull request #1692 from TheBlueMatt/2022-08-time-goes-backwards
Matt Corallo [Sat, 3 Sep 2022 16:15:33 +0000 (16:15 +0000)]
Merge pull request #1692 from TheBlueMatt/2022-08-time-goes-backwards

Handle monotonic clock going backwards during runtime

2 years agoMerge pull request #1650 from valentinewallace/2022-08-take-onionmsgs-public
Matt Corallo [Fri, 2 Sep 2022 22:20:32 +0000 (22:20 +0000)]
Merge pull request #1650 from valentinewallace/2022-08-take-onionmsgs-public

2 years agoHandle monotonic clock going backwards during runtime 2022-08-time-goes-backwards
Matt Corallo [Fri, 2 Sep 2022 21:57:32 +0000 (21:57 +0000)]
Handle monotonic clock going backwards during runtime

We've had some users complain that `duration_since` is panic'ing
for them. This is possible if the machine being run on is buggy and
the "monotonic clock" goes backwards, which sadly some ancient
systems can do.

Rust addressed this issue in 1.60 by forcing
`Instant::duration_since` to not panic if the machine is buggy
(and time goes backwards), but for users on older rust versions we
do the same by hand here.

2 years agoLimit OnionMessenger outbound buffer size
Valentine Wallace [Thu, 1 Sep 2022 19:26:17 +0000 (15:26 -0400)]
Limit OnionMessenger outbound buffer size

Drop OMs if they push us over the max OnionMessenger outbound buffer size

2 years agoDon't construct OnionMessage while holding peer lock
Valentine Wallace [Fri, 2 Sep 2022 17:41:41 +0000 (13:41 -0400)]
Don't construct OnionMessage while holding peer lock

2 years agoRefuse to send and forward OMs to disconnected peers
Valentine Wallace [Thu, 4 Aug 2022 15:05:07 +0000 (11:05 -0400)]
Refuse to send and forward OMs to disconnected peers

We also refuse to connect to peers that don't advertise onion message
forwarding support.

2 years agoExpose onion message module as public
Valentine Wallace [Wed, 3 Aug 2022 15:42:54 +0000 (11:42 -0400)]
Expose onion message module as public

And fix warnings

2 years agoMerge pull request #1643 from jurvis/jurvis/2022-07-inflights-htlcs-across-payments
Jeffrey Czyz [Thu, 1 Sep 2022 20:01:30 +0000 (15:01 -0500)]
Merge pull request #1643 from jurvis/jurvis/2022-07-inflights-htlcs-across-payments

Track in-flight HTLCs across payments when routing

2 years agoMake payment tests more realistic
jurvis [Sun, 28 Aug 2022 06:07:50 +0000 (23:07 -0700)]
Make payment tests more realistic

Made sure that every hop has a unique receipient. When we simulate
calling `channel_penalty_msat` in `TestRouter`’s find route, use
actual previous node ids instead of just using the payer’s.

2 years agoKeep track of inflight HTLCs across payments
jurvis [Tue, 30 Aug 2022 05:50:44 +0000 (22:50 -0700)]
Keep track of inflight HTLCs across payments

Added two methods, `process_path_inflight_htlcs` and
`remove_path_inflight_htlcs`, that updates that `payment_cache` map with
path information that may have failed, succeeded, or have been given up
on.

Introduced `AccountForInflightHtlcs`, which will wrap our user-provided
scorer. We move the `S:Score` type parameterization from the `Router` to
`find_route`, so we can use our newly introduced
`AccountForInflightHtlcs`.

`AccountForInflightHtlcs` keeps track of a map of inflight HTLCs by
their short channel id, direction, and give us the value that is being
used up.

This map will in turn be populated prior to calling `find_route`, where
we’ll use `create_inflight_map`, to generate a current map of all
inflight HTLCs based on what was stored in `payment_cache`.

2 years agoMerge pull request #1687 from TheBlueMatt/2022-08-fuzz-msrv
Matt Corallo [Tue, 30 Aug 2022 17:12:47 +0000 (17:12 +0000)]
Merge pull request #1687 from TheBlueMatt/2022-08-fuzz-msrv

Drop honggfuzz `arbitrary` dependency to meet MSRV

2 years agoDrop honggfuzz `arbitrary` dependency to meet MSRV 2022-08-fuzz-msrv
Matt Corallo [Tue, 30 Aug 2022 04:03:47 +0000 (04:03 +0000)]
Drop honggfuzz `arbitrary` dependency to meet MSRV

2 years agoChange `payment_cache` to accept `PaymentInfo`
jurvis [Tue, 30 Aug 2022 05:49:24 +0000 (22:49 -0700)]
Change `payment_cache` to accept `PaymentInfo`

Introduces a new `PaymentInfo` struct that contains both the previous
`attempts` count that was tracked as well as the paths that are also
currently inflight.

2 years agoMerge pull request #1604 from valentinewallace/2022-07-OMs-followup
Jeffrey Czyz [Mon, 29 Aug 2022 17:17:36 +0000 (12:17 -0500)]
Merge pull request #1604 from valentinewallace/2022-07-OMs-followup

Onion messages: add some initial rate limiting

2 years agoImplement buffering onion messages for peers.
Valentine Wallace [Thu, 25 Aug 2022 18:30:29 +0000 (14:30 -0400)]
Implement buffering onion messages for peers.

In this commit, we check if a peer's outbound buffer has room for onion
messages, and if so pulls them from an implementer of a new trait,
OnionMessageProvider.

Makes sure channel messages are prioritized over OMs, and OMs are prioritized
over gossip.

The onion_message module remains private until further rate limiting is added.

2 years agoImplement OnionMessageProvider for OnionMessenger
Valentine Wallace [Mon, 22 Aug 2022 16:22:22 +0000 (12:22 -0400)]
Implement OnionMessageProvider for OnionMessenger

2 years agoPeerManager: bump the read pause limit
Valentine Wallace [Tue, 16 Aug 2022 18:33:10 +0000 (14:33 -0400)]
PeerManager: bump the read pause limit

...to make sure we can still get channel messages out after enqueuing some big gossip messages.

2 years agoAdd boilerplate for sending and receiving onion messages in PeerManager
Valentine Wallace [Sat, 6 Aug 2022 04:33:48 +0000 (00:33 -0400)]
Add boilerplate for sending and receiving onion messages in PeerManager

Adds the boilerplate needed for PeerManager and OnionMessenger to work
together, with some corresponding docs and misc updates mostly due to the
PeerManager public API changing.

2 years agoMerge pull request #1683 from valentinewallace/2022-08-multiqueue-peerman
Matt Corallo [Thu, 25 Aug 2022 21:30:15 +0000 (21:30 +0000)]
Merge pull request #1683 from valentinewallace/2022-08-multiqueue-peerman

2 years agoMerge pull request #1673 from TheBlueMatt/2022-08-may-learn-preimage-balance
Matt Corallo [Thu, 25 Aug 2022 20:26:30 +0000 (20:26 +0000)]
Merge pull request #1673 from TheBlueMatt/2022-08-may-learn-preimage-balance

Expose a `Balance` for inbound HTLCs even without a preimage

2 years agoSeparate gossip broadcasts into their own queue in PeerManager
Valentine Wallace [Wed, 24 Aug 2022 22:04:58 +0000 (18:04 -0400)]
Separate gossip broadcasts into their own queue in PeerManager

This allows us to better prioritize channel messages over gossip broadcasts and
lays groundwork for rate limiting onion messages more simply, since they won't
be competing with gossip broadcasts for space in the main message queue.

2 years agoPeerMan: rename drop_gossip util to be more accurate
Valentine Wallace [Thu, 25 Aug 2022 17:58:49 +0000 (13:58 -0400)]
PeerMan: rename drop_gossip util to be more accurate

It's more accurate to name it as dropping gossip broadcasts, as it won't drop
all gossip.

2 years agoPeerMan: fix bug in drop_gossip util
Valentine Wallace [Thu, 25 Aug 2022 17:56:19 +0000 (13:56 -0400)]
PeerMan: fix bug in drop_gossip util

Fixes a flipped bool that was introduced in
4a1ee5f9a984c9b0c0892025d624ade734337b1a

2 years agoRename MaybeClaimableHTLCAwaitingTimeout for consistency 2022-08-may-learn-preimage-balance
Matt Corallo [Wed, 24 Aug 2022 22:16:04 +0000 (22:16 +0000)]
Rename MaybeClaimableHTLCAwaitingTimeout for consistency

As we now have a MaybePreimageClaimableHTLC, its more consistent
to rename `MaybeClaimableHTLCAwaitingTimeout` to
`MaybeTimeoutClaimableHTLC`.

2 years agoExpose a `Balance` for inbound HTLCs even without a preimage
Matt Corallo [Wed, 17 Aug 2022 20:15:23 +0000 (20:15 +0000)]
Expose a `Balance` for inbound HTLCs even without a preimage

If we don't currently have the preimage for an inbound HTLC, that
does not guarantee we can never claim it, but instead only that we
cannot claim it unless we receive the preimage from the channel we
forwarded the channel out on.

Thus, we cannot consider a channel to have no claimable balances if
the only remaining output on the commitment ransaction is an
inbound HTLC for which we do not have the preimage, as we may be
able to claim it in the future.

This commit addresses this issue by adding a new `Balance` variant
- `MaybePreimageClaimableHTLCAwaitingTimeout`, which is generated
until the HTLC output is spent.

Fixes #1620

2 years agoMerge pull request #1682 from tnull/2022-08-export-all-log-levels
Matt Corallo [Wed, 24 Aug 2022 18:41:46 +0000 (18:41 +0000)]
Merge pull request #1682 from tnull/2022-08-export-all-log-levels

2 years agoMerge pull request #1652 from valentinewallace/2022-08-reply-paths
Jeffrey Czyz [Wed, 24 Aug 2022 14:55:20 +0000 (09:55 -0500)]
Merge pull request #1652 from valentinewallace/2022-08-reply-paths

Onion messages: support reply paths

2 years agoExport and document all `log` macros.
Elias Rohrer [Wed, 24 Aug 2022 11:59:58 +0000 (13:59 +0200)]
Export and document all `log` macros.

Previously, only `log_error` and `log_trace` macros have been exported.
This change exports the macros of all log levels, which enables them to
be used downstream.

2 years agoSupport sending and receiving reply paths
Valentine Wallace [Fri, 5 Aug 2022 22:03:12 +0000 (18:03 -0400)]
Support sending and receiving reply paths

2 years agoFix bug in onion message payload decode
Valentine Wallace [Fri, 5 Aug 2022 21:50:02 +0000 (17:50 -0400)]
Fix bug in onion message payload decode

Previously, we were decoding payload lengths as a VarInt. Per the spec, this is
wrong -- it should be decoded as a BigSize.  This bug also exists in our
payment payload decoding, to be fixed separately.

Upcoming reply path tests caught this bug because we hadn't encoded a payload
greater than 253 before, so we hadn't hit the problem that VarInts are encoded
as little-endian whereas BigSizes are encoded as big-endian.

2 years agoOM functional tests: update util to take nodes by reference
Valentine Wallace [Fri, 5 Aug 2022 21:45:43 +0000 (17:45 -0400)]
OM functional tests: update util to take nodes by reference

And fix one test to be uniform with the others

2 years agoMerge pull request #1679 from TheBlueMatt/2022-08-no-double-access
Arik [Tue, 23 Aug 2022 20:11:36 +0000 (13:11 -0700)]
Merge pull request #1679 from TheBlueMatt/2022-08-no-double-access

Avoid querying the chain for outputs for channels we already have

2 years agoAvoid querying the chain for outputs for channels we already have 2022-08-no-double-access
Matt Corallo [Sun, 21 Aug 2022 20:34:22 +0000 (20:34 +0000)]
Avoid querying the chain for outputs for channels we already have

If we receive a ChannelAnnouncement message but we already have the
channel, there's no reason to do a chain lookup. Instead of
immediately calling the user-provided `chain::Access` when handling
a ChannelAnnouncement, we first check if we have the corresponding
channel in the graph.

Note that if we do have the corresponding channel but it was not
previously checked against the blockchain, we should still check
with the `chain::Access` and update if necessary.

2 years agoMerge pull request #1681 from dunxen/2022-08-githubactionsfix
Jeffrey Czyz [Tue, 23 Aug 2022 14:22:48 +0000 (09:22 -0500)]
Merge pull request #1681 from dunxen/2022-08-githubactionsfix

Workaround GH Actions issue for 1.45

2 years agoWorkaround GH Actions issue for 1.45
Duncan Dean [Tue, 23 Aug 2022 07:20:35 +0000 (09:20 +0200)]
Workaround GH Actions issue for 1.45

2 years agoMerge pull request #1677 from ok300/ok300-patch-1
Matt Corallo [Fri, 19 Aug 2022 22:18:59 +0000 (22:18 +0000)]
Merge pull request #1677 from ok300/ok300-patch-1

Bump bitcoin_hashes to v0.11

2 years agoMerge pull request #1676 from arik-so/2022-08-gossip-error-message-fix
Matt Corallo [Fri, 19 Aug 2022 19:03:08 +0000 (19:03 +0000)]
Merge pull request #1676 from arik-so/2022-08-gossip-error-message-fix

Fix script order in gossip key mismatch error.

2 years agoMerge pull request #1575 from NicolaLS/hash-invoice
Matt Corallo [Fri, 19 Aug 2022 17:45:11 +0000 (17:45 +0000)]
Merge pull request #1575 from NicolaLS/hash-invoice

2 years agoFix script order in gossip key mismatch error.
Arik Sosman [Fri, 19 Aug 2022 17:07:55 +0000 (10:07 -0700)]
Fix script order in gossip key mismatch error.

2 years agoderive Hash for Invoice
NicolaLS [Mon, 27 Jun 2022 12:41:46 +0000 (14:41 +0200)]
derive Hash for Invoice

2 years agoBump bitcoin_hashes to v0.11
ok300 [Fri, 19 Aug 2022 14:40:19 +0000 (16:40 +0200)]
Bump bitcoin_hashes to v0.11

2 years agoMerge pull request #1670 from TheBlueMatt/2022-08-mon-size-guidelines
Matt Corallo [Thu, 18 Aug 2022 16:33:20 +0000 (16:33 +0000)]
Merge pull request #1670 from TheBlueMatt/2022-08-mon-size-guidelines

Provide guidance on ChannelMonitorUpdate serialized size

2 years agoMerge pull request #1495 from TheBlueMatt/2022-04-all-claimables
Matt Corallo [Wed, 17 Aug 2022 22:50:49 +0000 (22:50 +0000)]
Merge pull request #1495 from TheBlueMatt/2022-04-all-claimables

Expose counterparty-revoked-outputs in `get_claimable_balance`

2 years agoProvide guidance on ChannelMonitorUpdate serialized size 2022-08-mon-size-guidelines
Matt Corallo [Mon, 15 Aug 2022 19:30:32 +0000 (19:30 +0000)]
Provide guidance on ChannelMonitorUpdate serialized size

Users need to make decisions about storage sizing and we need to
have advice on the maximum size of various things users need to
store. ChannelMonitorUpdates are likely the worst case of this,
they're usually at max a few KB, but can get up to a few hundred
KB for commitment transactions that have 400+ HTLCs pending.

We had one user report an update (likely) going over 400 KiB, which
isn't immediately obvious to me is practical, but its within a few
multiples of trivially-reachable sizes, so its likely that did
occur. To be on the safe side, we simply recommend users ensure
they can support "upwards of 1 MiB" here.

2 years agoMerge pull request #1648 from valentinewallace/2022-08-fuzz-onion-msgs
valentinewallace [Wed, 17 Aug 2022 19:48:29 +0000 (15:48 -0400)]
Merge pull request #1648 from valentinewallace/2022-08-fuzz-onion-msgs

Onion messages: add fuzz testing

2 years agoMerge pull request #1666 from TheBlueMatt/2022-08-fix-script-check
Matt Corallo [Wed, 17 Aug 2022 19:34:26 +0000 (19:34 +0000)]
Merge pull request #1666 from TheBlueMatt/2022-08-fix-script-check

Correct the on-chain script checked in gossip verification

2 years agoCorrect the on-chain script checked in gossip verification 2022-08-fix-script-check
Matt Corallo [Sat, 13 Aug 2022 17:29:06 +0000 (17:29 +0000)]
Correct the on-chain script checked in gossip verification

The `bitcoin_key_1` and `bitcoin_key_2` fields in
`channel_announcement` messages are sorted according to node_ids
rather than the keys themselves, however the on-chain funding
script is sorted according to the bitcoin keys themselves. Thus,
with some probability, we end up checking that the on-chain script
matches the wrong script and rejecting the channel announcement.

The correct solution is to use our existing channel funding script
generation function which ensure we always match what we generate.

This was found in testing the Java bindings, where a test checks
that retunring the generated funding script in `chain::Access`
results in the constructed channel ending up in our network graph.

2 years agoFuzz test onion messages
Valentine Wallace [Sun, 24 Jul 2022 18:47:31 +0000 (14:47 -0400)]
Fuzz test onion messages

Also update the fuzz ChaCha20Poly1305 to not mark as finished after a single
encrypt_in_place.  This is because more bytes may still need to be encrypted,
causing us to panic at the assertion that finished == false when we go to
encrypt more.

Also fix unused_mut warning in messenger + add log on OM forward for testing

2 years agoMerge pull request #1660 from TheBlueMatt/2022-08-cleanup-ratelimits
Matt Corallo [Tue, 16 Aug 2022 04:43:02 +0000 (04:43 +0000)]
Merge pull request #1660 from TheBlueMatt/2022-08-cleanup-ratelimits

Backfill gossip without buffering directly in LDK

2 years agoMove per-HTLC logic out of get_claimable_balances into a helper 2022-04-all-claimables
Matt Corallo [Sat, 16 Jul 2022 20:41:45 +0000 (20:41 +0000)]
Move per-HTLC logic out of get_claimable_balances into a helper

Val suggested this as an obvious cleanup to separate per_HTLC logic
from the total commitment transaction logic, separating the large
function into two.

2 years agoExpose counterparty-revoked-outputs in `get_claimable_balance`
Matt Corallo [Tue, 24 May 2022 23:57:56 +0000 (23:57 +0000)]
Expose counterparty-revoked-outputs in `get_claimable_balance`

This uses the various new tracking added in the prior commits to
expose a new `Balance` type - `CounterpartyRevokedOutputClaimable`.

Some nontrivial work is required, however, as we now have to track
HTLC outputs as spendable in a transaction that comes *after* an
HTLC-Success/HTLC-Timeout transaction, which we previously didn't
need to do. Thus, we have to check if an
`onchain_events_awaiting_threshold_conf` event spends a commitment
transaction's HTLC output while walking events. Further, because
we now need to track HTLC outputs after the
HTLC-Success/HTLC-Timeout confirms, and because we have to track
the counterparty's `to_self` output as a contentious output which
could be claimed by either party, we have to examine the
`OnchainTxHandler`'s set of outputs to spend when determining if
certain outputs are still spendable.

Two new tests are added which test various different transaction
formats, and hopefully provide good test coverage of the various
revoked output paths.

2 years agoDrop uneccessary `if {...; bool}` pattern in PeerManager 2022-08-cleanup-ratelimits
Matt Corallo [Mon, 15 Aug 2022 19:38:45 +0000 (19:38 +0000)]
Drop uneccessary `if {...; bool}` pattern in PeerManager

2 years agoReplace manual iteration with for loops in outbound gossip sync
Matt Corallo [Mon, 15 Aug 2022 18:08:13 +0000 (18:08 +0000)]
Replace manual iteration with for loops in outbound gossip sync

2 years agoSimplify claimable balance trivially with consistent accessors
Matt Corallo [Wed, 13 Jul 2022 01:20:09 +0000 (01:20 +0000)]
Simplify claimable balance trivially with consistent accessors

2 years agoScan `onchain_events_awaiting_threshold_conf` once in balance calc
Matt Corallo [Tue, 17 May 2022 20:45:17 +0000 (20:45 +0000)]
Scan `onchain_events_awaiting_threshold_conf` once in balance calc

Instead of a series of different
`onchain_events_awaiting_threshold_conf.iter()...` calls to scan
for HTLC status in balance calculation, pull them all out into one
`for ... { match ... }` to do it once and simplify the code
somewhat.

2 years agoTrack the txid that resolves HTLCs even after resolution completes
Matt Corallo [Sat, 21 May 2022 01:11:52 +0000 (01:11 +0000)]
Track the txid that resolves HTLCs even after resolution completes

We need this information when we look up if we still need to spend
a revoked output from an HTLC-Success/HTLC-Timeout transaction for
balance calculation.

2 years agoTrack HTLC-Success/HTLC-Timeout claims of revoked outputs
Matt Corallo [Thu, 19 May 2022 01:50:37 +0000 (01:50 +0000)]
Track HTLC-Success/HTLC-Timeout claims of revoked outputs

When a counterparty broadcasts a revoked commitment transaction,
followed immediately by HTLC-Success/-Timeout spends thereof, we'd
like to have an `onchain_events_awaiting_threshold_conf` entry
for them.

This does so using the `HTLCSpendConfirmation` entry, giving it
(slightly) new meaning. Because all existing uses of
`HTLCSpendConfirmation` already check if the relevant commitment
transaction is revoked first, this should be trivially backwards
compatible.

We will ultimately figure out if something is being spent via the
`OnchainTxHandler`, but to do so we need to look up the output via
the HTLC transaction txid, which this allows us to do.

2 years agoFix off-by-one in test_onchain_htlc_claim_reorg_remote_commitment
Matt Corallo [Tue, 17 May 2022 23:57:52 +0000 (23:57 +0000)]
Fix off-by-one in test_onchain_htlc_claim_reorg_remote_commitment

The test intended to disconnect a transaction previously connected
but didn't disconnect enough blocks to do so, leading to it
confirming two conflicting transactions.

In the next few commits this will become an assertion failure.

2 years agoClean up nesting in `get_counterparty_output_claim_info`
Matt Corallo [Thu, 14 Jul 2022 21:23:39 +0000 (21:23 +0000)]
Clean up nesting in `get_counterparty_output_claim_info`

2 years agoTrack counterparty payout info in counterparty commitment txn
Matt Corallo [Sat, 30 Apr 2022 20:29:31 +0000 (20:29 +0000)]
Track counterparty payout info in counterparty commitment txn

When handling a revoked counterparty commitment transaction which
was broadcast on-chain, we occasionally need to look up which
output (and its value) was to the counterparty (the `to_self`
output). This will allow us to generate `Balance`s for the user for
the revoked output.

2 years agoStore the full event transaction in `OnchainEvent` structs
Matt Corallo [Fri, 13 May 2022 05:11:14 +0000 (05:11 +0000)]
Store the full event transaction in `OnchainEvent` structs

When we see a transaction which generates some `OnchainEvent`, its
useful to have the full transaction around for later analysis.
Specifically, it lets us check the list of outputs which were spent
in the transaction, allowing us to look up, e.g. which HTLC
outpoint was spent in a transaction.

This will be used in a few commits to do exactly that - figure out
which HTLC a given `OnchainEvent` corresponds with.

2 years agoMerge pull request #1663 from tnull/2022-08-drop-register-output-return
Matt Corallo [Mon, 15 Aug 2022 23:10:32 +0000 (23:10 +0000)]
Merge pull request #1663 from tnull/2022-08-drop-register-output-return

Drop return value from `Filter::register_output`

2 years agoBackfill gossip without buffering directly in LDK
Matt Corallo [Tue, 9 Aug 2022 21:26:16 +0000 (21:26 +0000)]
Backfill gossip without buffering directly in LDK

Instead of backfilling gossip by buffering (up to) ten messages at
a time, only buffer one message at a time, as the peers' outbound
socket buffer drains. This moves the outbound backfill messages out
of `PeerHandler` and into the operating system buffer, where it
arguably belongs.

Not buffering causes us to walk the gossip B-Trees somewhat more
often, but avoids allocating vecs for the responses. While its
probably (without having benchmarked it) a net performance loss, it
simplifies buffer tracking and leaves us with more room to play
with the buffer sizing constants as we add onion message forwarding
which is an important win.

Note that because we change how often we check if we're out of
messages to send before pinging, we slightly change how many
messages are exchanged at once, impacting the
`test_do_attempt_write_data` constants.

2 years agoClarify 'should' vs 'will' in `get_relevant_txids`
Elias Rohrer [Thu, 11 Aug 2022 12:33:22 +0000 (14:33 +0200)]
Clarify 'should' vs 'will' in `get_relevant_txids`

2 years agoDrop return value from `Filter::register_output`
Elias Rohrer [Thu, 11 Aug 2022 12:27:45 +0000 (14:27 +0200)]
Drop return value from `Filter::register_output`

This commit removes the return value from `Filter::register_output` as
creating a suitable value almost always entails blocking operations
(e.g., lookups via network request), which however conflicts with the
requirement that user calls should avoid blocking calls at all cost.

Removing the return value also rendered quite a bit of test code for
dependent transaction handling superfluous, which is therefore also
removed with this commit.

2 years agoMerge pull request #1659 from valentinewallace/2022-08-fix-payload-decode
valentinewallace [Mon, 15 Aug 2022 18:32:47 +0000 (14:32 -0400)]
Merge pull request #1659 from valentinewallace/2022-08-fix-payload-decode

Fix payment onion payload decode

2 years agoFix fuzzer-found underflow
Valentine Wallace [Fri, 5 Aug 2022 17:15:57 +0000 (13:15 -0400)]
Fix fuzzer-found underflow

2 years agoRename HighZeroBytesDroppedVarInt to HighZeroBytesDroppedBigSize
Valentine Wallace [Sun, 7 Aug 2022 03:06:19 +0000 (23:06 -0400)]
Rename HighZeroBytesDroppedVarInt to HighZeroBytesDroppedBigSize

As observed by @wpaulino, this struct encodes its bytes as big-endian,
therefore it's a BigSize, not a VarInt.

2 years agoFix bug in onion payment payload decode
Valentine Wallace [Sat, 6 Aug 2022 02:19:50 +0000 (22:19 -0400)]
Fix bug in onion payment payload decode

Prior to this change, we could have failed to decode a valid payload of size
>253. This is because we were decoding the length (a BigSize, big-endian) as a
VarInt (little-endian).

Found in #1652.

2 years agoRename `PersistenceNotifier` to simply `Notifier`
Matt Corallo [Fri, 12 Aug 2022 23:53:50 +0000 (23:53 +0000)]
Rename `PersistenceNotifier` to simply `Notifier`

... as it is no longer persistence-specific (though still only used
for persistence).

2 years agoMerge pull request #1658 from lightning-signer/2022-08-bitcoin-0-29
Matt Corallo [Fri, 12 Aug 2022 23:51:06 +0000 (23:51 +0000)]
Merge pull request #1658 from lightning-signer/2022-08-bitcoin-0-29

Update bitcoin crate to 0.29.0

2 years agoModify ecdh to take Scalar
Devrandom [Wed, 10 Aug 2022 16:04:59 +0000 (18:04 +0200)]
Modify ecdh to take Scalar

2 years agoUpdate bitcoin crate to 0.29.0
Devrandom [Tue, 9 Aug 2022 15:39:51 +0000 (17:39 +0200)]
Update bitcoin crate to 0.29.0

2 years agoClarify comment on BUFFER_DRAIN_MSGS_PER_TICK.
Matt Corallo [Tue, 9 Aug 2022 21:45:19 +0000 (21:45 +0000)]
Clarify comment on BUFFER_DRAIN_MSGS_PER_TICK.

2 years agoUse util methods in `Peer` to decide when to forward
Valentine Wallace [Tue, 9 Aug 2022 21:20:56 +0000 (21:20 +0000)]
Use util methods in `Peer` to decide when to forward

This consolidates our various checks on peer buffer space into the
`Peer` impl itself, making the thresholds at which we stop taking
various actions on a peer more readable as a whole.

This commit was primarily authored by `Valentine Wallace
<vwallace@protonmail.com>` with some amendments by `Matt Corallo
<git@bluematt.me>`.

2 years agoMerge pull request #1655 from valentinewallace/2022-08-fix-ci-doc-links-check
valentinewallace [Tue, 9 Aug 2022 16:49:40 +0000 (12:49 -0400)]
Merge pull request #1655 from valentinewallace/2022-08-fix-ci-doc-links-check

Fix CI to error on doc links to private items

2 years agoMove PersistenceNotifier to a new util module
Matt Corallo [Tue, 9 Aug 2022 01:57:09 +0000 (01:57 +0000)]
Move PersistenceNotifier to a new util module

It was always somewhat strange to have a bunch of notification
logic in `channelmanager`, and with the next commit adding a bunch
more, its moved here first.

2 years agoMerge pull request #1651 from TheBlueMatt/2022-08-export-chan-list
Matt Corallo [Mon, 8 Aug 2022 16:59:53 +0000 (16:59 +0000)]
Merge pull request #1651 from TheBlueMatt/2022-08-export-chan-list

[C Bindings] Expose channel and nodes list in `ReadOnlyNetworkGraph`

2 years agoMerge pull request #1654 from TheBlueMatt/2022-08-update-libfuzzer
valentinewallace [Mon, 8 Aug 2022 15:37:42 +0000 (11:37 -0400)]
Merge pull request #1654 from TheBlueMatt/2022-08-update-libfuzzer

Update libfuzzer-sys to new upstream inclusion method

2 years agoUpdate libfuzzer-sys to new upstream inclusion method 2022-08-update-libfuzzer
Matt Corallo [Sun, 7 Aug 2022 19:02:33 +0000 (19:02 +0000)]
Update libfuzzer-sys to new upstream inclusion method

Dunno why they changed it, but the old "depend directly on git"
thing that cargo-fuzz used forever is now deprecated that that
repo is archived, they've now moved to another repo and publish
properly on crates.io.

2 years agoFix CI to error on doc links to private items
Valentine Wallace [Sun, 7 Aug 2022 17:49:10 +0000 (13:49 -0400)]
Fix CI to error on doc links to private items

Somehow we weren't doing this.

2 years agoMinor msgs::OnionHopData cleanups
Valentine Wallace [Sat, 6 Aug 2022 02:18:27 +0000 (22:18 -0400)]
Minor msgs::OnionHopData cleanups

2 years ago[C Bindings] Expose channel and nodes list in `ReadOnlyNetworkGraph` 2022-08-export-chan-list
Matt Corallo [Fri, 5 Aug 2022 03:20:53 +0000 (03:20 +0000)]
[C Bindings] Expose channel and nodes list in `ReadOnlyNetworkGraph`

2 years agoMerge pull request #1638 from ViktorTigerstrom/2022-07-update-decode-update-add-htlc...
Matt Corallo [Wed, 3 Aug 2022 17:44:46 +0000 (17:44 +0000)]
Merge pull request #1638 from ViktorTigerstrom/2022-07-update-decode-update-add-htlc-onion-return-parameters

Don't return `channel_state` from `decode_update_add_htlc_onion`

2 years agoMerge pull request #1619 from G8XSU/main
Matt Corallo [Wed, 3 Aug 2022 17:37:51 +0000 (17:37 +0000)]
Merge pull request #1619 from G8XSU/main

Add config support for 'their_channel_reserve_proportional_millionths' [#1498]

2 years agoMerge pull request #1503 from valentinewallace/2022-05-onion-msgs
Matt Corallo [Wed, 3 Aug 2022 04:39:56 +0000 (04:39 +0000)]
Merge pull request #1503 from valentinewallace/2022-05-onion-msgs

Onion messages v1

2 years agoFix possible incomplete read bug on onion packet decode
Valentine Wallace [Tue, 12 Jul 2022 20:04:42 +0000 (16:04 -0400)]
Fix possible incomplete read bug on onion packet decode

Pre-existing to this PR, we were reading next packet bytes with io::Read::read,
which is not guaranteed to read all the bytes we need, only guaranteed to read
*some* bytes.

We fix this to be read_exact, which is guaranteed to read all the next hop
packet bytes.

2 years agoError when attempting to send an OM to a blinded route with 0 hops
Valentine Wallace [Thu, 23 Jun 2022 19:51:43 +0000 (15:51 -0400)]
Error when attempting to send an OM to a blinded route with 0 hops

2 years agoAdd SendError enum for onion messages and error on too-big packets
Valentine Wallace [Thu, 23 Jun 2022 19:47:25 +0000 (15:47 -0400)]
Add SendError enum for onion messages and error on too-big packets

2 years agoAdd test utilities and integration tests for onion messages
Valentine Wallace [Sat, 4 Jun 2022 20:22:20 +0000 (13:22 -0700)]
Add test utilities and integration tests for onion messages

2 years agoSignificantly expand onion message documentation
Valentine Wallace [Thu, 23 Jun 2022 19:45:33 +0000 (15:45 -0400)]
Significantly expand onion message documentation

2 years agoImplement receiving and forwarding onion messages
Valentine Wallace [Sat, 28 May 2022 01:39:56 +0000 (18:39 -0700)]
Implement receiving and forwarding onion messages

This required adapting `onion_utils::decode_next_hop` to work for both payments
and onion messages.

Currently we just print out the path_id of any onion messages we receive. In
the future, these received onion messages will be redirected to their
respective handlers: i.e. an invoice_request will go to an InvoiceHandler,
custom onion messages will go to a custom handler, etc.

2 years agoSupport sending onion messages
Valentine Wallace [Sat, 28 May 2022 01:31:27 +0000 (18:31 -0700)]
Support sending onion messages

This adds several utilities in service of then adding
OnionMessenger::send_onion_message, which can send to either an unblinded
pubkey or a blinded route. Sending custom TLVs and sending an onion message
containing a reply path are not yet supported.

We also need to split the construct_keys_callback macro into two macros to
avoid an unused assignment warning.