]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
20 months agoAdd an `inbound` flag to the `peer_connected` message handlers
Matt Corallo [Thu, 26 Jan 2023 04:45:58 +0000 (04:45 +0000)]
Add an `inbound` flag to the `peer_connected` message handlers

Its useful for the message handlers to know if a peer is inbound
for DoS decision-making reasons.

20 months agoMerge pull request #2035 from TheBlueMatt/2023-02-fix-no-con-discon
Matt Corallo [Tue, 21 Feb 2023 21:28:05 +0000 (21:28 +0000)]
Merge pull request #2035 from TheBlueMatt/2023-02-fix-no-con-discon

Fix (and DRY) the conditionals before calling peer_disconnected

20 months agoMerge pull request #2040 from alecchendev/2023-02-indexed-map-btreeset-to-vec
Matt Corallo [Tue, 21 Feb 2023 19:57:51 +0000 (19:57 +0000)]
Merge pull request #2040 from alecchendev/2023-02-indexed-map-btreeset-to-vec

Replace `BTreeSet` in `IndexedMap` with sorted `Vec`

20 months agoRemove the `peer_disconnected` `no_connection_possible` flag 2023-02-fix-no-con-discon
Matt Corallo [Tue, 21 Feb 2023 19:10:43 +0000 (19:10 +0000)]
Remove the `peer_disconnected` `no_connection_possible` flag

Long ago, we used the `no_connection_possible` to signal that a
peer has some unknown feature set or some other condition prevents
us from ever connecting to the given peer. In that case we'd
automatically force-close all channels with the given peer. This
was somewhat surprising to users so we removed the automatic
force-close, leaving the flag serving no LDK-internal purpose.

Distilling the concept of "can we connect to this peer again in the
future" to a simple flag turns out to be ripe with edge cases, so
users actually using the flag to force-close channels would likely
cause surprising behavior.

Thus, there's really not a lot of reason to keep the flag,
especially given its untested and likely to be broken in subtle
ways anyway.

20 months agoAdd a further debug_assert that disconnecting peers are connected
Matt Corallo [Wed, 15 Feb 2023 06:29:29 +0000 (06:29 +0000)]
Add a further debug_assert that disconnecting peers are connected

20 months agoCorrect `funding_transaction_generated` err msg and fix fuzz check
Matt Corallo [Wed, 15 Feb 2023 01:23:20 +0000 (01:23 +0000)]
Correct `funding_transaction_generated` err msg and fix fuzz check

This fixes new errors in `full_stack_target` pointed out by
Chaincode's generous fuzzing infrastructure. Specifically, there's
no reason to check the error message in the
`funding_transaction_generated` return value - it can only return
a failure if the channel has closed since the funding transaction
was generated (which is fine) or if the signer refuses to sign
(which can't happen in fuzzing).

20 months agoCorrect the "is peer live" checks in `PeerManager`
Matt Corallo [Wed, 15 Feb 2023 01:20:38 +0000 (01:20 +0000)]
Correct the "is peer live" checks in `PeerManager`

In general, we should be checking if a `Peer` has `their_features`
set as the "is this peer connected and have they finished the
handshake" flag as it indicates an `Init` message was received.

While none of these appear to be reachable bugs, there were a
number of places where we checked other flags for this purpose,
which may lead to sending messages before `Init` in the future.

Here we clean these cases up to always use the correct check (via
the new util method).

20 months agoAdd test of an initial message other than `Init`
Matt Corallo [Wed, 15 Feb 2023 01:54:21 +0000 (01:54 +0000)]
Add test of an initial message other than `Init`

This test fails without the previous commit.

20 months agoFix (and DRY) the conditionals before calling `peer_disconnected`
Matt Corallo [Wed, 15 Feb 2023 01:13:57 +0000 (01:13 +0000)]
Fix (and DRY) the conditionals before calling `peer_disconnected`

If we have a peer that sends a non-`Init` first message, we'll call
`peer_disconnected` without ever having called `peer_connected`
(which has to wait until we have an `Init` message). This is a
violation of our API guarantees, though should generally not be an
issue.

Because this bug was repeated in a few places, we also take this
opportunity to DRY up the logic which checks the peer state before
calling `peer_disconnected`.

Found by the new `ChannelManager` assertions and the
`full_stack_target` fuzzer.

20 months agoReplace `BTreeSet` in `IndexedMap` with sorted `Vec`
Alec Chen [Thu, 16 Feb 2023 22:34:06 +0000 (16:34 -0600)]
Replace `BTreeSet` in `IndexedMap` with sorted `Vec`

The `Vec` is sorted not on `IndexedMap::insert`, but on
`IndexedMap::range` to avoid unnecessary work while reading a network
graph.

20 months agoMerge pull request #2026 from valentinewallace/2023-02-dedup-pending-forwardable-evs
Matt Corallo [Sun, 19 Feb 2023 18:25:03 +0000 (18:25 +0000)]
Merge pull request #2026 from valentinewallace/2023-02-dedup-pending-forwardable-evs

Deduplicate `PendingHTLCsForwardable` events on generation

20 months agoDeduplicate PendingHTLCsForwardable events when queueing
Valentine Wallace [Fri, 17 Feb 2023 22:41:21 +0000 (17:41 -0500)]
Deduplicate PendingHTLCsForwardable events when queueing

20 months agoOn retryable update_fail, don't queue redundant PendingHTLCsForwardable
Valentine Wallace [Fri, 17 Feb 2023 22:35:09 +0000 (17:35 -0500)]
On retryable update_fail, don't queue redundant PendingHTLCsForwardable

20 months agoCheck for abandon-able payments on startup
Valentine Wallace [Fri, 17 Feb 2023 22:14:43 +0000 (17:14 -0500)]
Check for abandon-able payments on startup

20 months agoMerge pull request #2009 from TheBlueMatt/2023-02-no-racey-retries
Matt Corallo [Thu, 16 Feb 2023 23:41:09 +0000 (23:41 +0000)]
Merge pull request #2009 from TheBlueMatt/2023-02-no-racey-retries

Fix (and test) threaded payment retries

20 months agoFix (and test) threaded payment retries 2023-02-no-racey-retries
Matt Corallo [Fri, 3 Feb 2023 23:05:58 +0000 (23:05 +0000)]
Fix (and test) threaded payment retries

The new in-`ChannelManager` retries logic does retries as two
separate steps, under two separate locks - first it calculates
the amount that needs to be retried, then it actually sends it.
Because the first step doesn't udpate the amount, a second thread
may come along and calculate the same amount and end up retrying
duplicatively.

Because we generally shouldn't ever be processing retries at the
same time, the fix is trivial - simply take a lock at the top of
the retry loop and hold it until we're done.

20 months agoAlways check `next_route` in `TestRouter` is fully consumed
Matt Corallo [Fri, 3 Feb 2023 02:17:41 +0000 (02:17 +0000)]
Always check `next_route` in `TestRouter` is fully consumed

...rather than only in std.

20 months agoTest if a given mutex is locked by the current thread in tests
Matt Corallo [Tue, 7 Feb 2023 19:46:08 +0000 (19:46 +0000)]
Test if a given mutex is locked by the current thread in tests

In anticipation of the next commit(s) adding threaded tests, we
need to ensure our lockorder checks work fine with multiple
threads. Sadly, currently we have tests in the form
`assert!(mutex.try_lock().is_ok())` to assert that a given mutex is
not locked by the caller to a function.

The fix is rather simple given we already track mutexes locked by a
thread in our `debug_sync` logic - simply replace the check with a
new extension trait which (for test builds) checks the locked state
by only looking at what was locked by the current thread.

20 months agoMerge pull request #2037 from valentinewallace/2023-02-fix-probe-leak
Matt Corallo [Thu, 16 Feb 2023 21:29:26 +0000 (21:29 +0000)]
Merge pull request #2037 from valentinewallace/2023-02-fix-probe-leak

Fix outbound payments probe leak

20 months agoMerge pull request #1921 from arik-so/2022-12-prohibit-old-rgs-updates
Matt Corallo [Thu, 16 Feb 2023 21:20:28 +0000 (21:20 +0000)]
Merge pull request #1921 from arik-so/2022-12-prohibit-old-rgs-updates

Throw error for RGS data that's more than two weeks old

20 months agoThrow error for RGS data that's more than two weeks old, fixing #1785
Arik Sosman [Fri, 16 Dec 2022 18:52:56 +0000 (10:52 -0800)]
Throw error for RGS data that's more than two weeks old, fixing #1785

20 months agoRemove pending probes on update_fail
Valentine Wallace [Wed, 15 Feb 2023 22:30:47 +0000 (17:30 -0500)]
Remove pending probes on update_fail

Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail

20 months agoMerge pull request #2008 from valentinewallace/2023-02-remove-manual-retries
Matt Corallo [Thu, 16 Feb 2023 00:49:04 +0000 (00:49 +0000)]
Merge pull request #2008 from valentinewallace/2023-02-remove-manual-retries

Abandon payments on behalf of the user and remove manual retries

20 months agoReword and fix grammar in PartialFailure docs
Valentine Wallace [Mon, 13 Feb 2023 16:36:46 +0000 (11:36 -0500)]
Reword and fix grammar in PartialFailure docs

20 months agoRemove retry_payments method
Valentine Wallace [Sun, 5 Feb 2023 22:05:12 +0000 (17:05 -0500)]
Remove retry_payments method

We're no longer supporting manual retries since
ChannelManager::send_payment_with_retry can be parameterized by a retry
strategy

This commit also updates all docs related to retry_payment and abandon_payment.
Since these docs frequently overlap with changes in preceding commits where we
start abandoning payments on behalf of the user, all the docs are updated in
one go.

20 months agoWhen processing pending htlcs, abandon outbounds that are not retryable
Valentine Wallace [Sat, 4 Feb 2023 02:44:08 +0000 (21:44 -0500)]
When processing pending htlcs, abandon outbounds that are not retryable

20 months agoAbandon payment on behalf of the user on payment path failure
Valentine Wallace [Fri, 3 Feb 2023 17:53:01 +0000 (12:53 -0500)]
Abandon payment on behalf of the user on payment path failure

Removed retry_single_path_payment, it's replaced by automatic_retries with
AutoRetry::Success

20 months agoAbandon payment if retry fails in process_pending_htlcs
Valentine Wallace [Fri, 3 Feb 2023 17:49:07 +0000 (12:49 -0500)]
Abandon payment if retry fails in process_pending_htlcs

20 months agoMerge pull request #1832 from jkczyz/2022-11-composite-handler
Matt Corallo [Wed, 15 Feb 2023 17:45:39 +0000 (17:45 +0000)]
Merge pull request #1832 from jkczyz/2022-11-composite-handler

Macro for composing custom message handlers

20 months agoPass pending events to outbound_payments::abandon_payment
Valentine Wallace [Fri, 3 Feb 2023 17:44:14 +0000 (12:44 -0500)]
Pass pending events to outbound_payments::abandon_payment

This makes it uniform with the outbound payment methods that generate events
and set us up for abandoning payments on behalf of the user.

20 months agoFix indentation in outbound payment mark_abandoned
Valentine Wallace [Fri, 3 Feb 2023 16:53:44 +0000 (11:53 -0500)]
Fix indentation in outbound payment mark_abandoned

20 months agoMerge pull request #2020 from valentinewallace/2023-02-test-inflight-scoring
Matt Corallo [Wed, 15 Feb 2023 01:25:09 +0000 (01:25 +0000)]
Merge pull request #2020 from valentinewallace/2023-02-test-inflight-scoring

Fix and test in-flight HTLC scoring in between retries

20 months agoAdd CI testing for lightning-custom-message crate
Jeffrey Czyz [Thu, 2 Feb 2023 03:35:38 +0000 (21:35 -0600)]
Add CI testing for lightning-custom-message crate

20 months agoRe-write CustomMessageHandler documentation
Jeffrey Czyz [Tue, 31 Jan 2023 16:44:19 +0000 (10:44 -0600)]
Re-write CustomMessageHandler documentation

Documentation for CustomMessageHandler wasn't clear how it is related to
PeerManager and contained some grammatical and factual errors. Re-write
the docs and link to the lightning_custom_message crate.

20 months agoFix compilation errors in some doc tests
Jeffrey Czyz [Tue, 3 Jan 2023 19:31:15 +0000 (13:31 -0600)]
Fix compilation errors in some doc tests

These are seen in newer versions of rustc.

20 months agoMacro for composing custom message handlers
Jeffrey Czyz [Tue, 3 Jan 2023 17:24:30 +0000 (11:24 -0600)]
Macro for composing custom message handlers

BOLT 1 specifies a custom message type range for use with experimental
or application-specific messages. While a `CustomMessageHandler` can be
defined to support more than one message type, defining such a handler
requires a significant amount of boilerplate and can be error prone.

Add a crate exporting a `composite_custom_message_handler` macro for
easily composing pre-defined custom message handlers. The resulting
handler can be further composed with other custom message handlers using
the same macro.

This requires a separate crate since the macro needs to support "or"
patterns in macro_rules, which is only available in edition 2021.

https://doc.rust-lang.org/edition-guide/rust-2021/or-patterns-macro-rules.html

Otherwise, a crate defining a handler for a set of custom messages could
not easily be reused with another custom message handler. Doing so would
require explicitly duplicating the reused handlers type ids, but those
may change when the crate is updated.

20 months agoMove `fairrwlock` to the `sync` module
Matt Corallo [Tue, 7 Feb 2023 19:39:24 +0000 (19:39 +0000)]
Move `fairrwlock` to the `sync` module

20 months agoMerge pull request #2029 from wpaulino/update-pgp-key
Matt Corallo [Tue, 14 Feb 2023 22:20:46 +0000 (22:20 +0000)]
Merge pull request #2029 from wpaulino/update-pgp-key

Update Wilmer Paulino's PGP key

20 months agoRemove unnecessary scoring methods from Router trait
Valentine Wallace [Thu, 9 Feb 2023 20:36:35 +0000 (14:36 -0600)]
Remove unnecessary scoring methods from Router trait

20 months agoFix computing in-flight HTLCs in between retries + test
Valentine Wallace [Tue, 7 Feb 2023 19:10:41 +0000 (14:10 -0500)]
Fix computing in-flight HTLCs in between retries + test

20 months agotest_utils: parameterize TestRouter by TestScorer
Valentine Wallace [Tue, 7 Feb 2023 19:04:55 +0000 (14:04 -0500)]
test_utils: parameterize TestRouter by TestScorer

This allows us set scoring expectations and ensure in-flight htlcs are factored
into scoring

20 months agoMerge pull request #1966 from ViktorTigerstrom/2023-01-store-channels-per-peer-followups
Matt Corallo [Tue, 14 Feb 2023 18:44:35 +0000 (18:44 +0000)]
Merge pull request #1966 from ViktorTigerstrom/2023-01-store-channels-per-peer-followups

1507 followups

20 months agoDon't serialize nodes which we have no channels to
Viktor Tigerström [Sat, 4 Feb 2023 18:14:42 +0000 (20:14 +0200)]
Don't serialize nodes which we have no channels to

20 months agoInitialize `list_channels_with_filter` result vec with capacity
Viktor Tigerström [Wed, 18 Jan 2023 01:30:36 +0000 (02:30 +0100)]
Initialize `list_channels_with_filter` result vec with capacity

20 months agoUpdate unknown peer passed to msg and api functions handling
Viktor Tigerström [Wed, 18 Jan 2023 01:20:09 +0000 (02:20 +0100)]
Update unknown peer passed to msg and api functions handling

20 months agoAlways remove disconnected peers with no channels
Viktor Tigerström [Tue, 17 Jan 2023 23:28:00 +0000 (00:28 +0100)]
Always remove disconnected peers with no channels

When a peer disconnects but still has channels, the peer's `peer_state`
entry in the `per_peer_state` is not removed by the `peer_disconnected`
function. If the channels with that peer is later closed while still
being disconnected (i.e. force closed), we therefore need to remove the
peer from `peer_state` separately.

To remove the peers separately, we push such peers to a separate HashSet
that holds peers awaiting removal, and remove the peers on a timer to
limit the negative effects on parallelism as much as possible.

20 months agoUpdate `ChannelManager` docs
Viktor Tigerström [Sun, 15 Jan 2023 16:01:29 +0000 (17:01 +0100)]
Update `ChannelManager` docs

Updates multiple instances of the `ChannelManager` docs related to the
previous change that moved the storage of the channels to the
`per_peer_state`. This docs update corrects some grammar errors and
incorrect information, as well as clarifies documentation that was
confusing.

20 months agoCheck `peer_state` existence more idiomatically
Viktor Tigerström [Fri, 13 Jan 2023 20:11:00 +0000 (21:11 +0100)]
Check `peer_state` existence more idiomatically

20 months agoRemove redundant Vec in `get_and_clear_pending_msg_events`
Viktor Tigerström [Thu, 12 Jan 2023 22:25:44 +0000 (23:25 +0100)]
Remove redundant Vec in `get_and_clear_pending_msg_events`

20 months agoDon't clone the Vec in `remove_first_msg_event_to_node`
Viktor Tigerström [Thu, 12 Jan 2023 21:50:43 +0000 (22:50 +0100)]
Don't clone the Vec in `remove_first_msg_event_to_node`

20 months agoRemove redundant hashmap lookups
Viktor Tigerström [Thu, 12 Jan 2023 20:51:18 +0000 (21:51 +0100)]
Remove redundant hashmap lookups

Avoid doing the same hashmap lookups twice in a row, when it's not
needed. Refactor `claim_funds_from_hop` in order to enable this change.

20 months agoUpdate Wilmer Paulino's PGP key
Wilmer Paulino [Sun, 12 Feb 2023 20:11:35 +0000 (12:11 -0800)]
Update Wilmer Paulino's PGP key

The old key (729E9D9D92C75A5FBFEEE057B5DD717BEF7CA5B1) has been revoked
as of 2023-02-11.

All commits going forward will be signed with the new key.

20 months agoMerge pull request #1980 from TheBlueMatt/2023-01-async-utxo-lookups
wpaulino [Sat, 11 Feb 2023 03:57:11 +0000 (19:57 -0800)]
Merge pull request #1980 from TheBlueMatt/2023-01-async-utxo-lookups

20 months agoMerge pull request #2019 from tnull/2023-02-expose-peer-addrs
Matt Corallo [Fri, 10 Feb 2023 21:20:36 +0000 (21:20 +0000)]
Merge pull request #2019 from tnull/2023-02-expose-peer-addrs

Expose peer addresses in `PeerManager`

20 months agoMerge pull request #2027 from danielgranhao/fix-create-inbound-payment-docs
Matt Corallo [Fri, 10 Feb 2023 17:41:18 +0000 (17:41 +0000)]
Merge pull request #2027 from danielgranhao/fix-create-inbound-payment-docs

Fix out-of-date `create_inbound_payment()` docs

20 months agoTest `get_peer_nodeids` returns peer addresses
Elias Rohrer [Fri, 10 Feb 2023 17:31:10 +0000 (11:31 -0600)]
Test `get_peer_nodeids` returns peer addresses

20 months agoExpose peer addresses via `get_peer_node_ids`
Elias Rohrer [Tue, 7 Feb 2023 17:22:12 +0000 (11:22 -0600)]
Expose peer addresses via `get_peer_node_ids`

20 months agoFix out-of-date `create_inbound_payment()` docs
Daniel Granhão [Fri, 10 Feb 2023 16:09:24 +0000 (16:09 +0000)]
Fix out-of-date `create_inbound_payment()` docs

20 months agoMerge pull request #1870 from tnull/2022-11-add-transaction-sync-crate
Matt Corallo [Fri, 10 Feb 2023 00:11:59 +0000 (00:11 +0000)]
Merge pull request #1870 from tnull/2022-11-add-transaction-sync-crate

Add transaction sync crate

20 months agoBuild and test transaction sync crate in CI
Elias Rohrer [Thu, 19 Jan 2023 17:45:46 +0000 (11:45 -0600)]
Build and test transaction sync crate in CI

20 months agoTest syncing against Esplora backend
Elias Rohrer [Fri, 6 Jan 2023 12:17:48 +0000 (13:17 +0100)]
Test syncing against Esplora backend

20 months agoAdd transaction sync crate
Elias Rohrer [Wed, 23 Nov 2022 08:33:37 +0000 (09:33 +0100)]
Add transaction sync crate

This crate provides utilities for syncing LDK via the transaction-based
`Confirm` interface. The initial implementation facilitates
synchronization with an Esplora backend server.

20 months agoMove the channel_announcement process log into `NetworkGraph` 2023-01-async-utxo-lookups
Matt Corallo [Thu, 26 Jan 2023 03:04:14 +0000 (03:04 +0000)]
Move the channel_announcement process log into `NetworkGraph`

This ensures its always written after we update the graph, no
matter how we updated the graph.

20 months agoAdd tests for the new async gossip checking internal APIs
Matt Corallo [Mon, 23 Jan 2023 04:59:13 +0000 (04:59 +0000)]
Add tests for the new async gossip checking internal APIs

20 months agoSupport async results in `TestChainSource`, count `get_utxo` calls
Matt Corallo [Wed, 8 Feb 2023 22:26:56 +0000 (22:26 +0000)]
Support async results in `TestChainSource`, count `get_utxo` calls

20 months agoSuggest a socket read buffer of 4KiB to limit message count
Matt Corallo [Wed, 1 Feb 2023 21:09:46 +0000 (21:09 +0000)]
Suggest a socket read buffer of 4KiB to limit message count

...and switch the same in `lightning-net-tokio`

20 months agoDon't apply gossip backpressure to non-channel-announcing peers
Matt Corallo [Mon, 30 Jan 2023 17:56:46 +0000 (17:56 +0000)]
Don't apply gossip backpressure to non-channel-announcing peers

When we apply the new gossip-async-check backpressure on peer
connections, if a peer has never sent us a `channel_announcement`
at all, we really shouldn't delay reading their messages.

This does so by tracking, on a per-peer basis, whether they've sent
us a channel_announcement, and resetting that state whenever we're
not backlogged.

20 months agoApply backpressure when we have too many gossip checks in-flight
Matt Corallo [Sun, 22 Jan 2023 18:08:33 +0000 (18:08 +0000)]
Apply backpressure when we have too many gossip checks in-flight

Now that the `RoutingMessageHandler` can signal that it needs to
apply message backpressure, we implement it here in the
`PeerManager`. There's not much complicated here, aside from noting
that we need to add the ability to call `send_data` with no data
to indicate that reading should resume (and track when we may need
to make such calls when updating the routing-backpressure state).

20 months agoAllow `RoutingMessageHandler` to signal backpressure
Matt Corallo [Sun, 22 Jan 2023 05:12:45 +0000 (05:12 +0000)]
Allow `RoutingMessageHandler` to signal backpressure

Now that we allow `handle_channel_announcement` to (indirectly)
spawn async tasks which will complete later, we have to ensure it
can apply backpressure all the way up to the TCP socket to ensure
we don't end up with too many buffers allocated for UTXO
validation.

We do this by adding a new method to `RoutingMessageHandler` which
allows it to signal if there are "many" checks pending and
`channel_announcement` messages should be delayed. The actual
`PeerManager` implementation thereof is done in the next commit.

20 months agoForward gossip messages which were verified asynchronously
Matt Corallo [Sun, 22 Jan 2023 04:14:58 +0000 (04:14 +0000)]
Forward gossip messages which were verified asynchronously

Gossip messages which were verified against the chain
asynchronously should still be forwarded to peers, but must now go
out via a new `P2PGossipSync` parameter in the
`AccessResolver::resolve` method, allowing us to wire them up to
the `P2PGossipSync`'s `MessageSendEventsProvider` implementation.

20 months agoAdd the ability to broadcast gossip msgs via the event pipeline
Matt Corallo [Sun, 22 Jan 2023 03:41:28 +0000 (03:41 +0000)]
Add the ability to broadcast gossip msgs via the event pipeline

When we process gossip messages asynchronously we may find that we
want to forward a gossip message to a peer after we've returned
from the existing `handle_*` method. In order to do so, we need to
be able to send arbitrary loose gossip messages back to the
`PeerManager` via `MessageSendEvent`.

This commit modifies `MessageSendEvent` in order to support this.

20 months agoProcess `channel_update`/`node_announcement` async if needed
Matt Corallo [Tue, 7 Feb 2023 20:38:20 +0000 (20:38 +0000)]
Process `channel_update`/`node_announcement` async if needed

If we have a `channel_announcement` which is waiting on a UTXO
lookup before we can process it, and we receive a `channel_update`
or `node_announcement` for the same channel or a node which is a
part of the channel, we have to wait until the lookup completes
until we can decide if we want to accept the new message.

Here, we store the new message in the pending lookup state and
process it asynchronously like the original `channel_announcement`.

20 months agoTrack in-flight `channel_announcement` lookups and avoid duplicates
Matt Corallo [Wed, 8 Feb 2023 22:11:56 +0000 (22:11 +0000)]
Track in-flight `channel_announcement` lookups and avoid duplicates

If we receive two `channel_announcement`s for the same channel at
the same time, we shouldn't spawn a second UTXO lookup for an
identical message. This likely isn't too rare - if we start syncing
the graph from two peers at the same time, it isn't unlikely that
we'll end up with the same messages around the same time.

In order to avoid this we keep a hash map of all the pending
`channel_announcement` messages by SCID and simply ignore duplicate
message lookups.

20 months agoAdd an async resolution option to `ChainAccess::get_utxo`
Matt Corallo [Wed, 8 Feb 2023 22:06:11 +0000 (22:06 +0000)]
Add an async resolution option to `ChainAccess::get_utxo`

For those operating in an async environment, requiring
`ChainAccess::get_utxo` return information about the requested UTXO
synchronously is incredibly painful. Requesting information about a
random UTXO is likely to go over the network, and likely to be a
rather slow request.

Thus, here, we change the return type of `get_utxo` to have both a
synchronous and asynchronous form. The asynchronous form requires
the user construct a `AccessFuture` which they `clone` and pass
back to us. Internally, an `AccessFuture` has an `Arc` to the
`channel_announcement` message which we need to process. When the
user completes their lookup, they call `resolve` on their
`AccessFuture` which we pull the `channel_announcement` from and
then apply to the network graph.

20 months agoClean up `check_channel_announcement` style
Matt Corallo [Sat, 21 Jan 2023 03:51:22 +0000 (03:51 +0000)]
Clean up `check_channel_announcement` style

`check_channel_announcement` had long lines, a (very-)stale TODO
and confusing variable assignment, which is all cleaned up here.

20 months agoMove logic to check a `ChannelAnnouncement` to `gossip_checking`
Matt Corallo [Sat, 21 Jan 2023 03:48:46 +0000 (03:48 +0000)]
Move logic to check a `ChannelAnnouncement` to `gossip_checking`

This commit is deliberately move-only, though the code being moved
is somewhat crufty.

20 months agoUpdate the `gossip` and `router` module docs which were out of date
Matt Corallo [Tue, 24 Jan 2023 05:04:21 +0000 (05:04 +0000)]
Update the `gossip` and `router` module docs which were out of date

20 months agoMove `chain::Access` to `routing` and rename it `UtxoLookup`
Matt Corallo [Sat, 21 Jan 2023 03:28:35 +0000 (03:28 +0000)]
Move `chain::Access` to `routing` and rename it `UtxoLookup`

The `chain::Access` trait (and the `chain::AccessError` enum) is a
bit strange - it only really makes sense if users import it via the
`chain` module, otherwise they're left with a trait just called
`Access`. Worse, for bindings users its always just called
`Access`, in part because many downstream languages don't have a
mechanism to import a module and then refer to it.

Further, its stuck dangling in the `chain` top-level mod.rs file,
sitting in a module that doesn't use it at all (it's only used in
`routing::gossip`).

Instead, we give it its full name - `UtxoLookup` (and rename the
error enum `UtxoLookupError`) and put it in the a new
`routing::utxo` module, next to `routing::gossip`.

20 months agoMerge pull request #2022 from alecchendev/2023-02-cache-peer-nodeid
Matt Corallo [Wed, 8 Feb 2023 23:50:59 +0000 (23:50 +0000)]
Merge pull request #2022 from alecchendev/2023-02-cache-peer-nodeid

Store a cached `NodeId` for each `Peer`

20 months agoCache `NodeId` by converting `their_node_id` to tuple
Alec Chen [Wed, 8 Feb 2023 17:55:16 +0000 (11:55 -0600)]
Cache `NodeId` by converting `their_node_id` to tuple

This is done to avoid redundantly serializing peer node
ids when forwarding gossip messages in
`PeerManager::forward_broadcast_msg`.

20 months agoMerge pull request #2016 from alecchendev/2023-02-gossip-msg-pubkey-to-nodeid
Matt Corallo [Tue, 7 Feb 2023 20:18:19 +0000 (20:18 +0000)]
Merge pull request #2016 from alecchendev/2023-02-gossip-msg-pubkey-to-nodeid

Swap gossip message `PublicKey` for `NodeId`

20 months agoMerge pull request #2007 from valentinewallace/2023-02-remove-invpayer-add-utils
Matt Corallo [Tue, 7 Feb 2023 17:37:38 +0000 (17:37 +0000)]
Merge pull request #2007 from valentinewallace/2023-02-remove-invpayer-add-utils

20 months agoSwap `PublicKey` for `NodeId` in `UnsignedNodeAnnouncement`
Alec Chen [Mon, 6 Feb 2023 21:33:02 +0000 (15:33 -0600)]
Swap `PublicKey` for `NodeId` in `UnsignedNodeAnnouncement`

Also swaps `PublicKey` for `NodeId` in `get_next_node_announcement`
and `InitSyncTracker` to avoid unnecessary deserialization that came
from changing `UnsignedNodeAnnouncement`.

20 months agoSwap `PublicKey` for `NodeId` in `UnsignedChannelAnnouncement`
Alec Chen [Mon, 6 Feb 2023 20:41:18 +0000 (14:41 -0600)]
Swap `PublicKey` for `NodeId` in `UnsignedChannelAnnouncement`

Adds the macro `get_pubkey_from_node_id`
to parse `PublicKey`s back from `NodeId`s for signature
verification, as well as `make_funding_redeemscript_from_slices`
to avoid parsing back and forth between types.

20 months agoExpand ChannelManager::send_spontaneous_payment_with_retry docs
Valentine Wallace [Mon, 6 Feb 2023 21:30:51 +0000 (16:30 -0500)]
Expand ChannelManager::send_spontaneous_payment_with_retry docs

20 months agoRemove InvoicePayer and replace with util methods
Valentine Wallace [Fri, 3 Feb 2023 01:08:00 +0000 (20:08 -0500)]
Remove InvoicePayer and replace with util methods

21 months agoDon't pass `per_peer_state` to `claim_funds_from_hop`
Viktor Tigerström [Thu, 12 Jan 2023 19:05:35 +0000 (20:05 +0100)]
Don't pass `per_peer_state` to `claim_funds_from_hop`

21 months agoTrack if a peer is connected or not in `ChannelManager`
Matt Corallo [Fri, 27 Jan 2023 04:06:07 +0000 (04:06 +0000)]
Track if a peer is connected or not in `ChannelManager`

21 months agoMerge pull request #1873 from jurvis/jurvis/2022-11-expose-pending-payments
Matt Corallo [Fri, 3 Feb 2023 19:16:02 +0000 (19:16 +0000)]
Merge pull request #1873 from jurvis/jurvis/2022-11-expose-pending-payments

Expose pending payments through `ChannelManager`

21 months agoMerge pull request #2002 from valentinewallace/2023-01-keysend-retries
valentinewallace [Fri, 3 Feb 2023 19:09:00 +0000 (14:09 -0500)]
Merge pull request #2002 from valentinewallace/2023-01-keysend-retries

Support auto-retrying keysend payments in `ChannelManager`

21 months agoMerge pull request #1996 from valentinewallace/2023-01-migrate-payment-scoring
Matt Corallo [Fri, 3 Feb 2023 18:44:13 +0000 (18:44 +0000)]
Merge pull request #1996 from valentinewallace/2023-01-migrate-payment-scoring

Score payment paths from events in `BackgroundProcessor`

21 months agoExpose pending payments through ChannelManager
jurvis [Mon, 5 Dec 2022 01:24:08 +0000 (17:24 -0800)]
Expose pending payments through ChannelManager

Adds a new method, `list_recent_payments ` to `ChannelManager` that
returns an array of `RecentPaymentDetails` containing the payment
status (Fulfilled/Retryable/Abandoned) and its total amount across all
paths.

21 months agoScore payment paths in BackgroundProcessor
Valentine Wallace [Fri, 3 Feb 2023 16:25:20 +0000 (11:25 -0500)]
Score payment paths in BackgroundProcessor

21 months agoUse TestScorer in BackgroundProcessor testing
Valentine Wallace [Fri, 3 Feb 2023 16:14:53 +0000 (11:14 -0500)]
Use TestScorer in BackgroundProcessor testing

21 months agoSpontaneous payments: make preimage construction more concise
Valentine Wallace [Fri, 3 Feb 2023 15:44:31 +0000 (10:44 -0500)]
Spontaneous payments: make preimage construction more concise

21 months agoSupport spontaneous payment retries in ChannelManager
Valentine Wallace [Tue, 24 Jan 2023 17:15:40 +0000 (12:15 -0500)]
Support spontaneous payment retries in ChannelManager

21 months agoStore keysend preimage in outbound payments
Valentine Wallace [Tue, 24 Jan 2023 16:31:50 +0000 (11:31 -0500)]
Store keysend preimage in outbound payments

This sets us up for spontaneous payment retries in ChannelManager.

Currently, retrying spontaneous payments is broken in InvoicePayer because it
does not include the keysend preimage on retry.

21 months agoMerge pull request #1994 from TheBlueMatt/2023-01-1916-followups
Matt Corallo [Thu, 2 Feb 2023 23:05:07 +0000 (23:05 +0000)]
Merge pull request #1994 from TheBlueMatt/2023-01-1916-followups

1916 Followups Part 1

21 months agoReduce the code in the commitment_signed_dance macro 2023-01-1916-followups
Matt Corallo [Sat, 28 Jan 2023 05:25:19 +0000 (05:25 +0000)]
Reduce the code in the commitment_signed_dance macro

This should marginally reduce compile times for the tests by
reducing the total volume of code across the tests in the lightning
crate.

21 months agoUse only the failed amount when retrying payments, not the full amt
Matt Corallo [Sat, 28 Jan 2023 02:14:26 +0000 (02:14 +0000)]
Use only the failed amount when retrying payments, not the full amt

When we landed the initial in-`ChannelManager` payment retries, we
stored the `RouteParameters` in the payment info, and then re-use
it as-is for new payments. `RouteParameters` is intended to store
the information specific to the *route*, `PaymentParameters` exists
to store information specific to a payment.

Worse, because we don't recalculate the amount stored in the
`RouteParameters` before fetching a new route with it, we end up
attempting to retry the full payment amount, rather than only the
failed part.

This issue brought to you by having redundant data in
datastructures, part 5,001.