]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
20 months agoRemove `build-no-std` CI flag now that its always true
Matt Corallo [Wed, 15 Mar 2023 18:17:02 +0000 (18:17 +0000)]
Remove `build-no-std` CI flag now that its always true

20 months agoRemove `build-net-tokio` flag in CI since its now always true
Matt Corallo [Wed, 15 Mar 2023 18:16:01 +0000 (18:16 +0000)]
Remove `build-net-tokio` flag in CI since its now always true

20 months agoBump MSRV to 1.48
Matt Corallo [Wed, 15 Mar 2023 18:08:35 +0000 (18:08 +0000)]
Bump MSRV to 1.48

1.48.0 was released at the end of 2020, nearly 2.5 years ago. It
has been the rustc available on Debian stable since bullseye,
released in 2021. supporting Debian oldstable for more than a year
seems more than sufficient time to give Debian folks to upgrade,
and bullseye is set to become `oldstable` later this year with the
release of `bookworm`, likely this summer.

This also allows us to clean up our MSRV substantially, having a
single MSRV across our crates rather than a number of separate
ones. Sadly, windows already requires 1.49.

20 months agoMerge pull request #2045 from wpaulino/fix-broken-commitment-test-vectors
Matt Corallo [Tue, 14 Mar 2023 19:01:59 +0000 (19:01 +0000)]
Merge pull request #2045 from wpaulino/fix-broken-commitment-test-vectors

Update commitment test vectors

20 months agoMerge pull request #2063 from valentinewallace/2023-03-remove-paymentpathfailed-retry
Matt Corallo [Mon, 13 Mar 2023 23:01:38 +0000 (23:01 +0000)]
Merge pull request #2063 from valentinewallace/2023-03-remove-paymentpathfailed-retry

Remove `PaymentPathFailed::retry`

20 months agoMerge pull request #2099 from tnull/2023-03-pin-ntapi-on-windows
Matt Corallo [Mon, 13 Mar 2023 17:44:08 +0000 (17:44 +0000)]
Merge pull request #2099 from tnull/2023-03-pin-ntapi-on-windows

Pin tokio to 1.26 on windows to fix CI

20 months agoMerge pull request #2091 from tnull/2023-03-expose-impl-writeable-tlv-based-enum...
Matt Corallo [Mon, 13 Mar 2023 16:32:19 +0000 (16:32 +0000)]
Merge pull request #2091 from tnull/2023-03-expose-impl-writeable-tlv-based-enum-common

20 months agoEnsure payment_params present when checking is_retryable_now
Valentine Wallace [Tue, 7 Mar 2023 22:13:57 +0000 (17:13 -0500)]
Ensure payment_params present when checking is_retryable_now

20 months agoRemove payment_params from send_payent_along_path
Valentine Wallace [Wed, 1 Mar 2023 17:11:35 +0000 (12:11 -0500)]
Remove payment_params from send_payent_along_path

It's unused since it no longer inserts it into HTLCSource

20 months agoRemove HTLCSource::OutboundRoute::payment_params
Valentine Wallace [Wed, 1 Mar 2023 17:02:52 +0000 (12:02 -0500)]
Remove HTLCSource::OutboundRoute::payment_params

No longer used since it is no longer used to construct PaymentPathFailed events

20 months agoRemove PaymentPathFailed::retry
Valentine Wallace [Wed, 1 Mar 2023 16:46:15 +0000 (11:46 -0500)]
Remove PaymentPathFailed::retry

We now support automatic retries in ChannelManager and no longer support manual
retries, so the field is useless.

20 months agoPin tokio to 1.26 on windows to fix CI
Elias Rohrer [Fri, 10 Mar 2023 21:24:39 +0000 (22:24 +0100)]
Pin tokio to 1.26 on windows to fix CI

20 months agoMerge pull request #2092 from TheBlueMatt/2023-03-find-route-id
Matt Corallo [Thu, 9 Mar 2023 21:49:09 +0000 (21:49 +0000)]
Merge pull request #2092 from TheBlueMatt/2023-03-find-route-id

Correct `outbound_payment` route-fetch calls to pass the hash + ID

20 months agoFurther simplify the `outbound_payments` failure macro 2023-03-find-route-id
Matt Corallo [Thu, 9 Mar 2023 20:06:23 +0000 (20:06 +0000)]
Further simplify the `outbound_payments` failure macro

20 months agoCorrect `outbound_payment` route-fetch calls to pass the hash + ID
Matt Corallo [Thu, 9 Mar 2023 19:23:58 +0000 (19:23 +0000)]
Correct `outbound_payment` route-fetch calls to pass the hash + ID

`Route::get_route_with_id` exists to provide users payment-specific
data when fetching a route, however we were failing to call it when
we have such info, opting for the simple `get_route` instead. This
defeats the purpose of the additional-metadata method, which we
swap to using here.

20 months agoExpose `_impl_writeable_tlv_based_enum_common` macro
Elias Rohrer [Thu, 9 Mar 2023 14:47:19 +0000 (15:47 +0100)]
Expose `_impl_writeable_tlv_based_enum_common` macro

As it isn't currently, and it's used by the exposed
`impl_writeable_tlv_based_enum`.

20 months agoMerge pull request #2079 from tnull/2023-03-add-list-channel-by-counterparty
Matt Corallo [Wed, 8 Mar 2023 21:55:32 +0000 (21:55 +0000)]
Merge pull request #2079 from tnull/2023-03-add-list-channel-by-counterparty

20 months agoMerge pull request #2078 from AdvaitPote/bigsize_tests
Matt Corallo [Wed, 8 Mar 2023 20:03:58 +0000 (20:03 +0000)]
Merge pull request #2078 from AdvaitPote/bigsize_tests

Added encoding and decoding tests for BigSize

20 months agoMerge pull request #2085 from tnull/2023-03-introduce-async-https-feature
Matt Corallo [Wed, 8 Mar 2023 19:41:10 +0000 (19:41 +0000)]
Merge pull request #2085 from tnull/2023-03-introduce-async-https-feature

Support HTTPS Esplora endpoints in `lightning-transaction-sync` via new feature

20 months agoencoding/decoding tests added for BigSize
Advait [Tue, 7 Mar 2023 07:10:46 +0000 (12:40 +0530)]
encoding/decoding tests added for BigSize

removed unnecessary debugging line

using io::Cursor in place of the std one

encoding/decoding tests added for BigSize

made the code concise

encoding/decoding tests added for BigSize

20 months agoRemove unnecessary logging imports
Elias Rohrer [Wed, 8 Mar 2023 11:11:51 +0000 (12:11 +0100)]
Remove unnecessary logging imports

20 months agoSupport HTTPS Esplora endpoints via new feature
Elias Rohrer [Wed, 8 Mar 2023 11:05:57 +0000 (12:05 +0100)]
Support HTTPS Esplora endpoints via new feature

To support HTTPS endpoints, the async HTTP library `reqwest` needs one of
the `-tls` features enabled. While the users could specify this in their
own cargo dependencies, we here provide a new `esplora-async-https`
feature for conveinience.

20 months agoMerge pull request #2084 from zpv/fix-changelog-all-paths-failed-typo
valentinewallace [Wed, 8 Mar 2023 16:59:36 +0000 (11:59 -0500)]
Merge pull request #2084 from zpv/fix-changelog-all-paths-failed-typo

Fix `all_paths_failed` typo in changelog

20 months agoMerge pull request #2081 from TheBlueMatt/2023-03-future-poll-panic
Matt Corallo [Wed, 8 Mar 2023 15:49:43 +0000 (15:49 +0000)]
Merge pull request #2081 from TheBlueMatt/2023-03-future-poll-panic

20 months agoFix `Event::PaymentPathFailed` typo in changelog
Steven Zhao [Wed, 8 Mar 2023 08:23:09 +0000 (00:23 -0800)]
Fix `Event::PaymentPathFailed` typo in changelog

20 months agoFix `all_paths_failed` typo in changelog
Steven [Wed, 8 Mar 2023 00:51:35 +0000 (16:51 -0800)]
Fix `all_paths_failed` typo in changelog

20 months agoAdd `list_channels_by_counterparty` method
Elias Rohrer [Tue, 7 Mar 2023 10:19:41 +0000 (11:19 +0100)]
Add `list_channels_by_counterparty` method

While we already provide a `list_channels` method, it could result in
quite a large `Vec<ChannelDetails>`. Here, we provide the means to query
our channels by `counterparty_node_id` and DRY up the code.

20 months agoAvoid `poll`ing completed futures in the `background-processor` 2023-03-future-poll-panic
Matt Corallo [Tue, 7 Mar 2023 18:06:12 +0000 (18:06 +0000)]
Avoid `poll`ing completed futures in the `background-processor`

`poll`ing completed futures invokes undefined behavior in Rust
(panics, etc, obviously not memory corruption as its not unsafe).
Sadly, in our futures-based version of
`lightning-background-processor` we have one case where we can
`poll` a completed future - if the timer for the network graph
prune + persist completes without a network graph to prune +
persist we'll happily poll the same future over and over again,
likely panicing in user code.

20 months agoMerge pull request #2049 from douglaz/run-clippy-fix
Matt Corallo [Mon, 6 Mar 2023 21:28:38 +0000 (21:28 +0000)]
Merge pull request #2049 from douglaz/run-clippy-fix

Run clippy fix

20 months agoRun clippy --fix and cleanup code
Allan Douglas R. de Oliveira [Mon, 27 Feb 2023 18:24:57 +0000 (18:24 +0000)]
Run clippy --fix and cleanup code

20 months agoUpdate same amount and preimage test vector
Wilmer Paulino [Wed, 22 Feb 2023 19:46:21 +0000 (11:46 -0800)]
Update same amount and preimage test vector

The amount for HTLC #6 was updated in the spec's test vectors, but the
"same amount and preimage" test vector itself was not updated, even
though the new HTLC amount resulted in a different commitment
transaction, and thus, different signatures.

20 months agoAdd missing test vector for anchors_zero_fee_htlc_tx
Wilmer Paulino [Wed, 22 Feb 2023 19:45:43 +0000 (11:45 -0800)]
Add missing test vector for anchors_zero_fee_htlc_tx

Tests the case where only one anchor output exists for the funder in the
commitment transaction due to the remote having a dust balance (in this
case, 0).

20 months agoMerge pull request #2074 from TheBlueMatt/2023-03-fix-threaded-test
Matt Corallo [Mon, 6 Mar 2023 17:26:59 +0000 (17:26 +0000)]
Merge pull request #2074 from TheBlueMatt/2023-03-fix-threaded-test

Make `fuzz_threaded_connections` more robust

20 months agoMerge pull request #2028 from TheBlueMatt/2023-02-macros-for-wilmer
Matt Corallo [Mon, 6 Mar 2023 16:56:08 +0000 (16:56 +0000)]
Merge pull request #2028 from TheBlueMatt/2023-02-macros-for-wilmer

Reduce macro usage in tests

20 months agoMerge pull request #1497 from TheBlueMatt/2022-05-no-default-opt
Matt Corallo [Sat, 4 Mar 2023 02:01:49 +0000 (02:01 +0000)]
Merge pull request #1497 from TheBlueMatt/2022-05-no-default-opt

Disable LTO builds in tests (and bump deps to -O2)

20 months agoMake `fuzz_threaded_connections` more robust 2023-03-fix-threaded-test
Matt Corallo [Sat, 4 Mar 2023 01:16:57 +0000 (01:16 +0000)]
Make `fuzz_threaded_connections` more robust

In `fuzz_threaded_connections`, if one thread is being run while
another is starved, and the running thread manages to call
`timer_tick_ocurred` twice after the starved thread constructs the
inbound connection but before it delivers the first bytes, we'll
receive an immediate error and `unwrap` it, causing failure.

The fix is trivial, simply remove the unwrap and return if we're
already disconnected when we do the initial read.

While we're here, we also reduce the frequency of the
`timer_tick_ocurred` calls to give us a chance to occasionally
deliver some additional messages.

Fixes #2073

20 months agoMerge pull request #2065 from TheBlueMatt/2023-02-0.0.114 0.0.114-bindings v0.0.114
Matt Corallo [Sat, 4 Mar 2023 01:00:06 +0000 (01:00 +0000)]
Merge pull request #2065 from TheBlueMatt/2023-02-0.0.114

Cut 0.0.114

20 months agoUpdate crate versions to 0.0.114/invoice 0.22 2023-02-0.0.114
Matt Corallo [Wed, 1 Mar 2023 05:06:03 +0000 (05:06 +0000)]
Update crate versions to 0.0.114/invoice 0.22

20 months agoAdd release notes for 0.0.114
Matt Corallo [Wed, 1 Mar 2023 05:03:21 +0000 (05:03 +0000)]
Add release notes for 0.0.114

20 months agoMerge pull request #2072 from jkczyz/2023-01-fix-scoring-div-by-zero
Matt Corallo [Sat, 4 Mar 2023 00:06:29 +0000 (00:06 +0000)]
Merge pull request #2072 from jkczyz/2023-01-fix-scoring-div-by-zero

Fix division by zero in `ProbabilisticScorer`

20 months agoMerge pull request #2071 from TheBlueMatt/2023-01-fix-fast-extra-ready-panic
Matt Corallo [Fri, 3 Mar 2023 23:32:42 +0000 (23:32 +0000)]
Merge pull request #2071 from TheBlueMatt/2023-01-fix-fast-extra-ready-panic

Fix panic on receiving `channel_ready` after 1st commitment update

20 months agoUse ProbabilisticScorer in router fuzzing, to cover overflows there
Matt Corallo [Wed, 18 Jan 2023 00:01:15 +0000 (18:01 -0600)]
Use ProbabilisticScorer in router fuzzing, to cover overflows there

20 months agoGuard against division by zero in scorer
Jeffrey Czyz [Fri, 6 Jan 2023 04:00:31 +0000 (22:00 -0600)]
Guard against division by zero in scorer

Since a node may announce that the htlc_maximum_msat of a channel is
zero, adding one to the denominator in the bucket formulas will prevent
the panic from ever happening. While the routing algorithm may never
select such a channel to score, this precaution may still be useful in
case the algorithm changes or if the scorer is used with a different
routing algorithm.

20 months agoUpdate scoring history buckets when no change
Jeffrey Czyz [Tue, 17 Jan 2023 23:36:03 +0000 (17:36 -0600)]
Update scoring history buckets when no change

Even when there is no change in min/max liquidity knowledge, tracking
should still be updated to include the additional data point.

20 months agoMerge pull request #2068 from jkczyz/2023-03-doc-fixes
Matt Corallo [Fri, 3 Mar 2023 22:19:59 +0000 (22:19 +0000)]
Merge pull request #2068 from jkczyz/2023-03-doc-fixes

Doc and build warning fixes

20 months agoMerge pull request #2069 from TheBlueMatt/2023-03-no-tx-sync-auto-std
Matt Corallo [Fri, 3 Mar 2023 22:19:46 +0000 (22:19 +0000)]
Merge pull request #2069 from TheBlueMatt/2023-03-no-tx-sync-auto-std

Do not auto-select the lightning `std` feature from tx-sync crate

20 months agoMerge pull request #2070 from TheBlueMatt/2023-03-get-key
Matt Corallo [Fri, 3 Mar 2023 22:19:37 +0000 (22:19 +0000)]
Merge pull request #2070 from TheBlueMatt/2023-03-get-key

Expose the node secret key in `{Phantom,}KeysManager`

20 months agoFix build warnings
Jeffrey Czyz [Fri, 3 Mar 2023 16:54:42 +0000 (10:54 -0600)]
Fix build warnings

20 months agoAdd more documentation about BlockSourceError
Jeffrey Czyz [Fri, 3 Mar 2023 16:09:58 +0000 (10:09 -0600)]
Add more documentation about BlockSourceError

Some BlockSource implementations provide more error details. Document
this in case users want to examine it further.

20 months agoExpose the node secret key in `{Phantom,}KeysManager` 2023-03-get-key
Matt Corallo [Fri, 3 Mar 2023 20:03:57 +0000 (20:03 +0000)]
Expose the node secret key in `{Phantom,}KeysManager`

When we removed the private keys from the signing interface we
forgot to re-add them in the public interface of our own
implementations, which users may need.

20 months agoMerge pull request #2067 from benthecarman/fix-typos
Matt Corallo [Fri, 3 Mar 2023 20:00:12 +0000 (20:00 +0000)]
Merge pull request #2067 from benthecarman/fix-typos

Fix typos in lightning-transaction-sync

20 months agoMerge pull request #2066 from TheBlueMatt/2023-02-no-enum-refs
Matt Corallo [Fri, 3 Mar 2023 19:44:40 +0000 (19:44 +0000)]
Merge pull request #2066 from TheBlueMatt/2023-02-no-enum-refs

Pass `FailureCode` to `fail_htlc_backwards` by ownership

20 months agoMerge pull request #2048 from TheBlueMatt/2023-02-send-persist-order-a
Matt Corallo [Fri, 3 Mar 2023 19:37:31 +0000 (19:37 +0000)]
Merge pull request #2048 from TheBlueMatt/2023-02-send-persist-order-a

Track claimed outbound HTLCs in ChannelMonitors

20 months agoDo not auto-select the lightning `std` feature from tx-sync crate 2023-03-no-tx-sync-auto-std
Matt Corallo [Fri, 3 Mar 2023 05:14:04 +0000 (05:14 +0000)]
Do not auto-select the lightning `std` feature from tx-sync crate

We have some downstream folks who are using LDK in wasm compiled
via the normal rust wasm path. To ensure nothing breaks they want
to use `no-std` on the lightning crate, disabling time calls as
those panic. However, the HTTP logic in
`lightning-transaction-sync` gets automatically stubbed out by the
HTTP client crates when targeting wasm via `wasm_bindgen`, so it
works fine despite the std restrictions.

In order to make both work, `lightning-transaction-sync` can remain
`std`, but needs to not automatically enable the `std` flag on the
`lightning` crate, ie by setting `default-features = false`. We do
so here.

20 months agoPass `FailureCode` to `fail_htlc_backwards` by ownership 2023-02-no-enum-refs
Matt Corallo [Fri, 3 Mar 2023 01:24:24 +0000 (01:24 +0000)]
Pass `FailureCode` to `fail_htlc_backwards` by ownership

`FaliureCode` is a trivial enum with no body, so we shouldn't be
passing it by reference. Its sufficiently strange that the Java
bindings aren't happy with it, which is fine, we should just fix it
here.

20 months agoRemove unused compat block in `provide_latest_holder_commitment_tx` 2023-02-send-persist-order-a
Matt Corallo [Wed, 1 Mar 2023 01:31:55 +0000 (01:31 +0000)]
Remove unused compat block in `provide_latest_holder_commitment_tx`

20 months agoTrack claimed outbound HTLCs in ChannelMonitors
Matt Corallo [Wed, 22 Feb 2023 02:40:59 +0000 (02:40 +0000)]
Track claimed outbound HTLCs in ChannelMonitors

When we receive an update_fulfill_htlc message, we immediately try
to "claim" the HTLC against the HTLCSource. If there is one, this
works great, we immediately generate a `ChannelMonitorUpdate` for
the corresponding inbound HTLC and persist that before we ever get
to processing our counterparty's `commitment_signed` and persisting
the corresponding `ChannelMonitorUpdate`.

However, if there isn't one (and this is the first successful HTLC
for a payment we sent), we immediately generate a `PaymentSent`
event and queue it up for the user. Then, a millisecond later, we
receive the `commitment_signed` from our peer, removing the HTLC
from the latest local commitment transaction as a side-effect of
the `ChannelMonitorUpdate` applied.

If the user has processed the `PaymentSent` event by that point,
great, we're done. However, if they have not, and we crash prior to
persisting the `ChannelManager`, on startup we get confused about
the state of the payment. We'll force-close the channel for being
stale, and see an HTLC which was removed and is no longer present
in the latest commitment transaction (which we're broadcasting).
Because we claim corresponding inbound HTLCs before updating a
`ChannelMonitor`, we assume such HTLCs have failed - attempting to
fail after having claimed should be a noop. However, in the
sent-payment case we now generate a `PaymentFailed` event for the
user, allowing an HTLC to complete without giving the user a
preimage.

Here we address this issue by storing the payment preimages for
claimed outbound HTLCs in the `ChannelMonitor`, in addition to the
existing inbound HTLC preimages already stored there. This allows
us to fix the specific issue described by checking for a preimage
and switching the type of event generated in response. In addition,
it reduces the risk of future confusion by ensuring we don't fail
HTLCs which were claimed but not fully committed to before a crash.

It does not, however, full fix the issue here - because the
preimages are removed after the HTLC has been fully removed from
available commitment transactions if we are substantially delayed
in persisting the `ChannelManager` from the time we receive the
`update_fulfill_htlc` until after a full commitment signed dance
completes we may still hit this issue. The full fix for this issue
is to delay the persistence of the `ChannelMonitorUpdate` until
after the `PaymentSent` event has been processed. This avoids the
issue entirely, ensuring we process the event before updating the
`ChannelMonitor`, the same as we ensure the upstream HTLC has been
claimed before updating the `ChannelMonitor` for forwarded
payments.

The full solution will be implemented in a later work, however this
change still makes sense at that point as well - if we were to
delay the initial `commitment_signed` `ChannelMonitorUpdate` util
after the `PaymentSent` event has been processed (which likely
requires a database update on the users' end), we'd hold our
`commitment_signed` + `revoke_and_ack` response for two DB writes
(i.e. `fsync()` calls), making our commitment transaction
processing a full `fsync` slower. By making this change first, we
can instead delay the `ChannelMonitorUpdate` from the
counterparty's final `revoke_and_ack` message until the event has
been processed, giving us a full network roundtrip to do so and
avoiding delaying our response as long as an `fsync` is faster than
a network roundtrip.

20 months agoReference `Router` in `ChannelManager` docs
Jeffrey Czyz [Fri, 3 Mar 2023 15:45:11 +0000 (09:45 -0600)]
Reference `Router` in `ChannelManager` docs

20 months agoDRY up historical bucket_idx calculation
Jeffrey Czyz [Fri, 6 Jan 2023 02:13:12 +0000 (20:13 -0600)]
DRY up historical bucket_idx calculation

20 months agoFix scorer panic when available capacity is zero
Jeffrey Czyz [Thu, 5 Jan 2023 17:50:24 +0000 (11:50 -0600)]
Fix scorer panic when available capacity is zero

ProbabilisticScorer takes a ChannelUsage when computing a penalty for a
channel. The formula for calculating the liquidity penalty reduces the
maximum capacity by the amount of in-flight HTLCs (available capacity)
and adds one to prevent division by zero.

However, since the available capacity is passed to
DirectedChannelLiquidity as the capacity, other penalty formulas may use
the available (i.e., reduced) capacity inadvertently. In practice, this
has two ramifications for the historical liquidity penalty computation:

1. The bucket formula doesn't have a consistent denominator for a given
   channel.
2. The bucket formula may divide by zero when the in-flight HTLC amount
   equals or exceeds the effective capacity.

Fixing this involves only using the available capacity when appropriate.

20 months agoFix typos in lightning-transaction-sync
benthecarman [Fri, 3 Mar 2023 05:36:54 +0000 (23:36 -0600)]
Fix typos in lightning-transaction-sync

20 months agoMerge pull request #2060 from TheBlueMatt/2023-02-peers-disconnect-consistency
Matt Corallo [Thu, 2 Mar 2023 23:44:23 +0000 (23:44 +0000)]
Merge pull request #2060 from TheBlueMatt/2023-02-peers-disconnect-consistency

Remove peers from the node_id_to_descriptor even without init

20 months agoAvoid removing stale preimages when hashes collide in fuzzing
Matt Corallo [Thu, 2 Mar 2023 04:48:14 +0000 (04:48 +0000)]
Avoid removing stale preimages when hashes collide in fuzzing

20 months agoFuzz rapid peer connection/disconnections in threads 2023-02-peers-disconnect-consistency
Matt Corallo [Thu, 2 Mar 2023 06:27:08 +0000 (06:27 +0000)]
Fuzz rapid peer connection/disconnections in threads

This test fails on the bug fixed two commits ago with the
additional assertions in the previous commit.

20 months agoImprove `PeerHandler` debug_assertions and checks
Matt Corallo [Tue, 28 Feb 2023 21:38:29 +0000 (21:38 +0000)]
Improve `PeerHandler` debug_assertions and checks

This removes two panics from `PeerHandler` which can trivially be
`debug_assert!(false); return Err;`s, and adds another
`debug_assertion` on internal state consistency during disconnect.

20 months agoMerge pull request #2064 from TheBlueMatt/2023-03-debug-futures
Wilmer Paulino [Thu, 2 Mar 2023 19:49:54 +0000 (11:49 -0800)]
Merge pull request #2064 from TheBlueMatt/2023-03-debug-futures

Make waking after a future completes propagates to the next future

20 months agoMerge pull request #2057 from johncantrell97/rpc-error-code
Jeffrey Czyz [Thu, 2 Mar 2023 14:26:29 +0000 (08:26 -0600)]
Merge pull request #2057 from johncantrell97/rpc-error-code

Surface bitcoind rpc error information

20 months agoMake waking after a future completes propagates to the next future 2023-03-debug-futures
Matt Corallo [Thu, 2 Mar 2023 07:50:16 +0000 (07:50 +0000)]
Make waking after a future completes propagates to the next future

In our `wakers`, if we first `notify` a future, which is then
`poll`ed complete, and then `notify` the same waker again before a
new future is fetched, that new future will be marked as
non-complete initially and wait for a third `notify`.

The fix is luckily rather trivial, when we `notify` a future, if it
is completed immediately, simply wipe the future state so that we
look at the pending-notify flag when we generate the next future.

20 months agoMerge pull request #2061 from TheBlueMatt/2023-02-114-upstream-bindings
Matt Corallo [Wed, 1 Mar 2023 18:57:12 +0000 (18:57 +0000)]
Merge pull request #2061 from TheBlueMatt/2023-02-114-upstream-bindings

`C-not exported` tags for 0.0.114

20 months agoReduce macro contents in `expect_pending_htlcs_forwardable*` macros 2023-02-macros-for-wilmer
Matt Corallo [Fri, 10 Feb 2023 20:38:14 +0000 (20:38 +0000)]
Reduce macro contents in `expect_pending_htlcs_forwardable*` macros

The `expect_pending_htlcs_forwardable*` macros don't need to be
macros so here we move much of the logic in them to a function and
leave the macro in place to avoid touching every line of code in
the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 301,915 LoC to 295,294 LoC.

20 months agoReplace `check_closed_event` macro with a function
Matt Corallo [Fri, 10 Feb 2023 20:17:16 +0000 (20:17 +0000)]
Replace `check_closed_event` macro with a function

The `check_closed_event!()` macro has no reason to be a macro so
here we move its logic to a function and leave the macro in place
to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 309,522 LoC to 301,915 LoC.

20 months agoReplace `check_closed_broadcast` macro with a function
Matt Corallo [Fri, 10 Feb 2023 20:07:54 +0000 (20:07 +0000)]
Replace `check_closed_broadcast` macro with a function

The `check_closed_broadcast!()` macro has no reason to be a macro
so here we move its logic to a function and leave the macro in
place to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 313,312 LoC to 309,522 LoC.

20 months agoMove parts of the `check_spends` macro into a function
Matt Corallo [Fri, 10 Feb 2023 19:57:20 +0000 (19:57 +0000)]
Move parts of the `check_spends` macro into a function

While we cannot move the entire `check_spends` macro into a
function, we can move parts of it out, which we do here.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 316,856 LoC to 313,312 LoC.

20 months agoReplace `get_htlc_update_msgs` macro with a function
Matt Corallo [Fri, 10 Feb 2023 19:57:00 +0000 (19:57 +0000)]
Replace `get_htlc_update_msgs` macro with a function

The `get_htlc_update_msgs!()` macro has no reason to be a macro
so here we move its logic to a function and leave the macro in
place to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 321,985 LoC to 316,856 LoC.

20 months agoReplace `get_err_msg` macro with a function
Matt Corallo [Fri, 10 Feb 2023 19:56:42 +0000 (19:56 +0000)]
Replace `get_err_msg` macro with a function

The `get_err_msg!()` macro has no reason to be a macro so here we
move its logic to a function and leave the macro in place to avoid
touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 322,183 LoC to 321,985 LoC.

20 months agoReplace `get_revoke_commit_msgs` macro with a function
Matt Corallo [Fri, 10 Feb 2023 19:39:09 +0000 (19:39 +0000)]
Replace `get_revoke_commit_msgs` macro with a function

The `get_revoke_commit_msgs!()` macro has no reason to be a macro
so here we move its logic to a function and leave the macro in
place to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 324,763 LoC to 322,183 LoC.

20 months agoReplace `get_route` macro with a function
Matt Corallo [Fri, 10 Feb 2023 19:29:13 +0000 (19:29 +0000)]
Replace `get_route` macro with a function

The `get_route!()` macro has no reason to be a macro so here we
move its logic to a function and leave the macro in place to
avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 326,588 LoC to 324,763 LoC.

20 months agoReplace `get_payment_preimage_hash` with a function
Matt Corallo [Fri, 10 Feb 2023 19:08:39 +0000 (19:08 +0000)]
Replace `get_payment_preimage_hash` with a function

The `get_payment_preimage_hash!()` macro has no reason to be a
macro so here we move its logic to a function and leave the macro
in place to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 329,119 LoC to 326,588 LoC.

20 months agoReplace `check_added_monitors` with a function
Matt Corallo [Fri, 10 Feb 2023 18:54:03 +0000 (18:54 +0000)]
Replace `check_added_monitors` with a function

The `check_added_monitors!()` macro has no reason to be a macro so
here we move its logic to a function and leave the macro in place
to avoid touching every line of code in the tests.

This reduces the `--profile=test --lib` `Zpretty=expanded` code
size from 338,710 LoC to 329,119 LoC.

20 months agoTag types used for the TLV macros with `(C-not exported)` 2023-02-114-upstream-bindings
Matt Corallo [Tue, 28 Feb 2023 19:59:34 +0000 (19:59 +0000)]
Tag types used for the TLV macros with `(C-not exported)`

Obviously bindings users can't use the Rust TLV-implementation
macros, so there's no reason to export typsed used exclusively by
them.

20 months agoMark `IndexedMap` types as `(C-not exported)`
Matt Corallo [Tue, 28 Feb 2023 19:42:31 +0000 (19:42 +0000)]
Mark `IndexedMap` types as `(C-not exported)`

While we could try to expose the type explicitly, we already have
alternative accessors for bindings, and mapping `Hash`, `Ord` and
the other requirements for `IndexedMap` would be a good chunk of
additional work.

20 months agoRemove peers from the `node_id_to_descriptor` even without init
Matt Corallo [Tue, 28 Feb 2023 21:28:13 +0000 (21:28 +0000)]
Remove peers from the `node_id_to_descriptor` even without init

When a peer has finished the noise handshake, but has not yet
completed the lightning `Init`-based handshake, they will be
present in the `node_id_to_descriptor` set, even though
`Peer::handshake_complete()` returns false. Thus, when we go to
disconnect such a peer, we must ensure that we remove it from the
descriptor set as well.

Failing to do so caused an `Inconsistent peers set state!` panic in
the C bindings network handler.

20 months agoMerge pull request #2046 from TheBlueMatt/2023-02-rgs-robust-and-log
Wilmer Paulino [Tue, 28 Feb 2023 19:36:04 +0000 (11:36 -0800)]
Merge pull request #2046 from TheBlueMatt/2023-02-rgs-robust-and-log

Do not fail to apply RGS updates for removed channels

20 months agoSurface bitcoind rpc error code
John Cantrell [Tue, 28 Feb 2023 16:39:29 +0000 (11:39 -0500)]
Surface bitcoind rpc error code

Users of the RpcClient had no way to access the error code
returned by bitcoind's rpc.  We embed a new RpcError struct
as the inner error for the returned io::Error. Users can access
both the code and the message using this inner struct.

20 months agoAdd some basic logging to Rapid Gossip Sync processing 2023-02-rgs-robust-and-log
Matt Corallo [Thu, 23 Feb 2023 19:20:41 +0000 (19:20 +0000)]
Add some basic logging to Rapid Gossip Sync processing

20 months agoMake log macros more usable outside of the `lightning` crate
Matt Corallo [Thu, 23 Feb 2023 19:20:09 +0000 (19:20 +0000)]
Make log macros more usable outside of the `lightning` crate

... by using explicit paths rather than requiring imports.

20 months agoDo not fail to apply RGS updates for removed channels
Matt Corallo [Thu, 23 Feb 2023 19:06:21 +0000 (19:06 +0000)]
Do not fail to apply RGS updates for removed channels

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

20 months agoMerge pull request #2006 from TheBlueMatt/2023-02-no-recursive-read-locks v0.0.114-beta
Wilmer Paulino [Tue, 28 Feb 2023 08:24:16 +0000 (00:24 -0800)]
Merge pull request #2006 from TheBlueMatt/2023-02-no-recursive-read-locks

Refuse recursive read locks

20 months agoMerge pull request #2015 from TheBlueMatt/2023-02-no-dumb-redundant-fields
Matt Corallo [Tue, 28 Feb 2023 07:55:56 +0000 (07:55 +0000)]
Merge pull request #2015 from TheBlueMatt/2023-02-no-dumb-redundant-fields

20 months agoMake sure individual mutexes are constructed on different lines 2023-02-no-recursive-read-locks
Matt Corallo [Sun, 26 Feb 2023 20:22:28 +0000 (20:22 +0000)]
Make sure individual mutexes are constructed on different lines

Our lockdep logic (on Windows) identifies a mutex based on which
line it was constructed on. Thus, if we have two mutexes
constructed on the same line it will generate false positives.

20 months agoExport RUST_BACKTRACE=1 in `--feature backtrace` CI test
Matt Corallo [Fri, 24 Feb 2023 20:29:45 +0000 (20:29 +0000)]
Export RUST_BACKTRACE=1 in `--feature backtrace` CI test

as this test often fails on windows which is hard to debug locally
for most contributors.

20 months agoDisallow taking two instances of the same mutex at the same time
Matt Corallo [Wed, 22 Feb 2023 22:54:38 +0000 (22:54 +0000)]
Disallow taking two instances of the same mutex at the same time

Taking two instances of the same mutex may be totally fine, but it
requires a total lockorder that we cannot (trivially) check. Thus,
its generally unsafe to do if we can avoid it.

To discourage doing this, here we default to panicing on such locks
in our lockorder tests, with a separate lock function added that is
clearly labeled "unsafe" to allow doing so when we can guarantee a
total lockorder.

This requires adapting a number of sites to the new API, including
fixing a bug this turned up in `ChannelMonitor`'s `PartialEq` where
no lockorder was guaranteed.

20 months agoRefuse recursive read locks in lockorder testing
Matt Corallo [Thu, 2 Feb 2023 22:38:54 +0000 (22:38 +0000)]
Refuse recursive read locks in lockorder testing

Our existing lockorder tests assume that a read lock on a thread
that is already holding the same read lock is totally fine. This
isn't at all true. The `std` `RwLock` behavior is
platform-dependent - on most platforms readers can starve writers
as readers will never block for a pending writer. However, on
platforms where this is not the case, one thread trying to take a
write lock may deadlock with another thread that both already has,
and is attempting to take again, a read lock.

Worse, our in-tree `FairRwLock` exhibits this behavior explicitly
on all platforms to avoid the starvation issue.

Thus, we shouldn't have any special handling for allowing recursive
read locks, so we simply remove it here.

20 months agoDon't `per_peer_state` read locks recursively in monitor updating
Matt Corallo [Wed, 22 Feb 2023 22:10:46 +0000 (22:10 +0000)]
Don't `per_peer_state` read locks recursively in monitor updating

When handling a `ChannelMonitor` update via the new
`handle_new_monitor_update` macro, we always call the macro with
the `per_peer_state` read lock held and have the macro drop the
per-peer state lock. Then, when handling the resulting updates, we
may take the `per_peer_state` read lock again in another function.

In a coming commit, recursive read locks will be disallowed, so we
have to drop the `per_peer_state` read lock before calling
additional functions in `handle_new_monitor_update`, which we do
here.

20 months agoExpect callers to hold read locks before `channel_monitor_updated`
Matt Corallo [Fri, 3 Feb 2023 00:46:50 +0000 (00:46 +0000)]
Expect callers to hold read locks before `channel_monitor_updated`

Our existing lockorder tests assume that a read lock on a thread
that is already holding the same read lock is totally fine. This
isn't at all true. The `std` `RwLock` behavior is
platform-dependent - on most platforms readers can starve writers
as readers will never block for a pending writer. However, on
platforms where this is not the case, one thread trying to take a
write lock may deadlock with another thread that both already has,
and is attempting to take again, a read lock.

Worse, our in-tree `FairRwLock` exhibits this behavior explicitly
on all platforms to avoid the starvation issue.

Sadly, a user ended up hitting this deadlock in production in the
form of a call to `get_and_clear_pending_msg_events` which holds
the `ChannelManager::total_consistency_lock` before calling
`process_pending_monitor_events` and eventually
`channel_monitor_updated`, which tries to take the same read lock
again.

Luckily, the fix is trivial, simply remove the redundand read lock
in `channel_monitor_updated`.

Fixes #2000

20 months agoHold the `total_consistency_lock` while in `outbound_payment` fns
Matt Corallo [Fri, 3 Feb 2023 00:33:27 +0000 (00:33 +0000)]
Hold the `total_consistency_lock` while in `outbound_payment` fns

We previously avoided holding the `total_consistency_lock` while
doing crypto operations to build onions. However, now that we've
abstracted out the outbound payment logic into a utility module,
ensuring the state is consistent at all times is now abstracted
away from code authors and reviewers, making it likely to break.

Further, because we now call `send_payment_along_path` both with,
and without, the `total_consistency_lock`, and because recursive
read locks may deadlock, it would now be quite difficult to figure
out which paths through `outbound_payment` need the lock and which
don't.

While it may slow writes somewhat, it's not really worth trying to
figure out this mess, instead we just hold the
`total_consistency_lock` before going into `outbound_payment`
functions.

20 months agoRemove the `final_cltv_expiry_delta` in `RouteParameters` entirely 2023-02-no-dumb-redundant-fields
Matt Corallo [Mon, 6 Feb 2023 22:12:09 +0000 (22:12 +0000)]
Remove the `final_cltv_expiry_delta` in `RouteParameters` entirely

fbc08477e8dcdd8f3f2ada8ca77388b6185febe2 purported to "move" the
`final_cltv_expiry_delta` field to `PaymentParamters` from
`RouteParameters`. However, for naive backwards-compatibility
reasons it left the existing on in place and only added a new,
redundant field in `PaymentParameters`.

It turns out there's really no reason for this - if we take a more
critical eye towards backwards compatibility we can figure out the
correct value in every `PaymentParameters` while deserializing.

We do this here - making `PaymentParameters` a `ReadableArgs`
taking a "default" `cltv_expiry_delta` when it goes to read. This
allows existing `RouteParameters` objects to pass the read
`final_cltv_expiry_delta` field in to be used if the new field
wasn't present.

20 months agoSupport `ReadableArgs` types across in the TLV struct serialization
Matt Corallo [Mon, 6 Feb 2023 21:56:39 +0000 (21:56 +0000)]
Support `ReadableArgs` types across in the TLV struct serialization

This adds `required` support for trait-wrapped reading (e.g. for
objects read via `ReadableArgs`) as well as support for the
trait-wrapped reading syntax across the TLV struct/enum
serialization macros.

20 months agoRequire a non-0 number of non-empty paths when deserializing routes
Matt Corallo [Mon, 6 Feb 2023 21:43:10 +0000 (21:43 +0000)]
Require a non-0 number of non-empty paths when deserializing routes

When we read a `Route` (or a list of `RouteHop`s), we should never
have zero paths or zero `RouteHop`s in a path. As such, its fine to
simply reject these at deserialization-time. Technically this could
lead to something which we can generate not round-trip'ing
serialization, but that seems okay here.

20 months agoMerge pull request #2055 from TheBlueMatt/2023-02-fning-msrv
Matt Corallo [Mon, 27 Feb 2023 22:30:14 +0000 (22:30 +0000)]
Merge pull request #2055 from TheBlueMatt/2023-02-fning-msrv

Remove the lightning-transaction-sync MSRV