]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
2 years agoAdd `ChannelReady` event
Elias Rohrer [Tue, 1 Nov 2022 08:57:37 +0000 (09:57 +0100)]
Add `ChannelReady` event

This adds a `ChannelReady` event that is emitted as soon as a new
channel becomes usable, i.e., after both sides have sent
`channel_ready`.

2 years agoMerge pull request #1815 from TheBlueMatt/2022-10-dead-gossip-code
valentinewallace [Tue, 1 Nov 2022 00:02:52 +0000 (20:02 -0400)]
Merge pull request #1815 from TheBlueMatt/2022-10-dead-gossip-code

Require directional updates for a DirectionalChannelInfo

2 years agoMerge pull request #1818 from TheBlueMatt/2022-10-stable-bench
Matt Corallo [Mon, 31 Oct 2022 21:28:53 +0000 (21:28 +0000)]
Merge pull request #1818 from TheBlueMatt/2022-10-stable-bench

Use rustc stable for benchmark CI run to fix current breakage

2 years agoUse rustc stable for benchmark CI run to fix current breakage 2022-10-stable-bench
Matt Corallo [Mon, 31 Oct 2022 18:46:31 +0000 (18:46 +0000)]
Use rustc stable for benchmark CI run to fix current breakage

2 years agoMake `htlc_maximum_msat` in `EffectiveCapacity` non-`Option`al 2022-10-dead-gossip-code
Matt Corallo [Sat, 29 Oct 2022 18:30:37 +0000 (18:30 +0000)]
Make `htlc_maximum_msat` in `EffectiveCapacity` non-`Option`al

Because we now never generate an `EffectiveCapacity` with an
`htlc_maximum_msat` set to `None`, making it non-`Option`al
effectively removes dead code, which we do here.

2 years agoRequire directional updates for a `DirectionalChannelInfo`
Matt Corallo [Sat, 29 Oct 2022 18:24:36 +0000 (18:24 +0000)]
Require directional updates for a `DirectionalChannelInfo`

We currently construct `DirectedChannelInfo`s for routing before
checking if the given direction has its directional info filled in.
We then always check for directional info before actually deciding
to route over a channel, as otherwise we assume the channel is not
online.

This makes for somewhat redundant checks, and `DirectedCHannelInfo`
isn't, by itself, a very useful API. Because fetching the HTLC-max
or effective channel capacity gives spurious data if no directional
info is available, there's little reason to have that data
available, and so we here check for directional info first. This
effectively merges `DirectionalChannelInfo` and
`DirectionalChannelInfoWithUpdate`.

2 years agoMerge pull request #1809 from valentinewallace/2022-10-custom-om-self
Matt Corallo [Thu, 27 Oct 2022 21:34:45 +0000 (21:34 +0000)]
Merge pull request #1809 from valentinewallace/2022-10-custom-om-self

Give us a self when reading a custom onion message

2 years agoGive us a self when reading a custom onion message
Matt Corallo [Wed, 26 Oct 2022 19:23:10 +0000 (19:23 +0000)]
Give us a self when reading a custom onion message

+ remove MaybeReadableArgs trait as it is now unused
+ remove onion_utils::DecodeInput as it would've now needed to be parameterized
by the CustomOnionMessageHandler trait, and we'd like to avoid either
implementing DecodeInput in messenger or having onion_utils depend on
onion_message::*

Co-authored-by: Matt Corallo <git@bluematt.me>
Co-authored-by: Valentine Wallace <vwallace@protonmail.com>
2 years agoMerge pull request #1805 from TheBlueMatt/2022-10-remote-useless-mut
Matt Corallo [Thu, 27 Oct 2022 16:11:28 +0000 (16:11 +0000)]
Merge pull request #1805 from TheBlueMatt/2022-10-remote-useless-mut

Drop useless `mut` in `KeysInterface::sign_holder_anchor_input`

2 years agoMerge pull request #1807 from TheBlueMatt/2022-10-errorkind-bindings
Matt Corallo [Thu, 27 Oct 2022 16:11:14 +0000 (16:11 +0000)]
Merge pull request #1807 from TheBlueMatt/2022-10-errorkind-bindings

Export io::ErrorKind in bindings

2 years agoExport io::ErrorKind in bindings 2022-10-errorkind-bindings
Matt Corallo [Wed, 26 Oct 2022 21:55:23 +0000 (21:55 +0000)]
Export io::ErrorKind in bindings

The bindings have exported `io::Error` as, basically,
`io::ErrorKind`, for quite some time, so there's little reason to
not just export `io::ErrorKind` as well.

2 years agoDrop useless `mut` in `KeysInterface::sign_holder_anchor_input` 2022-10-remote-useless-mut
Matt Corallo [Wed, 26 Oct 2022 20:46:34 +0000 (20:46 +0000)]
Drop useless `mut` in `KeysInterface::sign_holder_anchor_input`

The `Transaction` is not modified so there's no reason to make the
reference `mut`

2 years agoMerge pull request #1797 from arik-so/2022-10-channel-manager-deparameterization
valentinewallace [Wed, 26 Oct 2022 17:32:09 +0000 (13:32 -0400)]
Merge pull request #1797 from arik-so/2022-10-channel-manager-deparameterization

Deparametrize `ChannelManager` to infer `Signer` from its `KeysInterface`.

2 years agoMerge pull request #1802 from TheBlueMatt/2022-10-112 v0.0.112
Matt Corallo [Tue, 25 Oct 2022 21:33:34 +0000 (21:33 +0000)]
Merge pull request #1802 from TheBlueMatt/2022-10-112

Add release date and release name to CHANGELOG for 0.0.112

2 years agoRefer to `lnd` as `LND` in CHANGELOG 2022-10-112
Matt Corallo [Tue, 25 Oct 2022 18:59:41 +0000 (18:59 +0000)]
Refer to `lnd` as `LND` in CHANGELOG

for consistency since its an abbreviation for "lightning network
daemon".

2 years agoAdd release date and release name to CHANGELOG for 0.0.112
Matt Corallo [Tue, 25 Oct 2022 18:58:54 +0000 (18:58 +0000)]
Add release date and release name to CHANGELOG for 0.0.112

2 years agoMerge pull request #1798 from TheBlueMatt/2022-10-112
Matt Corallo [Tue, 25 Oct 2022 18:57:01 +0000 (18:57 +0000)]
Merge pull request #1798 from TheBlueMatt/2022-10-112

Cut 0.0.112

2 years agoDeparametrize `ChannelManager` to infer `Signer` from its `KeysInterface`.
Arik Sosman [Mon, 24 Oct 2022 19:11:12 +0000 (12:11 -0700)]
Deparametrize `ChannelManager` to infer `Signer` from its `KeysInterface`.

2 years agoAdd release notes for 0.0.112
Matt Corallo [Mon, 24 Oct 2022 18:50:47 +0000 (18:50 +0000)]
Add release notes for 0.0.112

2 years agoBump crate versions to 0.0.112/invoice 0.20
Matt Corallo [Mon, 24 Oct 2022 17:13:56 +0000 (17:13 +0000)]
Bump crate versions to 0.0.112/invoice 0.20

2 years agoDo not export new anchors structs in `events.rs` without anchors
Matt Corallo [Mon, 24 Oct 2022 18:12:30 +0000 (18:12 +0000)]
Do not export new anchors structs in `events.rs` without anchors

This avoids adding unused, effectively-dummy public structs in
`events.rs`.

2 years agoMerge pull request #1779 from valentinewallace/2022-10-node-pk-keysinterface
Matt Corallo [Mon, 24 Oct 2022 17:37:25 +0000 (17:37 +0000)]
Merge pull request #1779 from valentinewallace/2022-10-node-pk-keysinterface

Add `KeysInterface::get_node_id` method

2 years agoKeysInterface docs: note that Recipient type must be supported by impl
Valentine Wallace [Sat, 22 Oct 2022 00:30:49 +0000 (20:30 -0400)]
KeysInterface docs: note that Recipient type must be supported by impl

2 years agoMerge pull request #1780 from wpaulino/rust-edition-2018
Matt Corallo [Sat, 22 Oct 2022 01:05:26 +0000 (01:05 +0000)]
Merge pull request #1780 from wpaulino/rust-edition-2018

2 years agoBump workspace to rust edition 2018
Wilmer Paulino [Thu, 13 Oct 2022 06:35:48 +0000 (02:35 -0400)]
Bump workspace to rust edition 2018

Mostly motivated by the need of async/await.

2 years agoAdd KeysInterface::get_node_id method
Valentine Wallace [Tue, 18 Oct 2022 19:28:18 +0000 (15:28 -0400)]
Add KeysInterface::get_node_id method

Useful since we're working on getting rid of KeysInterface::get_node_secret to
complete support for remote signing.

Will be used in upcoming work to check whether an outbound onion message
blinded path has our node id as the introduction node id

2 years agoMerge pull request #1770 from jkczyz/2022-10-debug-route-hints
valentinewallace [Fri, 21 Oct 2022 17:04:45 +0000 (13:04 -0400)]
Merge pull request #1770 from jkczyz/2022-10-debug-route-hints

Add logging in route hint filtering

2 years agoMerge pull request #1788 from TheBlueMatt/2022-10-mtlc-writer
valentinewallace [Fri, 21 Oct 2022 15:28:15 +0000 (11:28 -0400)]
Merge pull request #1788 from TheBlueMatt/2022-10-mtlc-writer

Add `WriteableScore` bindings impl for `MultiThreadedLockableScore`

2 years agoAdd `WriteableScore` bindings impl for `MultiThreadedLockableScore` 2022-10-mtlc-writer
Matt Corallo [Thu, 29 Sep 2022 15:22:47 +0000 (15:22 +0000)]
Add `WriteableScore` bindings impl for `MultiThreadedLockableScore`

In 56b07e52aabdaca521987e765f1fa864966a5d49 we made
`MultiThreadedLockableScore` fully bindings-compatible. However, it
did not add a `WriteableScore` implementation for it. This was an
oversight as it is a `WriteableScore` in Rust and needs to be for
use in other parts of the API.

Here we add the required impl in a way that the bindings generator
is able to handle it and add conversion utilities.

2 years agoAdd logging in route hint filtering
Jeffrey Czyz [Fri, 14 Oct 2022 14:31:36 +0000 (10:31 -0400)]
Add logging in route hint filtering

Knowing why a channel was not included as an invoice route hint can be
valuable. Add logging on the decisions made when filtering channels.

2 years agoMerge pull request #1748 from valentinewallace/2022-10-custom-oms
Jeffrey Czyz [Wed, 19 Oct 2022 22:15:33 +0000 (17:15 -0500)]
Merge pull request #1748 from valentinewallace/2022-10-custom-oms

Support custom onion messages

2 years agoDisallow sending invalid custom OM TLVs
Valentine Wallace [Tue, 18 Oct 2022 17:33:45 +0000 (13:33 -0400)]
Disallow sending invalid custom OM TLVs

Onion message data TLV types must be >= 64, enforce this on send

2 years agoImplement sending and receiving custom onion messages
Valentine Wallace [Tue, 18 Oct 2022 17:29:43 +0000 (13:29 -0400)]
Implement sending and receiving custom onion messages

This uses the work done in the preceding commits to implement encoding a user's
custom TLV in outbound onion messages, and decoding custom TLVs in inbound
onion messages, to be provided to the new CustomOnionMessageHandler.

2 years agoMerge pull request #1678 from TheBlueMatt/2022-08-funding-locked-mon-persist-fail
Matt Corallo [Wed, 19 Oct 2022 16:55:01 +0000 (16:55 +0000)]
Merge pull request #1678 from TheBlueMatt/2022-08-funding-locked-mon-persist-fail

Handle async initial ChannelMonitor persistence failing on restart

2 years agoRename `ChannelState::MonitorUpdateFailed` `MonitorUpdateInProgress` 2022-08-funding-locked-mon-persist-fail
Matt Corallo [Wed, 5 Oct 2022 00:34:38 +0000 (00:34 +0000)]
Rename `ChannelState::MonitorUpdateFailed` `MonitorUpdateInProgress`

As we're moving towards monitor update async being a supported
use-case, we shouldn't call an async monitor update "failed", but
rather "in progress". This simply updates the internal channel.rs
enum name to reflect the new thinking.

2 years agoHandle async initial ChannelMonitor persistence failing on restart
Matt Corallo [Sat, 20 Aug 2022 01:03:27 +0000 (01:03 +0000)]
Handle async initial ChannelMonitor persistence failing on restart

If the initial ChannelMonitor persistence is done asynchronously
but does not complete before the node restarts (with a
ChannelManager persistence), we'll start back up with a channel
present but no corresponding ChannelMonitor.

Because the Channel is pending-monitor-update and has not yet
broadcasted its initial funding transaction or sent channel_ready,
this is not a violation of our API contract nor a safety violation.
However, the previous code would refuse to deserialize the
ChannelManager treating it as an API contract violation.

The solution is to test for this case explicitly and drop the
channel entirely as if the peer disconnected before we received
the funding_signed for outbound channels or before sending the
channel_ready for inbound channels.

2 years agoMerge pull request #1769 from TheBlueMatt/2022-10-disconnected-hints
valentinewallace [Wed, 19 Oct 2022 14:22:07 +0000 (10:22 -0400)]
Merge pull request #1769 from TheBlueMatt/2022-10-disconnected-hints

Include all channel route hints if no connected channels exist

2 years agoMerge pull request #1764 from G8XSU/rgs-ignore-error
Jeffrey Czyz [Wed, 19 Oct 2022 13:50:09 +0000 (08:50 -0500)]
Merge pull request #1764 from G8XSU/rgs-ignore-error

Ignore Duplicate Gossip Error while updating networkGraph from RGS

2 years agoMerge pull request #1777 from lexe-tech/max/best-block-header-best-block
Jeffrey Czyz [Wed, 19 Oct 2022 13:46:07 +0000 (08:46 -0500)]
Merge pull request #1777 from lexe-tech/max/best-block-header-best-block

Add `.to_best_block()` method to `ValidatedBlockHeader`

2 years agoMerge pull request #1689 from wpaulino/anchors-bump-channel-close-event
Matt Corallo [Wed, 19 Oct 2022 00:29:57 +0000 (00:29 +0000)]
Merge pull request #1689 from wpaulino/anchors-bump-channel-close-event

2 years agoClean up private channel route hint filtering
Jeffrey Czyz [Thu, 13 Oct 2022 20:49:58 +0000 (16:49 -0400)]
Clean up private channel route hint filtering

2 years agoAdd `ValidatedBlockHeader::to_best_block`
Max Fang [Tue, 18 Oct 2022 01:38:26 +0000 (18:38 -0700)]
Add `ValidatedBlockHeader::to_best_block`

2 years agoInclude all channel route hints if no connected channels exist 2022-10-disconnected-hints
Matt Corallo [Thu, 13 Oct 2022 21:57:09 +0000 (21:57 +0000)]
Include all channel route hints if no connected channels exist

Mobile clients often take a second or two before they reconnect to
their peers as its not always clear immediately that connections
have been killed (especially on iOS). This can cause us to
spuriously fail to include route hints in our invoices if we're on
mobile.

The fix is simple, if we're selecting channels to include in route
hints and we're not not connected to the peer for any of our
channels, we should simply include the hints for all channels, even
though we're disconencted.

Fixes #1768.

2 years agoAdd public helper to build anchor input witnesses
Wilmer Paulino [Tue, 11 Oct 2022 18:46:24 +0000 (14:46 -0400)]
Add public helper to build anchor input witnesses

2 years agoAdd holder anchor signing support to BaseSign
Wilmer Paulino [Wed, 13 Jul 2022 17:39:40 +0000 (10:39 -0700)]
Add holder anchor signing support to BaseSign

2 years agoDocument expected call frequency of ChainMonitor::process_pending_events
Wilmer Paulino [Tue, 11 Oct 2022 18:39:32 +0000 (14:39 -0400)]
Document expected call frequency of ChainMonitor::process_pending_events

2 years agoYield channel close bump events
Wilmer Paulino [Wed, 31 Aug 2022 18:39:47 +0000 (11:39 -0700)]
Yield channel close bump events

2 years agoGenerate ClaimEvent for HolderFundingOutput inputs from anchor channels
Wilmer Paulino [Wed, 31 Aug 2022 18:35:55 +0000 (11:35 -0700)]
Generate ClaimEvent for HolderFundingOutput inputs from anchor channels

2 years agoRefactor generate_claim_tx to return OnchainClaim enum
Wilmer Paulino [Wed, 27 Jul 2022 22:15:34 +0000 (15:15 -0700)]
Refactor generate_claim_tx to return OnchainClaim enum

2 years agoSplit finalize_package into separate methods per malleability
Wilmer Paulino [Wed, 31 Aug 2022 18:07:45 +0000 (11:07 -0700)]
Split finalize_package into separate methods per malleability

2 years agoTrack funding amount in HolderFundingOutput
Wilmer Paulino [Wed, 27 Jul 2022 21:58:43 +0000 (14:58 -0700)]
Track funding amount in HolderFundingOutput

This will be useful later on when determining the appropriate fee rate
to use on the anchor transaction to bump its commitment transaction.

2 years agoTrack whether HolderFundingOutput originated from an anchor channel
Wilmer Paulino [Wed, 14 Sep 2022 23:18:33 +0000 (16:18 -0700)]
Track whether HolderFundingOutput originated from an anchor channel

2 years agoStore anchors option for supported output types
Wilmer Paulino [Wed, 14 Sep 2022 19:35:26 +0000 (12:35 -0700)]
Store anchors option for supported output types

As we integrate the support of anchor outputs, we'll want to know if
each input we're working with came from an anchor outputs channel.
Instead of threading through a `opt_anchors` boolean across several
methods on `PackageSolvingData` and `PackageTemplate`, we decide to
store a reference in each `PackageSolvingData` variant instead that
features a change in behavior between channels with and without anchor
outputs.

2 years agoDedupe gossip msgs while updating networkGraph from RGS
Gursharan Singh [Tue, 11 Oct 2022 19:20:33 +0000 (15:20 -0400)]
Dedupe gossip msgs while updating networkGraph from RGS

While applying gossip info from RGS-server, number of harmless
errors might arise which should be ignored. E.g. client should not
fail if there is a duplicate gossip for same channel or duplicate
update.

2 years agoUpdate send_onion_message API to take new OnionMessageContents enum
Valentine Wallace [Tue, 18 Oct 2022 17:22:12 +0000 (13:22 -0400)]
Update send_onion_message API to take new OnionMessageContents enum

OnionMessageContents specifies the data TLV that the sender wants in the onion
message. This enum only has one variant for now, Custom. When offers are added,
additional variants for invoice, invoice_request, and invoice_error will be
added.

This commit does not actually implement sending the custom OM contents, just
the API change.

2 years agoFix accidental newline in OnionMessenger docs
Valentine Wallace [Tue, 18 Oct 2022 17:14:35 +0000 (13:14 -0400)]
Fix accidental newline in OnionMessenger docs

2 years agoTweak OnionMessenger log on inbound onion message receipt
Valentine Wallace [Tue, 18 Oct 2022 16:14:15 +0000 (12:14 -0400)]
Tweak OnionMessenger log on inbound onion message receipt

2 years agoParameterize OnionMessenger by new CustomOnionMessageHandler trait
Valentine Wallace [Mon, 17 Oct 2022 20:32:17 +0000 (16:32 -0400)]
Parameterize OnionMessenger by new CustomOnionMessageHandler trait

OnionMessenger::new will now take a custom onion message handler trait
implementation. This handler will be used in upcoming commit(s) to handle
inbound custom onion messages.

The new trait also specifies what custom messages are supported via its
associated type, CustomMessage. This associated type must implement a new
CustomOnionMessagesContents trait, which requires custom messages to support
being written, being read, and supplying their TLV type.

2 years agoOptionally parameterize decode_tlv_stream with custom decode closure
Valentine Wallace [Mon, 17 Oct 2022 20:07:40 +0000 (16:07 -0400)]
Optionally parameterize decode_tlv_stream with custom decode closure

Useful for decoding custom or user-provided TLVs. See macro docs for more info.

Used in upcoming commit(s) to support custom onion message TLVs

2 years agoser: Add MaybeReadableArgs trait
Valentine Wallace [Mon, 17 Oct 2022 20:05:32 +0000 (16:05 -0400)]
ser: Add MaybeReadableArgs trait

Useful in decoding a custom message, so (a) the message type can be provided to
the handler and (b) None can be returned if the message type is unknown.

Used in upcoming commit(s) to support custom onion messages.

2 years agoMerge pull request #1763 from gcomte/feature/derive-eq
Matt Corallo [Mon, 17 Oct 2022 16:03:23 +0000 (16:03 +0000)]
Merge pull request #1763 from gcomte/feature/derive-eq

Derive Eq for all structs that derive PartialEq

2 years agoMerge pull request #1774 from futurepaul/disable-rustfmt
Matt Corallo [Mon, 17 Oct 2022 14:51:22 +0000 (14:51 +0000)]
Merge pull request #1774 from futurepaul/disable-rustfmt

disable automatic formatting

2 years agodisable all formatting
Paul Miller [Sat, 15 Oct 2022 21:46:06 +0000 (17:46 -0400)]
disable all formatting

2 years agoDerive Eq for all structs that derive PartialEq
Gabriel Comte [Fri, 14 Oct 2022 11:24:02 +0000 (13:24 +0200)]
Derive Eq for all structs that derive PartialEq

2 years agoMerge pull request #1765 from gcomte/refactor/remove-redundant-field-names
valentinewallace [Wed, 12 Oct 2022 14:46:36 +0000 (10:46 -0400)]
Merge pull request #1765 from gcomte/refactor/remove-redundant-field-names

Remove redundant field names

2 years agoRemove redundant field names
Gabriel Comte [Tue, 11 Oct 2022 22:14:01 +0000 (00:14 +0200)]
Remove redundant field names

2 years agoMerge pull request #1754 from TheBlueMatt/2022-10-better-liq-halflife-docs
Matt Corallo [Tue, 11 Oct 2022 21:54:48 +0000 (21:54 +0000)]
Merge pull request #1754 from TheBlueMatt/2022-10-better-liq-halflife-docs

Rewrite some documentation on `ProbabilisticScorer` and increase half-life to 6 hours.

2 years agoIncrease the default `liquidity_offset_half_life` to six hours 2022-10-better-liq-halflife-docs
Matt Corallo [Wed, 5 Oct 2022 19:23:56 +0000 (19:23 +0000)]
Increase the default `liquidity_offset_half_life` to six hours

Even at relatively high payment volumes, decaying knowledge of each
individual channel every hour causes aggressive retrying of
channels as we quickly forget the state of a channel. Even with the
historical tracker, this isn't fully remedied, as we'll track the
history bounds with the decayed value.

Instead, we decay every six hours here, reducing how often we'll
retry a channel due to decay.

In addition to this, the decay likely needs to be substantially
more linear, as tracked in #1752.

2 years agoRewrite documentation some on `ProbabilisticScorer`
Matt Corallo [Wed, 5 Oct 2022 19:10:06 +0000 (19:10 +0000)]
Rewrite documentation some on `ProbabilisticScorer`

We had some user confusion on how the probabilistic scorer works,
especially in reference to the half-life parameter. This attempts
to clarify how the bounds work, and how they are decayed.

2 years agoMerge pull request #1744 from tnull/2022-09-fix-bp-futures-doc
Matt Corallo [Mon, 10 Oct 2022 20:29:11 +0000 (20:29 +0000)]
Merge pull request #1744 from tnull/2022-09-fix-bp-futures-doc

Fix `background-processor` doc build for `futures`

2 years agoAdd `futures` CI check
Elias Rohrer [Mon, 10 Oct 2022 18:47:18 +0000 (14:47 -0400)]
Add `futures` CI check

This adds testing of the `futures` feature to CI. In order to avoid
introducing a dependency on `std`, we switch to using the `futures-util`
crate directly enabling only a subset of features. To this end, we also
switch to using the `select_biased!` macro, which is equivalent to
`select!` except that it doesn't choose ready futures pseudo-randomly
at runtime.

2 years agoFix `background-processor` doc build for `futures`
Elias Rohrer [Mon, 26 Sep 2022 12:41:59 +0000 (14:41 +0200)]
Fix `background-processor` doc build for `futures`

2 years agoMerge pull request #1756 from TheBlueMatt/2022-10-rgs-no-std
valentinewallace [Fri, 7 Oct 2022 21:01:10 +0000 (17:01 -0400)]
Merge pull request #1756 from TheBlueMatt/2022-10-rgs-no-std

Fix `rapid-gossip-sync` `no-std` and properly test no-std in CI

2 years agoSpecify full import path in decode_tlv macro
Valentine Wallace [Fri, 7 Oct 2022 19:22:00 +0000 (15:22 -0400)]
Specify full import path in decode_tlv macro

Prevents unresolved import error

2 years agoMerge pull request #1649 from dunxen/2022-08-implementnodefailure
Matt Corallo [Fri, 7 Oct 2022 18:41:19 +0000 (18:41 +0000)]
Merge pull request #1649 from dunxen/2022-08-implementnodefailure

Implement permanent node failure handling

2 years agoMerge pull request #1693 from valentinewallace/2022-09-readme-updates
Jeffrey Czyz [Fri, 7 Oct 2022 17:44:52 +0000 (12:44 -0500)]
Merge pull request #1693 from valentinewallace/2022-09-readme-updates

Fix outdated README sections

2 years agoMerge pull request #1758 from TheBlueMatt/2022-10-fix-pre-notified-future
Matt Corallo [Fri, 7 Oct 2022 17:38:50 +0000 (17:38 +0000)]
Merge pull request #1758 from TheBlueMatt/2022-10-fix-pre-notified-future

Fix (and test) `Future` creation after a `Notifier` was notified

2 years agoTest a full `no-std` build in the `no-std-check` crate in CI 2022-10-rgs-no-std
Matt Corallo [Thu, 6 Oct 2022 15:44:03 +0000 (15:44 +0000)]
Test a full `no-std` build in the `no-std-check` crate in CI

Rust is incredibly forgiving in attempts to access `std`, making it
rather difficult to test `no-std` properly. In practice, the only
decent way to do so is to actually build for a platform that does
not have `std`, which we do here by building the `no-std-check`
crate for `arm-thumbv7m-none-eabi`.

2 years agoMerge pull request #1625 from TheBlueMatt/2022-07-history-tracking
Matt Corallo [Fri, 7 Oct 2022 16:32:10 +0000 (16:32 +0000)]
Merge pull request #1625 from TheBlueMatt/2022-07-history-tracking

2 years agoMerge pull request #1751 from TheBlueMatt/2022-10-fix-score-log-direction
valentinewallace [Fri, 7 Oct 2022 16:28:23 +0000 (12:28 -0400)]
Merge pull request #1751 from TheBlueMatt/2022-10-fix-score-log-direction

Correct the directionality of liquidity non-update messages

2 years agoMerge pull request #1760 from TheBlueMatt/2022-10-invoice-builder-round
Matt Corallo [Fri, 7 Oct 2022 16:25:25 +0000 (16:25 +0000)]
Merge pull request #1760 from TheBlueMatt/2022-10-invoice-builder-round

2 years agoFix outdated README sections
Valentine Wallace [Fri, 2 Sep 2022 21:55:53 +0000 (17:55 -0400)]
Fix outdated README sections

2 years agoRemove and track perm failed nodes & channels
Duncan Dean [Fri, 26 Aug 2022 10:54:16 +0000 (12:54 +0200)]
Remove and track perm failed nodes & channels

We never had the `NetworkGraph::node_failed` method implemented. The
scorer now handles non-permanent failures by downgrading nodes, so we
don't want that implemented.

The method is renamed to `node_failed_permanent` to explicitly indicate
that this is the only case it handles. We also add tracking in the form
of two maps as fields of `NetworkGraph`, namely, `removed_nodes` and
`removed_channels`. We track these removed graph entries to ensure we
don't just resync them right away from gossip soon after removing them.
We stop tracking these removed nodes whenever `remove_stale_channels_and_tracking()`
is called and the entries have been tracked for longer than
`REMOVED_ENTRIES_TRACKING_AGE_LIMIT_SECS` which is currently set to one
week.

2 years agoDrop the subsecond part of timestamps when constructing invoices 2022-10-invoice-builder-round
Matt Corallo [Fri, 7 Oct 2022 01:38:56 +0000 (01:38 +0000)]
Drop the subsecond part of timestamps when constructing invoices

We had a user who was confused why an invoice failed to round-trip
(i.e. was not `PartialEq` with itself after write/read) when a
subsecond creation time was used (e.g. via the `from_system_time`
constructor).

This commit should address this confusion by dropping subsecond
parts in easily-accessible constructors when creating BOLT 11
invoices.

Fixes #1759.

2 years agoSupport platforms with only 32-bit atomics
Matt Corallo [Thu, 6 Oct 2022 16:48:14 +0000 (16:48 +0000)]
Support platforms with only 32-bit atomics

Given there is only one instance in our code of `AtomicU64` its
simplest to just remove it rather than try to add some kind of
wrapper.

2 years agoTest `rapid-gossip-sync` in `no-std` in CI
Matt Corallo [Thu, 6 Oct 2022 15:41:58 +0000 (15:41 +0000)]
Test `rapid-gossip-sync` in `no-std` in CI

2 years agoUpdate `rapid-gossip-sync` docs to not reference `std`-only methods
Matt Corallo [Thu, 6 Oct 2022 19:38:17 +0000 (19:38 +0000)]
Update `rapid-gossip-sync` docs to not reference `std`-only methods

2 years agoFix (and test) `Future` creation after a `Notifier` was notified 2022-10-fix-pre-notified-future
Matt Corallo [Thu, 6 Oct 2022 23:54:52 +0000 (23:54 +0000)]
Fix (and test) `Future` creation after a `Notifier` was notified

After a `Notifier` has been `notify`'d, attempts to `get_future`
should return a `Future` which is pre-completed, however this was
not the case.

This commit simply fixes the behavior, adding a test to demonstrate
the issue.

2 years agoDecay historical liquidity tracking when no new data is added 2022-07-history-tracking
Matt Corallo [Mon, 22 Aug 2022 22:41:31 +0000 (22:41 +0000)]
Decay historical liquidity tracking when no new data is added

To avoid scoring based on incredibly old historical liquidity data,
we add a new half-life here which is used to (very slowly) decay
historical liquidity tracking buckets.

2 years agoTrack a reference to scoring parameters in DirectedChannelLiquidity
Matt Corallo [Mon, 22 Aug 2022 22:40:02 +0000 (22:40 +0000)]
Track a reference to scoring parameters in DirectedChannelLiquidity

This simplifies adding additional half lives in
DirectedChannelLiquidity by simply storing a reference to the full
parameter set rather than only the single current half-life.

2 years agoCalculate a new penalty based on historical channel liquidity range
Matt Corallo [Thu, 21 Jul 2022 02:08:28 +0000 (02:08 +0000)]
Calculate a new penalty based on historical channel liquidity range

Our current `ProbabilisticScorer` attempts to build a model of the
current liquidity across the payment channel network. This works
fine to ignore channels we *just* tried to pay through, but it
fails to remember patterns over longer time horizons.

Specifically, there are *many* channels within the network that are
very often either fully saturated in one direction, or are
regularly rebalanced and rarely saturated. While our model may
discover that, when it decays its offsets or if there is a
temporary change in liquidity, it effectively forgets the "normal"
state of the channel.

This causes substantially suboptimal routing in practice, and
avoiding discarding older knowledge when new datapoints come in is
a potential solution to this.

Here, we implement one such design, using the decaying buckets
added in the previous commit to calculate a probability of payment
success based on a weighted average of recent liquidity estimates
for a channel.

For each min/max liquidity bucket pair (where the min liquidity is
less than the max liquidity), we can calculate the probability that
a payment succeeds using our traditional `amount / capacity`
formula. From there, we weigh the probability by the number of
points in each bucket pair, calculating a total probability for the
payment, and assigning a penalty using the same log-probability
calculation used for the non-historical penalties.

2 years agoTrack history of where channel liquidities have been in the past
Matt Corallo [Tue, 19 Jul 2022 22:37:16 +0000 (22:37 +0000)]
Track history of where channel liquidities have been in the past

This introduces two new fields to the `ChannelLiquidity` struct -
`min_liquidity_offset_history` and `max_liquidity_offset_history`,
both an array of 8 `u16`s. Each entry represents the proportion of
time that we spent with the min or max liquidity offset in the
given 1/8th of the channel's liquidity range. ie the first bucket
in `min_liquidity_offset_history` represents the proportion of time
we've thought the channel's minimum liquidity is lower than 1/8th's
the channel's capacity.

Each bucket is stored, effectively, as a fixed-point number with
5 bits for the fractional part, which is incremented by one (ie 32)
each time we update our liquidity estimates and decide our
estimates are in that bucket. We then decay each bucket by
2047/2048.

Thus, memory of a payment sticks around for more than 8,000
data points, though the majority of that memory decays after 1,387
data points.

2 years agoCorrect the directionality of liquidity non-update messages 2022-10-fix-score-log-direction
Matt Corallo [Wed, 5 Oct 2022 02:11:00 +0000 (02:11 +0000)]
Correct the directionality of liquidity non-update messages

When we log liquidity updates, if we decline to update anything as
the new bounds are already within the old bounds, the
directionality of the log entries was reversed.

While we're at it, we also log the old values.

2 years agoCorrect `rapid-gossip-sync` `no-std` build and test
Matt Corallo [Thu, 6 Oct 2022 15:42:41 +0000 (15:42 +0000)]
Correct `rapid-gossip-sync` `no-std` build and test

While `rapid-gossip-sync` recently gained a `no-std` feature, it
didn't actually work, as there were still dangling references to
`std` and prelude assumptions. This makes `rapid-gossip-sync`
build (and test) properly in `no-std`.

2 years agoMerge pull request #1106 from TheBlueMatt/2021-10-no-perm-err-broadcast
Matt Corallo [Thu, 29 Sep 2022 22:02:07 +0000 (22:02 +0000)]
Merge pull request #1106 from TheBlueMatt/2021-10-no-perm-err-broadcast

Do not broadcast commitment txn on Permanent mon update failure

2 years agoClarify ambiguous comment in persist methods 2021-10-no-perm-err-broadcast
Matt Corallo [Wed, 24 Aug 2022 21:07:44 +0000 (21:07 +0000)]
Clarify ambiguous comment in persist methods

2 years agoRename APIError::MonitorUpdateFailed to MonitorUpdateInProgress
Matt Corallo [Thu, 29 Sep 2022 20:26:48 +0000 (20:26 +0000)]
Rename APIError::MonitorUpdateFailed to MonitorUpdateInProgress

This much more accurately represents the error, indicating that a
monitor update is in progress asynchronously and may complete at a
later time.

2 years agoRename `handle_monitor_err!()` `handle_monitor_update_res!`
Matt Corallo [Tue, 16 Aug 2022 22:16:48 +0000 (22:16 +0000)]
Rename `handle_monitor_err!()` `handle_monitor_update_res!`

2 years agoRename `Channel::monitor_update_failed` to `monitor_updating_paused`
Matt Corallo [Tue, 16 Aug 2022 22:15:32 +0000 (22:15 +0000)]
Rename `Channel::monitor_update_failed` to `monitor_updating_paused`

2 years agoAdd a TODO for an important issue for making async mon updates safe
Matt Corallo [Tue, 16 Aug 2022 21:58:06 +0000 (21:58 +0000)]
Add a TODO for an important issue for making async mon updates safe

If we receive a monitor event from a forwarded-to channel which
contains a preimage for an HTLC, we have to propogate that preimage
back to the forwarded-from channel monitor. However, once we have
that update, we're running in a relatively unsafe state - we have
the preimage in memory, but if we were to crash the forwarded-to
channel monitor will not regenerate the update with the preimage
for us. If we haven't managed to write the monitor update to the
forwarded-from channel by that point, we've lost the preimage, and,
thus, money!