]> git.bitcoin.ninja Git - rust-lightning/log
rust-lightning
3 years ago[router] Avoid re-processing peers when a score component decreases
Matt Corallo [Sat, 27 Mar 2021 16:49:42 +0000 (12:49 -0400)]
[router] Avoid re-processing peers when a score component decreases

While walking the graph doing Dijkstra's, we may decrease the
amount being sent along one path, and not others, based on the
htlc_minimum_msat value. This may result in a lower relative fees
on that path in comparison to others. In the extreme, this may
result in finding a second path to a node with a lower fee than the
first path found (normally impossible in a Dijkstra's
implementation, as we walk next hops in fee-order).

In such a case, we end up with parts of our state invalid as
further hops beyond that node have been filled in using the
original total fee information.

Instead, we simply track which nodes have been processed and ignore
and further updates to them (as it implies we've reduced the amount
we're sending to find a lower absolute fee, but a higher relative
fee, which isn't a better route).

We check that we are in exactly this case in test builds with new
in-line assertions. Note that these assertions successfully detect
the bug in the previous commit.

Sadly this requires an extra HashMap lookup every time we pop an
element off of our heap, though we can skip a number of heap pushes
during the channel walking.

3 years agoAdd a random real-world-network-graph test for the router
Matt Corallo [Sat, 27 Mar 2021 01:50:54 +0000 (21:50 -0400)]
Add a random real-world-network-graph test for the router

This is the same code as was recently failing in our benchmarks,
adapted to use a random starting seed instead of a fixed one and
a smaller iteration to reduce runtime.

3 years ago[router] Calc min-to-node fees based on min value, not est value
Matt Corallo [Sat, 27 Mar 2021 16:27:44 +0000 (12:27 -0400)]
[router] Calc min-to-node fees based on min value, not est value

When walking the network graph to calculate a route, we always
calculate the minimum fee which is required to make one further
hop. This avoids some extra hop processing at the end of each path
selection loop (saving around 10% runtime in our benchmarks).

However, if we use the real value which we expect
to send over a channel in that calculation, we may find an
alternate path to the same node which is more expensive but
capacity-constrained, resulting in us considering it cheaper as the
relative fee paid will be lower.

Instead, we can use the `minimal_value_contribution_msat`, which is
a constant through an entire path finding iteration, as the amount,
preventing any basis change in the relative fee paid.

3 years ago[router] Make Dijkstra's path scoring non-decreasing + consistent
Matt Corallo [Sat, 27 Mar 2021 02:56:37 +0000 (22:56 -0400)]
[router] Make Dijkstra's path scoring non-decreasing + consistent

Currently, the "best source" for a given node tracked during
Dijkstra's is updated with a different critera from the heap
sorting, resulting in loops in calculated paths.

This adds a test for the specific failure currently seen, utilizing
the new path-htlc-minimum tracking in the heap entries in place of
the per-hop htlc-minimum values which the MPP changeset partially
used.

3 years ago[router] Avoid re-selecting the same path over and over again
Matt Corallo [Sat, 27 Mar 2021 02:31:57 +0000 (22:31 -0400)]
[router] Avoid re-selecting the same path over and over again

If we walk the network graph and find a route that meets are
payment amount and has a liquidiy limit far in excess of our
payment amount, we may select the same path several times in the
main path-gathering loop.

Instead, if the path we selected was not limited by the available
liquidity, we set the middle hop in the path's liquidity available
to zero, disabling that channel in future path finding steps.

3 years ago[router] Track full-path htlc-minimum-msat while graph-walking
Matt Corallo [Sat, 27 Mar 2021 02:53:35 +0000 (22:53 -0400)]
[router] Track full-path htlc-minimum-msat while graph-walking

Previously, we'd happily send funds through a path where, while
generating the path, in some middle hope we reduce the value being
sent to meet an htlc_maximum, making a later hop invalid due to it
no longer meeting its htlc_minimum. Instead, we need to track the
path's htlc-minimum while we're transiting the graph.

3 years ago[router] Do not fail if we are exactly (or 3x) limited by a maximum
Matt Corallo [Sat, 27 Mar 2021 02:51:41 +0000 (22:51 -0400)]
[router] Do not fail if we are exactly (or 3x) limited by a maximum

The new MPP routing algorithm attempts to build paths with a higher
value than our payment to find paths which may allow us to pay a
fee to meet an htlc_minimum limit. This is great if we're
min-bounded, however it results in us rejecting paths where we are
bounded by a maximum near the end of the path, with fees, and then
bounded by a minimum earlier in the path. Further, it means we will
not find the cheapest path where paths have a lower relative fee
but a higher absolute fee.

Instead, we calculate routes using the actual amount we wish to
send. To maintain the previous behavior of searching for cheaper
paths where we can "pay the difference" to meet an htlc_minimum, we
detect if we were minimum-bounded during graph walking and, if we
are, we walk the graph again with a higher value.

3 years agoAdd comment in get_route describing our dijkstra's mods
Matt Corallo [Tue, 2 Mar 2021 01:20:16 +0000 (20:20 -0500)]
Add comment in get_route describing our dijkstra's mods

3 years agoMerge pull request #846 from TheBlueMatt/2021-03-test-chains
Matt Corallo [Sat, 20 Mar 2021 04:03:27 +0000 (04:03 +0000)]
Merge pull request #846 from TheBlueMatt/2021-03-test-chains

Require syntactically-valid blockchains in functional and unit tests

3 years ago[functional tests] Drop unused disconnect_block utility 2021-03-test-chains
Matt Corallo [Thu, 18 Mar 2021 17:16:58 +0000 (13:16 -0400)]
[functional tests] Drop unused disconnect_block utility

This also reduces some needless clones and indirections.

3 years agoDrop height parameter from [dis]connect_block in functional tests
Matt Corallo [Thu, 18 Mar 2021 02:00:47 +0000 (22:00 -0400)]
Drop height parameter from [dis]connect_block in functional tests

3 years ago[tests] Demonstrate that the commit is trivially safe
Matt Corallo [Thu, 18 Mar 2021 00:42:59 +0000 (20:42 -0400)]
[tests] Demonstrate that the commit is trivially safe

See comment in the code, This commit exists only to aid reviewers.

3 years ago[tests] Drop redundant parameters from connect_blocks
Matt Corallo [Wed, 17 Mar 2021 03:22:59 +0000 (23:22 -0400)]
[tests] Drop redundant parameters from connect_blocks

3 years ago[test] Demonstrate that the next commit is trivially safe
Matt Corallo [Wed, 17 Mar 2021 03:24:41 +0000 (23:24 -0400)]
[test] Demonstrate that the next commit is trivially safe

See comment in the code. This commit exists only to aid reviewers.

3 years agoEnforce block connection ordering in unit and functional tests
Matt Corallo [Mon, 15 Mar 2021 23:49:51 +0000 (19:49 -0400)]
Enforce block connection ordering in unit and functional tests

This expands the assertions on block ordering to apply to
`#[cfg(test)]` builds in addition to normal builds, requiring that
unit and functional tests have syntactically-valid (ie the previous
block hash pointer and the heights match the blocks) blockchains.

This requires a reasonably nontrivial diff in the functional tests
however it is mostly straightforward changes.

3 years agoFix block connection ordering in a number of functional tests
Matt Corallo [Wed, 17 Mar 2021 17:11:48 +0000 (13:11 -0400)]
Fix block connection ordering in a number of functional tests

Many functional tests rely on being able to call block_connected
arbitrarily, jumping back in time to confirm a transaction at a
specific height. Instead, this takes us one step towards having a
well-formed blockchain in the functional tests.

We also take this opportunity to reduce the number of blocks
connected during tests, requiring a number of constant tweaks in
various functional tests.

Co-authored-by: Valentine Wallace <vwallace@protonmail.com>
Co-authored-by: Matt Corallo <git@bluematt.me>
3 years agoAdd assertions for in-order block [dis]connection in ChannelManager
Matt Corallo [Fri, 5 Mar 2021 16:02:42 +0000 (11:02 -0500)]
Add assertions for in-order block [dis]connection in ChannelManager

Sadly the connected-in-order tests have to be skipped in our normal
test suite as many tests violate it. Luckily we can still enforce
it in the tests which run in other crates.

Co-authored-by: Matt Corallo <git@bluematt.me>
Co-authored-by: Jeffrey Czyz <jkczyz@gmail.com>
3 years agoAdd `Debug` to derive list in MessageSendEvent
Matt Corallo [Sat, 13 Mar 2021 02:30:26 +0000 (21:30 -0500)]
Add `Debug` to derive list in MessageSendEvent

It can be quite useful in debugging, and potentially also so for
users.

3 years agoClean up log warning in the event ChannelMonitor force-closed
Matt Corallo [Mon, 15 Mar 2021 03:26:15 +0000 (23:26 -0400)]
Clean up log warning in the event ChannelMonitor force-closed

3 years agoMerge pull request #849 from TheBlueMatt/2021-03-config-cltv-delta
Matt Corallo [Sat, 20 Mar 2021 02:52:51 +0000 (02:52 +0000)]
Merge pull request #849 from TheBlueMatt/2021-03-config-cltv-delta

Make cltv_expiry_delta configurable and reduce the min/default some

3 years agoIgnore patch codecov as long as total coverage is within 1% of base 2021-03-config-cltv-delta
Matt Corallo [Thu, 18 Mar 2021 03:12:47 +0000 (23:12 -0400)]
Ignore patch codecov as long as total coverage is within 1% of base

In some PRs, codecov gets mad that the coverage of the patch itself
is lower than the base. In most cases, we largely don't want a Big
Red X, at least as long as the total coverage has not gone down
substantially.

3 years agoTweak our_to_self_delay documentation wording to make it flow better
Matt Corallo [Fri, 19 Mar 2021 14:44:24 +0000 (10:44 -0400)]
Tweak our_to_self_delay documentation wording to make it flow better

3 years agoMake cltv_expiry_delta configurable and reduce the min/default some
Matt Corallo [Wed, 17 Mar 2021 16:49:49 +0000 (12:49 -0400)]
Make cltv_expiry_delta configurable and reduce the min/default some

We allow users to configure the to_self_delay, which is analogous to
the cltv_expiry_delta in terms of its security context, so we should
allow users to specify both.

We similarly bound it on the lower end, but reduce that bound
somewhat now that it is configurable.

3 years agoMerge pull request #848 from TheBlueMatt/2021-03-doc-cleanups
Matt Corallo [Thu, 18 Mar 2021 15:59:44 +0000 (15:59 +0000)]
Merge pull request #848 from TheBlueMatt/2021-03-doc-cleanups

Clean up doc links and enforce them in CI

3 years agoAlso correct/check links in private item documentation 2021-03-doc-cleanups
Matt Corallo [Wed, 17 Mar 2021 19:56:02 +0000 (15:56 -0400)]
Also correct/check links in private item documentation

3 years agoDrop all HTML-relative links since rustdoc now supports resolution
Matt Corallo [Wed, 17 Mar 2021 19:53:29 +0000 (15:53 -0400)]
Drop all HTML-relative links since rustdoc now supports resolution

3 years agoEnforce no missing docs in all crates (+ add docs that were missing)
Matt Corallo [Wed, 17 Mar 2021 18:18:37 +0000 (14:18 -0400)]
Enforce no missing docs in all crates (+ add docs that were missing)

3 years agoEnforce doc link correctness during `cargo doc` runs and in CI
Matt Corallo [Wed, 17 Mar 2021 18:05:09 +0000 (14:05 -0400)]
Enforce doc link correctness during `cargo doc` runs and in CI

3 years agoClean up some doc links in lightning_block_sync.
Matt Corallo [Wed, 17 Mar 2021 18:04:02 +0000 (14:04 -0400)]
Clean up some doc links in lightning_block_sync.

Relative HTML doc paths in doc links works locally, but breaks on
crates.io. Luckily, we can now use explicit full paths and rustdoc
will resolve them for us.

3 years agoExpose the constants mentioned in docs, and use doc links
Matt Corallo [Wed, 17 Mar 2021 17:40:10 +0000 (13:40 -0400)]
Expose the constants mentioned in docs, and use doc links

3 years agoMerge pull request #841 from valentinewallace/207-replacement
Matt Corallo [Wed, 17 Mar 2021 22:41:30 +0000 (22:41 +0000)]
Merge pull request #841 from valentinewallace/207-replacement

Expose counterparty forwarding info in ChannelDetails

3 years agoExpose counterparty forwarding info in ChannelDetails.
Valentine Wallace [Fri, 12 Mar 2021 21:02:17 +0000 (16:02 -0500)]
Expose counterparty forwarding info in ChannelDetails.

Useful for constructing route hints for private channels in invoices.

Co-authored-by: Valentine Wallace <vwallace@protonmail.com>
Co-authored-by: Antoine Riard <ariard@student.42.fr>
3 years agoPass channel updates to ChannelManager and Channel.
Valentine Wallace [Fri, 12 Mar 2021 20:25:56 +0000 (15:25 -0500)]
Pass channel updates to ChannelManager and Channel.

This will be used to expose forwarding info for route hints in the next commit.

Co-authored-by: Valentine Wallace <vwallace@protonmail.com>
Co-authored-by: Antoine Riard <ariard@student.42.fr>
3 years agoAdd CounterpartyForwardingInfo field to channel.
Valentine Wallace [Fri, 12 Mar 2021 19:23:20 +0000 (14:23 -0500)]
Add CounterpartyForwardingInfo field to channel.

This will be filled in in upcoming commits, then exposed in ChannelDetails
to allow constructing route hints for invoices.

Also update the cltv_expiry_deta comment in msgs::ChannelUpdate

Co-authored-by: Valentine Wallace <vwallace@protonmail.com>
Co-authored-by: Antoine Riard <ariard@student.42.fr>
3 years agoMerge pull request #828 from bmancini55/reply_channel_range
Matt Corallo [Wed, 17 Mar 2021 20:35:03 +0000 (20:35 +0000)]
Merge pull request #828 from bmancini55/reply_channel_range

Handle query_channel_range gossip queries

3 years agoSimplify sequencing of handle_query_channel_range
bmancini55 [Tue, 16 Mar 2021 20:30:22 +0000 (16:30 -0400)]
Simplify sequencing of handle_query_channel_range

Modify NetGraphMsgHandler::handle_query_channel_range to always use
first_blocknum=0 in replies. This is spec compliant after changes to
make sequence completion explicity using sync_complete.

3 years agoComment cleanup for handling query_channel_range
bmancini55 [Sun, 14 Mar 2021 19:45:45 +0000 (15:45 -0400)]
Comment cleanup for handling query_channel_range

Cleans up NetGraphMsgHandler::handle_query_channel_range

3 years agoUse constant MAX_REPLY_SCID
bmancini55 [Sat, 13 Mar 2021 19:51:36 +0000 (14:51 -0500)]
Use constant MAX_REPLY_SCID

Modifies NetGraphMsgHandler::handle_query_channel_range to use a constant
max value in replies. Modifies tests to generate 8000 channels instead
of making this value configurable.

3 years agoMerge pull request #836 from valentinewallace/invoice-features-methods
Matt Corallo [Fri, 12 Mar 2021 23:19:17 +0000 (23:19 +0000)]
Merge pull request #836 from valentinewallace/invoice-features-methods

Add methods to set invoice features in Features objects.

3 years agoAdd methods to set features in Features objects.
Valentine Wallace [Mon, 8 Mar 2021 00:26:01 +0000 (19:26 -0500)]
Add methods to set features in Features objects.

3 years agoMerge pull request #837 from TheBlueMatt/2021-03-0.0.13
Matt Corallo [Wed, 10 Mar 2021 23:26:50 +0000 (23:26 +0000)]
Merge pull request #837 from TheBlueMatt/2021-03-0.0.13

3 years agoUse constant in QueryChannelRange::end_blocknum
bmancini55 [Wed, 10 Mar 2021 11:46:44 +0000 (06:46 -0500)]
Use constant in QueryChannelRange::end_blocknum

Clarifies u32 max value used as the default.

3 years agoMerge pull request #834 from TheBlueMatt/2021-03-no-pub-internals
Matt Corallo [Wed, 10 Mar 2021 01:54:55 +0000 (01:54 +0000)]
Merge pull request #834 from TheBlueMatt/2021-03-no-pub-internals

Make `util::logger::macro_logger` pub(crate) not pub.

3 years agoUpdate licenses and add missing fields in Cargo.toml files 2021-03-0.0.13
Matt Corallo [Tue, 9 Mar 2021 21:51:14 +0000 (16:51 -0500)]
Update licenses and add missing fields in Cargo.toml files

3 years agoAdd missing feature to background-processor dev-dependencies
Matt Corallo [Tue, 9 Mar 2021 21:47:43 +0000 (16:47 -0500)]
Add missing feature to background-processor dev-dependencies

3 years agoClean up iteration in handle_query_channel_range
bmancini55 [Tue, 9 Mar 2021 21:30:52 +0000 (16:30 -0500)]
Clean up iteration in handle_query_channel_range

Refactor to use an enumerator in NetGraphMsgHandler::handle_query_channel_range

3 years agoMerge pull request #835 from TheBlueMatt/2021-03-0.0.13 v0.0.13
Matt Corallo [Tue, 9 Mar 2021 20:57:07 +0000 (20:57 +0000)]
Merge pull request #835 from TheBlueMatt/2021-03-0.0.13

Bump Version to 0.0.13

3 years agoUse scid_from_parts in Channel::block_connected
bmancini55 [Tue, 9 Mar 2021 20:42:45 +0000 (15:42 -0500)]
Use scid_from_parts in Channel::block_connected

Refactors validation and short_channel_id construction to use
the new scid_from_parts function.

3 years agoImprove short_channel_id utils
bmancini55 [Tue, 9 Mar 2021 20:34:52 +0000 (15:34 -0500)]
Improve short_channel_id utils

Modifies scid_from_parts to use u64 inputs allowing untruncated
validation. Adds public constants for limits.

3 years agoBump versions to 0.0.13 across the board
Matt Corallo [Mon, 8 Mar 2021 23:50:35 +0000 (18:50 -0500)]
Bump versions to 0.0.13 across the board

We also skip having different versions for different subcrates,
since that is confusing.

3 years agoRename `background-processor` to `lightning-background-processor`
Matt Corallo [Mon, 8 Mar 2021 23:48:24 +0000 (18:48 -0500)]
Rename `background-processor` to `lightning-background-processor`

This matches our naming scheme better and fits better on crates.io.
Also drop the version to 0.0.1 to match our versioning scheme
better.

3 years agoMerge pull request #827 from TheBlueMatt/2021-03-invoice-features
Matt Corallo [Tue, 9 Mar 2021 17:16:50 +0000 (17:16 +0000)]
Merge pull request #827 from TheBlueMatt/2021-03-invoice-features

Disable MPP routing when the payee does not support it

3 years agoMerge pull request #826 from valentinewallace/raise-max-to-self-delay
Matt Corallo [Mon, 8 Mar 2021 23:44:18 +0000 (15:44 -0800)]
Merge pull request #826 from valentinewallace/raise-max-to-self-delay

Raise max to_self_delay.

3 years agoRaise max to_self_delay.
Valentine Wallace [Thu, 4 Mar 2021 18:28:35 +0000 (13:28 -0500)]
Raise max to_self_delay.

lnd requires this to_self_delay for the max channel size
(excluding wumbo).

3 years agoSplit router benchmark into an MPP and a non-MPP route benchmark 2021-03-invoice-features
Matt Corallo [Fri, 5 Mar 2021 03:37:18 +0000 (22:37 -0500)]
Split router benchmark into an MPP and a non-MPP route benchmark

3 years agoDisable MPP routing when the payee does not support it
Matt Corallo [Fri, 5 Mar 2021 03:36:34 +0000 (22:36 -0500)]
Disable MPP routing when the payee does not support it

3 years agoAdd an Option<>al InvoiceFeatures object for the payee in get_route
Matt Corallo [Fri, 5 Mar 2021 03:13:18 +0000 (22:13 -0500)]
Add an Option<>al InvoiceFeatures object for the payee in get_route

We currently only use it to override the graph-specific features
returned in the route, though we should also use it to enable or
disable MPP.

Note that tests which relied on MPP behavior have had all of their
get_route calls upgraded to provide the MPP flag.

3 years agoMerge pull request #833 from TheBlueMatt/2021-03-no-xs
Matt Corallo [Mon, 8 Mar 2021 21:37:27 +0000 (13:37 -0800)]
Merge pull request #833 from TheBlueMatt/2021-03-no-xs

Drop C bindings (which are now in a separate repo)

3 years agoMerge pull request #830 from TheBlueMatt/2021-03-chanmon-deser-utils
Matt Corallo [Mon, 8 Mar 2021 21:25:56 +0000 (13:25 -0800)]
Merge pull request #830 from TheBlueMatt/2021-03-chanmon-deser-utils

Make chain::Filter slightly easier with some ChannelMonitor utilities.

3 years agoMake `util::logger::macro_logger` pub(crate) not pub. 2021-03-no-pub-internals
Matt Corallo [Sun, 7 Mar 2021 21:24:05 +0000 (16:24 -0500)]
Make `util::logger::macro_logger` pub(crate) not pub.

There is no reason for it to be pub, its a largely-internal
implementation detail of how we format our own objects for logging.

3 years agoDrop C bindings (which are now in a separate repo) 2021-03-no-xs
Matt Corallo [Mon, 8 Mar 2021 18:43:00 +0000 (13:43 -0500)]
Drop C bindings (which are now in a separate repo)

This allows us to drop the usually-failing C bindings CI check, and
makes Github code search more useful by dropping redundant things.

3 years agoCreate new `InvoiceFeatures` object for Invoice-specific features
Matt Corallo [Fri, 5 Mar 2021 03:10:48 +0000 (22:10 -0500)]
Create new `InvoiceFeatures` object for Invoice-specific features

In the past we skipped doing this since invoice parsing occurs in a
different crate. However, we need to accept InvoiceFeatures in routing
now that we support MPP route collection, to detect if we can select
multiple paths or not. Further, we should probably take
rust-lightning-invoice as either a module or a subcrate in this repo.

3 years agoAdd codecov.yml allowing coverage to drop slightly before "failing"
Matt Corallo [Mon, 8 Mar 2021 17:50:07 +0000 (12:50 -0500)]
Add codecov.yml allowing coverage to drop slightly before "failing"

This should remove one big red X in our builds, making it harder
to miss other failing tests.

3 years agoMerge pull request #832 from jkczyz/2021-03-typo-fix
Matt Corallo [Mon, 8 Mar 2021 16:48:15 +0000 (08:48 -0800)]
Merge pull request #832 from jkczyz/2021-03-typo-fix

Fix misspelling of 'occurred' in public interface

3 years agoUse the new `load_outputs_to_watch` util in `ChainMonitor` 2021-03-chanmon-deser-utils
Matt Corallo [Mon, 8 Mar 2021 05:06:31 +0000 (00:06 -0500)]
Use the new `load_outputs_to_watch` util in `ChainMonitor`

This is slightly more effecient as it avoids a clone, but its also
nice to use our own code more.

3 years agoMake `get_outputs_to_watch` return a `Vec` instead of a `HashMap`
Matt Corallo [Sun, 7 Mar 2021 18:00:11 +0000 (13:00 -0500)]
Make `get_outputs_to_watch` return a `Vec` instead of a `HashMap`

`get_outputs_to_watch` returned a reference to an existing
`HashMap` avoiding extra clones, but there isn't a huge reason to
do so now that we have to clone to copy it out of the
`ChannelMonitor` mutex. Instead, return a `Vec` since it may be
less memory and it allows us to have a bindings C mapping for the
function.

Co-authored-by: Jeffrey Czyz <jkczyz@gmail.com>
Co-authored-by: Matt Corallo <git@bluematt.me>
3 years agoFix misspelling of 'occurred' in public interface
Jeffrey Czyz [Mon, 8 Mar 2021 08:09:58 +0000 (00:09 -0800)]
Fix misspelling of 'occurred' in public interface

3 years agoMerge pull request #829 from TheBlueMatt/2021-03-0.0.13-bindings
Matt Corallo [Sun, 7 Mar 2021 20:49:15 +0000 (12:49 -0800)]
Merge pull request #829 from TheBlueMatt/2021-03-0.0.13-bindings

0.0.13 Bindings Updates

3 years agoAuto-generated bindings updates 2021-03-0.0.13-bindings
Matt Corallo [Sun, 7 Mar 2021 18:09:20 +0000 (13:09 -0500)]
Auto-generated bindings updates

3 years ago[bindings] Check for a wasm32 target before compiling in genbindings
Matt Corallo [Sat, 6 Mar 2021 14:50:43 +0000 (09:50 -0500)]
[bindings] Check for a wasm32 target before compiling in genbindings

3 years ago[bindings] Update demos for new upstream API
Matt Corallo [Sat, 6 Mar 2021 00:09:26 +0000 (19:09 -0500)]
[bindings] Update demos for new upstream API

3 years agoMake IgnoringMessageHandler and ErroringMessageHandler pub
Matt Corallo [Tue, 2 Mar 2021 15:25:37 +0000 (10:25 -0500)]
Make IgnoringMessageHandler and ErroringMessageHandler pub

This is largely useful for bindings, and the off-github discussion
around #814 concluded these should be pub, but the PR was not
updated to capture this. Now that the bindings support generation
for the structs, expose them.

3 years ago[bindings] Handle ::-containing paths in generics
Matt Corallo [Mon, 1 Mar 2021 22:08:51 +0000 (17:08 -0500)]
[bindings] Handle ::-containing paths in generics

Specifically, this is required for some paths which use
`Option<msgs::Message>`.

3 years ago[bindings] Be explicit when calling pointer.is_null()
Matt Corallo [Mon, 1 Mar 2021 22:07:29 +0000 (17:07 -0500)]
[bindings] Be explicit when calling pointer.is_null()

When the (somewhat anti-pattern)
`impl Deref for X { type Target = X; .. }` is used, this avoids an
infinite dereference exception trying to figure out what type to
resolve `is_null` against.

3 years ago[bindings] Avoid double-use'ing traits by writing out the path
Matt Corallo [Mon, 1 Mar 2021 22:06:25 +0000 (17:06 -0500)]
[bindings] Avoid double-use'ing traits by writing out the path

3 years agoChange ChannelManager::wait to be more descriptive
Matt Corallo [Fri, 26 Feb 2021 16:28:55 +0000 (11:28 -0500)]
Change ChannelManager::wait to be more descriptive

`wait` doesn't capture enough of what's going on, but also Java
Java doesn't accpet methods just called `wait`, as it conflicts
with existing sync primitives on all Objects.

3 years agoAdd utility in `ChannelMonitor` to reload `chain::Filter` data
Matt Corallo [Sun, 7 Mar 2021 17:58:14 +0000 (12:58 -0500)]
Add utility in `ChannelMonitor` to reload `chain::Filter` data

The deserialization process for `ChannelManager`/`ChannelMonitor`
data includes reloading any relevant `chain::Filter` with data
provided from the `ChannelMonitor`, but its nice if we adapt the
data to `chain::Filter` calls for users.

3 years ago[bindings] Support converting Network to rust, update variants
Matt Corallo [Fri, 5 Mar 2021 23:51:23 +0000 (18:51 -0500)]
[bindings] Support converting Network to rust, update variants

3 years agoMerge pull request #823 from jkczyz/2021-03-birthday-hash
Matt Corallo [Sat, 6 Mar 2021 00:15:05 +0000 (16:15 -0800)]
Merge pull request #823 from jkczyz/2021-03-birthday-hash

Replace default hash with birthday hash

3 years agoCorrectly update the last block hash on disconnect
Jeffrey Czyz [Thu, 4 Mar 2021 02:33:54 +0000 (18:33 -0800)]
Correctly update the last block hash on disconnect

When a block is disconnected, the hash of the disconnected block was
used to update the last connected block. However, this amounts to a
no-op because these hashes should be equal. Successive disconnections
would update the hash but leave it one block off.

Normally, this not a problem because the last block_disconnected should
be followed by block_connected since the former is triggered by a chain
re-org. However, this assumes the user calls the API correctly and that
no failure occurs that would prevent block_connected from being called
(e.g., if fetching the connected block fails).

Instead, update the last block hash with the disconnected block's
previous block hash.

3 years agoHold ChannelManager locks independently
Jeffrey Czyz [Fri, 5 Mar 2021 20:37:50 +0000 (12:37 -0800)]
Hold ChannelManager locks independently

ChannelManager reads channel_state and last_block_hash while processing
funding_created and funding_signed messages. It writes these while
processing block_connected and block_disconnected events. To avoid any
potential deadlocks, have each site hold these locks independent of one
another and in a consistent order.

Additionally, use a RwLock instead of Mutex for last_block_hash since
exclusive access is not needed in funding_created / funding_signed and
cannot be guaranteed in block_connected / block_disconnected because of
the reads in the former.

3 years agoRename header_hash to block_hash
Jeffrey Czyz [Thu, 4 Mar 2021 19:47:09 +0000 (11:47 -0800)]
Rename header_hash to block_hash

3 years agoPass along ChannelManager's last_block_hash
Jeffrey Czyz [Fri, 5 Mar 2021 02:22:37 +0000 (18:22 -0800)]
Pass along ChannelManager's last_block_hash

ChannelMonitor keeps track of the last block connected. However, it is
initialized with the default block hash, which is a problem if the
ChannelMonitor is serialized before a block is connected. Instead, pass
ChannelManager's last_block_hash, which is initialized with a "birthday"
hash, when creating a new ChannelMonitor.

3 years agoRemove last_block_connected from Channel
Jeffrey Czyz [Fri, 5 Mar 2021 01:39:21 +0000 (17:39 -0800)]
Remove last_block_connected from Channel

Tracking the last block was only used to de-duplicate block_connected
calls, but this is no longer required as of the previous commit.
Further, the ChannelManager can pass the latest block hash when needing
to create a ChannelMonitor rather than have each Channel maintain an
up-to-date copy. This is implemented in the next commit.

3 years agoRemove block_connected de-duplication logic
Jeffrey Czyz [Fri, 5 Mar 2021 00:58:26 +0000 (16:58 -0800)]
Remove block_connected de-duplication logic

Calling block_connected for the same block was necessary when clients
were expected to re-fetch filtered blocks with relevant transactions. At
the time, all listeners were notified of the connected block again for
re-scanning. Thus, de-duplication logic was put in place.

However, this requirement is no longer applicable for ChannelManager as
of commit bd39b20f642e042981e4fdd5f3600a357be51931, so remove this
logic.

3 years agoParameterize ChannelManager::new with a block hash
Jeffrey Czyz [Wed, 3 Mar 2021 19:24:55 +0000 (11:24 -0800)]
Parameterize ChannelManager::new with a block hash

When ChannelMonitors are persisted, they need to store the most recent
block hash seen. However, for newly created channels the default block
hash is used. If persisted before a block is connected, the funding
output may be missed when syncing after a restart. Instead, initialize
ChannelManager with a "birthday" hash so it can be used later when
creating channels.

3 years agoHandle query_channel_range message from peer
bmancini55 [Wed, 3 Mar 2021 21:48:19 +0000 (16:48 -0500)]
Handle query_channel_range message from peer

Initial implementation of handling query_channel_range messages in
NetGraphMsgHandler.  Enqueues a sequence of reply message in the pending
message events buffer.

3 years agoUpdate docs with correct hash type
Matt Corallo [Tue, 2 Mar 2021 18:07:43 +0000 (13:07 -0500)]
Update docs with correct hash type

Co-authored-by: Matt Corallo <git@bluematt.me>
Co-authored-by: Jeffrey Czyz <jkczyz@gmail.com>
3 years agoRevert "Merge pull request #819 from TheBlueMatt/2021-03-810-rebased"
Jeffrey Czyz [Fri, 5 Mar 2021 21:28:20 +0000 (13:28 -0800)]
Revert "Merge pull request #819 from TheBlueMatt/2021-03-810-rebased"

This reverts commit 793de5fe6944bb6ab414934e53a7ae80bb5a9a31, reversing
changes made to 03a518965100b6852f36e4f95ead4c1d93f5c4b0.

3 years agoMerge pull request #808 from TheBlueMatt/2021-02-791-order-fix
Matt Corallo [Fri, 5 Mar 2021 20:16:21 +0000 (12:16 -0800)]
Merge pull request #808 from TheBlueMatt/2021-02-791-order-fix

Process monitor update events in block_[dis]connected asynchronously

3 years agoCreate short_channel_id utils
bmancini55 [Wed, 3 Mar 2021 21:46:55 +0000 (16:46 -0500)]
Create short_channel_id utils

Util converts parts into u64 SCID
Util extracts block height from u64 SCID

Both will be used when processing gossip_query messages.

3 years agoAdd end_blocknum method to query_channel_range
bmancini55 [Tue, 2 Mar 2021 16:56:00 +0000 (11:56 -0500)]
Add end_blocknum method to query_channel_range

Overflow safe calculation of the ending block number for a query. Can
be used when processing the query.

3 years agoAdd SendReplyChannelRange message event
bmancini55 [Fri, 19 Feb 2021 21:56:48 +0000 (16:56 -0500)]
Add SendReplyChannelRange message event

Creates a MessageSendEvent for sending a reply_channel_range message.
This event will be fired when handling inbound query_channel_range
messages in the NetGraphMessageHandler.

3 years agoClarify ChannelManager docs somewhat around full blocks 2021-02-791-order-fix
Matt Corallo [Mon, 1 Mar 2021 22:30:41 +0000 (17:30 -0500)]
Clarify ChannelManager docs somewhat around full blocks

As suggested by Val.

3 years agoExpand tests to cover serialization with pending background events
Matt Corallo [Fri, 26 Feb 2021 17:50:46 +0000 (12:50 -0500)]
Expand tests to cover serialization with pending background events

3 years agoProcess monitor update events in block_[dis]connected asynchronously
Matt Corallo [Fri, 26 Feb 2021 17:02:11 +0000 (12:02 -0500)]
Process monitor update events in block_[dis]connected asynchronously

The instructions for `ChannelManagerReadArgs` indicate that you need
to connect blocks on a newly-deserialized `ChannelManager` in a
separate pass from the newly-deserialized `ChannelMontiors` as the
`ChannelManager` assumes the ability to update the monitors during
block [dis]connected events, saying that users need to:
```
4) Reconnect blocks on your ChannelMonitors
5) Move the ChannelMonitors into your local chain::Watch.
6) Disconnect/connect blocks on the ChannelManager.
```

This is fine for `ChannelManager`'s purpose, but is very awkward
for users. Notably, our new `lightning-block-sync` implemented
on-load reconnection in the most obvious (and performant) way -
connecting the blocks all at once, violating the
`ChannelManagerReadArgs` API.

Luckily, the events in question really don't need to be processed
with the same urgency as most channel monitor updates. The only two
monitor updates which can occur in block_[dis]connected is either
a) in block_connected, we identify a now-confirmed commitment
   transaction, closing one of our channels, or
b) in block_disconnected, the funding transaction is reorganized
   out of the chain, making our channel no longer funded.
In the case of (a), sending a monitor update which broadcasts a
conflicting holder commitment transaction is far from
time-critical, though we should still ensure we do it. In the case
of (b), we should try to broadcast our holder commitment transaction
when we can, but within a few minutes is fine on the scale of
block mining anyway.

Note that in both cases cannot simply move the logic to
ChannelMonitor::block[dis]_connected, as this could result in us
broadcasting a commitment transaction from ChannelMonitor, then
revoking the now-broadcasted state, and only then receiving the
block_[dis]connected event in the ChannelManager.

Thus, we move both events into an internal invent queue and process
them in timer_chan_freshness_every_min().

3 years agoMerge pull request #825 from jkczyz/2021-03-client-visibility
Matt Corallo [Fri, 5 Mar 2021 17:44:28 +0000 (09:44 -0800)]
Merge pull request #825 from jkczyz/2021-03-client-visibility

Expose RpcClient and RestClient interfaces as pub

3 years agoMerge pull request #821 from jkczyz/2021-03-fix-build-warnings
Matt Corallo [Fri, 5 Mar 2021 17:44:21 +0000 (09:44 -0800)]
Merge pull request #821 from jkczyz/2021-03-fix-build-warnings

Fix build warnings

3 years agoExpose RpcClient and RestClient interfaces as pub
Jeffrey Czyz [Thu, 4 Mar 2021 01:32:39 +0000 (17:32 -0800)]
Expose RpcClient and RestClient interfaces as pub

Useful for use outside of the BlockSource context, e.g., when
implementing fee estimation or transaction broadcasting.

3 years agoFix build warnings
Jeffrey Czyz [Wed, 3 Mar 2021 05:30:34 +0000 (21:30 -0800)]
Fix build warnings