Matt Corallo [Mon, 24 Feb 2020 23:47:38 +0000 (18:47 -0500)]
Dont treat a timer tick as no_connection_possible and log
In testing, due to other patches, I managed to flood the send queue
with messages and cause us not to be able to send pings, thus
getting a peer disconnected for ping timeout. To my surprise, this
also force-closed all of my channels with that peeer.
Obviously a ping timeout does not indicate that no future connection
with said peer will be possible, and we shouldn't be force-closing
channels as a result.
This also logs when a peer is disconnected to ping timeout to make
debug easier.
Matt Corallo [Wed, 12 Feb 2020 20:47:04 +0000 (15:47 -0500)]
Drop Clone from ChannelMonitor.
This removes the somewhat-easy-to-misuse Clone from ChannelMonitors,
opening us up to being able to track Events in ChannelMonitors with
less risk of misuse.
Sadly it doesn't remove the Clone requirement for ChannelKeys,
though gets us much closer - we now just need to request a second
copy once when we go to create the ChannelMonitors.
Matt Corallo [Tue, 11 Feb 2020 23:34:29 +0000 (18:34 -0500)]
Rm ChannelMonitor merge capabilities in favor of explicit add/update
This removes the ability to merge ChannelMonitors in favor of
explicit ChannelMonitorUpdates. It further removes
ChannelManager::test_restore_channel_monitor in favor of the new
ChannelManager::channel_monitor_updated method, which explicitly
confirms a set of updates instead of providing the latest copy of
each ChannelMonitor to the user.
This removes almost all need for Channels to have the latest
channel_monitor, except for broadcasting the latest local state.
Matt Corallo [Sat, 8 Feb 2020 22:22:58 +0000 (17:22 -0500)]
Use ChannelMonitorUpdate in fallen-behind handling during reestablish
This is a rather huge diff, almost entirely due to removing the
type parameter from ChannelError which was added in c20e930b31e973e0fb290322c9ac425002e3b672 due to holding the
ChannelKeys in ChannelMonitors.
Matt Corallo [Sat, 8 Feb 2020 21:17:59 +0000 (16:17 -0500)]
Set ChannelMonitor basic_channel_info on funding, not on accept
This prepares for only creating the ChannelMonitor on funding by
removing any channel_monitor calls from Channel open/accept-time to
funding-signed time.
Matt Corallo [Sat, 8 Feb 2020 01:08:31 +0000 (20:08 -0500)]
Use ChannelMonitorUpdates in commitment signing fns in Channel
This is a rather big step towards using the new ChannelMonitorUpdate
flow, using it in the various commitment signing and commitment
update message processing functions in Channel. Becase they all
often call each other, they all have to be updated as a group,
resulting in the somewhat large diff in this commit.
In order to keep the update_ids strictly increasing by one for
ease of use on the user end, we have to play some games with the
latest_monitor_update_id field, though its generally still pretty
readable, and the pattern of "get an update_id at the start, and
use the one we got at the start when returning, irrespective of
what other calls into the Channel during that time did" is
relatively straightforward.
Matt Corallo [Thu, 6 Feb 2020 18:53:56 +0000 (13:53 -0500)]
Update Channel::funding_signed to use ChannelMonitorUpdate
This is the first of several steps to update ChannelMonitor updates
to use the new ChannelMonitorUpdate objects, demonstrating how the
new flow works in Channel.
Matt Corallo [Thu, 6 Feb 2020 00:39:31 +0000 (19:39 -0500)]
Add types for updating ChannelMonitors without copying them.
This is the first step in migrating ChannelMonitor updating logic
to use incremental Update objects instead of copying the
ChannelMonitors themselves and insert_combine()ing them.
This adds most of the scaffolding and updates relevant comments to
refer to the new architecture, without changing how any actual
updates occur.
Matt Corallo [Thu, 6 Feb 2020 05:03:32 +0000 (00:03 -0500)]
Use Channel::funding_txo instead of its channel_monitor.funding_txo
Currently Channel relies on its own internal channel_monitor copy
to keep track of funding_txo information, which is both a bit
awkward and not ideal if we want to get rid of the ChannelMonitor
copy in Channel.
Instead, just duplicate it (its small) and keep it directly in
Channel, allowing us to remove the (super awkward)
ChannelMonitor::unset_funding_txo().
Matt Corallo [Fri, 7 Feb 2020 22:48:46 +0000 (17:48 -0500)]
Track counterparty's commitment secrets in Channel directly.
In the process of removing a local ChannelMonitor in each Channel,
we need to track our counterpartys' commitment secrets so that we
can check them locally instead of calling our channel monitor to
do that work for us.
Matt Corallo [Fri, 7 Feb 2020 21:05:24 +0000 (16:05 -0500)]
Swap out ChannelMonitor remote secret tracking for a struct.
In order to drop the ChannelMonitor from Channel, we need to track
remote per_commitment_secrets outside of the monitor to validate new
ones as they come in.
This just moves the current code from ChannelMonitor into a new
CounterpartyCommitmentSecrets struct in chan_utils.
Matt Corallo [Thu, 20 Feb 2020 20:12:42 +0000 (15:12 -0500)]
Fix incorrect docs around disconnect in peer_handler + rename fns
The way PeerHandler was written, it was supposed to remove from
self.peers iff the API docs indicate that disconnect_event should
NOT be called (and otherwise rely on disconnect_event to do so).
Sadly, the implementation was way out of whack with reality - in
the implementation, essentially anywhere where PeerHandler
originated the disconnection, the peer was removed and no
disconnect_event was expected. The docs, however, indicated that
disconnect_event should nearly only be called, only not doing so
when the initial handshake message never completed.
We opt to change the docs, mostly, as well as clean up the
ping/pong handling somewhat and rename a few functions to clarify
what they actually do.
Matt Corallo [Tue, 4 Feb 2020 04:46:29 +0000 (23:46 -0500)]
Move pending-HTLC-updated ChannelMonitor from ManyChannelMonitor
This is important for a number of reasons:
* Firstly, I hit this trying to implement rescan in the demo
bitcoinrpc client - if individual ChannelMonitors are out of
sync with each other, we cannot add them all into a
ManyChannelMonitor together and then rescan, but need to rescan
them individually without having to do a bunch of manual work.
Of the three return values in ChannelMonitor::block_connected,
only the HTLCsource stuff that is moved here makes no sense to
be exposed to the user.
* Secondly, the logic currently in ManyChannelMonitor cannot be
reproduced by the user! HTLCSource is deliberately an opaque
type but we use its data to decide which things to keep when
inserting into the HashMap. This would prevent a user from
properly implementing a replacement ManyChannelMonitor, which is
unacceptable.
* Finally, by moving the tracking into ChannelMonitor, we can
serialize them out, which prevents us from forgetting them when
loading from disk, though there are still other races which need
to be handled to make this fully safe (see TODOs in
ChannelManager).
This is safe as no two entries can have the same HTLCSource across
different channels (or, if they did, it would be a rather serious
bug), though note that, IIRC, when this code was added, the
HTLCSource field in the values was not present.
We also take this opportunity to rename the fetch function to match
our other event interfaces, makaing it clear that by calling the
function the set of HTLCUpdates will also be cleared.
Matt Corallo [Sat, 15 Feb 2020 17:12:50 +0000 (12:12 -0500)]
Allow deserialization of new Channels before we've seen a block
Previously, if we have a live ChannelManager (that has seen blocks)
and we open a new Channel, if we serialize that ChannelManager
before a new block comes in, we'll fail to deserialize it. This is
the result of an overly-ambigious last_block_connected check which
would see 0s for the new channel but the previous block for the
ChannelManager as a whole.
We add a new test which catches this error as well as hopefully
getting some test coverage for other similar issues in the future.
Matt Corallo [Mon, 17 Feb 2020 18:50:46 +0000 (13:50 -0500)]
Fix serialization rt bug in Channel and test in functional_tests
Previously, when attempting to write out a channel with some
RemoteAnnounced pending inbound HTLCs, we'd write out the count
without them, but write out some of their fields. We should drop
them as intended as they will need to be reannounced upon
reconnection.
This was found while attempting to simply reproduce a different
bug by adding tests for ChannelManager serialization rount-trip at
the end of each functional_test (in Node::drop). That test is
included here to prevent some classes of similar bugs in the future.
Matt Corallo [Tue, 21 Jan 2020 03:13:44 +0000 (22:13 -0500)]
Test that we don't forget to track any outputs at monitor-load
This tests, after each functional test, that if we serialize and
reload all of our ChannelMonitors we end up tracking the same set
of outputs as before.
Matt Corallo [Tue, 21 Jan 2020 03:05:29 +0000 (22:05 -0500)]
Track the full list of outpoints a chanmon wants monitoring for.
Upon deserialization/reload we need to be able to register each
outpoint which spends the commitment txo which a channelmonitor
believes to be on chain. While our other internal tracking is
likely sufficient to regenerate these, its much easier to simply
track all outpouts we've ever generated, so we do that here.
Antoine Riard [Thu, 23 Jan 2020 21:45:14 +0000 (16:45 -0500)]
Remove duplicata of broadcast txn from ChannelMonitor
Previously, if new ouputs were found to be watched as part
of channel operations, the block was rescan which triggers
again parser and generation of transactions already issued.
This commit first modifies the test framework without
altering further ChannelMonitor.
ChannelMonitor refactoring is introduced in a latter commit.
Matt Corallo [Sat, 28 Dec 2019 18:44:47 +0000 (13:44 -0500)]
Use RouteHop's new node_features to send TLV-encoded onion hops
This implements the new TLV variable-length encoding for onion hop
data, opting to send it if the RouteHop's node_features indicates
support. It also uses the new process_inline method in ChaCha20 to
optimize a few things (though it grows a new TODO for a
probably-important optimization).
Matt Corallo [Mon, 3 Feb 2020 01:44:54 +0000 (20:44 -0500)]
Add macros for building TLV (de)serializers.
There's quite a bit of machinery included here, but it neatly
avoids any dynamic allocation during TLV deserialization, and the
calling side looks nice and simple. The macro-generated code is
pretty nice, though has some redundant if statements (I haven't
checked if they get optimized out yet, but I can't imagine they
don't).
Matt Corallo [Mon, 3 Feb 2020 01:42:40 +0000 (20:42 -0500)]
Add new streams and serialization wrappers for TLV types.
This adds a number of new stream adapters to track and/or calculate
the number of bytes read/written to an underlying stream, as well
as wrappers for the two (?!) variable-length integer types that TLV
introduces.
Matt Corallo [Fri, 27 Dec 2019 22:38:15 +0000 (17:38 -0500)]
Add a ChaChaReader adapter to read an encrypted stream & use it
This prepares for variable-length per-hop-data by wrapping the full
hop_data field in a decrypting stream, with a few minor
optimizations and redundant allocations to boot.
Matt Corallo [Thu, 26 Dec 2019 18:45:44 +0000 (13:45 -0500)]
Move BogusHopData generation into test instead of OnionHopData.
This, as it should be, restricts OnionHopData to only being able to
represent valid states, while still allowing for tests to generate
bogus hop data fields to test deserialization.
Matt Corallo [Thu, 26 Dec 2019 18:43:43 +0000 (13:43 -0500)]
Pull hmac out of OnionHopData.
Its a bit awkward to have an hmac field covering the struct that
its in, and there is little difference in removing it, so just pull
it out and use a [u8; 32] where we care about the hmac.
Matt Corallo [Tue, 24 Dec 2019 20:52:47 +0000 (15:52 -0500)]
Flatten OnionHopData struct with the Realm0 struct.
Previously OnionHopData contained a OnionRealm0HopData field however
instead of bumping the realm number, it has been replaced with a
length, used to indicte the length of a TLV-formatted object.
Because a TLV-formatted hop data can contain the same information as
a realm-0 hop data, we flatten the field and simply keep track of
what format it was in.
Matt Corallo [Mon, 10 Feb 2020 09:27:23 +0000 (10:27 +0100)]
Fix sending funding_locked with 1 conf.
We previously tracked funding transaction confirmation by marking
funding_tx_confirmations to 1 when we see it in a block and
incrementing each block thereafter if its non-0. To avoid
double-incrementing the first confirmation, we did the increment
(and funding_locked check) after doing the first-confirmation
checks. Thus, we'd never hit the funding_locked case during the
first confirmation.
To address this, we simply swap the order of the checks, though
bumping the funding_tx_confirmations increment up to the top.
Jeffrey Czyz [Mon, 10 Feb 2020 19:13:41 +0000 (11:13 -0800)]
Move initial_routing_sync decision to the Router
PeerManager determines whether the initial_routing_sync feature bit
should be set when sending Init messages to peers. Move this to the
Router as it is better able to determine if a full sync is needed.
Matt Corallo [Sun, 12 Jan 2020 23:04:40 +0000 (18:04 -0500)]
Fix EnforcingChannelKeys panic when our counterparty burns their $.
If our counterparty burns their funds by revoking their current
commitment transaction before we've sent them a new one, we'll step
forward the remote commitment number. This would be otherwise fine
(and may even encourage them to broadcast their revoked state(s) on
chain), except that our new EnforcingChannelKeys expects us to not
jump forward in time. Since it isn't too important that we punish
our counterparty in such a corner-case, we opt to just close the
channel in such a case and move on.
Jeffrey Czyz [Fri, 24 Jan 2020 14:43:58 +0000 (06:43 -0800)]
Move message type parity logic to the wire module
Create a MessageType abstraction and use it throughout the wire module's
external interfaces. Include an is_even method for clients to determine
how to handle unknown messages.
Jeffrey Czyz [Tue, 21 Jan 2020 23:26:21 +0000 (15:26 -0800)]
Encapsulate message wire encoding into a module
Lightning messages are identified by a 2-byte type when encoded on the
wire. Rather than expecting callers to know message types when sending
messages to peers, have each message implement a trait defining the
message type. Provide an interface for reading and writing messages
as well as a Message enum for matching the decoded message, including
unknown messages.