From: Matt Corallo <649246+TheBlueMatt@users.noreply.github.com> Date: Tue, 31 Aug 2021 17:50:14 +0000 (+0000) Subject: Merge pull request #1040 from abhik-99/Issue#945 X-Git-Tag: v0.0.101~27 X-Git-Url: http://git.bitcoin.ninja/index.cgi?a=commitdiff_plain;h=a265f1e97a9eff25b3418daf4b12fc9c6ad4c036;hp=fd4d3bf3b76a9490c93d75a85e599989063b6575;p=rust-lightning Merge pull request #1040 from abhik-99/Issue#945 Multi-Hop Route Hint now considered. Added in unit tests for same. --- diff --git a/CHANGELOG.md b/CHANGELOG.md index dc2ea744..a23f9d76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,15 +1,94 @@ -# 0.0.100 - WIP +# 0.0.100 - 2021-08-17 + +## API Updates + * The `lightning` crate can now be built in no_std mode, making it easy to + target embedded hardware for rust users. Note that mutexes are replaced with + no-ops for such builds (#1008, #1028). + * LDK now supports sending and receiving "keysend" payments. This includes + modifications to `lightning::util::events::Event::PaymentReceived` to + indicate the type of payment (#967). + * A new variant, `lightning::util::events::Event::PaymentForwarded` has been + added which indicates a forwarded payment has been successfully claimed and + we've received a forwarding fee (#1004). + * `lightning::chain::keysinterface::KeysInterface::get_shutdown_pubkey` has + been renamed to `get_shutdown_scriptpubkey`, returns a script, and is now + called on channel open only if + `lightning::util::config::ChannelConfig::commit_upfront_shutdown_pubkey` is + set (#1019). + * Closing-signed negotiation is now more configurable, with an explicit + `lightning::util::config::ChannelConfig::force_close_avoidance_max_fee_satoshis` + field allowing you to select the maximum amount you are willing to pay to + avoid a force-closure. Further, we are now less restrictive on the fee + placed on the closing transaction when we are not the party paying it. To + control the feerate paid on a channel at close-time, use + `ChannelManager::close_channel_with_target_feerate` instead of + `close_channel` (#1011). + * `lightning_background_processor::BackgroundProcessor` now stops the + background thread when dropped (#1007). It is marked `#[must_use]` so that + Rust users will receive a compile-time warning when it is immediately + dropped after construction (#1029). + * Total potential funds burn on force-close due to dust outputs is now limited + to `lightning::util::config::ChannelConfig::max_dust_htlc_exposure_msat` per + channel (#1009). + * The interval on which + `lightning::ln::peer_handler::PeerManager::timer_tick_occurred` should be + called has been reduced to once every five seconds (#1035) and + `lightning::ln::channelmanager::ChannelManager::timer_tick_occurred` should + now be called on startup in addition to once per minute (#985). + * The rust-bitcoin and bech32 dependencies have been updated to their + respective latest versions (0.27 and 0.8, #1012). + +## Bug Fixes + * Fix panic when reading invoices generated by some versions of c-lightning + (#1002 and #1003). + * Fix panic when attempting to validate a signed message of incorrect length + (#1010). + * Do not ignore the route hints in invoices when the invoice is over 250k + sats (#986). + * Fees are automatically updated on outbound channels to ensure commitment + transactions are always broadcastable (#985). + * Fixes a rare case where a `lightning::util::events::Event::SpendableOutputs` + event is not generated after a counterparty commitment transaction is + confirmed in a reorg when a conflicting local commitment transaction is + removed in the same reorg (#1022). + * Fixes a remotely-triggerable force-closure of an origin channel after an + HTLC was forwarded over a next-hop channel and the next-hop channel was + force-closed by our counterparty (#1025). + * Fixes a rare force-closure case when sending a payment as a channel fundee + when overdrawing our remaining balance. Instead the send will fail (#998). + * Fixes a rare force-closure case when a payment was claimed prior to a + peer disconnection or restart, and later failed (#977). ## Serialization Compatibility + * Pending inbound keysend payments which have neither been failed nor claimed + when serialized will result in a `ChannelManager` which is not readable on + pre-0.0.100 clients (#967). + * Because + `lightning::chain::keysinterface::KeysInterface::get_shutdown_scriptpubkey` + has been updated to return a script instead of only a `PublicKey`, + `ChannelManager`s constructed with custom `KeysInterface` implementations on + 0.0.100 and later versions will not be readable on previous versions. + `ChannelManager`s created with 0.0.99 and prior versions will remain readable + even after the a serialization roundtrip on 0.0.100, as long as no new + channels are opened. Further, users using a + `lightning::chain::keysinterface::KeysManager` as their `KeysInterface` will + have `ChannelManager`s which are readable on prior versions as well (#1019). + * `ChannelMonitorUpdate`s created by 0.0.100 and later for channels when + `lightning::util::config::ChannelConfig::commit_upfront_shutdown_pubkey` is + not set may not be readable by versions prior to 0.0.100 (#1019). * HTLCs which were in the process of being claimed on-chain when a pre-0.0.100 `ChannelMonitor` was serialized may generate `PaymentForwarded` events with spurious `fee_earned_msat` values. This only applies to payments which were - unresolved at the time of the upgrade. - * 0.0.100 clients with pending PaymentForwarded events at serialization-time - will generate serialized `ChannelManager` objects which 0.0.99 and earlier - clients cannot read. The likelihood of this can be reduced by ensuring you - process all pending events immediately before serialization (as is done by - the `lightning-background-processor` crate). + unresolved at the time of the upgrade (#1004). + * 0.0.100 clients with pending `Event::PaymentForwarded` events at + serialization-time will generate serialized `ChannelManager` objects which + 0.0.99 and earlier clients cannot read. The likelihood of this can be reduced + by ensuring you process all pending events immediately before serialization + (as is done by the `lightning-background-processor` crate, #1004). + + +In total, this release features 59 files changed, 5861 insertions, and 2082 +deletions in 95 commits from 6 authors. # 0.0.99 - 2021-07-09 diff --git a/ci/check-compiles.sh b/ci/check-compiles.sh index 2bc31007..193c2b4e 100755 --- a/ci/check-compiles.sh +++ b/ci/check-compiles.sh @@ -5,5 +5,5 @@ echo Testing $(git log -1 --oneline) cargo check cargo doc cargo doc --document-private-items -cd fuzz && cargo check --features=stdin_fuzz +cd fuzz && RUSTFLAGS="--cfg=fuzzing" cargo check --features=stdin_fuzz cd ../lightning && cargo check --no-default-features --features=no-std diff --git a/fuzz/src/chanmon_consistency.rs b/fuzz/src/chanmon_consistency.rs index 70ddac5d..88826d65 100644 --- a/fuzz/src/chanmon_consistency.rs +++ b/fuzz/src/chanmon_consistency.rs @@ -37,9 +37,11 @@ use lightning::chain::chaininterface::{BroadcasterInterface, ConfirmationTarget, use lightning::chain::keysinterface::{KeysInterface, InMemorySigner}; use lightning::ln::{PaymentHash, PaymentPreimage, PaymentSecret}; use lightning::ln::channelmanager::{ChainParameters, ChannelManager, PaymentSendFailure, ChannelManagerReadArgs}; +use lightning::ln::channel::FEE_SPIKE_BUFFER_FEE_INCREASE_MULTIPLE; use lightning::ln::features::{ChannelFeatures, InitFeatures, NodeFeatures}; use lightning::ln::msgs::{CommitmentUpdate, ChannelMessageHandler, DecodeError, UpdateAddHTLC, Init}; -use lightning::util::enforcing_trait_impls::{EnforcingSigner, INITIAL_REVOKED_COMMITMENT_NUMBER}; +use lightning::ln::script::ShutdownScript; +use lightning::util::enforcing_trait_impls::{EnforcingSigner, EnforcementState}; use lightning::util::errors::APIError; use lightning::util::events; use lightning::util::logger::Logger; @@ -57,16 +59,27 @@ use bitcoin::secp256k1::recovery::RecoverableSignature; use bitcoin::secp256k1::Secp256k1; use std::mem; -use std::cmp::Ordering; +use std::cmp::{self, Ordering}; use std::collections::{HashSet, hash_map, HashMap}; use std::sync::{Arc,Mutex}; use std::sync::atomic; use std::io::Cursor; -struct FuzzEstimator {} +const MAX_FEE: u32 = 10_000; +struct FuzzEstimator { + ret_val: atomic::AtomicU32, +} impl FeeEstimator for FuzzEstimator { - fn get_est_sat_per_1000_weight(&self, _: ConfirmationTarget) -> u32 { - 253 + fn get_est_sat_per_1000_weight(&self, conf_target: ConfirmationTarget) -> u32 { + // We force-close channels if our counterparty sends us a feerate which is a small multiple + // of our HighPriority fee estimate or smaller than our Background fee estimate. Thus, we + // always return a HighPriority feerate here which is >= the maximum Normal feerate and a + // Background feerate which is <= the minimum Normal feerate. + match conf_target { + ConfirmationTarget::HighPriority => MAX_FEE, + ConfirmationTarget::Background => 253, + ConfirmationTarget::Normal => cmp::min(self.ret_val.load(atomic::Ordering::Acquire), MAX_FEE), + } } } @@ -131,7 +144,7 @@ impl chain::Watch for TestChainMonitor { }; let deserialized_monitor = <(BlockHash, channelmonitor::ChannelMonitor)>:: read(&mut Cursor::new(&map_entry.get().1), &*self.keys).unwrap().1; - deserialized_monitor.update_monitor(&update, &&TestBroadcaster{}, &&FuzzEstimator{}, &self.logger).unwrap(); + deserialized_monitor.update_monitor(&update, &&TestBroadcaster{}, &&FuzzEstimator { ret_val: atomic::AtomicU32::new(253) }, &self.logger).unwrap(); let mut ser = VecWriter(Vec::new()); deserialized_monitor.write(&mut ser).unwrap(); map_entry.insert((update.update_id, ser.0)); @@ -148,7 +161,7 @@ impl chain::Watch for TestChainMonitor { struct KeyProvider { node_id: u8, rand_bytes_id: atomic::AtomicU32, - revoked_commitments: Mutex>>>, + enforcement_states: Mutex>>>, } impl KeysInterface for KeyProvider { type Signer = EnforcingSigner; @@ -164,9 +177,11 @@ impl KeysInterface for KeyProvider { Builder::new().push_opcode(opcodes::all::OP_PUSHBYTES_0).push_slice(&our_channel_monitor_claim_key_hash[..]).into_script() } - fn get_shutdown_pubkey(&self) -> PublicKey { + fn get_shutdown_scriptpubkey(&self) -> ShutdownScript { let secp_ctx = Secp256k1::signing_only(); - PublicKey::from_secret_key(&secp_ctx, &SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, self.node_id]).unwrap()) + let secret_key = SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, self.node_id]).unwrap(); + let pubkey_hash = WPubkeyHash::hash(&PublicKey::from_secret_key(&secp_ctx, &secret_key).serialize()); + ShutdownScript::new_p2wpkh(&pubkey_hash) } fn get_channel_signer(&self, _inbound: bool, channel_value_satoshis: u64) -> EnforcingSigner { @@ -183,7 +198,7 @@ impl KeysInterface for KeyProvider { channel_value_satoshis, [0; 32], ); - let revoked_commitment = self.make_revoked_commitment_cell(keys.commitment_seed); + let revoked_commitment = self.make_enforcement_state_cell(keys.commitment_seed); EnforcingSigner::new_with_revoked(keys, revoked_commitment, false) } @@ -198,14 +213,11 @@ impl KeysInterface for KeyProvider { let mut reader = std::io::Cursor::new(buffer); let inner: InMemorySigner = Readable::read(&mut reader)?; - let revoked_commitment = self.make_revoked_commitment_cell(inner.commitment_seed); - - let last_commitment_number = Readable::read(&mut reader)?; + let state = self.make_enforcement_state_cell(inner.commitment_seed); Ok(EnforcingSigner { inner, - last_commitment_number: Arc::new(Mutex::new(last_commitment_number)), - revoked_commitment, + state, disable_revocation_policy_check: false, }) } @@ -216,10 +228,10 @@ impl KeysInterface for KeyProvider { } impl KeyProvider { - fn make_revoked_commitment_cell(&self, commitment_seed: [u8; 32]) -> Arc> { - let mut revoked_commitments = self.revoked_commitments.lock().unwrap(); + fn make_enforcement_state_cell(&self, commitment_seed: [u8; 32]) -> Arc> { + let mut revoked_commitments = self.enforcement_states.lock().unwrap(); if !revoked_commitments.contains_key(&commitment_seed) { - revoked_commitments.insert(commitment_seed, Arc::new(Mutex::new(INITIAL_REVOKED_COMMITMENT_NUMBER))); + revoked_commitments.insert(commitment_seed, Arc::new(Mutex::new(EnforcementState::new()))); } let cell = revoked_commitments.get(&commitment_seed).unwrap(); Arc::clone(cell) @@ -244,12 +256,14 @@ fn check_api_err(api_err: APIError) { _ if err.starts_with("Cannot send value that would put counterparty balance under holder-announced channel reserve value") => {}, _ if err.starts_with("Cannot send value that would overdraw remaining funds.") => {}, _ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => {}, + _ if err.starts_with("Cannot send value that would put our exposure to dust HTLCs at") => {}, _ => panic!("{}", err), } }, APIError::MonitorUpdateFailed => { // We can (obviously) temp-fail a monitor update }, + APIError::IncompatibleShutdownScript { .. } => panic!("Cannot send an incompatible shutdown script"), } } #[inline] @@ -329,14 +343,13 @@ fn send_hop_payment(source: &ChanMan, middle: &ChanMan, middle_chan_id: u64, des #[inline] pub fn do_test(data: &[u8], out: Out) { - let fee_est = Arc::new(FuzzEstimator{}); let broadcast = Arc::new(TestBroadcaster{}); macro_rules! make_node { - ($node_id: expr) => { { + ($node_id: expr, $fee_estimator: expr) => { { let logger: Arc = Arc::new(test_logger::TestLogger::new($node_id.to_string(), out.clone())); - let keys_manager = Arc::new(KeyProvider { node_id: $node_id, rand_bytes_id: atomic::AtomicU32::new(0), revoked_commitments: Mutex::new(HashMap::new()) }); - let monitor = Arc::new(TestChainMonitor::new(broadcast.clone(), logger.clone(), fee_est.clone(), Arc::new(TestPersister{}), Arc::clone(&keys_manager))); + let keys_manager = Arc::new(KeyProvider { node_id: $node_id, rand_bytes_id: atomic::AtomicU32::new(0), enforcement_states: Mutex::new(HashMap::new()) }); + let monitor = Arc::new(TestChainMonitor::new(broadcast.clone(), logger.clone(), $fee_estimator.clone(), Arc::new(TestPersister{}), Arc::clone(&keys_manager))); let mut config = UserConfig::default(); config.channel_options.forwarding_fee_proportional_millionths = 0; @@ -346,16 +359,16 @@ pub fn do_test(data: &[u8], out: Out) { network, best_block: BestBlock::from_genesis(network), }; - (ChannelManager::new(fee_est.clone(), monitor.clone(), broadcast.clone(), Arc::clone(&logger), keys_manager.clone(), config, params), + (ChannelManager::new($fee_estimator.clone(), monitor.clone(), broadcast.clone(), Arc::clone(&logger), keys_manager.clone(), config, params), monitor, keys_manager) } } } macro_rules! reload_node { - ($ser: expr, $node_id: expr, $old_monitors: expr, $keys_manager: expr) => { { + ($ser: expr, $node_id: expr, $old_monitors: expr, $keys_manager: expr, $fee_estimator: expr) => { { let keys_manager = Arc::clone(& $keys_manager); let logger: Arc = Arc::new(test_logger::TestLogger::new($node_id.to_string(), out.clone())); - let chain_monitor = Arc::new(TestChainMonitor::new(broadcast.clone(), logger.clone(), fee_est.clone(), Arc::new(TestPersister{}), Arc::clone(& $keys_manager))); + let chain_monitor = Arc::new(TestChainMonitor::new(broadcast.clone(), logger.clone(), $fee_estimator.clone(), Arc::new(TestPersister{}), Arc::clone(& $keys_manager))); let mut config = UserConfig::default(); config.channel_options.forwarding_fee_proportional_millionths = 0; @@ -374,7 +387,7 @@ pub fn do_test(data: &[u8], out: Out) { let read_args = ChannelManagerReadArgs { keys_manager, - fee_estimator: fee_est.clone(), + fee_estimator: $fee_estimator.clone(), chain_monitor: chain_monitor.clone(), tx_broadcaster: broadcast.clone(), logger, @@ -393,6 +406,9 @@ pub fn do_test(data: &[u8], out: Out) { let mut channel_txn = Vec::new(); macro_rules! make_channel { ($source: expr, $dest: expr, $chan_id: expr) => { { + $source.peer_connected(&$dest.get_our_node_id(), &Init { features: InitFeatures::known() }); + $dest.peer_connected(&$source.get_our_node_id(), &Init { features: InitFeatures::known() }); + $source.create_channel($dest.get_our_node_id(), 100_000, 42, 0, None).unwrap(); let open_channel = { let events = $source.get_and_clear_pending_msg_events(); @@ -489,11 +505,18 @@ pub fn do_test(data: &[u8], out: Out) { } } } + let fee_est_a = Arc::new(FuzzEstimator { ret_val: atomic::AtomicU32::new(253) }); + let mut last_htlc_clear_fee_a = 253; + let fee_est_b = Arc::new(FuzzEstimator { ret_val: atomic::AtomicU32::new(253) }); + let mut last_htlc_clear_fee_b = 253; + let fee_est_c = Arc::new(FuzzEstimator { ret_val: atomic::AtomicU32::new(253) }); + let mut last_htlc_clear_fee_c = 253; + // 3 nodes is enough to hit all the possible cases, notably unknown-source-unknown-dest // forwarding. - let (node_a, mut monitor_a, keys_manager_a) = make_node!(0); - let (node_b, mut monitor_b, keys_manager_b) = make_node!(1); - let (node_c, mut monitor_c, keys_manager_c) = make_node!(2); + let (node_a, mut monitor_a, keys_manager_a) = make_node!(0, fee_est_a); + let (node_b, mut monitor_b, keys_manager_b) = make_node!(1, fee_est_b); + let (node_c, mut monitor_c, keys_manager_c) = make_node!(2, fee_est_c); let mut nodes = [node_a, node_b, node_c]; @@ -629,10 +652,10 @@ pub fn do_test(data: &[u8], out: Out) { had_events = true; match event { events::MessageSendEvent::UpdateHTLCs { node_id, updates: CommitmentUpdate { update_add_htlcs, update_fail_htlcs, update_fulfill_htlcs, update_fail_malformed_htlcs, update_fee, commitment_signed } } => { - for dest in nodes.iter() { + for (idx, dest) in nodes.iter().enumerate() { if dest.get_our_node_id() == node_id { - assert!(update_fee.is_none()); for update_add in update_add_htlcs.iter() { + out.locked_write(format!("Delivering update_add_htlc to node {}.\n", idx).as_bytes()); if !$corrupt_forward { dest.handle_update_add_htlc(&nodes[$node].get_our_node_id(), update_add); } else { @@ -647,14 +670,21 @@ pub fn do_test(data: &[u8], out: Out) { } } for update_fulfill in update_fulfill_htlcs.iter() { + out.locked_write(format!("Delivering update_fulfill_htlc to node {}.\n", idx).as_bytes()); dest.handle_update_fulfill_htlc(&nodes[$node].get_our_node_id(), update_fulfill); } for update_fail in update_fail_htlcs.iter() { + out.locked_write(format!("Delivering update_fail_htlc to node {}.\n", idx).as_bytes()); dest.handle_update_fail_htlc(&nodes[$node].get_our_node_id(), update_fail); } for update_fail_malformed in update_fail_malformed_htlcs.iter() { + out.locked_write(format!("Delivering update_fail_malformed_htlc to node {}.\n", idx).as_bytes()); dest.handle_update_fail_malformed_htlc(&nodes[$node].get_our_node_id(), update_fail_malformed); } + if let Some(msg) = update_fee { + out.locked_write(format!("Delivering update_fee to node {}.\n", idx).as_bytes()); + dest.handle_update_fee(&nodes[$node].get_our_node_id(), &msg); + } let processed_change = !update_add_htlcs.is_empty() || !update_fulfill_htlcs.is_empty() || !update_fail_htlcs.is_empty() || !update_fail_malformed_htlcs.is_empty(); if $limit_events != ProcessMessages::AllMessages && processed_change { @@ -669,21 +699,24 @@ pub fn do_test(data: &[u8], out: Out) { } }); break; } + out.locked_write(format!("Delivering commitment_signed to node {}.\n", idx).as_bytes()); dest.handle_commitment_signed(&nodes[$node].get_our_node_id(), &commitment_signed); break; } } }, events::MessageSendEvent::SendRevokeAndACK { ref node_id, ref msg } => { - for dest in nodes.iter() { + for (idx, dest) in nodes.iter().enumerate() { if dest.get_our_node_id() == *node_id { + out.locked_write(format!("Delivering revoke_and_ack to node {}.\n", idx).as_bytes()); dest.handle_revoke_and_ack(&nodes[$node].get_our_node_id(), msg); } } }, events::MessageSendEvent::SendChannelReestablish { ref node_id, ref msg } => { - for dest in nodes.iter() { + for (idx, dest) in nodes.iter().enumerate() { if dest.get_our_node_id() == *node_id { + out.locked_write(format!("Delivering channel_reestablish to node {}.\n", idx).as_bytes()); dest.handle_channel_reestablish(&nodes[$node].get_our_node_id(), msg); } } @@ -816,7 +849,9 @@ pub fn do_test(data: &[u8], out: Out) { } } } - match get_slice!(1)[0] { + let v = get_slice!(1)[0]; + out.locked_write(format!("READ A BYTE! HANDLING INPUT {:x}...........\n", v).as_bytes()); + match v { // In general, we keep related message groups close together in binary form, allowing // bit-twiddling mutations to have similar effects. This is probably overkill, but no // harm in doing so. @@ -920,7 +955,7 @@ pub fn do_test(data: &[u8], out: Out) { node_a_ser.0.clear(); nodes[0].write(&mut node_a_ser).unwrap(); } - let (new_node_a, new_monitor_a) = reload_node!(node_a_ser, 0, monitor_a, keys_manager_a); + let (new_node_a, new_monitor_a) = reload_node!(node_a_ser, 0, monitor_a, keys_manager_a, fee_est_a); nodes[0] = new_node_a; monitor_a = new_monitor_a; }, @@ -939,7 +974,7 @@ pub fn do_test(data: &[u8], out: Out) { bc_events.clear(); cb_events.clear(); } - let (new_node_b, new_monitor_b) = reload_node!(node_b_ser, 1, monitor_b, keys_manager_b); + let (new_node_b, new_monitor_b) = reload_node!(node_b_ser, 1, monitor_b, keys_manager_b, fee_est_b); nodes[1] = new_node_b; monitor_b = new_monitor_b; }, @@ -953,7 +988,7 @@ pub fn do_test(data: &[u8], out: Out) { node_c_ser.0.clear(); nodes[2].write(&mut node_c_ser).unwrap(); } - let (new_node_c, new_monitor_c) = reload_node!(node_c_ser, 2, monitor_c, keys_manager_c); + let (new_node_c, new_monitor_c) = reload_node!(node_c_ser, 2, monitor_c, keys_manager_c, fee_est_c); nodes[2] = new_node_c; monitor_c = new_monitor_c; }, @@ -1015,6 +1050,33 @@ pub fn do_test(data: &[u8], out: Out) { 0x6c => { send_hop_payment(&nodes[0], &nodes[1], chan_a, &nodes[2], chan_b, 1, &mut payment_id); }, 0x6d => { send_hop_payment(&nodes[2], &nodes[1], chan_b, &nodes[0], chan_a, 1, &mut payment_id); }, + 0x80 => { + let max_feerate = last_htlc_clear_fee_a * FEE_SPIKE_BUFFER_FEE_INCREASE_MULTIPLE as u32; + if fee_est_a.ret_val.fetch_add(250, atomic::Ordering::AcqRel) + 250 > max_feerate { + fee_est_a.ret_val.store(max_feerate, atomic::Ordering::Release); + } + nodes[0].maybe_update_chan_fees(); + }, + 0x81 => { fee_est_a.ret_val.store(253, atomic::Ordering::Release); nodes[0].maybe_update_chan_fees(); }, + + 0x84 => { + let max_feerate = last_htlc_clear_fee_b * FEE_SPIKE_BUFFER_FEE_INCREASE_MULTIPLE as u32; + if fee_est_b.ret_val.fetch_add(250, atomic::Ordering::AcqRel) + 250 > max_feerate { + fee_est_b.ret_val.store(max_feerate, atomic::Ordering::Release); + } + nodes[1].maybe_update_chan_fees(); + }, + 0x85 => { fee_est_b.ret_val.store(253, atomic::Ordering::Release); nodes[1].maybe_update_chan_fees(); }, + + 0x88 => { + let max_feerate = last_htlc_clear_fee_c * FEE_SPIKE_BUFFER_FEE_INCREASE_MULTIPLE as u32; + if fee_est_c.ret_val.fetch_add(250, atomic::Ordering::AcqRel) + 250 > max_feerate { + fee_est_c.ret_val.store(max_feerate, atomic::Ordering::Release); + } + nodes[2].maybe_update_chan_fees(); + }, + 0x89 => { fee_est_c.ret_val.store(253, atomic::Ordering::Release); nodes[2].maybe_update_chan_fees(); }, + 0xff => { // Test that no channel is in a stuck state where neither party can send funds even // after we resolve all pending events. @@ -1070,6 +1132,10 @@ pub fn do_test(data: &[u8], out: Out) { assert!( send_payment(&nodes[1], &nodes[2], chan_b, 10_000_000, &mut payment_id) || send_payment(&nodes[2], &nodes[1], chan_b, 10_000_000, &mut payment_id)); + + last_htlc_clear_fee_a = fee_est_a.ret_val.load(atomic::Ordering::Acquire); + last_htlc_clear_fee_b = fee_est_b.ret_val.load(atomic::Ordering::Acquire); + last_htlc_clear_fee_c = fee_est_c.ret_val.load(atomic::Ordering::Acquire); }, _ => test_return!(), } diff --git a/fuzz/src/full_stack.rs b/fuzz/src/full_stack.rs index d1adf06e..28592ffd 100644 --- a/fuzz/src/full_stack.rs +++ b/fuzz/src/full_stack.rs @@ -34,14 +34,15 @@ use lightning::chain::transaction::OutPoint; use lightning::chain::keysinterface::{InMemorySigner, KeysInterface}; use lightning::ln::{PaymentHash, PaymentPreimage, PaymentSecret}; use lightning::ln::channelmanager::{ChainParameters, ChannelManager}; -use lightning::ln::peer_handler::{MessageHandler,PeerManager,SocketDescriptor}; +use lightning::ln::peer_handler::{MessageHandler,PeerManager,SocketDescriptor,IgnoringMessageHandler}; use lightning::ln::msgs::DecodeError; +use lightning::ln::script::ShutdownScript; use lightning::routing::router::get_route; use lightning::routing::network_graph::NetGraphMsgHandler; use lightning::util::config::UserConfig; use lightning::util::errors::APIError; use lightning::util::events::Event; -use lightning::util::enforcing_trait_impls::EnforcingSigner; +use lightning::util::enforcing_trait_impls::{EnforcingSigner, EnforcementState}; use lightning::util::logger::Logger; use lightning::util::ser::Readable; @@ -159,7 +160,7 @@ type ChannelMan = ChannelManager< EnforcingSigner, Arc, Arc, Arc, Arc, Arc>>, Arc, Arc, Arc, Arc>; -type PeerMan<'a> = PeerManager, Arc, Arc, Arc>>, Arc>; +type PeerMan<'a> = PeerManager, Arc, Arc, Arc>>, Arc, IgnoringMessageHandler>; struct MoneyLossDetector<'a> { manager: Arc, @@ -271,9 +272,11 @@ impl KeysInterface for KeyProvider { Builder::new().push_opcode(opcodes::all::OP_PUSHBYTES_0).push_slice(&our_channel_monitor_claim_key_hash[..]).into_script() } - fn get_shutdown_pubkey(&self) -> PublicKey { + fn get_shutdown_scriptpubkey(&self) -> ShutdownScript { let secp_ctx = Secp256k1::signing_only(); - PublicKey::from_secret_key(&secp_ctx, &SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]).unwrap()) + let secret_key = SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]).unwrap(); + let pubkey_hash = WPubkeyHash::hash(&PublicKey::from_secret_key(&secp_ctx, &secret_key).serialize()); + ShutdownScript::new_p2wpkh(&pubkey_hash) } fn get_channel_signer(&self, inbound: bool, channel_value_satoshis: u64) -> EnforcingSigner { @@ -312,8 +315,15 @@ impl KeysInterface for KeyProvider { (ctr >> 8*7) as u8, (ctr >> 8*6) as u8, (ctr >> 8*5) as u8, (ctr >> 8*4) as u8, (ctr >> 8*3) as u8, (ctr >> 8*2) as u8, (ctr >> 8*1) as u8, 14, (ctr >> 8*0) as u8] } - fn read_chan_signer(&self, data: &[u8]) -> Result { - EnforcingSigner::read(&mut std::io::Cursor::new(data)) + fn read_chan_signer(&self, mut data: &[u8]) -> Result { + let inner: InMemorySigner = Readable::read(&mut data)?; + let state = Arc::new(Mutex::new(EnforcementState::new())); + + Ok(EnforcingSigner::new_with_revoked( + inner, + state, + false + )) } fn sign_invoice(&self, _invoice_preimage: Vec) -> Result { @@ -374,7 +384,7 @@ pub fn do_test(data: &[u8], logger: &Arc) { let mut loss_detector = MoneyLossDetector::new(&peers, channelmanager.clone(), monitor.clone(), PeerManager::new(MessageHandler { chan_handler: channelmanager.clone(), route_handler: net_graph_msg_handler.clone(), - }, our_network_key, &[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 0], Arc::clone(&logger))); + }, our_network_key, &[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 0], Arc::clone(&logger), IgnoringMessageHandler{})); let mut should_forward = false; let mut payments_received: Vec = Vec::new(); diff --git a/fuzz/src/msg_targets/gen_target.sh b/fuzz/src/msg_targets/gen_target.sh index 044f1a1e..0c1d061a 100755 --- a/fuzz/src/msg_targets/gen_target.sh +++ b/fuzz/src/msg_targets/gen_target.sh @@ -14,7 +14,6 @@ echo "mod utils;" > mod.rs GEN_TEST AcceptChannel test_msg "" GEN_TEST AnnouncementSignatures test_msg "" GEN_TEST ChannelReestablish test_msg "" -GEN_TEST ClosingSigned test_msg "" GEN_TEST CommitmentSigned test_msg "" GEN_TEST DecodedOnionErrorPacket test_msg "" GEN_TEST FundingCreated test_msg "" @@ -40,6 +39,7 @@ GEN_TEST UpdateAddHTLC test_msg_hole ", 85, 33" GEN_TEST ErrorMessage test_msg_hole ", 32, 2" GEN_TEST ChannelUpdate test_msg_hole ", 108, 1" +GEN_TEST ClosingSigned test_msg_simple "" GEN_TEST Init test_msg_simple "" GEN_TEST OnionHopData test_msg_simple "" GEN_TEST Ping test_msg_simple "" diff --git a/fuzz/src/msg_targets/mod.rs b/fuzz/src/msg_targets/mod.rs index e11e3eb2..0f273cb7 100644 --- a/fuzz/src/msg_targets/mod.rs +++ b/fuzz/src/msg_targets/mod.rs @@ -2,7 +2,6 @@ mod utils; pub mod msg_accept_channel; pub mod msg_announcement_signatures; pub mod msg_channel_reestablish; -pub mod msg_closing_signed; pub mod msg_commitment_signed; pub mod msg_decoded_onion_error_packet; pub mod msg_funding_created; @@ -25,6 +24,7 @@ pub mod msg_gossip_timestamp_filter; pub mod msg_update_add_htlc; pub mod msg_error_message; pub mod msg_channel_update; +pub mod msg_closing_signed; pub mod msg_init; pub mod msg_onion_hop_data; pub mod msg_ping; diff --git a/fuzz/src/msg_targets/msg_closing_signed.rs b/fuzz/src/msg_targets/msg_closing_signed.rs index 47881d32..52f39af2 100644 --- a/fuzz/src/msg_targets/msg_closing_signed.rs +++ b/fuzz/src/msg_targets/msg_closing_signed.rs @@ -17,11 +17,11 @@ use utils::test_logger; #[inline] pub fn msg_closing_signed_test(data: &[u8], _out: Out) { - test_msg!(msgs::ClosingSigned, data); + test_msg_simple!(msgs::ClosingSigned, data); } #[no_mangle] pub extern "C" fn msg_closing_signed_run(data: *const u8, datalen: usize) { let data = unsafe { std::slice::from_raw_parts(data, datalen) }; - test_msg!(msgs::ClosingSigned, data); + test_msg_simple!(msgs::ClosingSigned, data); } diff --git a/lightning-background-processor/Cargo.toml b/lightning-background-processor/Cargo.toml index 1659ffd3..98d9ade9 100644 --- a/lightning-background-processor/Cargo.toml +++ b/lightning-background-processor/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lightning-background-processor" -version = "0.0.99" +version = "0.0.100" authors = ["Valentine Wallace "] license = "MIT OR Apache-2.0" repository = "http://github.com/rust-bitcoin/rust-lightning" @@ -11,9 +11,9 @@ edition = "2018" [dependencies] bitcoin = "0.27" -lightning = { version = "0.0.99", path = "../lightning", features = ["allow_wallclock_use"] } -lightning-persister = { version = "0.0.99", path = "../lightning-persister" } +lightning = { version = "0.0.100", path = "../lightning", features = ["allow_wallclock_use"] } +lightning-persister = { version = "0.0.100", path = "../lightning-persister" } [dev-dependencies] -lightning = { version = "0.0.99", path = "../lightning", features = ["_test_utils"] } +lightning = { version = "0.0.100", path = "../lightning", features = ["_test_utils"] } diff --git a/lightning-background-processor/src/lib.rs b/lightning-background-processor/src/lib.rs index afa3633b..34cddd1a 100644 --- a/lightning-background-processor/src/lib.rs +++ b/lightning-background-processor/src/lib.rs @@ -16,6 +16,7 @@ use lightning::chain::keysinterface::{Sign, KeysInterface}; use lightning::ln::channelmanager::ChannelManager; use lightning::ln::msgs::{ChannelMessageHandler, RoutingMessageHandler}; use lightning::ln::peer_handler::{PeerManager, SocketDescriptor}; +use lightning::ln::peer_handler::CustomMessageHandler; use lightning::util::events::{EventHandler, EventsProvider}; use lightning::util::logger::Logger; use std::sync::Arc; @@ -50,6 +51,14 @@ const FRESHNESS_TIMER: u64 = 60; #[cfg(test)] const FRESHNESS_TIMER: u64 = 1; +#[cfg(not(debug_assertions))] +const PING_TIMER: u64 = 5; +/// Signature operations take a lot longer without compiler optimisations. +/// Increasing the ping timer allows for this but slower devices will be disconnected if the +/// timeout is reached. +#[cfg(debug_assertions)] +const PING_TIMER: u64 = 30; + /// Trait which handles persisting a [`ChannelManager`] to disk. /// /// [`ChannelManager`]: lightning::ln::channelmanager::ChannelManager @@ -121,7 +130,8 @@ impl BackgroundProcessor { CMP: 'static + Send + ChannelManagerPersister, M: 'static + Deref> + Send + Sync, CM: 'static + Deref> + Send + Sync, - PM: 'static + Deref> + Send + Sync, + UMH: 'static + Deref + Send + Sync, + PM: 'static + Deref> + Send + Sync, > (persister: CMP, event_handler: EH, chain_monitor: M, channel_manager: CM, peer_manager: PM, logger: L) -> Self where @@ -134,11 +144,16 @@ impl BackgroundProcessor { P::Target: 'static + channelmonitor::Persist, CMH::Target: 'static + ChannelMessageHandler, RMH::Target: 'static + RoutingMessageHandler, + UMH::Target: 'static + CustomMessageHandler, { let stop_thread = Arc::new(AtomicBool::new(false)); let stop_thread_clone = stop_thread.clone(); let handle = thread::spawn(move || -> Result<(), std::io::Error> { - let mut current_time = Instant::now(); + log_trace!(logger, "Calling ChannelManager's timer_tick_occurred on startup"); + channel_manager.timer_tick_occurred(); + + let mut last_freshness_call = Instant::now(); + let mut last_ping_call = Instant::now(); loop { peer_manager.process_events(); channel_manager.process_pending_events(&event_handler); @@ -153,11 +168,27 @@ impl BackgroundProcessor { log_trace!(logger, "Terminating background processor."); return Ok(()); } - if current_time.elapsed().as_secs() > FRESHNESS_TIMER { - log_trace!(logger, "Calling ChannelManager's and PeerManager's timer_tick_occurred"); + if last_freshness_call.elapsed().as_secs() > FRESHNESS_TIMER { + log_trace!(logger, "Calling ChannelManager's timer_tick_occurred"); channel_manager.timer_tick_occurred(); + last_freshness_call = Instant::now(); + } + if last_ping_call.elapsed().as_secs() > PING_TIMER * 2 { + // On various platforms, we may be starved of CPU cycles for several reasons. + // E.g. on iOS, if we've been in the background, we will be entirely paused. + // Similarly, if we're on a desktop platform and the device has been asleep, we + // may not get any cycles. + // In any case, if we've been entirely paused for more than double our ping + // timer, we should have disconnected all sockets by now (and they're probably + // dead anyway), so disconnect them by calling `timer_tick_occurred()` twice. + log_trace!(logger, "Awoke after more than double our ping timer, disconnecting peers."); + peer_manager.timer_tick_occurred(); + peer_manager.timer_tick_occurred(); + last_ping_call = Instant::now(); + } else if last_ping_call.elapsed().as_secs() > PING_TIMER { + log_trace!(logger, "Calling PeerManager's timer_tick_occurred"); peer_manager.timer_tick_occurred(); - current_time = Instant::now(); + last_ping_call = Instant::now(); } } }); @@ -224,8 +255,8 @@ mod tests { use lightning::get_event_msg; use lightning::ln::channelmanager::{BREAKDOWN_TIMEOUT, ChainParameters, ChannelManager, SimpleArcChannelManager}; use lightning::ln::features::InitFeatures; - use lightning::ln::msgs::ChannelMessageHandler; - use lightning::ln::peer_handler::{PeerManager, MessageHandler, SocketDescriptor}; + use lightning::ln::msgs::{ChannelMessageHandler, Init}; + use lightning::ln::peer_handler::{PeerManager, MessageHandler, SocketDescriptor, IgnoringMessageHandler}; use lightning::util::config::UserConfig; use lightning::util::events::{Event, MessageSendEventsProvider, MessageSendEvent}; use lightning::util::ser::Writeable; @@ -253,7 +284,7 @@ mod tests { struct Node { node: Arc>, - peer_manager: Arc, Arc, Arc>>, + peer_manager: Arc, Arc, Arc, IgnoringMessageHandler>>, chain_monitor: Arc, persister: Arc, tx_broadcaster: Arc, @@ -294,10 +325,18 @@ mod tests { let params = ChainParameters { network, best_block }; let manager = Arc::new(ChannelManager::new(fee_estimator.clone(), chain_monitor.clone(), tx_broadcaster.clone(), logger.clone(), keys_manager.clone(), UserConfig::default(), params)); let msg_handler = MessageHandler { chan_handler: Arc::new(test_utils::TestChannelMessageHandler::new()), route_handler: Arc::new(test_utils::TestRoutingMessageHandler::new() )}; - let peer_manager = Arc::new(PeerManager::new(msg_handler, keys_manager.get_node_secret(), &seed, logger.clone())); + let peer_manager = Arc::new(PeerManager::new(msg_handler, keys_manager.get_node_secret(), &seed, logger.clone(), IgnoringMessageHandler{})); let node = Node { node: manager, peer_manager, chain_monitor, persister, tx_broadcaster, logger, best_block }; nodes.push(node); } + + for i in 0..num_nodes { + for j in (i+1)..num_nodes { + nodes[i].node.peer_connected(&nodes[j].node.get_our_node_id(), &Init { features: InitFeatures::known() }); + nodes[j].node.peer_connected(&nodes[i].node.get_our_node_id(), &Init { features: InitFeatures::known() }); + } + } + nodes } @@ -441,8 +480,10 @@ mod tests { let bg_processor = BackgroundProcessor::start(persister, event_handler, nodes[0].chain_monitor.clone(), nodes[0].node.clone(), nodes[0].peer_manager.clone(), nodes[0].logger.clone()); loop { let log_entries = nodes[0].logger.lines.lock().unwrap(); - let desired_log = "Calling ChannelManager's and PeerManager's timer_tick_occurred".to_string(); - if log_entries.get(&("lightning_background_processor".to_string(), desired_log)).is_some() { + let desired_log = "Calling ChannelManager's timer_tick_occurred".to_string(); + let second_desired_log = "Calling PeerManager's timer_tick_occurred".to_string(); + if log_entries.get(&("lightning_background_processor".to_string(), desired_log)).is_some() && + log_entries.get(&("lightning_background_processor".to_string(), second_desired_log)).is_some() { break } } diff --git a/lightning-block-sync/Cargo.toml b/lightning-block-sync/Cargo.toml index 36bb5af9..2a8ff0c6 100644 --- a/lightning-block-sync/Cargo.toml +++ b/lightning-block-sync/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lightning-block-sync" -version = "0.0.99" +version = "0.0.100" authors = ["Jeffrey Czyz", "Matt Corallo"] license = "MIT OR Apache-2.0" repository = "http://github.com/rust-bitcoin/rust-lightning" @@ -15,7 +15,7 @@ rpc-client = [ "serde", "serde_json", "chunked_transfer" ] [dependencies] bitcoin = "0.27" -lightning = { version = "0.0.99", path = "../lightning" } +lightning = { version = "0.0.100", path = "../lightning" } tokio = { version = "1.0", features = [ "io-util", "net", "time" ], optional = true } serde = { version = "1.0", features = ["derive"], optional = true } serde_json = { version = "1.0", optional = true } diff --git a/lightning-block-sync/src/convert.rs b/lightning-block-sync/src/convert.rs index 37b2c432..e8e1427b 100644 --- a/lightning-block-sync/src/convert.rs +++ b/lightning-block-sync/src/convert.rs @@ -4,7 +4,7 @@ use crate::utils::hex_to_uint256; use bitcoin::blockdata::block::{Block, BlockHeader}; use bitcoin::consensus::encode; -use bitcoin::hash_types::{BlockHash, TxMerkleNode}; +use bitcoin::hash_types::{BlockHash, TxMerkleNode, Txid}; use bitcoin::hashes::hex::{ToHex, FromHex}; use serde::Deserialize; @@ -156,11 +156,37 @@ impl TryInto<(BlockHash, Option)> for JsonResponse { } } +impl TryInto for JsonResponse { + type Error = std::io::Error; + fn try_into(self) -> std::io::Result { + match self.0.as_str() { + None => Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "expected JSON string", + )), + Some(hex_data) => match Vec::::from_hex(hex_data) { + Err(_) => Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "invalid hex data", + )), + Ok(txid_data) => match encode::deserialize(&txid_data) { + Err(_) => Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "invalid txid", + )), + Ok(txid) => Ok(txid), + }, + }, + } + } +} + #[cfg(test)] pub(crate) mod tests { use super::*; use bitcoin::blockdata::constants::genesis_block; use bitcoin::consensus::encode; + use bitcoin::hashes::Hash; use bitcoin::network::constants::Network; /// Converts from `BlockHeaderData` into a `GetHeaderResponse` JSON value. @@ -469,4 +495,50 @@ pub(crate) mod tests { }, } } + + #[test] + fn into_txid_from_json_response_with_unexpected_type() { + let response = JsonResponse(serde_json::json!({ "result": "foo" })); + match TryInto::::try_into(response) { + Err(e) => { + assert_eq!(e.kind(), std::io::ErrorKind::InvalidData); + assert_eq!(e.get_ref().unwrap().to_string(), "expected JSON string"); + } + Ok(_) => panic!("Expected error"), + } + } + + #[test] + fn into_txid_from_json_response_with_invalid_hex_data() { + let response = JsonResponse(serde_json::json!("foobar")); + match TryInto::::try_into(response) { + Err(e) => { + assert_eq!(e.kind(), std::io::ErrorKind::InvalidData); + assert_eq!(e.get_ref().unwrap().to_string(), "invalid hex data"); + } + Ok(_) => panic!("Expected error"), + } + } + + #[test] + fn into_txid_from_json_response_with_invalid_txid_data() { + let response = JsonResponse(serde_json::json!("abcd")); + match TryInto::::try_into(response) { + Err(e) => { + assert_eq!(e.kind(), std::io::ErrorKind::InvalidData); + assert_eq!(e.get_ref().unwrap().to_string(), "invalid txid"); + } + Ok(_) => panic!("Expected error"), + } + } + + #[test] + fn into_txid_from_json_response_with_valid_txid_data() { + let target_txid = Txid::from_slice(&[1; 32]).unwrap(); + let response = JsonResponse(serde_json::json!(encode::serialize_hex(&target_txid))); + match TryInto::::try_into(response) { + Err(e) => panic!("Unexpected error: {:?}", e), + Ok(txid) => assert_eq!(txid, target_txid), + } + } } diff --git a/lightning-invoice/Cargo.toml b/lightning-invoice/Cargo.toml index 404f12d7..8c6623b8 100644 --- a/lightning-invoice/Cargo.toml +++ b/lightning-invoice/Cargo.toml @@ -1,7 +1,7 @@ [package] name = "lightning-invoice" description = "Data structures to parse and serialize BOLT11 lightning invoices" -version = "0.7.0" +version = "0.8.0" authors = ["Sebastian Geisler "] documentation = "https://docs.rs/lightning-invoice/" license = "MIT OR Apache-2.0" @@ -10,10 +10,10 @@ readme = "README.md" [dependencies] bech32 = "0.8" -lightning = { version = "0.0.99", path = "../lightning" } +lightning = { version = "0.0.100", path = "../lightning" } secp256k1 = { version = "0.20", features = ["recovery"] } num-traits = "0.2.8" bitcoin_hashes = "0.10" [dev-dependencies] -lightning = { version = "0.0.99", path = "../lightning", features = ["_test_utils"] } +lightning = { version = "0.0.100", path = "../lightning", features = ["_test_utils"] } diff --git a/lightning-net-tokio/Cargo.toml b/lightning-net-tokio/Cargo.toml index 928eab7a..053766e1 100644 --- a/lightning-net-tokio/Cargo.toml +++ b/lightning-net-tokio/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lightning-net-tokio" -version = "0.0.99" +version = "0.0.100" authors = ["Matt Corallo"] license = "MIT OR Apache-2.0" repository = "https://github.com/rust-bitcoin/rust-lightning/" @@ -12,7 +12,7 @@ edition = "2018" [dependencies] bitcoin = "0.27" -lightning = { version = "0.0.99", path = "../lightning" } +lightning = { version = "0.0.100", path = "../lightning" } tokio = { version = "1.0", features = [ "io-util", "macros", "rt", "sync", "net", "time" ] } [dev-dependencies] diff --git a/lightning-net-tokio/src/lib.rs b/lightning-net-tokio/src/lib.rs index 5f5fece0..25c161d2 100644 --- a/lightning-net-tokio/src/lib.rs +++ b/lightning-net-tokio/src/lib.rs @@ -80,6 +80,7 @@ use tokio::io::{AsyncReadExt, AsyncWrite, AsyncWriteExt}; use lightning::ln::peer_handler; use lightning::ln::peer_handler::SocketDescriptor as LnSocketTrait; +use lightning::ln::peer_handler::CustomMessageHandler; use lightning::ln::msgs::{ChannelMessageHandler, RoutingMessageHandler}; use lightning::util::logger::Logger; @@ -119,10 +120,11 @@ struct Connection { id: u64, } impl Connection { - async fn schedule_read(peer_manager: Arc, Arc, Arc>>, us: Arc>, mut reader: io::ReadHalf, mut read_wake_receiver: mpsc::Receiver<()>, mut write_avail_receiver: mpsc::Receiver<()>) where + async fn schedule_read(peer_manager: Arc, Arc, Arc, Arc>>, us: Arc>, mut reader: io::ReadHalf, mut read_wake_receiver: mpsc::Receiver<()>, mut write_avail_receiver: mpsc::Receiver<()>) where CMH: ChannelMessageHandler + 'static, RMH: RoutingMessageHandler + 'static, - L: Logger + 'static + ?Sized { + L: Logger + 'static + ?Sized, + UMH: CustomMessageHandler + 'static { // 8KB is nice and big but also should never cause any issues with stack overflowing. let mut buf = [0; 8192]; @@ -141,30 +143,23 @@ impl Connection { PeerDisconnected } let disconnect_type = loop { - macro_rules! shutdown_socket { - ($err: expr, $need_disconnect: expr) => { { - println!("Disconnecting peer due to {}!", $err); - break $need_disconnect; - } } - } - let read_paused = { let us_lock = us.lock().unwrap(); if us_lock.rl_requested_disconnect { - shutdown_socket!("disconnect_socket() call from RL", Disconnect::CloseConnection); + break Disconnect::CloseConnection; } us_lock.read_paused }; tokio::select! { v = write_avail_receiver.recv() => { assert!(v.is_some()); // We can't have dropped the sending end, its in the us Arc! - if let Err(e) = peer_manager.write_buffer_space_avail(&mut our_descriptor) { - shutdown_socket!(e, Disconnect::CloseConnection); + if let Err(_) = peer_manager.write_buffer_space_avail(&mut our_descriptor) { + break Disconnect::CloseConnection; } }, _ = read_wake_receiver.recv() => {}, read = reader.read(&mut buf), if !read_paused => match read { - Ok(0) => shutdown_socket!("Connection closed", Disconnect::PeerDisconnected), + Ok(0) => break Disconnect::PeerDisconnected, Ok(len) => { let read_res = peer_manager.read_event(&mut our_descriptor, &buf[0..len]); let mut us_lock = us.lock().unwrap(); @@ -174,10 +169,10 @@ impl Connection { us_lock.read_paused = true; } }, - Err(e) => shutdown_socket!(e, Disconnect::CloseConnection), + Err(_) => break Disconnect::CloseConnection, } }, - Err(e) => shutdown_socket!(e, Disconnect::PeerDisconnected), + Err(_) => break Disconnect::PeerDisconnected, }, } peer_manager.process_events(); @@ -222,10 +217,11 @@ impl Connection { /// The returned future will complete when the peer is disconnected and associated handling /// futures are freed, though, because all processing futures are spawned with tokio::spawn, you do /// not need to poll the provided future in order to make progress. -pub fn setup_inbound(peer_manager: Arc, Arc, Arc>>, stream: StdTcpStream) -> impl std::future::Future where +pub fn setup_inbound(peer_manager: Arc, Arc, Arc, Arc>>, stream: StdTcpStream) -> impl std::future::Future where CMH: ChannelMessageHandler + 'static + Send + Sync, RMH: RoutingMessageHandler + 'static + Send + Sync, - L: Logger + 'static + ?Sized + Send + Sync { + L: Logger + 'static + ?Sized + Send + Sync, + UMH: CustomMessageHandler + 'static + Send + Sync { let (reader, write_receiver, read_receiver, us) = Connection::new(stream); #[cfg(debug_assertions)] let last_us = Arc::clone(&us); @@ -262,10 +258,11 @@ pub fn setup_inbound(peer_manager: Arc(peer_manager: Arc, Arc, Arc>>, their_node_id: PublicKey, stream: StdTcpStream) -> impl std::future::Future where +pub fn setup_outbound(peer_manager: Arc, Arc, Arc, Arc>>, their_node_id: PublicKey, stream: StdTcpStream) -> impl std::future::Future where CMH: ChannelMessageHandler + 'static + Send + Sync, RMH: RoutingMessageHandler + 'static + Send + Sync, - L: Logger + 'static + ?Sized + Send + Sync { + L: Logger + 'static + ?Sized + Send + Sync, + UMH: CustomMessageHandler + 'static + Send + Sync { let (reader, mut write_receiver, read_receiver, us) = Connection::new(stream); #[cfg(debug_assertions)] let last_us = Arc::clone(&us); @@ -332,10 +329,11 @@ pub fn setup_outbound(peer_manager: Arc(peer_manager: Arc, Arc, Arc>>, their_node_id: PublicKey, addr: SocketAddr) -> Option> where +pub async fn connect_outbound(peer_manager: Arc, Arc, Arc, Arc>>, their_node_id: PublicKey, addr: SocketAddr) -> Option> where CMH: ChannelMessageHandler + 'static + Send + Sync, RMH: RoutingMessageHandler + 'static + Send + Sync, - L: Logger + 'static + ?Sized + Send + Sync { + L: Logger + 'static + ?Sized + Send + Sync, + UMH: CustomMessageHandler + 'static + Send + Sync { if let Ok(Ok(stream)) = time::timeout(Duration::from_secs(10), async { TcpStream::connect(&addr).await.map(|s| s.into_std().unwrap()) }).await { Some(setup_outbound(peer_manager, their_node_id, stream)) } else { None } @@ -563,7 +561,7 @@ mod tests { let a_manager = Arc::new(PeerManager::new(MessageHandler { chan_handler: Arc::clone(&a_handler), route_handler: Arc::clone(&a_handler), - }, a_key.clone(), &[1; 32], Arc::new(TestLogger()))); + }, a_key.clone(), &[1; 32], Arc::new(TestLogger()), Arc::new(lightning::ln::peer_handler::IgnoringMessageHandler{}))); let (b_connected_sender, mut b_connected) = mpsc::channel(1); let (b_disconnected_sender, mut b_disconnected) = mpsc::channel(1); @@ -577,7 +575,7 @@ mod tests { let b_manager = Arc::new(PeerManager::new(MessageHandler { chan_handler: Arc::clone(&b_handler), route_handler: Arc::clone(&b_handler), - }, b_key.clone(), &[2; 32], Arc::new(TestLogger()))); + }, b_key.clone(), &[2; 32], Arc::new(TestLogger()), Arc::new(lightning::ln::peer_handler::IgnoringMessageHandler{}))); // We bind on localhost, hoping the environment is properly configured with a local // address. This may not always be the case in containers and the like, so if this test is diff --git a/lightning-persister/Cargo.toml b/lightning-persister/Cargo.toml index 81c69d0f..cc585b9c 100644 --- a/lightning-persister/Cargo.toml +++ b/lightning-persister/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lightning-persister" -version = "0.0.99" +version = "0.0.100" authors = ["Valentine Wallace", "Matt Corallo"] license = "MIT OR Apache-2.0" repository = "https://github.com/rust-bitcoin/rust-lightning/" @@ -13,11 +13,11 @@ unstable = ["lightning/unstable"] [dependencies] bitcoin = "0.27" -lightning = { version = "0.0.99", path = "../lightning" } +lightning = { version = "0.0.100", path = "../lightning" } libc = "0.2" [target.'cfg(windows)'.dependencies] winapi = { version = "0.3", features = ["winbase"] } [dev-dependencies] -lightning = { version = "0.0.99", path = "../lightning", features = ["_test_utils"] } +lightning = { version = "0.0.100", path = "../lightning", features = ["_test_utils"] } diff --git a/lightning/Cargo.toml b/lightning/Cargo.toml index 8580a53a..1ef4c74f 100644 --- a/lightning/Cargo.toml +++ b/lightning/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lightning" -version = "0.0.99" +version = "0.0.100" authors = ["Matt Corallo"] license = "MIT OR Apache-2.0" repository = "https://github.com/rust-bitcoin/rust-lightning/" diff --git a/lightning/src/chain/chaininterface.rs b/lightning/src/chain/chaininterface.rs index 83fee4d6..8ccfb945 100644 --- a/lightning/src/chain/chaininterface.rs +++ b/lightning/src/chain/chaininterface.rs @@ -23,6 +23,7 @@ pub trait BroadcasterInterface { /// An enum that represents the speed at which we want a transaction to confirm used for feerate /// estimation. +#[derive(Clone, Copy, PartialEq, Eq)] pub enum ConfirmationTarget { /// We are happy with this transaction confirming slowly when feerate drops some. Background, diff --git a/lightning/src/chain/chainmonitor.rs b/lightning/src/chain/chainmonitor.rs index 12bfebbc..8969427a 100644 --- a/lightning/src/chain/chainmonitor.rs +++ b/lightning/src/chain/chainmonitor.rs @@ -162,6 +162,7 @@ where fn block_connected(&self, block: &Block, height: u32) { let header = &block.header; let txdata: Vec<_> = block.txdata.iter().enumerate().collect(); + log_debug!(self.logger, "New best block {} at height {} provided via block_connected", header.block_hash(), height); self.process_chain_data(header, &txdata, |monitor, txdata| { monitor.block_connected( header, txdata, height, &*self.broadcaster, &*self.fee_estimator, &*self.logger) @@ -170,6 +171,7 @@ where fn block_disconnected(&self, header: &BlockHeader, height: u32) { let monitors = self.monitors.read().unwrap(); + log_debug!(self.logger, "Latest block {} at height {} removed via block_disconnected", header.block_hash(), height); for monitor in monitors.values() { monitor.block_disconnected( header, height, &*self.broadcaster, &*self.fee_estimator, &*self.logger); @@ -187,6 +189,7 @@ where P::Target: channelmonitor::Persist, { fn transactions_confirmed(&self, header: &BlockHeader, txdata: &TransactionData, height: u32) { + log_debug!(self.logger, "{} provided transactions confirmed at height {} in block {}", txdata.len(), height, header.block_hash()); self.process_chain_data(header, txdata, |monitor, txdata| { monitor.transactions_confirmed( header, txdata, height, &*self.broadcaster, &*self.fee_estimator, &*self.logger) @@ -194,6 +197,7 @@ where } fn transaction_unconfirmed(&self, txid: &Txid) { + log_debug!(self.logger, "Transaction {} reorganized out of chain", txid); let monitors = self.monitors.read().unwrap(); for monitor in monitors.values() { monitor.transaction_unconfirmed(txid, &*self.broadcaster, &*self.fee_estimator, &*self.logger); @@ -201,6 +205,7 @@ where } fn best_block_updated(&self, header: &BlockHeader, height: u32) { + log_debug!(self.logger, "New best block {} at height {} provided via best_block_updated", header.block_hash(), height); self.process_chain_data(header, &[], |monitor, txdata| { // While in practice there shouldn't be any recursive calls when given empty txdata, // it's still possible if a chain::Filter implementation returns a transaction. diff --git a/lightning/src/chain/channelmonitor.rs b/lightning/src/chain/channelmonitor.rs index 7904d9bd..dc009c4d 100644 --- a/lightning/src/chain/channelmonitor.rs +++ b/lightning/src/chain/channelmonitor.rs @@ -106,7 +106,9 @@ impl Readable for ChannelMonitorUpdate { let len: u64 = Readable::read(r)?; let mut updates = Vec::with_capacity(cmp::min(len as usize, MAX_ALLOC_SIZE / ::core::mem::size_of::())); for _ in 0..len { - updates.push(Readable::read(r)?); + if let Some(upd) = MaybeReadable::read(r)? { + updates.push(upd); + } } read_tlv_fields!(r, {}); Ok(Self { update_id, updates }) @@ -394,13 +396,36 @@ enum OnchainEvent { }, } -impl_writeable_tlv_based!(OnchainEventEntry, { - (0, txid, required), - (2, height, required), - (4, event, required), -}); +impl Writeable for OnchainEventEntry { + fn write(&self, writer: &mut W) -> Result<(), io::Error> { + write_tlv_fields!(writer, { + (0, self.txid, required), + (2, self.height, required), + (4, self.event, required), + }); + Ok(()) + } +} -impl_writeable_tlv_based_enum!(OnchainEvent, +impl MaybeReadable for OnchainEventEntry { + fn read(reader: &mut R) -> Result, DecodeError> { + let mut txid = Default::default(); + let mut height = 0; + let mut event = None; + read_tlv_fields!(reader, { + (0, txid, required), + (2, height, required), + (4, event, ignorable), + }); + if let Some(ev) = event { + Ok(Some(Self { txid, height, event: ev })) + } else { + Ok(None) + } + } +} + +impl_writeable_tlv_based_enum_upgradable!(OnchainEvent, (0, HTLCUpdate) => { (0, source, required), (1, onchain_value_satoshis, option), @@ -409,7 +434,7 @@ impl_writeable_tlv_based_enum!(OnchainEvent, (1, MaturingOutput) => { (0, descriptor, required), }, -;); +); #[cfg_attr(any(test, feature = "fuzztarget", feature = "_test_utils"), derive(PartialEq))] #[derive(Clone)] @@ -438,9 +463,12 @@ pub(crate) enum ChannelMonitorUpdateStep { /// think we've fallen behind! should_broadcast: bool, }, + ShutdownScript { + scriptpubkey: Script, + }, } -impl_writeable_tlv_based_enum!(ChannelMonitorUpdateStep, +impl_writeable_tlv_based_enum_upgradable!(ChannelMonitorUpdateStep, (0, LatestHolderCommitmentTXInfo) => { (0, commitment_tx, required), (2, htlc_outputs, vec_type), @@ -461,7 +489,10 @@ impl_writeable_tlv_based_enum!(ChannelMonitorUpdateStep, (4, ChannelForceClosed) => { (0, should_broadcast, required), }, -;); + (5, ShutdownScript) => { + (0, scriptpubkey, required), + }, +); /// A ChannelMonitor handles chain events (blocks connected and disconnected) and generates /// on-chain transactions to ensure no loss of funds occurs. @@ -493,7 +524,7 @@ pub(crate) struct ChannelMonitorImpl { destination_script: Script, broadcasted_holder_revokable_script: Option<(Script, PublicKey, PublicKey)>, counterparty_payment_script: Script, - shutdown_script: Script, + shutdown_script: Option