Matt Corallo [Thu, 18 Feb 2021 19:58:05 +0000 (14:58 -0500)]
only dir?
Matt Corallo [Thu, 18 Feb 2021 19:37:43 +0000 (14:37 -0500)]
temp paths
Matt Corallo [Thu, 18 Feb 2021 18:27:56 +0000 (13:27 -0500)]
swap src and dst
Valentine Wallace [Wed, 17 Feb 2021 20:21:22 +0000 (15:21 -0500)]
try FlushFileBuffers
Valentine Wallace [Wed, 17 Feb 2021 20:12:18 +0000 (15:12 -0500)]
sync_all final file
Valentine Wallace [Wed, 17 Feb 2021 19:25:48 +0000 (14:25 -0500)]
check # write_to_file calls
Valentine Wallace [Wed, 17 Feb 2021 19:22:02 +0000 (14:22 -0500)]
check that all updates are being attempted to persist
Valentine Wallace [Wed, 17 Feb 2021 19:18:27 +0000 (14:18 -0500)]
confirm both mons same upd id
Valentine Wallace [Wed, 17 Feb 2021 19:06:18 +0000 (14:06 -0500)]
log error on persisting new channel
Valentine Wallace [Wed, 17 Feb 2021 19:02:27 +0000 (14:02 -0500)]
don't build block sync
Valentine Wallace [Wed, 17 Feb 2021 18:59:34 +0000 (13:59 -0500)]
only run failing test
Valentine Wallace [Wed, 17 Feb 2021 18:56:13 +0000 (13:56 -0500)]
which monitor is failing?
Valentine Wallace [Wed, 17 Feb 2021 18:43:34 +0000 (13:43 -0500)]
re-enable CI
Valentine Wallace [Wed, 17 Feb 2021 18:33:21 +0000 (13:33 -0500)]
log error on test_rename_failure
Valentine Wallace [Wed, 17 Feb 2021 18:31:10 +0000 (13:31 -0500)]
no backup file
Valentine Wallace [Wed, 17 Feb 2021 18:13:09 +0000 (13:13 -0500)]
ok try w/ all tests
Valentine Wallace [Wed, 17 Feb 2021 17:17:51 +0000 (12:17 -0500)]
check if file exists
Valentine Wallace [Wed, 17 Feb 2021 17:09:01 +0000 (12:09 -0500)]
fix dwReplaceFlags
Valentine Wallace [Wed, 17 Feb 2021 17:04:50 +0000 (12:04 -0500)]
print error
Valentine Wallace [Wed, 17 Feb 2021 03:14:16 +0000 (22:14 -0500)]
fixups
Valentine Wallace [Wed, 17 Feb 2021 03:12:30 +0000 (22:12 -0500)]
ReplaceFileW w/ backup
Valentine Wallace [Wed, 17 Feb 2021 03:06:05 +0000 (22:06 -0500)]
use ReplaceFileW
Valentine Wallace [Wed, 17 Feb 2021 02:40:14 +0000 (21:40 -0500)]
add sleep
Valentine Wallace [Wed, 17 Feb 2021 02:31:06 +0000 (21:31 -0500)]
switch back to fs::rename
Valentine Wallace [Wed, 17 Feb 2021 02:25:58 +0000 (21:25 -0500)]
print perms right before rename
Valentine Wallace [Wed, 17 Feb 2021 02:14:23 +0000 (21:14 -0500)]
print readonly
Valentine Wallace [Wed, 17 Feb 2021 02:09:52 +0000 (21:09 -0500)]
set dir readonly false
Valentine Wallace [Wed, 17 Feb 2021 02:06:54 +0000 (21:06 -0500)]
print all the perms
Valentine Wallace [Wed, 17 Feb 2021 01:51:09 +0000 (20:51 -0500)]
make writeable
Valentine Wallace [Wed, 17 Feb 2021 01:22:53 +0000 (20:22 -0500)]
trying atomic rename thing
Valentine Wallace [Tue, 16 Feb 2021 20:44:20 +0000 (15:44 -0500)]
prints and nocapture and faster CI
Valentine Wallace [Mon, 11 Jan 2021 23:03:32 +0000 (18:03 -0500)]
Add BackgroundProcessor for ChannelManager persistence and other
Other includes calling timer_chan_freshness_every_minute() and in the
future, possibly persisting channel graph data.
This struct is suitable for things that need to happen periodically and
can happen in the background.
Valentine Wallace [Thu, 19 Nov 2020 17:53:16 +0000 (12:53 -0500)]
Add PersistenceNotifier to ChannelManager
This will allow the ChannelManager to signal when it has new
updates to persist, and adds a way for ChannelManager persisters
to be notified when they should re-persist the ChannelManager
to disk/backups.
Feature-gate the wait_timeout function because the core
lightning crate shouldn't depend on wallclock time unless
users opt into it.
Valentine Wallace [Thu, 19 Nov 2020 16:26:21 +0000 (11:26 -0500)]
Abstract out disk-writing utilities from FilesystemPersister
Valentine Wallace [Wed, 2 Dec 2020 22:18:17 +0000 (17:18 -0500)]
Make logger macros public
These will be used in upcoming commits for the BackgroundProcessor
to log.
Matt Corallo [Tue, 16 Feb 2021 17:33:37 +0000 (09:33 -0800)]
Merge pull request #786 from TheBlueMatt/2021-02-chansigner-util
Expand documentation and fields in SpendableOutputDescriptors
Matt Corallo [Tue, 16 Feb 2021 16:05:26 +0000 (08:05 -0800)]
Merge pull request #797 from TheBlueMatt/2021-02-no-addr-order
Drop address ordering enforcement in NodeAnnouncement deser
Matt Corallo [Mon, 15 Feb 2021 23:32:10 +0000 (15:32 -0800)]
Merge pull request #798 from TheBlueMatt/2021-02-route-bench
Matt Corallo [Mon, 15 Feb 2021 21:49:44 +0000 (16:49 -0500)]
Fetch network graph cache and run benchmarks on CI
Matt Corallo [Mon, 15 Feb 2021 21:49:02 +0000 (16:49 -0500)]
Add a trivial benchmark of calculating routes on today's graph
Sadly rust upstream never really figured out the benchmark story,
and it looks like the API we use here may not be long for this
world. Luckily, we can switch to criterion with largely the same
API if that happens before upstream finishes ongoing work with the
custom test framework stuff.
Sadly, it requires fetching the current network graph, which I did
using Val's route-testing script written to test the MPP router.
Matt Corallo [Mon, 15 Feb 2021 20:25:37 +0000 (15:25 -0500)]
Allow gossip messages to have 1KB of uninterpreted data and relay
Matt Corallo [Fri, 12 Feb 2021 21:06:14 +0000 (16:06 -0500)]
Drop trailing semicolons which rustc nightly generates warnings for
Matt Corallo [Sat, 6 Feb 2021 19:06:07 +0000 (14:06 -0500)]
Adapt C++ bindings demo to new API and keys (new funding tx)
Matt Corallo [Fri, 12 Feb 2021 23:58:59 +0000 (18:58 -0500)]
Update auto-generated bindings
Matt Corallo [Tue, 2 Feb 2021 21:40:59 +0000 (16:40 -0500)]
Slightly expand documentation on KeysInterface
Specifically, this notes when methods can or can not return the
same value on each call.
Matt Corallo [Sat, 6 Feb 2021 18:19:40 +0000 (13:19 -0500)]
Expand documentation and fields in SpendableOutputDescriptors
This adds a channel_value_satoshis field to
SpendableOutputDescriptors as it is required to recreate our
InMemoryChannelKeys. It also slightly expands documentation.
Matt Corallo [Sat, 6 Feb 2021 18:11:23 +0000 (13:11 -0500)]
Swap key_derivation_params (u64, u64) for channel_keys_id [u8; 32]
Instead of `key_derivation_params` being a rather strange type, we
call it `channel_keys_id` and give it a generic 32 byte array. This
should be much clearer for users and also more flexible.
Matt Corallo [Fri, 12 Feb 2021 18:05:59 +0000 (13:05 -0500)]
Drop address ordering enforcement in NodeAnnouncement deser
It seems many other nodes never bothered to enforce these
requirements, so there's little reason that we should either.
cc https://github.com/lightningnetwork/lightning-rfc/pull/842
Matt Corallo [Thu, 11 Feb 2021 04:31:43 +0000 (20:31 -0800)]
Merge pull request #788 from TheBlueMatt/2020-02-concrete-bindings
Concretize bindings templates
Matt Corallo [Thu, 11 Feb 2021 03:25:47 +0000 (22:25 -0500)]
Update auto-generated bindings
Matt Corallo [Thu, 11 Feb 2021 03:25:42 +0000 (22:25 -0500)]
Add additional Clone derives
The only API change outside of additional derives is to change
the inner field in `DecodeError::Io()` to an `std::io::ErrorKind`
instead of an `std::io::Error`. While `std::io::Error` obviously
makes more sense in context, it doesn't support Clone, and the
inner error largely doesn't have a lot of value on its own.
Matt Corallo [Sat, 30 Jan 2021 22:15:22 +0000 (17:15 -0500)]
[bindings] Allow cloning opaque types when inner is NULL
Previously we'd segfault trying to deref the NULL page, but there
is no reason to not simply clone by creating another opaque instance
with a null inner. This comes up specifically when cloning
ChannelSigners as the pubkeys instance is NULL on construction
before get_pubkeys is called.
Matt Corallo [Tue, 5 Jan 2021 02:08:53 +0000 (21:08 -0500)]
[C++ bindings] Update demo app due to removed TupleTempl generics
Matt Corallo [Tue, 5 Jan 2021 01:58:58 +0000 (20:58 -0500)]
[bindings] Drop some stale code from bindings generic generation
Matt Corallo [Wed, 3 Feb 2021 23:14:39 +0000 (18:14 -0500)]
[bindings] Move to manual write-out for Tuples, too
Matt Corallo [Tue, 5 Jan 2021 00:26:12 +0000 (19:26 -0500)]
[bindings] Move to manual write-out for Vec, too
Matt Corallo [Mon, 4 Jan 2021 22:29:25 +0000 (17:29 -0500)]
[bindings] Concretize Result types without type aliasing
While the type aliasing trick works great for cbindgen,
wasm_bindgen doesn't support it and requires fully-concrete types.
In order to better support wasm_bindgen in the future, we do so
here, adding a function which manually writes out almost the exact
thing which was templated previously in concrete form.
As a nice side-effect, we no longer have to allocate and free a u8
for generic parameters which were `()` (though we still do in some
conversion functions, which we can get rid of when we similarly
concretize all generics fully).
Matt Corallo [Wed, 3 Feb 2021 22:46:29 +0000 (17:46 -0500)]
[bindings] Keep track of all types which derive[Clone]
Matt Corallo [Sat, 30 Jan 2021 22:19:33 +0000 (17:19 -0500)]
[bindings] Add additional clone methods on manually-defined types
Matt Corallo [Wed, 10 Feb 2021 17:09:53 +0000 (09:09 -0800)]
Merge pull request #787 from TheBlueMatt/2021-02-check-close-source
Matt Corallo [Mon, 1 Feb 2021 01:53:28 +0000 (20:53 -0500)]
Fix documentation on PeerHandleError
Matt Corallo [Sun, 7 Feb 2021 23:49:48 +0000 (18:49 -0500)]
Add test for error message hangline resulting in force-close
Matt Corallo [Sat, 16 Jan 2021 02:34:17 +0000 (21:34 -0500)]
Check the PK of the source of an error before closing chans from it
When we receive an error message from a peer, it can indicate a
channel which we should close. However, we previously did not
check that the counterparty who sends us such a message is the
counterparty with whom we have the channel, allowing any
connected peer to make us force-close any channel we have as long
as they know the channel id.
This commit simply changes the force-close logic to check that the
sender matches the channel's counterparty node_id, though as noted
in #105, we eventually need to change the indexing anyway to allow
absurdly terrible peers to open channels with us.
Found during review of #777.
Matt Corallo [Fri, 5 Feb 2021 14:17:46 +0000 (06:17 -0800)]
Merge pull request #793 from galderz/t_double_validation_792
Galder Zamarreño [Fri, 5 Feb 2021 12:29:56 +0000 (13:29 +0100)]
Unknown bits feature only needs to be validated once #792
Matt Corallo [Thu, 4 Feb 2021 17:18:52 +0000 (09:18 -0800)]
Merge pull request #790 from bmancini55/sync_complete
Interpret sync_complete in reply_channel_range
Matt Corallo [Thu, 4 Feb 2021 17:16:14 +0000 (09:16 -0800)]
Merge pull request #774 from jkczyz/2021-01-http-client
HTTP-based block source clients
Jeffrey Czyz [Tue, 12 Jan 2021 07:33:42 +0000 (23:33 -0800)]
Implement BlockSource for REST and RPC clients
Interprets HTTP responses as either binary or JSON format, which are
then converted to the appropriate data types.
Jeffrey Czyz [Tue, 12 Jan 2021 00:07:47 +0000 (16:07 -0800)]
Add REST and RPC clients to lightning-block-sync
Implements a simple HTTP client that can issue GET and POST requests.
Used to implement REST and RPC clients, respectively. Both clients
support either blocking or non-blocking I/O.
bmancini55 [Wed, 3 Feb 2021 16:34:48 +0000 (11:34 -0500)]
Interpret sync_complete in reply_channel_range
LN Spec PR #826 changes full_information to indicate completion of a
sequence of reply_channel_range messages.
Jeffrey Czyz [Mon, 11 Jan 2021 18:50:54 +0000 (10:50 -0800)]
Add lightning-block-sync package and library
Defines an interface and related types for fetching block headers and
data from a block source (e.g., Bitcoin Core). Used to keep lightning in
sync with chain activity.
Matt Corallo [Wed, 3 Feb 2021 15:43:34 +0000 (07:43 -0800)]
Merge pull request #768 from TheBlueMatt/2020-12-chanman-bindings-deser
Expose ChannelManager/Monitor read methods in bindings
Matt Corallo [Wed, 3 Feb 2021 00:28:40 +0000 (19:28 -0500)]
[bindings] Fix genbindings.sh compile issues on OSX
There were two issues on OSX - we need to give gcc the clang
warnings flags because `gcc` *is* clang on OSX and we missed an
`-std=c++11` on one of the clang++ calls, causing compile failures.
Matt Corallo [Thu, 31 Dec 2020 22:48:55 +0000 (17:48 -0500)]
Update C++ bindings demo to use `ChannelManager` de/serialization
This demonstrates (and tests) the newly-exposed `ChannelManager`
de/serialization functions. Best revewied with -b --color-moved.
Matt Corallo [Thu, 31 Dec 2020 21:45:55 +0000 (16:45 -0500)]
[C++ bindings demo] Add now-missing std::moves
Matt Corallo [Mon, 4 Jan 2021 19:23:16 +0000 (14:23 -0500)]
Auto-generated bindings updates
Matt Corallo [Thu, 31 Dec 2020 20:34:50 +0000 (15:34 -0500)]
[C++ bindings] Add move-assign operator, require rvalue for move
This adds a move-assignment operator (`A& operator=(A&& o)`) to our
C++ wrapper classes as well as requiring an rvalue for the move
auto-convert operator (`operator CStruct()() &&`).
The second makes the C++ wrapper classes much easier to work with
by requiring an explicit `std::move` when the bindings will
automatically move a C++-wrapper object into a C object.
Matt Corallo [Thu, 31 Dec 2020 04:27:36 +0000 (23:27 -0500)]
[bindings] Use references in a few places instead of pointers
Previously, references and pointers ended up identical in C, so
there was little reason to differentiate. With the addition of
nullability annotations, there is a (very slight) reason to prefer
references, so use them in a few places where its a trivial change.
Matt Corallo [Thu, 31 Dec 2020 03:09:26 +0000 (22:09 -0500)]
[bindings] Use new non-null annotation feature in cbindgen
This adds a new annotation for objects we take by reference in the
C header indicating the pointers must not be null. We have to
disable some warning clang now dumps that we haven't annotated all
pointers, as cbindgen is not yet able to add a nullable annotation.
Matt Corallo [Thu, 31 Dec 2020 16:14:22 +0000 (11:14 -0500)]
[bindings] Don't require trait impl for-structs to have no generics
This (finally) exposes `ChannelManager`/`ChannelMonitor` _write
methods, which were (needlessly) excluded as the structs themselves
have generic parameters. Sadly, we also now need to parse
`(C-not exported)` doc comments on impl blocks as we otherwise try
to expose _write methods for `&Vec<RouteHop>`, which doesn't work
(and isn't particularly interesting for users anyway). We add such
doc comments there.
Matt Corallo [Thu, 31 Dec 2020 00:47:32 +0000 (19:47 -0500)]
[bindings] Implement ReadableArgs mapping, try impl mapping for ()s
This is most of the code to expose `ChannelManager`/`ChannelMonitor`
deserialization in our C bindings, using the new infrastructure to
map types in `maybe_convert_trait_impl` and passing generics in
from the callsites.
We also call `maybe_convert_trait_impl` for tuple types, as the
`ChannelManager`/`ChannelMonitor` deserialization returns a
`(BlockHash, T)` to indicate the block hash at which users need to
start resyncing the chain.
The final step to expose them is in the next commit.
Matt Corallo [Thu, 31 Dec 2020 00:46:39 +0000 (19:46 -0500)]
[bindings] Allow `write_rust_type` to handle leading-colon paths
It just stubs out to `write_rust_path` in this case anyway, which
handles leading-colons just fine, so no need to panic on them.
Matt Corallo [Wed, 30 Dec 2020 22:50:43 +0000 (17:50 -0500)]
[bindings] Use common conv in `_write` impls, drop type restrictions
This expands the manual implementation logic for `*_write` and
`*_read` methods to most types, converting the `*_write` path to
the common type-conversion logic to ensure it works.
Note that `*_write_void` is still only implemented for has-inner
types, as its unclear what the `void*` would point to for others.
Matt Corallo [Wed, 30 Dec 2020 22:30:59 +0000 (17:30 -0500)]
[bindings] Convert manual `_read` implementations to return Results
Previously, manual `*_read` implementations were only defined for
types with inner fields, which were set to NULL to indicate read
errors. This prevents exposing `*_read` for several other types,
including tuples (which are needed for `ChannelManager`/
`ChannelMonitors`) and enums (which includes `Event`s, though users
likely never need to call that directly). Further, this means we
don't expose the actual error enum (which is likely no big deal,
but is still nice).
Here, we instead create the `Result<Object, DecodeError>` type and
then pass it through the normal type conversion functions, giving
us access to any types which we can convert normally.
Matt Corallo [Wed, 30 Dec 2020 20:25:02 +0000 (15:25 -0500)]
[bindings] Pipe errors back from write_template_constructor
We can fail to resolve a part of a tuple, resulting in a panic in
write_template_constructor even if we're calling
`understood_c_type` with the intent of figuring out whether we can
print a type at all. Instead, we should pipe errors back and let
`understood_c_type` return false as a result.
Matt Corallo [Wed, 3 Feb 2021 03:49:28 +0000 (19:49 -0800)]
Merge pull request #767 from TheBlueMatt/2020-12-chansigner-read-bindings
Handle `read_chan_signer` in bindings (761 bindings updates)
Matt Corallo [Tue, 2 Feb 2021 22:10:05 +0000 (14:10 -0800)]
Merge pull request #781 from TheBlueMatt/2021-01-tokio-1
Update tokio to 1.0
Matt Corallo [Fri, 1 Jan 2021 01:37:27 +0000 (20:37 -0500)]
[bindings] Update C++ demo for new NetGraphMsgHandler::new() params
Matt Corallo [Tue, 2 Feb 2021 21:53:33 +0000 (16:53 -0500)]
Update auto-generated bindings, including bumping cbindgen version
Matt Corallo [Mon, 4 Jan 2021 20:25:23 +0000 (15:25 -0500)]
[bindings] Allow unused fns that the bindings can call but don't
We no longer have any public `Option<Signatures>` in our code, and
thus get warnings that the two functions which support it are
unused. Instead of removing support for them (which we may need in
the future), we add `#[allow(unused)]`.
Matt Corallo [Wed, 30 Dec 2020 20:14:42 +0000 (15:14 -0500)]
[bindings] Drop useless `#[no_mangle]` from `pub type` definitions
Newer rustc complains that "attribute should be applied to a function or
static"
Matt Corallo [Wed, 25 Nov 2020 18:25:18 +0000 (13:25 -0500)]
[bindings] Be explicit with take_ptr calls
If you try to call take_ptr on a pointer to an object which
implements Deref, rustc hits the deref recursion limit.
To avoid this, we can explicitly tell rustc that we want to treat
the pointer as a pointer and call take_ptr on it directly.
Matt Corallo [Wed, 25 Nov 2020 18:20:15 +0000 (13:20 -0500)]
[bindings] Separate take_ptr and take_inner
It is confusing to have two utility methods on different classes
of types which do two different, but related, things with the same
name.
Matt Corallo [Mon, 4 Jan 2021 20:12:37 +0000 (15:12 -0500)]
[bindings] Support exposing `bitcoin::OutPoint`s as our common type
Matt Corallo [Mon, 4 Jan 2021 20:52:18 +0000 (15:52 -0500)]
[bindings] Figure out in-file structs and enums before processing
Previously, types which were declared and used in the same file
would fail if the use was before the declaration. This makes sense
in a few cases where a "parent" class returns a reference to a
"child" class and there's no reason we shouldn't support it.
This change adds a second pass to our file processing which gathers
the structs and enums whicha re declared in the file and adds them
to the type resolver first, before doing the real conversion.
Matt Corallo [Mon, 4 Jan 2021 19:15:36 +0000 (14:15 -0500)]
[bindings] Don't export new functions with unexportable types
`CommitmentTransaction::new_with_auxiliary_htlc_data()` includes a
unbounded generic parameter which we can't concretize and it's of
limited immediate use for users in any case. We should eventually
add a non-generic version which uses `()` for the generic but that
can come later.
`CommitmentTransaction::htlcs()` returns a reference to a Vec,
which we cannot currently map. It should, however, be exposed to
users, so in the future we'll need to have a duplication function
which returns Vec of references or a cloned Vec.
Matt Corallo [Mon, 4 Jan 2021 19:13:59 +0000 (14:13 -0500)]
[bindings] Expose secp256k1::Message as ThirtyTwoBytes
Matt Corallo [Wed, 25 Nov 2020 17:22:31 +0000 (12:22 -0500)]
[bindings] Add support for mapping Write as a supertrait
Matt Corallo [Wed, 25 Nov 2020 17:22:03 +0000 (12:22 -0500)]
[bindings] Handle MessageSendEventsProvider impl blocks in a util fn
Instead of having manually-written lightning-specific code in a
supertrait walk in the middle of a large function, move it to a
utility function up next to the other manually-written-impl-block
functions.
Matt Corallo [Wed, 25 Nov 2020 16:59:58 +0000 (11:59 -0500)]
[bindings] Always resolve supertrait types during supertrait walks
This is a rather trivial cleanup to ensure we always have the full
path when we walk supertraits even if the supertrait is specified
with only a single ident.