rapid-gossip-sync-server
11 months agoRemoved unused_mut rejection and fix some unused `mut`s
Matt Corallo [Mon, 5 Jun 2023 23:26:38 +0000 (23:26 +0000)]
Removed unused_mut rejection and fix some unused `mut`s

Making warnings a hard failure is generally bad practice as it can
result in new compiler versions failing to compile
otherwise-totally-acceptable code, which in this case is happening
on rustc beta, which is now warning for new cases of unused mut.

11 months agoMerge pull request #37 from marctyndel/configurable-caches-path
Matt Corallo [Mon, 5 Jun 2023 23:24:38 +0000 (23:24 +0000)]
Merge pull request #37 from marctyndel/configurable-caches-path

optionally set path for caches from env

11 months agoquick fix to be able to set the caches path via env variables like we do for the...
Marc Tyndel [Mon, 5 Jun 2023 17:54:51 +0000 (13:54 -0400)]
quick fix to be able to set the caches path via env variables like we do for the DB and bitcoind configuration

11 months agoMerge pull request #35 from andrei-21/fix/dummy-symlink
Matt Corallo [Tue, 16 May 2023 19:34:20 +0000 (19:34 +0000)]
Merge pull request #35 from andrei-21/fix/dummy-symlink

Fix dummy symlink

11 months agoFix dummy symlink
Andrei [Tue, 16 May 2023 00:00:00 +0000 (00:00 +0000)]
Fix dummy symlink

The commit changes the symlink for the snapshot for the current date
from `./res/snapshots_pending/empty_delta.lngossip` to
`../snapshots/empty_delta.lngossip` such that nginx does not
retrun 404, but 200 with the dummy snapshot

12 months agoMerge pull request #29 from arik-so/2023-04-empty-blob-generator
Matt Corallo [Sat, 6 May 2023 23:57:09 +0000 (23:57 +0000)]
Merge pull request #29 from arik-so/2023-04-empty-blob-generator

Create a utility method to generate noöp RGS blobs.

12 months agoCreate a utility method to generate noöp RGS blobs.
Arik Sosman [Tue, 4 Apr 2023 23:20:58 +0000 (16:20 -0700)]
Create a utility method to generate noöp RGS blobs.

12 months agoMerge pull request #31 from benthecarman/ignore-cargo-lock
Matt Corallo [Mon, 24 Apr 2023 17:34:22 +0000 (17:34 +0000)]
Merge pull request #31 from benthecarman/ignore-cargo-lock

Include Cargo.lock file

12 months agoRemove need to pin deps
benthecarman [Sat, 15 Apr 2023 19:30:31 +0000 (14:30 -0500)]
Remove need to pin deps

13 months agoMerge pull request #21 from jurvis/jurvis/2022-11-rgs-docker-compose
Matt Corallo [Mon, 3 Apr 2023 23:12:32 +0000 (23:12 +0000)]
Merge pull request #21 from jurvis/jurvis/2022-11-rgs-docker-compose

Create a docker-compose for running an RGS server

13 months agoMerge pull request #25 from TheBlueMatt/main
Arik [Sat, 18 Mar 2023 13:56:29 +0000 (06:56 -0700)]
Merge pull request #25 from TheBlueMatt/main

Update to LDK 0.0.114

13 months agoWrite an `update_time.txt` file out to make monitoring simpler
Matt Corallo [Wed, 15 Mar 2023 17:52:13 +0000 (17:52 +0000)]
Write an `update_time.txt` file out to make monitoring simpler

13 months agoDelay prior to reconnecting peers that have disconnected
Matt Corallo [Tue, 28 Feb 2023 18:14:22 +0000 (18:14 +0000)]
Delay prior to reconnecting peers that have disconnected

13 months agoCall `PeerManager::timer_tick_occurred` regularly to ping peers
Matt Corallo [Wed, 8 Feb 2023 16:59:09 +0000 (16:59 +0000)]
Call `PeerManager::timer_tick_occurred` regularly to ping peers

14 months agoUpgrade to latest lightning crate, switching to async chain resolution
Matt Corallo [Wed, 8 Feb 2023 16:58:06 +0000 (16:58 +0000)]
Upgrade to latest lightning crate, switching to async chain resolution

14 months agoStop pruning the network graph on startup
Matt Corallo [Wed, 8 Feb 2023 16:55:23 +0000 (16:55 +0000)]
Stop pruning the network graph on startup

There's not a lot of reason for this and better to keep things
around until we go to build the gossip output

14 months agoMerge pull request #26 from andrei-21/feature/configure-network
Matt Corallo [Thu, 9 Feb 2023 00:08:30 +0000 (00:08 +0000)]
Merge pull request #26 from andrei-21/feature/configure-network

14 months agoMerge pull request #24 from andrei-21/feature/configure-peers
Arik [Wed, 8 Feb 2023 17:27:52 +0000 (09:27 -0800)]
Merge pull request #24 from andrei-21/feature/configure-peers

Make ln peers configurable

14 months agoLowercase network values
Andrei [Wed, 8 Feb 2023 00:00:00 +0000 (00:00 +0000)]
Lowercase network values

14 months agoMake ln peers configurable
Andrei [Wed, 8 Feb 2023 00:00:00 +0000 (00:00 +0000)]
Make ln peers configurable

14 months agoConfigure network
Andrei [Wed, 8 Feb 2023 00:00:00 +0000 (00:00 +0000)]
Configure network

17 months agoCreate simple docker-compose for testnet
Jurvis Tan [Tue, 15 Nov 2022 03:59:17 +0000 (19:59 -0800)]
Create simple docker-compose for testnet

17 months agoMerge pull request #19 from TheBlueMatt/main
Matt Corallo [Tue, 8 Nov 2022 19:00:42 +0000 (19:00 +0000)]
Merge pull request #19 from TheBlueMatt/main

Update LDK to 0.0.112 and workaround Postgres being broken

17 months agoPin once_cell to 1.14 to fix MSRV
Matt Corallo [Tue, 8 Nov 2022 01:06:51 +0000 (01:06 +0000)]
Pin once_cell to 1.14 to fix MSRV

17 months agoSet per-table autovacuum thresholds low
Matt Corallo [Tue, 8 Nov 2022 00:53:41 +0000 (00:53 +0000)]
Set per-table autovacuum thresholds low

Fixes #20

18 months agoPrune the network graph immediately prior to generating updates
Matt Corallo [Thu, 27 Oct 2022 18:01:44 +0000 (18:01 +0000)]
Prune the network graph immediately prior to generating updates

18 months agoUpdate LDK to 0.0.112
Matt Corallo [Thu, 27 Oct 2022 18:01:37 +0000 (18:01 +0000)]
Update LDK to 0.0.112

19 months agoMerge pull request #14 from TheBlueMatt/main
Arik [Wed, 14 Sep 2022 20:32:19 +0000 (13:32 -0700)]
Merge pull request #14 from TheBlueMatt/main

Upgrade to LDK 0.0.111, Improve SQL Types

19 months agoAdd one additional index which postgres prefers as the DB fills
Matt Corallo [Wed, 14 Sep 2022 20:12:17 +0000 (20:12 +0000)]
Add one additional index which postgres prefers as the DB fills

If postgres decides walking the full `channel_updates_scid_dir_seen`
index and removing old `seen` values is slower than just walking
the full table (or this new index) it does so. Sadly this causes
re-sorting (usually on-disk), but there doesn't seem to be a way to
avoid this.

19 months agoDrop useless indexes, add (very) useful indxes after benchmarking
Matt Corallo [Wed, 14 Sep 2022 19:22:49 +0000 (19:22 +0000)]
Drop useless indexes, add (very) useful indxes after benchmarking

19 months agoDrop unused columns from SELECT queries
Matt Corallo [Tue, 13 Sep 2022 15:48:31 +0000 (15:48 +0000)]
Drop unused columns from SELECT queries

19 months agoDrop the composite_index in favor of a multi-column unique index
Matt Corallo [Tue, 13 Sep 2022 15:43:53 +0000 (15:43 +0000)]
Drop the composite_index in favor of a multi-column unique index

Also set all the channel_update fields to NOT NULL, because they
should be non-null always.

19 months agoAdd some further indexes on channel_updates to match queries
Matt Corallo [Tue, 13 Sep 2022 00:07:49 +0000 (00:07 +0000)]
Add some further indexes on channel_updates to match queries

19 months agoDrop redundant block_height column on announcements, use i16 for flags
Matt Corallo [Tue, 13 Sep 2022 15:33:41 +0000 (15:33 +0000)]
Drop redundant block_height column on announcements, use i16 for flags

19 months agoSimplify composite index construction somewhat
Matt Corallo [Mon, 12 Sep 2022 04:21:24 +0000 (04:21 +0000)]
Simplify composite index construction somewhat

19 months agoUpdate to LDK 0.0.111, rust-bitcoin 0.29
Matt Corallo [Sun, 11 Sep 2022 22:39:59 +0000 (22:39 +0000)]
Update to LDK 0.0.111, rust-bitcoin 0.29

19 months agoUse more appropriate data types for SCIDs, direction, composite_index
Matt Corallo [Sun, 11 Sep 2022 21:06:34 +0000 (21:06 +0000)]
Use more appropriate data types for SCIDs, direction, composite_index

19 months agoDrop the chain_hash column
Matt Corallo [Sun, 11 Sep 2022 21:07:57 +0000 (21:07 +0000)]
Drop the chain_hash column

19 months agoAdd upgrade query support
Matt Corallo [Sun, 11 Sep 2022 16:41:12 +0000 (16:41 +0000)]
Add upgrade query support

19 months agoDrop several unused dependencies
Matt Corallo [Sun, 11 Sep 2022 18:14:32 +0000 (18:14 +0000)]
Drop several unused dependencies

20 months agoMerge pull request #13 from arik-so/2022-08-envvar-renaming
Matt Corallo [Tue, 23 Aug 2022 04:07:31 +0000 (04:07 +0000)]
Merge pull request #13 from arik-so/2022-08-envvar-renaming

20 months agoChange bitcoin REST endpoint default port to the standard mainnet default.
Arik Sosman [Tue, 23 Aug 2022 03:21:11 +0000 (20:21 -0700)]
Change bitcoin REST endpoint default port to the standard mainnet default.

20 months agoMerge pull request #11 from arik-so/2022-08-envvar-renaming
Matt Corallo [Tue, 23 Aug 2022 03:08:12 +0000 (03:08 +0000)]
Merge pull request #11 from arik-so/2022-08-envvar-renaming

Rename environment variables to reflect the name of this project.

20 months agoRename environment variables to reflect the name of this project.
Arik Sosman [Tue, 23 Aug 2022 00:46:14 +0000 (17:46 -0700)]
Rename environment variables to reflect the name of this project.

20 months agoMerge pull request #10 from TheBlueMatt/2022-08-no-pin-cpufeatures
Arik [Tue, 23 Aug 2022 00:37:53 +0000 (17:37 -0700)]
Merge pull request #10 from TheBlueMatt/2022-08-no-pin-cpufeatures

Skip pinning cpufeatures, which has fixed its MSRV

20 months agoMerge pull request #7 from TheBlueMatt/2022-08-fix-deadlock
Arik [Tue, 23 Aug 2022 00:36:53 +0000 (17:36 -0700)]
Merge pull request #7 from TheBlueMatt/2022-08-fix-deadlock

Fix Counter Lock Deadlock, Further Cleanups

20 months agoDrop spurious comments 2022-08-fix-deadlock
Matt Corallo [Mon, 22 Aug 2022 04:19:52 +0000 (04:19 +0000)]
Drop spurious comments

20 months agoDon't hold the counter lock while verifying gossip/waiting on DB
Matt Corallo [Mon, 22 Aug 2022 04:14:24 +0000 (04:14 +0000)]
Don't hold the counter lock while verifying gossip/waiting on DB

This resolves a deadlock if we block on the DB where we have one
thread blocked waiting on DB in a blocking thread, and the tokio
reactor blocked waiting on the counter lock which the blocking
thread holds.

20 months agoRemove unused fields and function in lib.rs
Matt Corallo [Mon, 22 Aug 2022 03:46:49 +0000 (03:46 +0000)]
Remove unused fields and function in lib.rs

20 months agoRemove indirection in initial sync completion
Matt Corallo [Mon, 22 Aug 2022 03:37:49 +0000 (03:37 +0000)]
Remove indirection in initial sync completion

Rather than sending a GossipMessage::InitialSyncComplete, which
causes the persistence logic to push to another queue to wake the
snapshoter, we can simply wake the snapshoter directly.

20 months agoSkip pinning cpufeatures, which has fixed its MSRV 2022-08-no-pin-cpufeatures
Matt Corallo [Tue, 23 Aug 2022 00:03:34 +0000 (00:03 +0000)]
Skip pinning cpufeatures, which has fixed its MSRV

20 months agoMerge pull request #9 from arik-so/2022-08-snapshot-timestamp-fix
Matt Corallo [Tue, 23 Aug 2022 00:02:03 +0000 (00:02 +0000)]
Merge pull request #9 from arik-so/2022-08-snapshot-timestamp-fix

Reference canonical timestamps when generating snapshots.

20 months agoAlways pin dependencies for 1.48.0.
Arik Sosman [Mon, 22 Aug 2022 23:48:49 +0000 (16:48 -0700)]
Always pin dependencies for 1.48.0.

20 months agoReference canonical timestamps when generating snapshots.
Arik Sosman [Mon, 22 Aug 2022 23:42:20 +0000 (16:42 -0700)]
Reference canonical timestamps when generating snapshots.

20 months agoMerge pull request #8 from arik-so/2022-08-add-0-symlink
Matt Corallo [Mon, 22 Aug 2022 23:12:37 +0000 (23:12 +0000)]
Merge pull request #8 from arik-so/2022-08-add-0-symlink

Add 0.bin symlink for clients requiring an initial rapid sync.

20 months agoMerge pull request #6 from TheBlueMatt/2022-08-cleanups
Arik [Mon, 22 Aug 2022 22:48:44 +0000 (15:48 -0700)]
Merge pull request #6 from TheBlueMatt/2022-08-cleanups

Various Code Cleanups

20 months agoAdd 0.bin symlink for clients requiring an initial rapid sync.
Arik Sosman [Mon, 22 Aug 2022 21:39:02 +0000 (14:39 -0700)]
Add 0.bin symlink for clients requiring an initial rapid sync.

20 months agoCleanup network graph persisting and ensure write succeeds 2022-08-cleanups
Matt Corallo [Mon, 22 Aug 2022 03:30:35 +0000 (03:30 +0000)]
Cleanup network graph persisting and ensure write succeeds

20 months agoTrivial cleanups in lookup.rs
Matt Corallo [Mon, 22 Aug 2022 03:27:50 +0000 (03:27 +0000)]
Trivial cleanups in lookup.rs

20 months agoCorrect error-matching in downloader
Matt Corallo [Mon, 22 Aug 2022 03:24:27 +0000 (03:24 +0000)]
Correct error-matching in downloader

20 months agoDrop Arcs around the TestLogger, as its an empty struct anyway
Matt Corallo [Mon, 22 Aug 2022 03:22:09 +0000 (03:22 +0000)]
Drop Arcs around the TestLogger, as its an empty struct anyway

20 months agoSubstantially reduce boilerplate and Arcs in download_gossip
Matt Corallo [Mon, 22 Aug 2022 03:17:09 +0000 (03:17 +0000)]
Substantially reduce boilerplate and Arcs in download_gossip

20 months agoRemove unnecessary Arcs in verifier.rs
Matt Corallo [Mon, 22 Aug 2022 03:04:41 +0000 (03:04 +0000)]
Remove unnecessary Arcs in verifier.rs

20 months agoClean up the flow and variables in lib.rs and main.rs
Matt Corallo [Mon, 22 Aug 2022 02:39:26 +0000 (02:39 +0000)]
Clean up the flow and variables in lib.rs and main.rs

20 months agoMerge pull request #5 from TheBlueMatt/2022-08-block-on-slow-db
Arik [Mon, 22 Aug 2022 03:03:03 +0000 (20:03 -0700)]
Merge pull request #5 from TheBlueMatt/2022-08-block-on-slow-db

Block tasks if the DB writes get behind, rather than growing the queue.

20 months agoBlock tasks if the DB writes get behind, rather than growing the queue 2022-08-block-on-slow-db
Matt Corallo [Mon, 22 Aug 2022 02:40:23 +0000 (02:40 +0000)]
Block tasks if the DB writes get behind, rather than growing the queue

20 months agoMerge pull request #4 from TheBlueMatt/2022-08-fix-executors
Arik [Mon, 22 Aug 2022 02:43:03 +0000 (19:43 -0700)]
Merge pull request #4 from TheBlueMatt/2022-08-fix-executors

Don't spawn new single-threaded executors, stay inside tokio

20 months agoDon't spawn new single-threaded executors, stay inside tokio 2022-08-fix-executors
Matt Corallo [Mon, 22 Aug 2022 02:18:30 +0000 (02:18 +0000)]
Don't spawn new single-threaded executors, stay inside tokio

20 months agoMerge pull request #3 from lightningdevkit/readme_fixups
Arik [Mon, 22 Aug 2022 02:13:45 +0000 (19:13 -0700)]
Merge pull request #3 from lightningdevkit/readme_fixups

Fix readme links.

20 months agoFix readme links.
Arik Sosman [Mon, 22 Aug 2022 02:13:10 +0000 (19:13 -0700)]
Fix readme links.

20 months agoMerge pull request #2 from lightningdevkit/readme_fixups
Arik [Mon, 22 Aug 2022 02:10:45 +0000 (19:10 -0700)]
Merge pull request #2 from lightningdevkit/readme_fixups

Fix readme.

20 months agoFix readme.
Arik Sosman [Mon, 22 Aug 2022 02:10:23 +0000 (19:10 -0700)]
Fix readme.

20 months agoInitial commit.
Arik Sosman [Mon, 22 Aug 2022 01:18:35 +0000 (18:18 -0700)]
Initial commit.

23 months agoInitial commit
Arik Sosman [Fri, 20 May 2022 03:02:48 +0000 (20:02 -0700)]
Initial commit