Matt Corallo [Thu, 19 Oct 2023 19:50:38 +0000 (19:50 +0000)]
Merge pull request #63 from arik-so/2023/10/ordering_fix
Fix update recency issue with expanded test coverage
Arik Sosman [Thu, 19 Oct 2023 18:52:32 +0000 (11:52 -0700)]
Optimize indices for expired value fix lookup.
Arik Sosman [Thu, 19 Oct 2023 18:52:06 +0000 (11:52 -0700)]
Fix expired value being sent as the latest update.
Arik Sosman [Thu, 19 Oct 2023 18:49:27 +0000 (11:49 -0700)]
Reproduce test for expired value being sent as the latest update.
Arik Sosman [Wed, 18 Oct 2023 00:37:55 +0000 (17:37 -0700)]
Create snapshot generation test.
Arik Sosman [Tue, 17 Oct 2023 20:49:48 +0000 (13:49 -0700)]
Extract snapshot generation method.
Arik Sosman [Mon, 16 Oct 2023 15:27:45 +0000 (08:27 -0700)]
Replicate Mutiny test scenarios.
Arik Sosman [Sat, 14 Oct 2023 01:39:27 +0000 (18:39 -0700)]
Allow setting custom gossip seen timestamps.
Matt Corallo [Sun, 15 Oct 2023 19:54:49 +0000 (19:54 +0000)]
Merge pull request #62 from arik-so/2023/10/ordering_fix
Create Postgres step in CI.
Arik Sosman [Sat, 14 Oct 2023 01:01:21 +0000 (18:01 -0700)]
Assert that schema is uninitialized.
Arik Sosman [Fri, 13 Oct 2023 19:54:34 +0000 (12:54 -0700)]
Create Postgres step in CI.
Matt Corallo [Thu, 12 Oct 2023 17:35:33 +0000 (17:35 +0000)]
Merge pull request #60 from arik-so/2023/08/first_unit_test
First unit test
Arik Sosman [Tue, 12 Sep 2023 19:25:09 +0000 (12:25 -0700)]
Assert schema sanitization.
Arik Sosman [Fri, 8 Sep 2023 17:47:11 +0000 (10:47 -0700)]
Use schema as test logger id.
Arik Sosman [Tue, 15 Aug 2023 21:39:12 +0000 (14:39 -0700)]
Create first unit test.
Arik Sosman [Tue, 15 Aug 2023 21:38:46 +0000 (14:38 -0700)]
Upgrade to LDK 117.
Arik Sosman [Tue, 15 Aug 2023 21:38:17 +0000 (14:38 -0700)]
Create TestLogger.
Arik Sosman [Tue, 15 Aug 2023 21:37:25 +0000 (14:37 -0700)]
Create test-aware db env vars.
Matt Corallo [Wed, 6 Sep 2023 23:51:06 +0000 (23:51 +0000)]
Merge pull request #56 from arik-so/2023/08/connect-to-peers-asynchronously
Track gossip data prior to peer connection loop.
Arik Sosman [Wed, 6 Sep 2023 23:26:59 +0000 (16:26 -0700)]
Always wait 10 seconds prior to attempting reconnection.
Arik Sosman [Wed, 6 Sep 2023 04:28:15 +0000 (21:28 -0700)]
Don't special-case initial connection attempt.
Arik Sosman [Mon, 28 Aug 2023 21:40:35 +0000 (14:40 -0700)]
Connect peers independently.
Previously, we wouldn't start monitoring the gossip data until all
intended peers had either failed or succeeded to connect.
Instead, we're now spawn each peer connection independently, and only
await the first 5 successful peer connections.
Matt Corallo [Wed, 6 Sep 2023 22:16:26 +0000 (22:16 +0000)]
Merge pull request #57 from arik-so/2023/08/old-incremental-update-fix
Send full updates after old last seen updates.
Arik Sosman [Tue, 29 Aug 2023 01:01:03 +0000 (18:01 -0700)]
Send full updates after old last seen updates.
Previously, whenever we saw that there was a previous update that a
client would have seen, we simply calculated the delta set based on
which properties have changed, and would most likely send an
incremental update set (excepting the case of a new or newly sent
announcement, in which case all sent updates are full).
However, if the last seen update was old, and there's a chance that
a user may have run RGS since, it is possible that due to the
7-day-backdating-mechanism included on the client, the reference
update would no longer be present.
To fix that, anytime we see that a last seen update is more than six
days old, we automatically include a full update.
Matt Corallo [Sat, 2 Sep 2023 16:27:08 +0000 (16:27 +0000)]
Merge pull request #59 from arik-so/2023/08/custom-intervals
Distinction between snapshot and symlink intervals
Arik Sosman [Fri, 1 Sep 2023 21:42:40 +0000 (14:42 -0700)]
Dynamically adjust snapshot scopes.
Previously, we had hard-coded factors for the default snapshot
generation interval, which also served as the minimum snapshot
scope. In this commit, we substitute that with a doubling
mechanism that stops once it reaches or exceeds the
21-day-mark, which can be configured using an additional flag.
Arik Sosman [Thu, 31 Aug 2023 22:59:32 +0000 (15:59 -0700)]
Distinguish between snapshot and symlink intervals.
Snapshots may be generated every day or even every couple days,
but symlinks will be generated at the given symlink granularity.
To avoid gcd complications, all snapshot generation intervals
must be multiples of the symlink granularity interval.
Graham Krizek [Thu, 6 Jul 2023 05:31:24 +0000 (00:31 -0500)]
Allow the snapshot interval to be configurable in the environment variables
Matt Corallo [Mon, 28 Aug 2023 21:20:51 +0000 (21:20 +0000)]
Merge pull request #55 from arik-so/2023/08/multiplication-overflow-fix
Fix multiplication overflow bug
Arik Sosman [Mon, 28 Aug 2023 16:07:19 +0000 (09:07 -0700)]
Fix multiplication overflow bug.
The `snapshot_sync_day_factors` array is sorted
ascendingly, so find() will return on the first
iteration that is at least equal to the requested
interval.
However, the last value in the array is u64::max,
which means that multiplying it with DAY_SECONDS
will overflow. To avoid that, we use saturating_mul.
Matt Corallo [Thu, 17 Aug 2023 20:55:48 +0000 (20:55 +0000)]
Merge pull request #54 from arik-so/2023/08/incremental-update-fix
Only send full updates with announcements
Arik Sosman [Thu, 17 Aug 2023 20:14:18 +0000 (13:14 -0700)]
Create indices for schema version 12.
Arik Sosman [Thu, 17 Aug 2023 04:35:55 +0000 (21:35 -0700)]
Create method for obtaining UTC-prepared Postgres client.
Arik Sosman [Thu, 17 Aug 2023 04:50:43 +0000 (21:50 -0700)]
Send full updates whenever the channel announcement is sent.
Arik Sosman [Thu, 17 Aug 2023 03:55:51 +0000 (20:55 -0700)]
Fix ordering of intermediate commits to ensure the latest comes first and is used as the authoritative source of what's latest.
Matt Corallo [Thu, 17 Aug 2023 00:16:14 +0000 (00:16 +0000)]
Merge pull request #53 from arik-so/2023/08/query-timestamp-fix
Switch timestamp queries to use f64
Arik Sosman [Thu, 17 Aug 2023 00:04:06 +0000 (17:04 -0700)]
Switch timestamp reads to use i64.
Arik Sosman [Wed, 16 Aug 2023 23:18:40 +0000 (16:18 -0700)]
Switch timestamp queries to use f64.
Matt Corallo [Sun, 13 Aug 2023 16:37:54 +0000 (16:37 +0000)]
Merge pull request #49 from TheBlueMatt/main
Matt Corallo [Sun, 16 Jul 2023 20:41:28 +0000 (20:41 +0000)]
Make snapshotter a bit more amenable to changes in interval
By separating the concept of "seconds in a day" from the snapshot
interval.
Matt Corallo [Sun, 16 Jul 2023 20:41:00 +0000 (20:41 +0000)]
Only generate as many symlinks as we need (rather than 27 years worth)
Matt Corallo [Sun, 13 Aug 2023 05:30:42 +0000 (05:30 +0000)]
Merge pull request #51 from arik-so/2023/08/pre_test_refactors
Allow arbitrary logger types.
Arik Sosman [Fri, 4 Aug 2023 21:30:02 +0000 (14:30 -0700)]
Allow log threshold configuration.
Arik Sosman [Thu, 3 Aug 2023 01:34:01 +0000 (18:34 -0700)]
Remove println from verifier.rs
Arik Sosman [Thu, 3 Aug 2023 01:26:55 +0000 (18:26 -0700)]
Remove println from tracking.rs
Arik Sosman [Thu, 3 Aug 2023 01:21:29 +0000 (18:21 -0700)]
Remove println from snapshot.rs
Arik Sosman [Thu, 3 Aug 2023 01:19:13 +0000 (18:19 -0700)]
Remove println from persistence.rs
Arik Sosman [Thu, 3 Aug 2023 01:16:24 +0000 (18:16 -0700)]
Remove println from lookup.rs
Arik Sosman [Thu, 3 Aug 2023 01:11:06 +0000 (18:11 -0700)]
Remove println from lib.rs
Arik Sosman [Wed, 2 Aug 2023 23:00:36 +0000 (16:00 -0700)]
Switch Logger from Arc to Deref.
Arik Sosman [Wed, 2 Aug 2023 22:41:14 +0000 (15:41 -0700)]
Allow custom logger types.
Arik Sosman [Wed, 2 Aug 2023 21:33:05 +0000 (14:33 -0700)]
Upgrade to 116 release.
Arik [Sun, 16 Jul 2023 17:36:13 +0000 (10:36 -0700)]
Merge pull request #48 from TheBlueMatt/2023-07-further-opt
Further optimize intermediate row fetching
Matt Corallo [Sun, 16 Jul 2023 17:25:10 +0000 (17:25 +0000)]
Correct upgrade logic to ignore missing indexes
If we upgrade multiple versions in one go, the indexes we're trying
to drop may not exist, and that shouldn't cause us to fail to
upgrade.
Matt Corallo [Sun, 16 Jul 2023 17:20:56 +0000 (17:20 +0000)]
Drop overly optimistic index
The `channel_updates_id_with_scid_dir_blob` index allows the
intermediate-row-fetching logic to be index-only, but there's very
little reason to do so - we now use subqueries to build the exact
set of rows we want, by id, and then fetch various colums. Having
an index that lets us look up those columns without hitting the
regular table is fine, but there's not a ton of cost to hitting the
table by primary key and maintaining yet another index isn't free.
Matt Corallo [Sun, 16 Jul 2023 17:13:10 +0000 (17:13 +0000)]
Further optimize intermediate row fetching by filtering scids first
Postgresql actually thinks this version is more expensive, but when
I run it its about 4x as fast:
```
ln-gossip=# EXPLAIN ANALYZE SELECT id, direction, blob_signed FROM channel_updates
WHERE id IN (
SELECT DISTINCT ON (short_channel_id, direction) id
FROM channel_updates
WHERE seen < '2023-07-08 00:00:00' AND short_channel_id IN (
SELECT DISTINCT ON (short_channel_id) short_channel_id
FROM channel_updates
WHERE seen >= '2023-07-08 00:00:00'
)
ORDER BY short_channel_id ASC, direction ASC, seen DESC
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=
3771382.74..
3772300.66 rows=172186 width=145) (actual time=25270.727..34954.993 rows=129921 loops=1)
-> HashAggregate (cost=
3771382.18..
3771384.18 rows=200 width=4) (actual time=25267.060..25313.378 rows=129921 loops=1)
Group Key: channel_updates_1.id
Batches: 1 Memory Usage: 12321kB
-> Unique (cost=189510.13..
3769229.85 rows=172186 width=21) (actual time=693.154..25177.896 rows=129921 loops=1)
-> Merge Join (cost=189510.13..
3480148.53 rows=
57816264 width=21) (actual time=693.152..23970.052 rows=
31781122 loops=1)
Merge Cond: (channel_updates_1.short_channel_id = channel_updates_2.short_channel_id)
-> Index Only Scan using channel_updates_scid_dir_seen_desc_with_id on channel_updates channel_updates_1 (cost=0.56..
2560497.30 rows=
57816264 width=21) (actual time=0.154..19198.277 rows=
57605314 loops=1)
Index Cond: (seen < '2023-07-08 00:00:00'::timestamp without time zone)
Heap Fetches: 0
-> Unique (cost=189509.57..195871.76 rows=86093 width=8) (actual time=594.634..787.364 rows=68181 loops=1)
-> Sort (cost=189509.57..192690.67 rows=
1272439 width=8) (actual time=594.621..722.739 rows=
1452420 loops=1)
Sort Key: channel_updates_2.short_channel_id
Sort Method: external merge Disk: 17664kB
-> Index Only Scan using channel_updates_seen_scid on channel_updates channel_updates_2 (cost=0.56..43091.07 rows=
1272439 width=8) (actual time=3.461..348.146 rows=
1502697 loops=1)
Index Cond: (seen >= '2023-07-08 00:00:00'::timestamp without time zone)
Heap Fetches: 40000
-> Index Only Scan using channel_updates_id_with_scid_dir_blob on channel_updates (cost=0.56..4.60 rows=1 width=145) (actual time=0.074..0.074 rows=1 loops=129921)
Index Cond: (id = channel_updates_1.id)
Heap Fetches: 0
Planning Time: 20.757 ms
JIT:
Functions: 18
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.770 ms, Inlining 51.503 ms, Optimization 25.848 ms, Emission 21.086 ms, Total 99.208 ms
Execution Time: 34995.683 ms
(26 rows)
ln-gossip=# EXPLAIN ANALYZE SELECT id, direction, blob_signed FROM channel_updates
WHERE id IN (
SELECT DISTINCT ON (short_channel_id, direction) id
FROM channel_updates
WHERE seen < '2023-07-08 00:00:00'
ORDER BY short_channel_id ASC, direction ASC, seen DESC
) AND short_channel_id IN (
SELECT DISTINCT ON (short_channel_id) short_channel_id
FROM channel_updates
WHERE seen >= '2023-07-08 00:00:00'
);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hash Join (cost=
3049540.37..
3050910.30 rows=172186 width=145) (actual time=18697.102..123187.066 rows=129921 loops=1)
Hash Cond: (channel_updates.short_channel_id = channel_updates_2.short_channel_id)
-> Nested Loop (cost=
2851731.51..
2852649.44 rows=172186 width=153) (actual time=18029.529..122265.952 rows=393938 loops=1)
-> HashAggregate (cost=
2851730.95..
2851732.95 rows=200 width=4) (actual time=18027.857..18309.986 rows=393938 loops=1)
Group Key: channel_updates_1.id
Batches: 5 Memory Usage: 16449kB Disk Usage: 7104kB
-> Result (cost=0.56..
2849578.62 rows=172186 width=21) (actual time=0.070..17851.034 rows=393938 loops=1)
-> Unique (cost=0.56..
2849578.62 rows=172186 width=21) (actual time=0.044..17809.361 rows=393938 loops=1)
-> Index Only Scan using channel_updates_scid_dir_seen_desc_with_id on channel_updates channel_updates_1 (cost=0.56..
2560497.30 rows=
57816264 width=21) (actual time=0.043..15596.783 rows=
57605314 loops=1)
Index Cond: (seen < '2023-07-08 00:00:00'::timestamp without time zone)
Heap Fetches: 0
-> Index Only Scan using channel_updates_id_with_scid_dir_blob on channel_updates (cost=0.56..4.60 rows=1 width=153) (actual time=0.263..0.263 rows=1 loops=393938)
Index Cond: (id = channel_updates_1.id)
Heap Fetches: 0
-> Hash (cost=196732.69..196732.69 rows=86093 width=8) (actual time=662.832..662.833 rows=70509 loops=1)
Buckets: 131072 Batches: 1 Memory Usage: 3779kB
-> Unique (cost=189509.57..195871.76 rows=86093 width=8) (actual time=542.007..657.269 rows=70509 loops=1)
-> Sort (cost=189509.57..192690.67 rows=
1272439 width=8) (actual time=542.004..610.886 rows=
1502697 loops=1)
Sort Key: channel_updates_2.short_channel_id
Sort Method: external merge Disk: 17664kB
-> Index Only Scan using channel_updates_seen_scid on channel_updates channel_updates_2 (cost=0.56..43091.07 rows=
1272439 width=8) (actual time=67.791..274.989 rows=
1502697 loops=1)
Index Cond: (seen >= '2023-07-08 00:00:00'::timestamp without time zone)
Heap Fetches: 39999
Planning Time: 0.324 ms
JIT:
Functions: 23
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.976 ms, Inlining 8.080 ms, Optimization 37.602 ms, Emission 26.389 ms, Total 73.048 ms
Execution Time: 123206.373 ms
(29 rows)
```
Matt Corallo [Sun, 16 Jul 2023 03:24:53 +0000 (03:24 +0000)]
Enforce panic = abort in all build modes
Matt Corallo [Sun, 16 Jul 2023 03:23:34 +0000 (03:23 +0000)]
Upgrade to LDK 0.0.116-alpha1
This includes a fix for the deadlock in #32.
Matt Corallo [Sun, 16 Jul 2023 03:20:52 +0000 (03:20 +0000)]
Don't hold the `NetworkGraph` read lock across an await point
Holding the `NetworkGraph` read lock across a query await point
can cause a deadlock if another task tries to handle a gossip
message at the same time.
Matt Corallo [Sun, 16 Jul 2023 00:37:05 +0000 (00:37 +0000)]
Switch to streaming queries
In order to use streaming queries we have to use `tokio-postgres`'s
`query_raw` command, rather than `query`. This should reduce our
memory footprint from 10+GB to well under one.
Matt Corallo [Sun, 16 Jul 2023 00:34:55 +0000 (00:34 +0000)]
Bump tokio to latest LTS version now that we have a higher MSRV.
Matt Corallo [Sun, 16 Jul 2023 00:32:01 +0000 (00:32 +0000)]
Switch to Rust edition 2021
This increases the MSRV to 1.56, which is now almost two years old,
and lets us use array::IntoIter in the next commit.
Matt Corallo [Sat, 15 Jul 2023 07:01:16 +0000 (07:01 +0000)]
Remove unused `consider_intermediate_updates` flag..optimizing query
The `consider_intermediate_updates` flag is always set, and must be
set for correctness, so we remove it. Further, we optimize the
query that hung on it somewhat by removing an uneccessary
`ORDER BY` clause which was only neccessary if
`consider_intermediate_updates` were unset.
Matt Corallo [Sat, 15 Jul 2023 06:42:33 +0000 (06:42 +0000)]
Substantially optimize reference-row-fetching
By first fetching the rows we need from a smaller index, we avoid
walking a large index which contained the full `blob_signed`. This
reduces reference-row-fetching from 680 seconds to 152 seconds when
searching today for reference rows against 7 days ago.
Old:
```
ln-gossip=# EXPLAIN ANALYZE SELECT DISTINCT ON (short_channel_id, direction) id, blob_signed, direction
FROM channel_updates
WHERE seen < '2023-07-07 00:00:00' AND short_channel_id IN (
SELECT DISTINCT ON (short_channel_id) short_channel_id
FROM channel_updates
WHERE seen >= '2023-07-07 00:00:00'
)
ORDER BY short_channel_id ASC, direction ASC, seen DESC;
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Unique (cost=186279.46..
11921204.82 rows=168910 width=161) (actual time=732.365..680504.173 rows=129985 loops=1)
-> Merge Join (cost=186279.46..
11632998.93 rows=
57641177 width=161) (actual time=732.364..679193.755 rows=
31714061 loops=1)
Merge Cond: (channel_updates.short_channel_id = channel_updates_1.short_channel_id)
-> Index Only Scan using channel_updates_scid_dir_seen on channel_updates (cost=0.56..
10718853.69 rows=
57641177 width=161) (actual time=0.638..673675.749 rows=
57408667 loops=1)
Index Cond: (seen < '2023-07-07 00:00:00'::timestamp without time zone)
Heap Fetches: 0
-> Unique (cost=186278.90..192574.84 rows=84455 width=8) (actual time=478.881..750.241 rows=68210 loops=1)
-> Sort (cost=186278.90..189426.87 rows=
1259188 width=8) (actual time=478.878..653.035 rows=
1452661 loops=1)
Sort Key: channel_updates_1.short_channel_id
Sort Method: external merge Disk: 17680kB
-> Index Only Scan using channel_updates_seen_scid on channel_updates channel_updates_1 (cost=0.56..41481.08 rows=
1259188 width=8) (actual time=0.885..264.333 rows=
1504495 loops=1)
Index Cond: (seen >= '2023-07-07 00:00:00'::timestamp without time zone)
Heap Fetches: 2273
Planning Time: 0.164 ms
JIT:
Functions: 9
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 21.265 ms, Inlining 37.914 ms, Optimization 113.040 ms, Emission 101.901 ms, Total 274.121 ms
Execution Time: 680601.155 ms
(19 rows)
```
New:
```
ln-gossip=# EXPLAIN ANALYZE SELECT id, direction, blob_signed FROM channel_updates
WHERE id IN (
SELECT DISTINCT ON (short_channel_id, direction) id
FROM channel_updates
WHERE seen < '2023-07-07 00:00:00'
ORDER BY short_channel_id ASC, direction ASC, seen DESC
) AND short_channel_id IN (
SELECT DISTINCT ON (short_channel_id) short_channel_id
FROM channel_updates
WHERE seen >= '2023-07-07 00:00:00'
);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hash Join (cost=
2942503.92..
2943867.77 rows=169870 width=145) (actual time=22862.627..152436.685 rows=130116 loops=1)
Hash Cond: (channel_updates.short_channel_id = channel_updates_2.short_channel_id)
-> Nested Loop (cost=
2738282.26..
2739200.18 rows=169870 width=153) (actual time=22141.452..151504.140 rows=393250 loops=1)
-> HashAggregate (cost=
2738281.69..
2738283.69 rows=200 width=4) (actual time=22139.440..22339.035 rows=393250 loops=1)
Group Key: channel_updates_1.id
Batches: 1 Memory Usage: 45089kB
-> Result (cost=0.56..
2736158.32 rows=169870 width=21) (actual time=0.102..21984.409 rows=393250 loops=1)
-> Unique (cost=0.56..
2736158.32 rows=169870 width=21) (actual time=0.074..21943.089 rows=393250 loops=1)
-> Index Only Scan using channel_updates_scid_dir_seen_desc_with_id on channel_updates channel_updates_1 (cost=0.56..
2448011.03 rows=
57629457 width=21) (actual time=0.073..19776.181 rows=
57408667 loops=1)
Index Cond: (seen < '2023-07-07 00:00:00'::timestamp without time zone)
Heap Fetches: 0
-> Index Only Scan using channel_updates_id_with_scid_dir_blob on channel_updates (cost=0.56..4.60 rows=1 width=153) (actual time=0.328..0.328 rows=1 loops=393250)
Index Cond: (id = channel_updates_1.id)
Heap Fetches: 0
-> Hash (cost=203159.97..203159.97 rows=84935 width=8) (actual time=721.105..721.107 rows=70731 loops=1)
Buckets: 131072 Batches: 1 Memory Usage: 3787kB
-> Unique (cost=195708.67..202310.62 rows=84935 width=8) (actual time=552.965..713.465 rows=70731 loops=1)
-> Sort (cost=195708.67..199009.65 rows=
1320391 width=8) (actual time=552.962..650.323 rows=
1537141 loops=1)
Sort Key: channel_updates_2.short_channel_id
Sort Method: external merge Disk: 18064kB
-> Index Only Scan using channel_updates_seen_scid on channel_updates channel_updates_2 (cost=0.56..43421.19 rows=
1320391 width=8) (actual time=66.736..324.130 rows=
1537141 loops=1)
Index Cond: (seen >= '2023-07-07 00:00:00'::timestamp without time zone)
Heap Fetches: 68
Planning Time: 0.520 ms
JIT:
Functions: 21
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.643 ms, Inlining 7.055 ms, Optimization 33.167 ms, Emission 25.782 ms, Total 66.648 ms
Execution Time: 152458.777 ms
(29 rows)
```
Arik [Mon, 10 Jul 2023 18:11:00 +0000 (11:11 -0700)]
Merge pull request #43 from TheBlueMatt/main
Require DB insertions to complete in five seconds
Matt Corallo [Thu, 6 Jul 2023 16:43:05 +0000 (16:43 +0000)]
Require DB insertions to complete in fifteen seconds
For some reason the mainnet server hung, seemingly on the DB
insertion task. This will improve debugging by simply crashing if
an insertion takes longer than five seconds.
Arik [Sun, 2 Jul 2023 19:43:01 +0000 (12:43 -0700)]
Merge pull request #42 from TheBlueMatt/main
#39 Followups
Matt Corallo [Sun, 2 Jul 2023 18:45:10 +0000 (18:45 +0000)]
Redo indexes for new SELECTs and trivially tweak one old query
Matt Corallo [Sun, 2 Jul 2023 17:17:07 +0000 (17:17 +0000)]
Build reminder updates with correct SCID field
When the reminder updates were added, a dummy `ChannelUpdate` with
a number of zero'd fields were created under the assumption that
the zero'd fields would be ignored downstream when building
serialized updates. However, the SCID field was `assert`'ed on (and
serialized in the update), causing any reminder updates to cause an
assertion panic.
Instead, we do it the Right Way (tm) here and move the
only-sometimes-available fields into the update type enum, ensuring
we can't access "poison" fields downstream.
Matt Corallo [Sun, 2 Jul 2023 16:12:25 +0000 (16:12 +0000)]
Let postgres filter timestamps rather than doing it in Rust
Matt Corallo [Sun, 2 Jul 2023 16:13:26 +0000 (16:13 +0000)]
Fix query compat with Postgres 13
...which requires DISTINCT ON to match the start of ORDER BY.
Matt Corallo [Sun, 2 Jul 2023 17:39:35 +0000 (17:39 +0000)]
Clean up SQL query formatting to make them somewhat more readable
Matt Corallo [Sun, 2 Jul 2023 16:02:11 +0000 (16:02 +0000)]
Reduce data loaded from the `channel_update` table when not needed
Matt Corallo [Sun, 2 Jul 2023 15:52:53 +0000 (15:52 +0000)]
Merge pull request #39 from arik-so/2023-06-announcement_inclusion_fixes
Announcement inclusion fixes & reminders
Matt Corallo [Wed, 28 Jun 2023 15:03:27 +0000 (15:03 +0000)]
Merge pull request #40 from arik-so/2023-06-0.0.115
Update lightning to 0.0.115
Arik Sosman [Tue, 27 Jun 2023 20:56:56 +0000 (16:56 -0400)]
Omit reminders for channels with incomplete data.
Arik Sosman [Tue, 27 Jun 2023 20:51:56 +0000 (16:51 -0400)]
Create dummy flags without secondary lookup query.
Arik Sosman [Tue, 27 Jun 2023 19:58:12 +0000 (15:58 -0400)]
Send reminders for stale-ish channels.
Arik Sosman [Tue, 27 Jun 2023 19:57:37 +0000 (15:57 -0400)]
Fix first update lookup.
Arik Sosman [Tue, 27 Jun 2023 20:26:15 +0000 (16:26 -0400)]
Update to 0.0.115.
Matt Corallo [Mon, 12 Jun 2023 18:19:58 +0000 (18:19 +0000)]
Merge pull request #38 from TheBlueMatt/main
Removed unused_mut rejection and fix some unused `mut`s
Matt Corallo [Mon, 5 Jun 2023 23:26:38 +0000 (23:26 +0000)]
Removed unused_mut rejection and fix some unused `mut`s
Making warnings a hard failure is generally bad practice as it can
result in new compiler versions failing to compile
otherwise-totally-acceptable code, which in this case is happening
on rustc beta, which is now warning for new cases of unused mut.
Matt Corallo [Mon, 5 Jun 2023 23:24:38 +0000 (23:24 +0000)]
Merge pull request #37 from marctyndel/configurable-caches-path
optionally set path for caches from env
Marc Tyndel [Mon, 5 Jun 2023 17:54:51 +0000 (13:54 -0400)]
quick fix to be able to set the caches path via env variables like we do for the DB and bitcoind configuration
Matt Corallo [Tue, 16 May 2023 19:34:20 +0000 (19:34 +0000)]
Merge pull request #35 from andrei-21/fix/dummy-symlink
Fix dummy symlink
Andrei [Tue, 16 May 2023 00:00:00 +0000 (00:00 +0000)]
Fix dummy symlink
The commit changes the symlink for the snapshot for the current date
from `./res/snapshots_pending/empty_delta.lngossip` to
`../snapshots/empty_delta.lngossip` such that nginx does not
retrun 404, but 200 with the dummy snapshot
Matt Corallo [Sat, 6 May 2023 23:57:09 +0000 (23:57 +0000)]
Merge pull request #29 from arik-so/2023-04-empty-blob-generator
Create a utility method to generate noöp RGS blobs.
Arik Sosman [Tue, 4 Apr 2023 23:20:58 +0000 (16:20 -0700)]
Create a utility method to generate noöp RGS blobs.
Matt Corallo [Mon, 24 Apr 2023 17:34:22 +0000 (17:34 +0000)]
Merge pull request #31 from benthecarman/ignore-cargo-lock
Include Cargo.lock file
benthecarman [Sat, 15 Apr 2023 19:30:31 +0000 (14:30 -0500)]
Remove need to pin deps
Matt Corallo [Mon, 3 Apr 2023 23:12:32 +0000 (23:12 +0000)]
Merge pull request #21 from jurvis/jurvis/2022-11-rgs-docker-compose
Create a docker-compose for running an RGS server
Arik [Sat, 18 Mar 2023 13:56:29 +0000 (06:56 -0700)]
Merge pull request #25 from TheBlueMatt/main
Update to LDK 0.0.114
Matt Corallo [Wed, 15 Mar 2023 17:52:13 +0000 (17:52 +0000)]
Write an `update_time.txt` file out to make monitoring simpler
Matt Corallo [Tue, 28 Feb 2023 18:14:22 +0000 (18:14 +0000)]
Delay prior to reconnecting peers that have disconnected
Matt Corallo [Wed, 8 Feb 2023 16:59:09 +0000 (16:59 +0000)]
Call `PeerManager::timer_tick_occurred` regularly to ping peers
Matt Corallo [Wed, 8 Feb 2023 16:58:06 +0000 (16:58 +0000)]
Upgrade to latest lightning crate, switching to async chain resolution
Matt Corallo [Wed, 8 Feb 2023 16:55:23 +0000 (16:55 +0000)]
Stop pruning the network graph on startup
There's not a lot of reason for this and better to keep things
around until we go to build the gossip output
Matt Corallo [Thu, 9 Feb 2023 00:08:30 +0000 (00:08 +0000)]
Merge pull request #26 from andrei-21/feature/configure-network
Arik [Wed, 8 Feb 2023 17:27:52 +0000 (09:27 -0800)]
Merge pull request #24 from andrei-21/feature/configure-peers
Make ln peers configurable
Andrei [Wed, 8 Feb 2023 00:00:00 +0000 (00:00 +0000)]
Lowercase network values