dnsseed-rust
9 months agoDrop memory limit. It was useful to debug OOMs but is now unnecessary master
Matt Corallo [Wed, 28 Jul 2021 14:29:02 +0000 (14:29 +0000)]
Drop memory limit. It was useful to debug OOMs but is now unnecessary

9 months agoSupport specifying the BGP peer asn on the CLI, increase max connections
Matt Corallo [Tue, 27 Jul 2021 20:47:13 +0000 (20:47 +0000)]
Support specifying the BGP peer asn on the CLI, increase max connections

9 months agoParallelize rolling filter checks/updates, reduce hashes
Matt Corallo [Tue, 27 Jul 2021 20:41:24 +0000 (20:41 +0000)]
Parallelize rolling filter checks/updates, reduce hashes

10 months agoManually unroll bloom clearing loop since LLVM doesn't do it
Matt Corallo [Wed, 14 Jul 2021 14:17:23 +0000 (14:17 +0000)]
Manually unroll bloom clearing loop since LLVM doesn't do it

10 months agoWrite to stdout all at once to reduce "flashing" on remote monitoring
Matt Corallo [Wed, 14 Jul 2021 13:55:48 +0000 (13:55 +0000)]
Write to stdout all at once to reduce "flashing" on remote monitoring

10 months agoTweak default settings slightly
Matt Corallo [Wed, 14 Jul 2021 02:16:44 +0000 (02:16 +0000)]
Tweak default settings slightly

10 months agoKeep more log lines in case of giant terminals
Matt Corallo [Tue, 13 Jul 2021 16:18:27 +0000 (16:18 +0000)]
Keep more log lines in case of giant terminals

10 months agoStore nodes that timed out in a rolling bloom filter
Matt Corallo [Tue, 13 Jul 2021 13:41:44 +0000 (13:41 +0000)]
Store nodes that timed out in a rolling bloom filter

10 months agoUse last_services to store if we ever connected
Matt Corallo [Tue, 13 Jul 2021 01:26:43 +0000 (01:26 +0000)]
Use last_services to store if we ever connected

Also drain to-test queue faster

10 months agoDecrease the size of Node from 48 to 16 (and a map entry from 72 to 36)
Matt Corallo [Mon, 12 Jul 2021 02:23:27 +0000 (02:23 +0000)]
Decrease the size of Node from 48 to 16 (and a map entry from 72 to 36)

10 months agoBump rand, drop SockAddr size by an additional 4 bytes
Matt Corallo [Mon, 12 Jul 2021 01:28:41 +0000 (01:28 +0000)]
Bump rand, drop SockAddr size by an additional 4 bytes

14 months agoBump rust-bitcoin, add addrv2
Matt Corallo [Thu, 25 Feb 2021 05:47:57 +0000 (00:47 -0500)]
Bump rust-bitcoin, add addrv2

19 months agoFewer full vec allocations
Matt Corallo [Thu, 15 Oct 2020 17:05:34 +0000 (13:05 -0400)]
Fewer full vec allocations

19 months agoOne less Vec indirection
Matt Corallo [Thu, 15 Oct 2020 17:01:22 +0000 (13:01 -0400)]
One less Vec indirection

20 months agoREADME
Matt Corallo [Mon, 7 Sep 2020 20:27:18 +0000 (16:27 -0400)]
README

22 months agoBump nodes_buff prealloc to match current avg char/line
Matt Corallo [Wed, 22 Jul 2020 18:25:16 +0000 (14:25 -0400)]
Bump nodes_buff prealloc to match current avg char/line

22 months agoPrint ASN lookups in output zonefile comments
Matt Corallo [Tue, 21 Jul 2020 23:10:58 +0000 (19:10 -0400)]
Print ASN lookups in output zonefile comments

22 months agoTry reducing memory footprint a bit further
Matt Corallo [Tue, 21 Jul 2020 19:28:25 +0000 (15:28 -0400)]
Try reducing memory footprint a bit further

22 months agoFix AS path detection
Matt Corallo [Tue, 21 Jul 2020 19:28:17 +0000 (15:28 -0400)]
Fix AS path detection

22 months agoSort fields in size order
Matt Corallo [Tue, 21 Jul 2020 18:09:28 +0000 (14:09 -0400)]
Sort fields in size order

22 months agoDont track earliest-scan-time, since we scan num/interval anyway
Matt Corallo [Tue, 21 Jul 2020 17:45:51 +0000 (13:45 -0400)]
Dont track earliest-scan-time, since we scan num/interval anyway

22 months agopre-alloc routing table
Matt Corallo [Tue, 21 Jul 2020 17:34:27 +0000 (13:34 -0400)]
pre-alloc routing table

22 months agoAdd an extra hop in the path length now that we have fewer threads
Matt Corallo [Mon, 20 Jul 2020 00:03:59 +0000 (20:03 -0400)]
Add an extra hop in the path length now that we have fewer threads

22 months agoFix BGP parameter usage
Matt Corallo [Sun, 19 Jul 2020 23:42:18 +0000 (19:42 -0400)]
Fix BGP parameter usage

22 months agoUse released bgp-rs
Matt Corallo [Sun, 19 Jul 2020 23:01:47 +0000 (19:01 -0400)]
Use released bgp-rs

22 months agoGlobablly limit alloc to 1.75GB
Matt Corallo [Sun, 19 Jul 2020 21:17:03 +0000 (17:17 -0400)]
Globablly limit alloc to 1.75GB

22 months agoDecrease thread count somewhat
Matt Corallo [Sun, 19 Jul 2020 21:15:35 +0000 (17:15 -0400)]
Decrease thread count somewhat

23 months agoUpdate to latest rust-bitcoin
Matt Corallo [Sat, 13 Jun 2020 22:09:04 +0000 (18:09 -0400)]
Update to latest rust-bitcoin

23 months agoUpdate to upstream bgp-rs
Matt Corallo [Sun, 31 May 2020 17:30:07 +0000 (13:30 -0400)]
Update to upstream bgp-rs

2 years agoFurther reduce memory in bgp_client
Matt Corallo [Thu, 5 Dec 2019 00:48:28 +0000 (19:48 -0500)]
Further reduce memory in bgp_client

2 years agoAlign routing table entries to single bytes
Matt Corallo [Thu, 5 Dec 2019 00:29:22 +0000 (19:29 -0500)]
Align routing table entries to single bytes

2 years agoImpose a max conns/sec
Matt Corallo [Tue, 3 Dec 2019 06:46:35 +0000 (01:46 -0500)]
Impose a max conns/sec

2 years agoDrop processing threads a bit
Matt Corallo [Mon, 2 Dec 2019 20:09:47 +0000 (15:09 -0500)]
Drop processing threads a bit

2 years agoMake sure we dont ever queue things twice
Matt Corallo [Mon, 2 Dec 2019 20:09:36 +0000 (15:09 -0500)]
Make sure we dont ever queue things twice

2 years agoAdd some basic COMPACT_FILTERS support
Matt Corallo [Mon, 2 Dec 2019 19:55:06 +0000 (14:55 -0500)]
Add some basic COMPACT_FILTERS support

2 years agoDrop a needless Vec
Matt Corallo [Mon, 2 Dec 2019 19:37:00 +0000 (14:37 -0500)]
Drop a needless Vec

2 years agoClean up print format
Matt Corallo [Mon, 2 Dec 2019 19:11:17 +0000 (14:11 -0500)]
Clean up print format

2 years agoFix BGP table lookup
Matt Corallo [Mon, 2 Dec 2019 18:17:11 +0000 (13:17 -0500)]
Fix BGP table lookup

2 years agoDrop Arcs around paths, and add more debug info for looked-up ASNs
Matt Corallo [Mon, 2 Dec 2019 04:40:22 +0000 (23:40 -0500)]
Drop Arcs around paths, and add more debug info for looked-up ASNs

2 years agoDrop prepends in BGP paths since it doesn't change our results
Matt Corallo [Sun, 1 Dec 2019 23:08:59 +0000 (18:08 -0500)]
Drop prepends in BGP paths since it doesn't change our results

2 years agoDrop v6 tclass and scope to cut down on mem usage a bit
Matt Corallo [Sun, 1 Dec 2019 22:31:33 +0000 (17:31 -0500)]
Drop v6 tclass and scope to cut down on mem usage a bit

2 years agoWe don't need BTreeMaps
Carl Dong [Fri, 23 Aug 2019 19:07:34 +0000 (15:07 -0400)]
We don't need BTreeMaps

2 years agoInclude debug symbols in release mode, so backtraces still work
Matt Corallo [Wed, 2 Oct 2019 01:34:17 +0000 (21:34 -0400)]
Include debug symbols in release mode, so backtraces still work

2 years agoFix a minor timing issue, load nodes at start with an old time
Matt Corallo [Wed, 2 Oct 2019 01:19:18 +0000 (21:19 -0400)]
Fix a minor timing issue, load nodes at start with an old time

2 years agoDrop the connections/sec arg and just make it fit in the interval
Matt Corallo [Wed, 18 Sep 2019 17:54:51 +0000 (13:54 -0400)]
Drop the connections/sec arg and just make it fit in the interval

2 years agoBump bgp-rs
Matt Corallo [Sat, 7 Sep 2019 04:51:27 +0000 (00:51 -0400)]
Bump bgp-rs

2 years agoSupport scanning by .onion hostname
Matt Corallo [Mon, 26 Aug 2019 20:27:19 +0000 (16:27 -0400)]
Support scanning by .onion hostname

2 years agoDont flush with a lock held
Matt Corallo [Mon, 26 Aug 2019 19:57:24 +0000 (15:57 -0400)]
Dont flush with a lock held

2 years agoAlways return two tor addresses
Matt Corallo [Mon, 26 Aug 2019 19:55:45 +0000 (15:55 -0400)]
Always return two tor addresses

2 years agoUse Option<NetworkMessage> so short buffer and unknown msg are diff
Matt Corallo [Mon, 26 Aug 2019 19:10:44 +0000 (15:10 -0400)]
Use Option<NetworkMessage> so short buffer and unknown msg are diff

2 years agoAllow up to 2 onion addresses to bypass BGP checks
Matt Corallo [Mon, 26 Aug 2019 05:13:12 +0000 (01:13 -0400)]
Allow up to 2 onion addresses to bypass BGP checks

2 years agoImplement connecting to Tor peers over SOCKS5 in Peer
Matt Corallo [Mon, 26 Aug 2019 05:02:22 +0000 (01:02 -0400)]
Implement connecting to Tor peers over SOCKS5 in Peer

2 years agoRejigger BGP timeouts
Matt Corallo [Fri, 23 Aug 2019 04:54:39 +0000 (00:54 -0400)]
Rejigger BGP timeouts

2 years agoPrint stats about routing table size in UI
Matt Corallo [Fri, 23 Aug 2019 04:50:24 +0000 (00:50 -0400)]
Print stats about routing table size in UI

2 years agoNever allow non-resolving-to-ASN IPs
Matt Corallo [Fri, 23 Aug 2019 04:39:27 +0000 (00:39 -0400)]
Never allow non-resolving-to-ASN IPs

2 years agoSelect only ASNs visible in all paths for an IP
Matt Corallo [Fri, 23 Aug 2019 04:38:47 +0000 (00:38 -0400)]
Select only ASNs visible in all paths for an IP

2 years agoBump dep
Matt Corallo [Fri, 23 Aug 2019 04:14:56 +0000 (00:14 -0400)]
Bump dep

2 years agoReset routing tables on reconnect
Matt Corallo [Fri, 23 Aug 2019 03:39:20 +0000 (23:39 -0400)]
Reset routing tables on reconnect

2 years agoAllow bgp lookups from CLI
Matt Corallo [Fri, 23 Aug 2019 01:22:56 +0000 (21:22 -0400)]
Allow bgp lookups from CLI

2 years agoFix handling failed route lookups
Matt Corallo [Fri, 23 Aug 2019 01:14:53 +0000 (21:14 -0400)]
Fix handling failed route lookups

2 years agoFix route lookup masking
Matt Corallo [Fri, 23 Aug 2019 01:01:25 +0000 (21:01 -0400)]
Fix route lookup masking

2 years agoAvoid storing unused attrs
Matt Corallo [Fri, 23 Aug 2019 00:51:29 +0000 (20:51 -0400)]
Avoid storing unused attrs

2 years agopanic = abort on debug, too
Matt Corallo [Fri, 23 Aug 2019 00:19:36 +0000 (20:19 -0400)]
panic = abort on debug, too

2 years agoFilter DNS results by common ASN
Matt Corallo [Fri, 23 Aug 2019 00:19:24 +0000 (20:19 -0400)]
Filter DNS results by common ASN

2 years agoSplit DNS results writing and data store writing
Matt Corallo [Thu, 22 Aug 2019 21:41:55 +0000 (17:41 -0400)]
Split DNS results writing and data store writing

2 years agoHold the nodes lock less while writing DNS
Matt Corallo [Thu, 22 Aug 2019 21:28:57 +0000 (17:28 -0400)]
Hold the nodes lock less while writing DNS

2 years agoDetect some simple classes of evil spy nodes
Matt Corallo [Wed, 21 Aug 2019 22:29:38 +0000 (18:29 -0400)]
Detect some simple classes of evil spy nodes

2 years agoBump bitcoin{,_hashes}
Matt Corallo [Fri, 16 Aug 2019 23:12:12 +0000 (19:12 -0400)]
Bump bitcoin{,_hashes}

2 years agoAdd a category for timedout pong
Matt Corallo [Mon, 10 Jun 2019 10:38:02 +0000 (06:38 -0400)]
Add a category for timedout pong

2 years agoFix some printing for new rules
Matt Corallo [Mon, 10 Jun 2019 10:21:07 +0000 (06:21 -0400)]
Fix some printing for new rules

2 years agoDrop unrequested tx peers,, request ping/pong cycle
Matt Corallo [Mon, 10 Jun 2019 10:10:44 +0000 (06:10 -0400)]
Drop unrequested tx peers,, request ping/pong cycle

2 years agoBump rust-bitcoin dep
Matt Corallo [Mon, 10 Jun 2019 10:10:28 +0000 (06:10 -0400)]
Bump rust-bitcoin dep

2 years agoRequest 216, not 1008 blocks ago
Matt Corallo [Wed, 5 Jun 2019 09:24:35 +0000 (05:24 -0400)]
Request 216, not 1008 blocks ago

2 years agoEnsure we always hit our scan target, if only UNTESTED are avail
Matt Corallo [Mon, 3 Jun 2019 19:22:43 +0000 (15:22 -0400)]
Ensure we always hit our scan target, if only UNTESTED are avail

2 years agoRequire some number of addrs to count, not just one
Matt Corallo [Mon, 3 Jun 2019 18:38:35 +0000 (14:38 -0400)]
Require some number of addrs to count, not just one

2 years agoTrack which element timeout was hit for in different categories
Matt Corallo [Mon, 3 Jun 2019 11:59:48 +0000 (07:59 -0400)]
Track which element timeout was hit for in different categories

2 years agoClose recv stream after receiving block
Matt Corallo [Sun, 2 Jun 2019 18:33:19 +0000 (14:33 -0400)]
Close recv stream after receiving block

2 years agoFix split point calculation
Matt Corallo [Sun, 2 Jun 2019 14:41:11 +0000 (10:41 -0400)]
Fix split point calculation

2 years agoPrint scan set len on new scan set
Matt Corallo [Sun, 2 Jun 2019 14:28:27 +0000 (10:28 -0400)]
Print scan set len on new scan set

2 years agoMake good_node_services a Vec, not HashMap<u8, >...cause, duh
Matt Corallo [Sun, 2 Jun 2019 10:06:37 +0000 (06:06 -0400)]
Make good_node_services a Vec, not HashMap<u8, >...cause, duh

2 years agoDrop nodes mutex before shuffling result
Matt Corallo [Sun, 2 Jun 2019 09:56:54 +0000 (05:56 -0400)]
Drop nodes mutex before shuffling result

2 years agoConvert state_next_scan to vec to make addr prioritization determ.
Matt Corallo [Mon, 27 May 2019 23:53:36 +0000 (19:53 -0400)]
Convert state_next_scan to vec to make addr prioritization determ.

2 years agoAlways print result of manual scan
Matt Corallo [Mon, 27 May 2019 23:41:12 +0000 (19:41 -0400)]
Always print result of manual scan

2 years agoUse more threads to avoid locks blocking all progress
Matt Corallo [Sun, 26 May 2019 18:03:35 +0000 (14:03 -0400)]
Use more threads to avoid locks blocking all progress

3 years agoBump bitcoin dep
Matt Corallo [Thu, 23 May 2019 17:18:53 +0000 (13:18 -0400)]
Bump bitcoin dep

3 years agoFix manual scan
Matt Corallo [Thu, 23 May 2019 00:52:21 +0000 (20:52 -0400)]
Fix manual scan

3 years agoBump rust-bitcoin with better ser/deser
Matt Corallo [Wed, 22 May 2019 22:28:01 +0000 (18:28 -0400)]
Bump rust-bitcoin with better ser/deser

3 years agopanic = abort
Matt Corallo [Wed, 22 May 2019 19:41:18 +0000 (15:41 -0400)]
panic = abort

3 years agocompare to known-good copy of blocks instead of checking merkle roots
Matt Corallo [Wed, 22 May 2019 19:36:49 +0000 (15:36 -0400)]
compare to known-good copy of blocks instead of checking merkle roots

3 years agoPrint timeout during request too
Matt Corallo [Wed, 22 May 2019 16:54:29 +0000 (12:54 -0400)]
Print timeout during request too

3 years agoPrint more lines of log
Matt Corallo [Tue, 21 May 2019 19:25:52 +0000 (15:25 -0400)]
Print more lines of log

3 years agoRegularly poll other dnsseeds for new addrs
Matt Corallo [Tue, 21 May 2019 19:25:33 +0000 (15:25 -0400)]
Regularly poll other dnsseeds for new addrs

3 years agoPreallocate the header maps
Matt Corallo [Tue, 21 May 2019 15:19:06 +0000 (11:19 -0400)]
Preallocate the header maps

3 years agoNever run the scan loop more than once every 15 seconds
Matt Corallo [Tue, 21 May 2019 15:07:53 +0000 (11:07 -0400)]
Never run the scan loop more than once every 15 seconds

3 years agoSend trusted peer a sendheaders
Matt Corallo [Tue, 21 May 2019 14:53:29 +0000 (10:53 -0400)]
Send trusted peer a sendheaders

3 years agoDont ever run two scan threads
Matt Corallo [Tue, 21 May 2019 14:35:18 +0000 (10:35 -0400)]
Dont ever run two scan threads

3 years agoFix printing Good on not-Good
Matt Corallo [Tue, 21 May 2019 03:59:35 +0000 (23:59 -0400)]
Fix printing Good on not-Good

3 years agoAlways get 30 seconds of nodes to scan at a time
Matt Corallo [Tue, 21 May 2019 03:19:08 +0000 (23:19 -0400)]
Always get 30 seconds of nodes to scan at a time

3 years agoBetter log messages
Matt Corallo [Tue, 21 May 2019 02:46:03 +0000 (22:46 -0400)]
Better log messages

3 years agoPrint time more logically
Matt Corallo [Tue, 21 May 2019 02:38:46 +0000 (22:38 -0400)]
Print time more logically