This change simply logs an error if there is a problem loading a COB
from storage, instead of bailing on the cache update process.
We also ensure that objects that can't be loaded from storage are
removed from the cache.
Otherwise, previously, when an error occurred before `logger::init()`
then no message about the error would be shown, because `log` ignores
messages generated before the logger is initialized. E.g. when
`$RAD_HOME/config.json` is missing, invoking `radicle-node`
directly (i.e. not via `rad node start`) would fail without any
explanation being logged nor printed. There also are some other
possible errors that can occur before `logger::init()` where no message
would've been shown.
The fix is to detect if the logger is enabled, which it won't be when it
hasn't been initialized, and if not then fallback to printing directly
to stderr.
To show the lower-level source of an error, like previously, and to
avoid now needing more conditionals with more format strings for all
possibilities of `err.source().is_some()` and `log_enabled!()`, the
"alternate" form (`{:#}`) of formatting `anyhow::Error` is now used.
This also introduces a change in behavior such that the entire chain of
source errors will now be shown, instead of only the first in the chain,
which seems more desirable for errors that cause fatal exiting of
`radicle-node`. Note that this form still formats as only a single
line, like previously.
The prefix "Error: " is used for the new fallback printing, because in
this case it's not a log message (though, it might be written to the log
file), and because that prefix is consistent with how Rust errors that
cause immediate termination are usually printed directly. For the
opposite case, the "Fatal: " prefix serves to distinguish it as being
fatal among the many other various log messages.
While it would be possible to instead return `anyhow::Result<()>` from
`main`, to achieve the printing of early errors (and exiting with
failure code), that would cause undesirable duplication, for non-early
errors that occur after `logger::init()`, of the error message where it
would be written to both the log file and to stderr which often is the
same as the log file.
Signed-off-by: Derick Eddington <kcired@pm.me>
Ensure that when a node initialises multiple repositories, those
repositories can successfully be fetched by another peer once they
connect.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Fix "signals" thread to continue upon other signals.
Otherwise, previously, when one of the non-shutdown signals was
received, that would cause the "signals" thread to finish, but then a
subsequent signal that should cause shutdown would not because the
thread no longer existed to do so. E.g. when SIGHUP or SIGWINCH was
received first and ignored and then SIGTERM or SIGINT was received next,
the program should still be shutdown, but it wouldn't be. This is now
fixed by simply looping to continue handling subsequent signals.
Logging is now done for:
- Receiving of SIGHUP, at log level `debug`, because the default action
for that signal would be to terminate the process but that is not done
by `radicle_signals`. Someone sending that signal might want to debug
why the process isn't being terminated.
- Disconnection of the signal-notifications channel, at log level
`warn`, because, even though that should be impossible, if it somehow
occurs then a warning is warranted.
Logging is not done for receiving of SIGWINCH, because the default
action for that signal would be to ignore it, which is what
`radicle-node` does.
A wildcard pattern is not used in the match arms, so that any future
changes to the `Signal` variants will require reviewing their handling
in the "signals" thread.
Signed-off-by: Derick Eddington <kcired@pm.me>
In the current design it's possible for one peer to fill the fetch queue
so that fetches from other peers are delayed.
To improve fairness, we move to a queue per peer. We then try to dequeue
from all peers in a random order for good measure.
We were comparing the available peers with the target, so the comparison
would always succeed. We now compare it with the missing count, and use
debug logging since it's not something of concern yet.
To avoid buffering large amounts of data in the process, we set the
worker channel size to `64`. Keep in mind that this is one
`ChannelEvent`, not one byte. `ChannelEvent::Data` can already contain
an arbitrary amount of data via its `Vec<u8>`, but in practice the
maximum is 8192, due to the use of `io::copy`.
We also remove an unused function, and move the flushing to the
`ChannelFlushWriter`.
If two peers are persistent for each other and experience
disconnections, it's possible that an outbound connection is turned into
an inbound one. But if we don't set that correctly, we'll eventually run
into a crash where an already connected peer re-connects, since the
disconnection logic checks for the link direction.
The fetch queue could sometimes not drain due to skipped fetches not
signaling that to the dequeue loop.
We now return a `bool` from `try_fetch` and keep trying to dequeue
fetches until one of them actually is initiated.
Additionally, we try to dequeue fetches at a regular interval
(`IDLE_INTERVAL`) to ensure that the queue is drained.
There are cases where the service state doesn't match the state
of the underlying wire protocol. We remedy this by transitioning peers
to a "connected" state when a message is received, if they are in a
"connecting" state.
The long term solution to this will likely be to merge the service and
protocol layer so that there are no inconsistencies.
The other case happens when a persistent peer is in "disconnected" state
and attempts an inbound connection to us.
1. Adds a user-agent string to the node announcement. This lets us keep
track of what software version everyone is running. Especially useful
to see what percentage of the network has upgraded to a new release.
For old nodes, this defaults to the string `/radicle/`.
The user agent format is roughly based on BIP 0014.
2. Advertize protocol version in node announcement.
By advertizing the protocol version, nodes can decide to only connect to
peers with a compatible version. The default and current version is `1`.
We bundle these two changes as they both modify the node announcement
and node database.
3. Swap the order of the fields in announcements to make the message
extensible.
This prevents potential memory leaks. We also ensure that sends fail
instead of blocking, in case the channels are full.
Additionally, we add some metrics to report on channel size.
Previously, our advertized inventory was not always correct: when we
stopped seeding a repository, it was still advertised as part of our
inventory until node restart, because the `Storage` type doesn't have
access to seeding policies.
In this change, we remove the concept of inventory from `Storage` and
make the authoritative place for it the routing table in our database.
To make this work, we have to add our node to the database on profile
creation, to not violate the foreign key constraint on the routing
table. Hence, the tests are changed to include our alias which is now
always available.
This is a change to the `config.json` file which removes certain options
that are best not touched and changes some options to make them easier
to work with.
1. We change the default journaling mode to "wal" and remove the config
option.
2. We remove the option to set the peer connection "target", as it's not
a good idea to set this to a different value, as it affects the
network.
3. We combine seeding policy & scope, since there's no need to set the
scope when the policy is "block". In that case, we always want to
block "all" remotes. The new policy configuration has the following
schema:
{ seedingPolicy: { default: "block" | "allow", scope?: "all" | "followed" } }
The "scope" key is not used when "default" is set to "block".
4. We add an `extra` field to the node config for options that are not
recognized. We use this to warn the user.
We use the *staggered broadcast* technique when relaying gossip messages
to reduce message amplification. In our basic simulation, it brings
amplification down from 3.5 to 1.5.
The idea is simply to accumulate gossip messages and relay them after a fixed
interval of time. This has two effects:
1) Old messages that haven't been sent yet are replaced by newer ones,
so only one message is sent after the delay.
2) Because nodes have their own intervals, messages are not all sent at
the same time, which reduces the chance of messages crossing paths.
For now we only turn this on for inventory messages, since this is the
least likely to be noticed by users, and also the most problematic
currently.
Ensures dead connections are not kept around longer than necessary.
Note that the ping/pong mechanism is more expensive and checks that the
session is active and healthy, while TCP_KEEPALIVE is lower level,
cheaper, and checks that the connection is still alive.
On FreeBSD systems, the file limits are of type `i64` in comparison to
`u64` on Linux systems.
Also fix `rad web` command name on Windows and FreeBSD.
Co-authored-by: cloudhead <cloudhead@radicle.xyz>
Co-authored-by: Shawn Webb <shawn.webb@hardenedbsd.org>
In the case of an out-of-sync local node, we were showing the wrong tip.
Additionally, mark your tip as unannounced if it isn't announced. This
isn't fool-proof, but can be useful to debug certain issues.
From empirical testing, the `FETCH_TIMEOUT` can be set to a lower value of 3s.
For a repository that is about 300MiB, the process will successfully send all
packfile bytes and the references will be written.
The `--timeout` option is also used in the `git-upload-pack` process for good
measure.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `join` method is equivalent to the custom loop that was written for joining
the `reader` thread in upload-pack.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Sebastian Martinez
cloudhead
Derick Eddington
Fintan Halpenny
Lorenz Leutgeb
Alexis Sellier