1. Adds a user-agent string to the node announcement. This lets us keep
track of what software version everyone is running. Especially useful
to see what percentage of the network has upgraded to a new release.
For old nodes, this defaults to the string `/radicle/`.
The user agent format is roughly based on BIP 0014.
2. Advertize protocol version in node announcement.
By advertizing the protocol version, nodes can decide to only connect to
peers with a compatible version. The default and current version is `1`.
We bundle these two changes as they both modify the node announcement
and node database.
3. Swap the order of the fields in announcements to make the message
extensible.
This prevents potential memory leaks. We also ensure that sends fail
instead of blocking, in case the channels are full.
Additionally, we add some metrics to report on channel size.
Previously, our advertized inventory was not always correct: when we
stopped seeding a repository, it was still advertised as part of our
inventory until node restart, because the `Storage` type doesn't have
access to seeding policies.
In this change, we remove the concept of inventory from `Storage` and
make the authoritative place for it the routing table in our database.
To make this work, we have to add our node to the database on profile
creation, to not violate the foreign key constraint on the routing
table. Hence, the tests are changed to include our alias which is now
always available.
This is a change to the `config.json` file which removes certain options
that are best not touched and changes some options to make them easier
to work with.
1. We change the default journaling mode to "wal" and remove the config
option.
2. We remove the option to set the peer connection "target", as it's not
a good idea to set this to a different value, as it affects the
network.
3. We combine seeding policy & scope, since there's no need to set the
scope when the policy is "block". In that case, we always want to
block "all" remotes. The new policy configuration has the following
schema:
{ seedingPolicy: { default: "block" | "allow", scope?: "all" | "followed" } }
The "scope" key is not used when "default" is set to "block".
4. We add an `extra` field to the node config for options that are not
recognized. We use this to warn the user.
We use the *staggered broadcast* technique when relaying gossip messages
to reduce message amplification. In our basic simulation, it brings
amplification down from 3.5 to 1.5.
The idea is simply to accumulate gossip messages and relay them after a fixed
interval of time. This has two effects:
1) Old messages that haven't been sent yet are replaced by newer ones,
so only one message is sent after the delay.
2) Because nodes have their own intervals, messages are not all sent at
the same time, which reduces the chance of messages crossing paths.
For now we only turn this on for inventory messages, since this is the
least likely to be noticed by users, and also the most problematic
currently.
On FreeBSD systems, the file limits are of type `i64` in comparison to
`u64` on Linux systems.
Also fix `rad web` command name on Windows and FreeBSD.
Co-authored-by: cloudhead <cloudhead@radicle.xyz>
Co-authored-by: Shawn Webb <shawn.webb@hardenedbsd.org>
Introduce an `UploadPack` type for keeping track of relevant data from
upload-pack events and displaying progress to the terminal.
It keeps track of the number of remotes that are being uploaded to as well as
the throughput of transmitted data.
The `Progress` events are logged rather than being displayed since they don't
provide any useful details to the user.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Breaking change: when writing to the node socket, command names should
be specified via the `"command"` field instead of the `"type"` field.
This was changed to be more readable.
* Remove `rad sync --force` flag
* Remove `rad clone --private` flag
These flags are no longer necessary. Simplify specifying which seed
to fetch from with `--seed` is enough.
*Breaking change* The `tor` configuration field is no longer
supported. Users must remove that field from their config.
Allows for a mixed mode, where regular addresses bypass the Tor proxy,
while onion addresses go through it.
When `onion.mode` is set to "forward", Tor connections are fowarded to
the global proxy, and if it isn't set, to the OS's DNS resolution.
For mixed mode, a global proxy simply isn't set, so that IP/DNS is not
proxied.
The previous banning implementation didn't work, since it included port
numbers in the address value.
Here, we keep a separate table with (node, ip) pairs and update this
table when a node connects or is banned.
The timeout for a `UnixStream` will interrupt a `read` even when more data is
available. The preference would be that the timeout applies to each read, where
the timeout is reset again once a read is successful. This better supports
streaming semantics expected for the `Handle::subscribe` method.
To achieve this, the `LineIter` type holds the `UnixStream` and timeout
`Duration`. The `Iterator` implementation can then use `read_line` to progress
the stream one line at a time, and the timeout only applies to each read.
Co-authored-by: Alexis Sellier <alexis@radicle.xyz>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a variant for upload-pack events to allow reporting from the upload-pack
process.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Move the events Emitter to the `radicle::node::events` module to allow it to be
used outside of `radicle-node`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of resetting the number of attempts on connection, reset it when
the connection is stable, ie. after some time has passed.
This prevents connection retries from happening in succession when a
peer is disconnected during the initial handshake, as is the case with
invalid subscribe timestamps.
If our inventory hasn't updated, don't issue a new inventory
announcement, simply use the previous message.
This avoids identical inventories propagating over the network and using
up resources.
Allow unknown fields to pass through when deserializing the `Project` payload.
This allows the deserialization to be more resilient when unknown fields are
added, e.g. from the CLI, and improves backwards-compatability when new fields
may be added in the future.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The rate-limitter gets hit on connect, due to historical gossip messages
being exchanged. This results in a high initial burst that needs to be
accounted for.
Relay now has three values: "always", "never" and "auto".
The first two operate like the previous `true`/`false`. The
new (default) value turns relaying on for nodes with public (external)
addresses and off for nodes without. This should reduce redundant traffic on
the network, that was causing the rate limiters to trigger.
With the increase in traffic on the network after launch, the old
defaults don't make sense anymore.
Longer term, we should look into throttling outgoing messages as well,
prioritizing original messages over relayed messages.
* Try to use constants instead of strings
* Move crypto seed override out of radicle-crypto
* Use GIT_COMMITTER_DATE for commit overrides
* Use RAD_LOCAL_TIME as general time override
* Allow variables to be used in release mode
In radicle/src/test.rs, we have a helper type for a Node, which
creates a temporary directory using tempfile::tempdir, which creates a
TempDir. When the TempDir is dropped, it deletes its associated
directory. Before that happens, the Node has already stored the path.
Right after that the TempDir is dropped, and the directory deleted.
When the test code later opens the repository, using Storage::open,
it recreates the directory. As a result, there is nothing that would
delete the directory when the test finishes.
This means tests that use the helper Node type leave temporary files
behind.
Fix this by storing the TempDir in the helper Node, not just its path.
This means the TempDir is only dropped at the end of the test its used
in, and no temporary files are left behind.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
cloudhead
Alexis Sellier
Lorenz Leutgeb
Fintan Halpenny
Dr Maxim Orlovsky
Lars Wirzenius