Previously, our advertized inventory was not always correct: when we
stopped seeding a repository, it was still advertised as part of our
inventory until node restart, because the `Storage` type doesn't have
access to seeding policies.
In this change, we remove the concept of inventory from `Storage` and
make the authoritative place for it the routing table in our database.
To make this work, we have to add our node to the database on profile
creation, to not violate the foreign key constraint on the routing
table. Hence, the tests are changed to include our alias which is now
always available.
```
--- Expected
++++ actual: stdout
1 - branch 'master' set up to track 'origin/master'.
1 + Branch 'master' set up to track remote branch 'master' from 'origin'.
Exit status: 0
```
As per @fintohaps' suggestion, this gets the tests passing again on git 2.34.
Co-Authored-By: fintohaps <fintan@monoidal.tech>
Signed-off-by: Yorgos Saslis <yorgos.work@proton.me>
This is a change to the `config.json` file which removes certain options
that are best not touched and changes some options to make them easier
to work with.
1. We change the default journaling mode to "wal" and remove the config
option.
2. We remove the option to set the peer connection "target", as it's not
a good idea to set this to a different value, as it affects the
network.
3. We combine seeding policy & scope, since there's no need to set the
scope when the policy is "block". In that case, we always want to
block "all" remotes. The new policy configuration has the following
schema:
{ seedingPolicy: { default: "block" | "allow", scope?: "all" | "followed" } }
The "scope" key is not used when "default" is set to "block".
4. We add an `extra` field to the node config for options that are not
recognized. We use this to warn the user.
We use the *staggered broadcast* technique when relaying gossip messages
to reduce message amplification. In our basic simulation, it brings
amplification down from 3.5 to 1.5.
The idea is simply to accumulate gossip messages and relay them after a fixed
interval of time. This has two effects:
1) Old messages that haven't been sent yet are replaced by newer ones,
so only one message is sent after the delay.
2) Because nodes have their own intervals, messages are not all sent at
the same time, which reduces the chance of messages crossing paths.
For now we only turn this on for inventory messages, since this is the
least likely to be noticed by users, and also the most problematic
currently.
In the case of an out-of-sync local node, we were showing the wrong tip.
Additionally, mark your tip as unannounced if it isn't announced. This
isn't fool-proof, but can be useful to debug certain issues.
Introduce an `UploadPack` type for keeping track of relevant data from
upload-pack events and displaying progress to the terminal.
It keeps track of the number of remotes that are being uploaded to as well as
the throughput of transmitted data.
The `Progress` events are logged rather than being displayed since they don't
provide any useful details to the user.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Remove `rad sync --force` flag
* Remove `rad clone --private` flag
These flags are no longer necessary. Simplify specifying which seed
to fetch from with `--seed` is enough.
*Breaking change* The `tor` configuration field is no longer
supported. Users must remove that field from their config.
Allows for a mixed mode, where regular addresses bypass the Tor proxy,
while onion addresses go through it.
When `onion.mode` is set to "forward", Tor connections are fowarded to
the global proxy, and if it isn't set, to the OS's DNS resolution.
For mixed mode, a global proxy simply isn't set, so that IP/DNS is not
proxied.
To ensure that the `rad id` command does not modify the project payload without
it being able to be deserialized into the `Project` type, the command performs
this check throught the `project` method, and returns an error if this fails.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
As a response to Issue # c4134924cf2ce84444c6f7235a27651ad49311b5.
A no-op command in rad inbox defaults to show, but the inbox argument parser
only permits --repo in instances where the operation is explicity declared.
Here, the check for an explicit operation is removed so one is never required.
To improve the user experience of `rad init`, the events listener waits for
upload-pack messages indefinitely. The spinner reports back the upload progress
over time.
This process can be cancelled via Ctrl+C without affecting the announcement,
since that is happening in the background.
To also improve the experience, the remote NIDs that have successfully synced
are reported to the terminal.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The timeout for a `UnixStream` will interrupt a `read` even when more data is
available. The preference would be that the timeout applies to each read, where
the timeout is reset again once a read is successful. This better supports
streaming semantics expected for the `Handle::subscribe` method.
To achieve this, the `LineIter` type holds the `UnixStream` and timeout
`Duration`. The `Iterator` implementation can then use `read_line` to progress
the stream one line at a time, and the timeout only applies to each read.
Co-authored-by: Alexis Sellier <alexis@radicle.xyz>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a test to confirm that the identity can still be deserialized when an
unknown field is added.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The rate-limitter gets hit on connect, due to historical gossip messages
being exchanged. This results in a high initial burst that needs to be
accounted for.
Relay now has three values: "always", "never" and "auto".
The first two operate like the previous `true`/`false`. The
new (default) value turns relaying on for nodes with public (external)
addresses and off for nodes without. This should reduce redundant traffic on
the network, that was causing the rate limiters to trigger.
With the increase in traffic on the network after launch, the old
defaults don't make sense anymore.
Longer term, we should look into throttling outgoing messages as well,
prioritizing original messages over relayed messages.
* Try to use constants instead of strings
* Move crypto seed override out of radicle-crypto
* Use GIT_COMMITTER_DATE for commit overrides
* Use RAD_LOCAL_TIME as general time override
* Allow variables to be used in release mode
If multiple ids are specified for `rad inbox clear`, it would report
that only 1 item was cleared, when in fact multiple were.
This was found to be due to the SQL code returning a change count of
`1`. This is likely due to the way the statement is being reset each
time. The `sqlite` library does not seem to easily support the usage
of `IN` in a `WHERE` clause. So instead, the `count` is aggregated in
the loop and returned instead.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a `Patch` or `Issue` is entirely removed from the Git repository,
then there is no update to be made -- instead it should also be
removed from the cache.
This covers the case where the original author of the COB deletes the
COB.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Add `--force` flag to force a fetch even if the node is not a seed.
* Don't fail if we can't connect to a node during a fetch.
* More accurately exit when we hit our replica count.
When `--private` is passed to `clone`, we force a fetch from nodes that
aren't known seeds.
When it isn't specified, we skip nodes that aren't known seeds, even if
they are in the preferred seeds list. Previously, we would still fetch
from them, which would cause errors.
cloudhead
Yorgos Saslis
Alexis Sellier
Lars Wirzenius
Lorenz Leutgeb
Fintan Halpenny
RadsammyT
Sebastian Martinez