We add the necessary database table to keep track of which node has
synced which of our repos. This will be useful for giving the user
information on the sync status of every local repository.
We improve the API so that error messages can be sent back and are
always expected as potential responses to commands. This means that
`CommandResult` is always the type that is sent for commands.
This includes some breaking changes, for example the default "success"
message is simply `{}` instead of `{"status":"ok"}`.
Instead of storing received gossip messages in an in-memory `BTreeMap`,
we persist them to the database (`addresses.db`). There are two reasons
for this:
1. Node restarts do not wipe the state, meaning that it is much less
likely that some gossip message will get lost before reaching all
nodes.
2. This will allow us in the next commit to know how far behind we are
after a restart, to be able to request the correct time range of
gossip messages that we missed while offline.
Note that we now share the addresses.db file between two connections and
stores, which we don't do anywhere else. Having multiple connections to
the same database is fine, but the way we're having to do it is a bit
awkward. In a future patch, we could pull out the database schema from the
`address` module and rename the database to make things clearer.
The `service::tracking` module has nothing specific to the
radicle-node code. This means that it can be moved to radicle under
the `node::tracking` module.
This makes it more resuable in other contexts, for example, for
fetching logic.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Private repos are implemented by extending the identity document with a
`visibility` attribute, that can either be `"public"` (default) or
`"private"`.
In case of `private` visibility, only the delegates are allowed to view
the repo, as well as any DIDs added to the allow list.
To implement repo visibility, we simply block fetches from and
announcements to peers for whom the repo should remain invisible.
Private repos are also not announced in the `inventory` message, since
the full list of peers that *may* have the repo is retrievable from the
repo identity. This could cause errors if eg. a peer who is allowed to
view the repo doesn't actually have it. However this is an ok trade-off
for now to keep the complexity low. For repos to truly be private, it's
important that the RIDs don't leak either.
Finally, we modify `radicle-httpd` for now to only list public repos.
Eventually, we would want to change this depending on whether an allowed
peer is authenticated with the service or not.
---
It's also worth mentioning why this approach was taken, vs. end-to-end
encryption. The reasons are as follows:
1. Nodes that do not have access to a private repo will generally not want to
replicate encrypted data that they cannot examine or use.
2. The chosen solution is trivial, while encrypting git objects isn't.
3. Performance of the chosen solution is much better, there is no
overhead.
4. Privacy of the chosen solution is better: RIDs are never leaked, and
neither is the existence of a private repo, nor who has access to it.
There is one downside: Paying for storage of private repos is no better
in terms of privacy than what GitHub offers. Hosting providers will have
access to your private repos, if this solution is used.
Add `deny.toml` with a few exceptions, and update certain dependencies
to get rid of duplicates.
Running `cargo deny check` will yield some warnings still, but no
errors.
This adds functionality so that on `rad sync` and `rad clone`, we
connect to known seeds if necessary, before attempting to fetch.
In passing, some related changes were made:
* The `rad sync` command's arguments were reworked:
`--replicas <count>` can be used to specify a replica count when
fetching
* The `rad patch` command no longer has a `--fetch` option
* It's now possible to tell the node not to automatically connect to
peers in the background
* `AddressBook` is being used underneath `Seeds` to allow for shuffled
iteration
Only return once the connection is established or failed.
It's still not perfect, as several failure scenarios are not handled,
but it's an improvement.
As a starting point, we populate the address book (if empty) with some
bootstrap nodes which are trusted.
To prevent these nodes being connected to during tests (including e2e
tests), we create a new constructor for `Config` that is used in tests.
Note that a `cfg(test)` check is not enough given that e2e tests don't
have that set.
It's easier if we use camel-case everywhere, because some types may be
used on the web as well as on the control socket. This way, all
json-encoded data looks the same.
The tracking database can be opened in read-only or read-write
mode. Knowing which is being used is significant in concurrent
scenarios, e.g. it's safer to have a read-only handle on multiple
threads.
Provide static type information by signifying what kind of database
handle is being worked with, read or read-write.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This is a fairly substantial change, which adds a new configuration file
to the user's `$RAD_HOME` that includes node configuration, including
the alias, and also makes node aliases required.
When running `rad auth`, the user is now prompted for an alias, which
defaults to `$USER`.
When running `rad self`, the alias is now shown.
If the user runs radicle without a config file, the defaults are loaded,
and `$USER` is used as the alias.
* Return the correct alias from a node announcement, stripping '\0'
bytes.
* Accept `AliasStore` types as references and as owned.
* Store a `null` in the database if the alias received is empty.
Previously, we only sent the announcement when there were external
addresses. But it contains other information like the node alias,
therefore we always send it.
Since the proof-of-work can be expensive to compute everytime, we add
the ability to load the local node announcement from the file system.
We also change the proof-of-work parameters so that they are relaxed
when running in debug mode, instead of when running tests.
Finally, we remove the boolean validation of the PoW. Instead, we store
the work per-node, in the database, to be used in the future in case of
DoS attacks or network congestion.
This command announces refs to peers and waits for them to be
in sync.
Adds a subscribe method to `Handle` so that we can get the events
from the seed to allow us to confirm that the configured seeds fetched
from us.
Previously we were announcing inventories all the time, due to
relying on the timestamp changing on insertion.
We change this to only announce our inventory when it actually changes.
This fixes an issue that came up in tests where we would be reading
and writing to the database at the same time, and it would fail with
an error due to the lock being contended on.
The tracking configuration has no way of consulting for a configured
scope.
This change allows the storing of this scope inside the tracking
`Config` -- currently unused.
The default Scope is chosen as Trusted, so that nodes do not
implicitly track any other node. If this behaviour is desired -- for
example, for seed nodes -- then All should be chosen.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The DelegatesOnly scope is very limited and causes more confusion than
it solves.
The Trusted scope subsumes DelegatesOnly since it implies delegates
along with any tracked nodes. This feels more natural to use and so
DelegatesOnly is removed -- leaving only Trusted and All.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously the alias field was simply a string which could possibly be
empty. Some checks were made to see if this string was empty.
Instead, make the logic more obvious by making the alias field
optional.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The routing function for Node was not implemented and there were no
functions for getting any tracking information.
This implements routing, tracked_repos, and tracked_nodes. To
accomplish this it was necessary to add associated types to the
Handler trait so that implementations could differ in the types they
use for returning this data, i.e. the control socket needs to use a
channel while the Node needs to return serializable data.
Currently, the Node uses a `Vec` but it would be more beneficial to
return an Iterator directly to ensure we are not always hitting the
heap.
To allow for the tracking types to be used across radicle-node (the
crate) and in radicle::node (the module), it was necessary to move
the types to the common place, i.e. radicle::node. At the same time,
new structs were added to make it easier to refer to tracked repos vs
tracked nodes.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
cloudhead
Fintan Halpenny
Michalis Zampetakis
Alexis Sellier
xphoniex
Alexis Sellier