Only return once the connection is established or failed.
It's still not perfect, as several failure scenarios are not handled,
but it's an improvement.
We move the session address to the top-level struct, since it's needed
in a bunch of places. This change is required for the rate-limiting
code that is coming next.
The tracking database can be opened in read-only or read-write
mode. Knowing which is being used is significant in concurrent
scenarios, e.g. it's safer to have a read-only handle on multiple
threads.
Provide static type information by signifying what kind of database
handle is being worked with, read or read-write.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This is a fairly substantial change, which adds a new configuration file
to the user's `$RAD_HOME` that includes node configuration, including
the alias, and also makes node aliases required.
When running `rad auth`, the user is now prompted for an alias, which
defaults to `$USER`.
When running `rad self`, the alias is now shown.
If the user runs radicle without a config file, the defaults are loaded,
and `$USER` is used as the alias.
* Return the correct alias from a node announcement, stripping '\0'
bytes.
* Accept `AliasStore` types as references and as owned.
* Store a `null` in the database if the alias received is empty.
Previously, we only sent the announcement when there were external
addresses. But it contains other information like the node alias,
therefore we always send it.
Since the proof-of-work can be expensive to compute everytime, we add
the ability to load the local node announcement from the file system.
We also change the proof-of-work parameters so that they are relaxed
when running in debug mode, instead of when running tests.
Finally, we remove the boolean validation of the PoW. Instead, we store
the work per-node, in the database, to be used in the future in case of
DoS attacks or network congestion.
When attempting to connect to a peer, and when succeeding, write the timestamp
to the database.
This will allow us to better select addresses and not retry peers to
soon.
We were allowing remotes to be constructed via a function that didn't
check its inputs.
This is now fixed by moving the public key into the `SignedRefs` struct.
This command announces refs to peers and waits for them to be
in sync.
Adds a subscribe method to `Handle` so that we can get the events
from the seed to allow us to confirm that the configured seeds fetched
from us.
To improve the reliability and flexibility of the protocol, we introduce
multiplexing over peer connections. This involves a new `Frame` type
that carries a stream-id and payload.
Three stream types are made available:
1. Control
2. Gossip
3. Git
This change brings the following improvements:
* Removed need to queue fetch requests
* Removed need to queue gossip messages
* Removed need for `Fetch` and `FetchOk` messages
* Service doesn't need to know about inbound fetches
* Removed one round-trip for fetch negotiation
* Removed special `done` git packet
* Removed session logic and state around Git/Fetch protocols
* Removed code around upgrading/downgrading transport
* Worker in responder mode is able to process any number of fetches
* Connections support any number of concurrent fetches
We had to introduce a few extra things however to make it all work:
* A `VarInt` type for variable-length integers, since we want the frames
to be lightweight.
* A custom "tunnel" implementation, since we couldn't use the existing
one anymore.
Overall the change removes more complexity than it adds, while improving
the protocol along the way.
The One variant caused many paint points for fetching logic, where it
was not necessary. It was only constructed in one place, which could
be replaced by using the variant that holds a set of keys.
Remove the Namespaces::One variant and rename Namespaces::Many to
Namespaces::Trusted.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a repository does not yet exist during a fetch, i.e. a clone,
then only delegates are being fetched.
Augment Namespaces::Many to hold the trusted and optional delegate peers
separately. Doing so allows the construction of the variant without
the repository existing, but tracking relationships existing.
This variant can then be used to build refspecs for both the trusted
peers and delegates when doing a cloning fetch.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
First, we make sure to always attempt reconnection to persistent peers,
no matter the reason.
Second, we don't re-attempt connections when an inbound peer
disconnects. This minimizes the chance of connections crossing.
Third, we clean up `session::Error`, and don't use it to signal a
missing session, since it doesn't really belong there.
This change ensures that on `Subscribe`, we only relay the latest
announcement of each node, per announcement category.
Previously, we would relay all announcement, while only the last one
was relevant.
It's not a good idea to have fallible operations run after `FetchOk`
is received, since there's no easy way to communicate the error
to the remote at that point. What would then happen is that the fetch
just "hangs", since nothing is there to cancel it.
Hence, we compute namespaces earlier in the process, and store the
namespaces in the session. If there is a failure, the fetch is
never initiated.
It was possible to trigger multiple connection attempts to the same
peer, since we didn't track connection attempts that hadn't yet
returned to the service via `Service::attempted`.
With this change, we add an extra `Initial` state to sessions, so
that redundant connections are not attempted when one is already under
way.
It's possible for a peer to disconnect before responding with `FetchOk`.
In that case, we make sure to return to any pending fetch request with
an error.
To determine the correct Namespaces to use, the repo Policy and Scope
are consulted.
If the policy is Block then this is a failure state and is reported to
the other end.
Otherwise, if the policy is Track, then the scope is checked. For the
Trusted Scope then the Namespaces should consist of all tracked nodes
and delegates for the given repository. However, the repository might
not exist and so it is necessary for the whole repository to be
fetched.
For the All Scope, all Namespaces are fetched.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The fetch implementation will potentially fetch the local operator's
remote, which is unnecessary and could result in deleted data.
Fix this by threading the local operator's public key and add an
ignore refspec to the fetch arguments, i.e.
^refs/namespaces/<local>/*
The implementation also strictly fetches all or one namespace. This is
improved upon by introducing a new variant for specifying a set of
namespaces.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of returning tuples for `repo_entry` and `node_entry`, return
the `Repo` and `Node` types, respectively.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The tracking configuration has no way of consulting for a configured
scope.
This change allows the storing of this scope inside the tracking
`Config` -- currently unused.
The default Scope is chosen as Trusted, so that nodes do not
implicitly track any other node. If this behaviour is desired -- for
example, for seed nodes -- then All should be chosen.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The DelegatesOnly scope is very limited and causes more confusion than
it solves.
The Trusted scope subsumes DelegatesOnly since it implies delegates
along with any tracked nodes. This feels more natural to use and so
DelegatesOnly is removed -- leaving only Trusted and All.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously the alias field was simply a string which could possibly be
empty. Some checks were made to see if this string was empty.
Instead, make the logic more obvious by making the alias field
optional.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
A fetch is a client-server interaction and so there are two sides to
this interaction. These sides are generally referred to as receive --
the client side -- and upload -- the server side.
The `Fetch` type indicated whether it was doing a receive or upload by
using a field `initiated: bool`. However, this can be confusing when
you pair it with the `namespaces: Namespaces` field, since only the
receive side should be sending what refspecs it wants and the upload
side responds with those matching refspecs.
A better way to represent this is using an enum for which direction
the fetch is being considered, where the receive side contains the
Namespaces. To borrow from Noise, the variant names are 'Initiator'
for the client and 'Responder' for the server.
The resulting code then makes decisions based on what variant of the
enum was passed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It is possible for a pair of nodes to concurrently ask each other
to fetch some repo. These "crossing" requests are problematic,
because the nodes end up waiting for each other to reply, and
the reply never comes.
With this change, the node pair agrees on proceeding with one of
the fetches, while canceling the other.
The routing function for Node was not implemented and there were no
functions for getting any tracking information.
This implements routing, tracked_repos, and tracked_nodes. To
accomplish this it was necessary to add associated types to the
Handler trait so that implementations could differ in the types they
use for returning this data, i.e. the control socket needs to use a
channel while the Node needs to return serializable data.
Currently, the Node uses a `Vec` but it would be more beneficial to
return an Iterator directly to ensure we are not always hitting the
heap.
To allow for the tracking types to be used across radicle-node (the
crate) and in radicle::node (the module), it was necessary to move
the types to the common place, i.e. radicle::node. At the same time,
new structs were added to make it easier to refer to tracked repos vs
tracked nodes.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, the seeds information would only consist of the connected
NodeIds.
Improve on this by adding any disconnected and fetching seeds as
well. A seed can either be disconnected, connected, or fetching --
where fetching implies connected.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a specific tracking policy isn't found, we allow setting a default.
To preserve existing behavior, the default is `Block`, ie. if a node
isn't tracked explicitly, it is blocked.
However, this change allows for using `Track` as the default, which
is a way to track everything without having to know the entities in
advance.
We weren't updating the sessions map when a persistent peer
was re-attempted. It would then panic when `connected` was called,
because the state would be `Disconnected`.
Rust 1.67 was announced[0]. Update the necessary files for running
1.67 and fix the clippy suggestions for string interpolation.
[0]: https://blog.rust-lang.org/2023/01/26/Rust-1.67.0.html
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We shouldn't be marking attempted peers as "connected". We should wait
for the connection to be properly established.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
There are few unrelated changes in this commit that I wasn't able to
split out, mostly due to laziness, so I'll list them here:
* The node startup logic is moved out of the main function, and into a
dedicated functionin `client.rs`.
* There is a tentative worker implementation that is untested, and
handles git replication both ways. It's likely lacking in various ways
but I wanted to get it in there anyway.
* `Address` was moved from `radicle-node` to `radicle`, because it's
needed to send `Connect` commands from the node handle.
* The `netservices` dependency was updated.
* Finally, there is a first end-to-end node test, which currently
doesn't use NoiseXK, but uses the new reactor in `netservices`.
Signed-off-by: Alexis Sellier <self@cloudhead.io>
This is a large change that introduces transport encryption via
a Noise_XK handshake. The inner `Service` now works directly
with `NodeId` and doesn't need to interact with socket addresses.
The transport logic takes place in `wire::transport`, and the
established sessions are passed down to `Service`.
Note that the i/o implementation in the `poll-reactor` crate still
needs testing, and there is still work to be done to integrate the
Git transport as part of this architecture.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Use the BoundedVec to limit the supported number of refs in
RefAnnouncement messages. The size (235) is the maximum supported by
message wire size.
To reduce the complexity of the changes, BoundedVec is used again
instead of using a new bounded Refs or BTreeMap. This requires
defining new Decode/Encode logic for Ref's element types, and
workarounds for arbitrary instances.
Signed-off-by: Slack Coder <slackcoder@server.ky>
The latter is a fork of the former, which adds const generics
and a bunch of other quality of life improvements that aren't
in the original crate due to it being unmaintained.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Restrict message size decoding by limiting Message's vector size.
Introduce the BoundedVec type to do this on the type level for
convenience and clarity.
Signed-off-by: Slack Coder <slackcoder@server.ky>
Avoid wasting persistent memory by cleaning up stale routing entries
when the routing table is considered constrained. Limit pruning to
improve resiliency in the face of long intervals between service
downtimes.
Signed-off-by: Slack Coder <slackcoder@server.ky>
Keep the node's understanding of project hosting location current by
updating how Inventory Announcement messages are processed.
The Inventory Announcements contain a node's entire listing of projects.
Replace the routing table information for the announcer with those from the
message.
Signed-off-by: Slack Coder <slackcoder@server.ky>
Include the node's IP address in the Node's announcements. Source it
from the service's configuration which can assumed to be populated with
default values when available.
Signed-off-by: Slack Coder <slackcoder@server.ky>
To support ssh-agent, add a signing method that can fail.
Also cleanup usage so that we always accept signer references.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
We implement a small proof-of-work on the node announcement message
to prevent the routing table from being filled by fake entries.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Rename the module to session from peer to be consistent with recent
renaming of the type from Peer to Session. Remove stutter by renaming
session::SessionState to session::State, and session::SessionError to
session::Error.
Signed-off-by: Slack Coder <slackcoder@server.ky>
The crypto types are useful to use in other components that are
separate from the radicle crate, e.g. the incoming radicle-cobs
crate.
Separate out the types into a radicle-crypto crate. This crate defines
the usual PublicKey, SecretKey, and Signature types. It also provides
the ssh functionality, under the `ssh` feature. It is also necessary
to provide the `From` impl for `Component` here, which again is
guarded by the `git-ref-format` feature.
The `Arbitrary` implementation are moved. The arbitrary decision of
redefining `ByteArray` in both radicle-crypto and radicle was made.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The changes in this commit are all intertwined and it was hard to
split them into smaller commits:
* Use custom transports everywhere
Instead of `file://` and instead of using a child process for fetching
and pushing; we go through our custom transports, which use the `rad://`
and `heartwood://` schemes.
* Introduce a 'mock' remote transport
To test the remote transport without needing to spawn TCP streams,
we add a mock transport that works between storages on the file
system.
* Get rid of node's `git-url`
Instead of git urls, nodes simply use their node-ids to replicate.
This type that was wrapping `Message` isn't very useful here.
If we decide that we do need something like that, it's best handled one
layer below.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
cloudhead
Fintan Halpenny
Alexis Sellier
xphoniex
Han Xu
Alexis Sellier
Dr. Maxim Orlovsky
Dr Maxim Orlovsky
Slack Coder