In the current design it's possible for one peer to fill the fetch queue
so that fetches from other peers are delayed.
To improve fairness, we move to a queue per peer. We then try to dequeue
from all peers in a random order for good measure.
There are cases where the service state doesn't match the state
of the underlying wire protocol. We remedy this by transitioning peers
to a "connected" state when a message is received, if they are in a
"connecting" state.
The long term solution to this will likely be to merge the service and
protocol layer so that there are no inconsistencies.
The other case happens when a persistent peer is in "disconnected" state
and attempts an inbound connection to us.
Instead of resetting the number of attempts on connection, reset it when
the connection is stable, ie. after some time has passed.
This prevents connection retries from happening in succession when a
peer is disconnected during the initial handshake, as is the case with
invalid subscribe timestamps.
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.
We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.
When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
The previous system allowed the same RID to be fetched from multiple
nodes concurrently, which caused issues, especially when cloning.
Worker code assumes that there is only one fetch per repository going on
at any given time. This change ensures that.
We keep a global fetch queue, instead of having a queue per session. We
also enqueue fetches if a session is at capacity. Finally, we allow
multiple fetch requests to receive results for the same fetch.
Only return once the connection is established or failed.
It's still not perfect, as several failure scenarios are not handled,
but it's an improvement.
We move the session address to the top-level struct, since it's needed
in a bunch of places. This change is required for the rate-limiting
code that is coming next.
This is a fairly substantial change, which adds a new configuration file
to the user's `$RAD_HOME` that includes node configuration, including
the alias, and also makes node aliases required.
When running `rad auth`, the user is now prompted for an alias, which
defaults to `$USER`.
When running `rad self`, the alias is now shown.
If the user runs radicle without a config file, the defaults are loaded,
and `$USER` is used as the alias.
When attempting to connect to a peer, and when succeeding, write the timestamp
to the database.
This will allow us to better select addresses and not retry peers to
soon.
To improve the reliability and flexibility of the protocol, we introduce
multiplexing over peer connections. This involves a new `Frame` type
that carries a stream-id and payload.
Three stream types are made available:
1. Control
2. Gossip
3. Git
This change brings the following improvements:
* Removed need to queue fetch requests
* Removed need to queue gossip messages
* Removed need for `Fetch` and `FetchOk` messages
* Service doesn't need to know about inbound fetches
* Removed one round-trip for fetch negotiation
* Removed special `done` git packet
* Removed session logic and state around Git/Fetch protocols
* Removed code around upgrading/downgrading transport
* Worker in responder mode is able to process any number of fetches
* Connections support any number of concurrent fetches
We had to introduce a few extra things however to make it all work:
* A `VarInt` type for variable-length integers, since we want the frames
to be lightweight.
* A custom "tunnel" implementation, since we couldn't use the existing
one anymore.
Overall the change removes more complexity than it adds, while improving
the protocol along the way.
First, we make sure to always attempt reconnection to persistent peers,
no matter the reason.
Second, we don't re-attempt connections when an inbound peer
disconnects. This minimizes the chance of connections crossing.
Third, we clean up `session::Error`, and don't use it to signal a
missing session, since it doesn't really belong there.
It's not a good idea to have fallible operations run after `FetchOk`
is received, since there's no easy way to communicate the error
to the remote at that point. What would then happen is that the fetch
just "hangs", since nothing is there to cancel it.
Hence, we compute namespaces earlier in the process, and store the
namespaces in the session. If there is a failure, the fetch is
never initiated.
It was possible to trigger multiple connection attempts to the same
peer, since we didn't track connection attempts that hadn't yet
returned to the service via `Service::attempted`.
With this change, we add an extra `Initial` state to sessions, so
that redundant connections are not attempted when one is already under
way.
It's possible for a peer to disconnect before responding with `FetchOk`.
In that case, we make sure to return to any pending fetch request with
an error.
The fetch implementation will potentially fetch the local operator's
remote, which is unnecessary and could result in deleted data.
Fix this by threading the local operator's public key and add an
ignore refspec to the fetch arguments, i.e.
^refs/namespaces/<local>/*
The implementation also strictly fetches all or one namespace. This is
improved upon by introducing a new variant for specifying a set of
namespaces.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It is possible for a pair of nodes to concurrently ask each other
to fetch some repo. These "crossing" requests are problematic,
because the nodes end up waiting for each other to reply, and
the reply never comes.
With this change, the node pair agrees on proceeding with one of
the fetches, while canceling the other.
Previously, the seeds information would only consist of the connected
NodeIds.
Improve on this by adding any disconnected and fetching seeds as
well. A seed can either be disconnected, connected, or fetching --
where fetching implies connected.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We weren't updating the sessions map when a persistent peer
was re-attempted. It would then panic when `connected` was called,
because the state would be `Disconnected`.
We shouldn't be marking attempted peers as "connected". We should wait
for the connection to be properly established.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
There are few unrelated changes in this commit that I wasn't able to
split out, mostly due to laziness, so I'll list them here:
* The node startup logic is moved out of the main function, and into a
dedicated functionin `client.rs`.
* There is a tentative worker implementation that is untested, and
handles git replication both ways. It's likely lacking in various ways
but I wanted to get it in there anyway.
* `Address` was moved from `radicle-node` to `radicle`, because it's
needed to send `Connect` commands from the node handle.
* The `netservices` dependency was updated.
* Finally, there is a first end-to-end node test, which currently
doesn't use NoiseXK, but uses the new reactor in `netservices`.
Signed-off-by: Alexis Sellier <self@cloudhead.io>
This is a large change that introduces transport encryption via
a Noise_XK handshake. The inner `Service` now works directly
with `NodeId` and doesn't need to interact with socket addresses.
The transport logic takes place in `wire::transport`, and the
established sessions are passed down to `Service`.
Note that the i/o implementation in the `poll-reactor` crate still
needs testing, and there is still work to be done to integrate the
Git transport as part of this architecture.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Rename the module to session from peer to be consistent with recent
renaming of the type from Peer to Session. Remove stutter by renaming
session::SessionState to session::State, and session::SessionError to
session::Error.
Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-10-28 09:57:56 -05:00
Renamed from radicle-node/src/service/peer.rs (Browse further)
Lorenz Leutgeb
cloudhead
Alexis Sellier
Han Xu
Fintan Halpenny
Alexis Sellier
Dr. Maxim Orlovsky
Dr Maxim Orlovsky
Slack Coder