The tracked nodes include nodes marked tracked in the db, and
the delegates and `Namespaces::All` if it is a new repo.
Signed-off-by: Han Xu <keepsimple@gmail.com>
First, we make sure to always attempt reconnection to persistent peers,
no matter the reason.
Second, we don't re-attempt connections when an inbound peer
disconnects. This minimizes the chance of connections crossing.
Third, we clean up `session::Error`, and don't use it to signal a
missing session, since it doesn't really belong there.
1. We were announcing inventory every 30 seconds. This is now changed
to 1 hour.
2. We were using a variable to track whether our inventory has been
announced to the network, but this var was set based on what we
tracked, not the inventory we had. This var has been removed.
Better handle case where the connection to the daemon fails, in the
responder scenario.
Instead of trying to read the `done` packet once, we loop until we've
read it. This handles a scenario where the uploader fails before the
git protocol is initiated, and the fetcher is still sending git packets.
This change ensures that on `Subscribe`, we only relay the latest
announcement of each node, per announcement category.
Previously, we would relay all announcement, while only the last one
was relevant.
If there is an unexpected error, eg. `BrokenPipe` in the upload process
process of the worker, we may have more than four bytes to read from the
remote socket. This change ensures that we don't attempt to read more
into a buffer of size `4`.
For whatever reason, we weren't using the deserializer for message
decoding. This meant that if a message came in two pieces, it would
cause decoding problems, since the first piece would be discarded.
It's possible that the daemon or stream closes the git connection
without reading the final `done` packet. If this is the case, we
try to read it after either of the connections is closed.
This may potentially mitigate the issue where the node then receives
the `done` packet while in gossip mode, and disconnects the remote
for misbehavior.
Due to never removing peers from the peer map in the wire
protocol module, it was possible for there to be multiple
connected peers with the same Node ID. Some might be in a
"connected" state, while others might be "disconnected".
This caused a bug where a peer, although connected would
be considered disconnected by the wire module, because
when searching for a peer with that Node ID, the disconnected
one would return first.
To fix this, we make sure to remove peers from the map once
the connection is dropped.
It's not a good idea to have fallible operations run after `FetchOk`
is received, since there's no easy way to communicate the error
to the remote at that point. What would then happen is that the fetch
just "hangs", since nothing is there to cancel it.
Hence, we compute namespaces earlier in the process, and store the
namespaces in the session. If there is a failure, the fetch is
never initiated.
It was possible to trigger multiple connection attempts to the same
peer, since we didn't track connection attempts that hadn't yet
returned to the service via `Service::attempted`.
With this change, we add an extra `Initial` state to sessions, so
that redundant connections are not attempted when one is already under
way.
The radicle-node was forced into using Scope::Trusted for its default
scope.
Allow the Scope to be specified at the command line using a
`--tracking-scope` option.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Periodically, make sure we try to fetch inventory that is tracked
but missing from local storage.
We decide not to fetch immediately on `TrackRepo`, as this limits
the command's flexibility. For example, when `rad clone` is used,
we want to be able to fetch manually after tracking.
This avoids race conditions, especially in tests, where we try to
connect to the node right after spawning it, but since the
`Runtime::run` function is often called in a different thread, the
socket is not created in time.
It's possible for a peer to disconnect before responding with `FetchOk`.
In that case, we make sure to return to any pending fetch request with
an error.
To ensure the behaviour of ignoring references that are owned by the
local peer a test case is added.
The case has two peers: alice and bob, where alice creates a new
repository. bob fetches to be up-to-date with alice. alice, in turn,
creates an issue and fetches from bob -- which should be successful.
If the refspec:
^refs/namespaces/<alice>/*
was not in place, her fetch from bob would fail since the signed
references would not be a fast-forward.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Adds a test for fetching from trusted delegates.
Currently marked as failing. When the fetch is initiated from a client
that does not have the repository then it will fetch all namespaces,
instead of the trusted set.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To determine the correct Namespaces to use, the repo Policy and Scope
are consulted.
If the policy is Block then this is a failure state and is reported to
the other end.
Otherwise, if the policy is Track, then the scope is checked. For the
Trusted Scope then the Namespaces should consist of all tracked nodes
and delegates for the given repository. However, the repository might
not exist and so it is necessary for the whole repository to be
fetched.
For the All Scope, all Namespaces are fetched.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The fetch implementation will potentially fetch the local operator's
remote, which is unnecessary and could result in deleted data.
Fix this by threading the local operator's public key and add an
ignore refspec to the fetch arguments, i.e.
^refs/namespaces/<local>/*
The implementation also strictly fetches all or one namespace. This is
improved upon by introducing a new variant for specifying a set of
namespaces.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The method `track_repo` only accepted the RID as the argument for
tracking.
Instead, allow passing Scope as an argument for tracking a
repository.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of returning tuples for `repo_entry` and `node_entry`, return
the `Repo` and `Node` types, respectively.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The tracking configuration has no way of consulting for a configured
scope.
This change allows the storing of this scope inside the tracking
`Config` -- currently unused.
The default Scope is chosen as Trusted, so that nodes do not
implicitly track any other node. If this behaviour is desired -- for
example, for seed nodes -- then All should be chosen.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The DelegatesOnly scope is very limited and causes more confusion than
it solves.
The Trusted scope subsumes DelegatesOnly since it implies delegates
along with any tracked nodes. This feels more natural to use and so
DelegatesOnly is removed -- leaving only Trusted and All.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously the alias field was simply a string which could possibly be
empty. Some checks were made to see if this string was empty.
Instead, make the logic more obvious by making the alias field
optional.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `routes_to` function can loop infinitely during test.
To prevent this from happening a number of tries variable is added. It
checks that the number of tries does not exceed 30, which gives the
function about 3 seconds to complete -- due to the call to
`thread::sleep` using 100ms.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The MockStorage holds an inventory of repositories and their
identities but when asking for a MockRepository, none of this
information is transferred.
Add `id` and `doc` fields to MockRepository for defining some of its
functionality.
In addition, add a generator for a nonempty MockStorage to use in
tests that will expect the storage to be populated.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
A fetch is a client-server interaction and so there are two sides to
this interaction. These sides are generally referred to as receive --
the client side -- and upload -- the server side.
The `Fetch` type indicated whether it was doing a receive or upload by
using a field `initiated: bool`. However, this can be confusing when
you pair it with the `namespaces: Namespaces` field, since only the
receive side should be sending what refspecs it wants and the upload
side responds with those matching refspecs.
A better way to represent this is using an enum for which direction
the fetch is being considered, where the receive side contains the
Namespaces. To borrow from Noise, the variant names are 'Initiator'
for the client and 'Responder' for the server.
The resulting code then makes decisions based on what variant of the
enum was passed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It is possible for a pair of nodes to concurrently ask each other
to fetch some repo. These "crossing" requests are problematic,
because the nodes end up waiting for each other to reply, and
the reply never comes.
With this change, the node pair agrees on proceeding with one of
the fetches, while canceling the other.
Fintan Halpenny
Han Xu
Alexis Sellier
Alexis Sellier
Adam Szkoda
xphoniex