Similar to the `Fetcher` sans-IO approach, an `Announcer` sans-IO approach is
introduced. The previous `Handler::announce` was already quite close to using a
sans-IO approach, but using internally held state. This change makes the
business logic more reusable – and testable.
Similarly to the `Fetcher`, the `Announcer` is configured and then driven by
telling it when nodes have been synchronized with. It will continuously check if
it has reached its target, yielding to the caller to decide whether to keep
going. In contrast, it also provides a way to mark the process as timed out,
since announcing relies on waiting for events to come back in a timely manner.
Reword the help message of `--replicas` and `--max-replicas`, while also
improving the documentation on how the fetch and announce functionality works.
This patch changes separates the business logic of fetching from the process of
fetching itself. It does this through a sans-IO approach, where a `Fetcher`
provides the necessary state to help drive a fetch forward, without performing
any of the IO itself.
What the `Fetcher` cares about is:
- What nodes are going to be attempted to be fetched from
- In what order should they be attempted
- When should the fetching process be considered finished
The `Fetcher` is then used in `radicle-cli` to drive forward the `sync::fetch`
function, allowing it to only care about the IO.
This gives the possibility to see the inventory of another node than self.
This does replicatee the functionality of 'rad node routing --nid', but
feels like a more intuitive command to try.
Update the rust toolchain version to 1.85.
Note that this is not the latest, however, it is the latest available in nixpgs,
so this version is good enough for now.
In the case of an out-of-sync local node, we were showing the wrong tip.
Additionally, mark your tip as unannounced if it isn't announced. This
isn't fool-proof, but can be useful to debug certain issues.
* Remove `rad sync --force` flag
* Remove `rad clone --private` flag
These flags are no longer necessary. Simplify specifying which seed
to fetch from with `--seed` is enough.
* Add `--force` flag to force a fetch even if the node is not a seed.
* Don't fail if we can't connect to a node during a fetch.
* More accurately exit when we hit our replica count.
When `--private` is passed to `clone`, we force a fetch from nodes that
aren't known seeds.
When it isn't specified, we skip nodes that aren't known seeds, even if
they are in the preferred seeds list. Previously, we would still fetch
from them, which would cause errors.
The inventory operation of the `rad sync` command does not require an
RID. Localise resolving the RID in the match arms that require it.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of re-loading the full inventory on `rad init`, we simply add
the new repository to the in-memory cache.
We also keep track of whether a fetch was a full clone, to know when
to update our routing table.
The command now works with both a replica target *and* a seeds target.
This is especially useful to eg. ensure that a preferred seed has been
synced, while still trying to hit a higher replica count.
In the case where we don't find any seeds, display a better error
message.
We also don't bail out on error, since the announcement step might still
be worth trying.
We were using the two pretty interchangeably. "Project" should only be
used to refer to the repository payload. "Repository" should be used
when referring to the resource that is fetched, synced, cloned, checked
out etc.
If the expected replica count had exceeded the amount of seeds already
synced with the following message would appear:
Not connected to any seeds for {rid}.
This is not true, since there are seeds that were already synced.
This is fixed by calculating the `synced` and `connected` nodes,
without the local peer in the set. This means that the check for
`synced.len() >= connected.len()` will get a hit and the function will
correctly return early.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To allow `rad sync` to break early from the call to `announce`,
control flow is introduced for the callback.
In `rad sync`, the number of synced nodes is checked against the
`replicas` -- if specified -- and sends a `Break`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
By default, this command now:
* Follows the peer if not already followed
* Syncs refs of the peer, if it wasn't followed before
* Fetches refs into working copy
This functionality can be disabled via `--no-sync` and `--no-fetch`.
Allow the caller of `rad sync` to `--sort-by` the NID, alias, or
status of the seeds.
This prevents random ordering, due to timestamps, in CLI testing.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Currently, if refs are updated while the node is stopped, these refs are
not immediately announced when the node starts, due to the node being
unaware of the updates.
To remedy this, on startup, we compare the refs we have in storage with
the refs we announced to the network. Since only our own refs could have
changed while the node was stopped, this change only applies to owned
refs. If we detect that storage has changed while the node was stopped,
we create ref announcements for the updated repos and store them in our
gossip database. These announcements will be included with other
historical gossip announcements when nodes connect to us.
To keep track of what was announced, we store our local node's sync
status in the database. This is also useful when using the `rad sync status`
command, as it tells us what oid our local node is at, and whether it
is aware of our local changes yet.
In addition to this, we make a small fix: previously, inventory and ref
announcements originating from our local node were not stored in the
announcement database. We ensure that they are via the new `announce`
method.
This command shows the sync status of a repository by consulting the
database via the node.
To make it work, we call into the address store to update the
`repo-sync-status` table when receiving ref announcements that are
relevant to us.
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
When a fetch is user-requested, a timeout can be supplied that is passed
down to the worker.
When a fetch is service-requested, a default timeout is used.
This adds functionality so that on `rad sync` and `rad clone`, we
connect to known seeds if necessary, before attempting to fetch.
In passing, some related changes were made:
* The `rad sync` command's arguments were reworked:
`--replicas <count>` can be used to specify a replica count when
fetching
* The `rad patch` command no longer has a `--fetch` option
* It's now possible to tell the node not to automatically connect to
peers in the background
* `AddressBook` is being used underneath `Seeds` to allow for shuffled
iteration
Lorenz Leutgeb
Fintan Halpenny
Richard Levitte
Lorenz Leutgeb
cloudhead
Alexis Sellier
Lars Wirzenius