Instead of having a routing database, and an addresses database,
consolidate that in a single database called `node.db`.
We also simplify database access, by providing a single type for opening
the database, and use traits to access the various functions.
We create a new `seed::Store` that has seed-related methods split out of
the `address::Store`.
Note that tracking is still in its own database currently, as it isn't
ephemeral state. This makes it easy to delete the new `node.db` without
risk of losing any important information.
To avoid renaming all database methods, we create a `Stores` wrapper
that allows us to access routing, addresses, seeds etc. without type
ambiguity.
This is a breaking change and requires `addresses.db` and `routing.db`
to be deleted.
Allow the caller of `rad sync` to `--sort-by` the NID, alias, or
status of the seeds.
This prevents random ordering, due to timestamps, in CLI testing.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If the local peer does not have any `rad/sigrefs` then there is no
risk of any data corruption. In this case the repository can be
removed entirely.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `remove` method is no longer being used, with `clean` taking its
place.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a `clean` method to remove the remotes of a repository that are
not the local peer nor the delegates.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Distinguish between repos that are simply tracked, vs. repos that are
cloned (and thus forked). By default, we don't show repos that don't
have a local fork.
Add the `--name` option to `rad patch checkout`. This allows the user
to name the branch they are checking out, rather than use the
`patch/<short-id>` format.
This change includes returning early for setting the upstream branch,
if the upstream already existed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Currently, if refs are updated while the node is stopped, these refs are
not immediately announced when the node starts, due to the node being
unaware of the updates.
To remedy this, on startup, we compare the refs we have in storage with
the refs we announced to the network. Since only our own refs could have
changed while the node was stopped, this change only applies to owned
refs. If we detect that storage has changed while the node was stopped,
we create ref announcements for the updated repos and store them in our
gossip database. These announcements will be included with other
historical gossip announcements when nodes connect to us.
To keep track of what was announced, we store our local node's sync
status in the database. This is also useful when using the `rad sync status`
command, as it tells us what oid our local node is at, and whether it
is aware of our local changes yet.
In addition to this, we make a small fix: previously, inventory and ref
announcements originating from our local node were not stored in the
announcement database. We ensure that they are via the new `announce`
method.
This command shows the sync status of a repository by consulting the
database via the node.
To make it work, we call into the address store to update the
`repo-sync-status` table when receiving ref announcements that are
relevant to us.
An informational message for the protocol is useful for sending
messages to a node that is not required to be relayed.
This message is introduced by starting with an ACK message for a set
of references being synced. This fixes an issue for announcing to a
node that is already synced, since it can now ACK that it received the
message and replies that it is synced.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It can be useful to return the `RefsAt` that was announced for this
node, for example, to compare it to node events.
Change the `Handle` method to return `RefsAt` so that when it is
called it returns the local node's `RefsAt`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It can be useful to be able to check if a repository contains a given
commit. Add a `contains` method to the `ReadRepository` trait to
handle this.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It's possible that a temporary directory can be created on a different
mount, and when moved during a clone, creates an I/O error.
To prevent this, add a method to `Storage` which creates a temporary
directory and `Repository` with the path in the form
`<rid>.lock`.
This `.lock` path name can easily be skipped when listing
repositories. Further tooling can be added for detecting leftover
`.lock` directories if, for some reason, they were not cleaned up by
the `tempfile` crate.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of creating the tree to check if `rad/sigrefs` has changed, it
is possible to check that the signature of the signed refs being
updated is the same as the old one. If so, then there are no
changes. Otherwise, there must be changes and the update can continue
to create the necessary Git objects.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, we would process any valid inventory or refs announcement.
This could cause problems for keeping sync statuses, as we could get
a refs announcement from a node that isn't in our database.
In practice, there is no good reason why we should receive a refs or
inventory announcement before the announcer's node announcement, unless
nodes are not following the protocol.
Implementing this change meant that many of the tests needed updating,
as it wasn't assumed that a node announcement was needed before
receiving inventories etc. from a peer.
We also had to implement a basic form of node address banning to make
the tests pass after this change.
We add the necessary database table to keep track of which node has
synced which of our repos. This will be useful for giving the user
information on the sync status of every local repository.
We improve the API so that error messages can be sent back and are
always expected as potential responses to commands. This means that
`CommandResult` is always the type that is sent for commands.
This includes some breaking changes, for example the default "success"
message is simply `{}` instead of `{"status":"ok"}`.
Instead of trying to handle cases where delegate branches diverge, we
simply throw an error in those cases, which prevents the canonical head
from updating.
This simplifies the code a lot, and forces the user to take action when
such an event occures.
The RefsAnnouncement previous announced its complete set of
`rad/sigrefs`. This was useful in that verification can occur on the
receiveing side immediately by verifying the refs and proceeding with
the fetch -- note that it did nothing with the ref information it
received.
This can be minimised to only advertising the namespace, i.e. the
NID, and the new SHA of the `rad/sigrefs`. This is represented by a
new message type, `RefsAt`, which contains the NID and the Oid.
For checking if an announcement is stale or fresh, the receiving
side can check the existence of the SHA in its Git repository.
The tests are appropriately modified to use NIDs and Oids.
To prepare for a follow-up change, the announcement data is also
threaded through for the fetch protocol to focus fetching only the
advertised set of Oids.
N.B.: because the refs aren't communicated, the verification happens
when fetching since the data needs to be fetched before
verifying. This is a trade-off with minimising the message payload for
verifying earlier.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.
Change the error type from `git::ext::Error` to `git::raw::Error`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The fetch V2 improvements introduced a `UserInfo` type that is used to
set the Git config's name and email when cloning a repository. This
`UserInfo` can also be used when creating a new `Repository`.
The `Storage` type now takes a `UserInfo` so that it can be passed to
`Repository::create`, for creating a new repository.
For testing, this requires adding a new `fixtures::user` so that
testing `Storage` instances can be easily constructed.
It also means that the `radicle-cli` tests all changed with new SHAs
since the name and email are used as part of the hashing input.
N.b. the order of the timelines while testing the identity cob
changed. It's not clear why that is.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of storing received gossip messages in an in-memory `BTreeMap`,
we persist them to the database (`addresses.db`). There are two reasons
for this:
1. Node restarts do not wipe the state, meaning that it is much less
likely that some gossip message will get lost before reaching all
nodes.
2. This will allow us in the next commit to know how far behind we are
after a restart, to be able to request the correct time range of
gossip messages that we missed while offline.
Note that we now share the addresses.db file between two connections and
stores, which we don't do anywhere else. Having multiple connections to
the same database is fine, but the way we're having to do it is a bit
awkward. In a future patch, we could pull out the database schema from the
`address` module and rename the database to make things clearer.
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.
The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.
[0]: https://github.com/Byron/gitoxide
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add the ability to remove a repository, given its RID.
This is useful in the case of attempting to clone a repository, and if
that process fails -- for example, if the repository is private --
then it's necessary to clean up the initially created repository.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `identity_doc_at` and `identity_doc` methods on `ReadRepository`
returned the unverified identity document. Every usage of these
methods would always verify the document after.
Simplify the code by calling verified in the methods and directly return the
verified identity document.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add an enum to represent the special references, `rad/id` and
`rad/sigrefs`. These can be used for cases where it's more manageable
to work with the enum type over checking for reference string equality.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Make the validation logic customisable by extracting it out into a
separate trait that can be defined by other crates.
The Repository validation logic stays the same.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of returning the unsigned references, the validation of a
repository's remotes returns the set of validation errors. The caller
is then expected to handle these validation errors appropriately, for
example, by logging them and re-raising an error.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a type to capture user information that can be used for Git
configuration and signatures.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `service::tracking` module has nothing specific to the
radicle-node code. This means that it can be moved to radicle under
the `node::tracking` module.
This makes it more resuable in other contexts, for example, for
fetching logic.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Similar to the `id` function, add a `sigrefs` function to construct
the reference name for a given remote's `rad/sigrefs`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduce a SignedRefsAt type that combines a SignedRefs along with the Oid
it was loaded at.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
cloudhead
Fintan Halpenny
Sebastian Martinez