Add the `--revision` option for `rad patch checkout`. This allows the
user to specify a particular revision of the patch to checkout. If the
revision is not specified it will use the latest revision of the patch.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add the `--name` option to `rad patch checkout`. This allows the user
to name the branch they are checking out, rather than use the
`patch/<short-id>` format.
This change includes returning early for setting the upstream branch,
if the upstream already existed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a `--force` flag to forcibly checkout the branch to the latest
revision, if the branch already exists.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Currently, if refs are updated while the node is stopped, these refs are
not immediately announced when the node starts, due to the node being
unaware of the updates.
To remedy this, on startup, we compare the refs we have in storage with
the refs we announced to the network. Since only our own refs could have
changed while the node was stopped, this change only applies to owned
refs. If we detect that storage has changed while the node was stopped,
we create ref announcements for the updated repos and store them in our
gossip database. These announcements will be included with other
historical gossip announcements when nodes connect to us.
To keep track of what was announced, we store our local node's sync
status in the database. This is also useful when using the `rad sync status`
command, as it tells us what oid our local node is at, and whether it
is aware of our local changes yet.
In addition to this, we make a small fix: previously, inventory and ref
announcements originating from our local node were not stored in the
announcement database. We ensure that they are via the new `announce`
method.
This command shows the sync status of a repository by consulting the
database via the node.
To make it work, we call into the address store to update the
`repo-sync-status` table when receiving ref announcements that are
relevant to us.
An informational message for the protocol is useful for sending
messages to a node that is not required to be relayed.
This message is introduced by starting with an ACK message for a set
of references being synced. This fixes an issue for announcing to a
node that is already synced, since it can now ACK that it received the
message and replies that it is synced.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It can be useful to return the `RefsAt` that was announced for this
node, for example, to compare it to node events.
Change the `Handle` method to return `RefsAt` so that when it is
called it returns the local node's `RefsAt`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The bounds on `collect_from` were overly restrictive. Allow any
`IntoIterator` be used as an argument, while taking ownership, rather
mutably borrowed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It can be useful to be able to check if a repository contains a given
commit. Add a `contains` method to the `ReadRepository` trait to
handle this.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The previous system allowed the same RID to be fetched from multiple
nodes concurrently, which caused issues, especially when cloning.
Worker code assumes that there is only one fetch per repository going on
at any given time. This change ensures that.
We keep a global fetch queue, instead of having a queue per session. We
also enqueue fetches if a session is at capacity. Finally, we allow
multiple fetch requests to receive results for the same fetch.
The pack-writer perfectly consumes all input of the pack without
overshoot, which it can as it knows how many objects it ought
to read.
Thus, the last packetline of the pack is consumed to the last
byte without ever asking for more data that would then be denied
as the flush packet is encountered.
This needs to be accounted for in the calling code.
Signed-off-by: Sebastian Thiel <sebastian.thiel@icloud.com>
Previously, it was possible for the `wire::serialize` function to panic
due to large ref announcements. Now that ref announcements are properly
bounded in size, this shouldn't happen anymore.
It's possible that a temporary directory can be created on a different
mount, and when moved during a clone, creates an I/O error.
To prevent this, add a method to `Storage` which creates a temporary
directory and `Repository` with the path in the form
`<rid>.lock`.
This `.lock` path name can easily be skipped when listing
repositories. Further tooling can be added for detecting leftover
`.lock` directories if, for some reason, they were not cleaned up by
the `tempfile` crate.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The test suite for `radicle-httpd` assumed that `master` would be used
for the working copy. However, if a user set their Git default branch
to be something other than `master` then the test suite will fail for
them.
To remedy this, initialise the working copy with the `master` branch
as well.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of creating the tree to check if `rad/sigrefs` has changed, it
is possible to check that the signature of the signed refs being
updated is the same as the old one. If so, then there are no
changes. Otherwise, there must be changes and the update can continue
to create the necessary Git objects.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, we would process any valid inventory or refs announcement.
This could cause problems for keeping sync statuses, as we could get
a refs announcement from a node that isn't in our database.
In practice, there is no good reason why we should receive a refs or
inventory announcement before the announcer's node announcement, unless
nodes are not following the protocol.
Implementing this change meant that many of the tests needed updating,
as it wasn't assumed that a node announcement was needed before
receiving inventories etc. from a peer.
We also had to implement a basic form of node address banning to make
the tests pass after this change.
We add the necessary database table to keep track of which node has
synced which of our repos. This will be useful for giving the user
information on the sync status of every local repository.
* Set peer's `last_active` even if they are being throttled
* Make sure not to break out of loop in wire module
* Add some more logging
* Use different variable for session filtering to make it clearer
We improve the API so that error messages can be sent back and are
always expected as potential responses to commands. This means that
`CommandResult` is always the type that is sent for commands.
This includes some breaking changes, for example the default "success"
message is simply `{}` instead of `{"status":"ok"}`.
Instead of trying to handle cases where delegate branches diverge, we
simply throw an error in those cases, which prevents the canonical head
from updating.
This simplifies the code a lot, and forces the user to take action when
such an event occures.
It is desirable to keep the Git repositories in Radicle efficient by
ensuring objects are packed and unreachable objects are removed.
Add a process call to `git gc` to achieve this. Two flags are used for
garbage collection: `--auto` and `--prune`. Their details are outlined below:
--auto
With this option, git gc checks whether any housekeeping is
required; if not, it exits without performing any work.
--prune=<date>
Prune loose objects older than date (default is 2 weeks
ago, overridable by the config variable
gc.pruneExpire). --prune=now prunes loose objects
regardless of their age and increases the risk of
corruption if another process is writing to the repository
concurrently; see "NOTES" below. --prune is on by default.
For `--prune` the default used is `1.hours.ago`. This was chosen over
`now` since that may be too aggressive after just fetching new data.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Ref announcements contain the namespace (NID) and the SHA of the new
`rad/sigrefs`. This information can be used by the fetch protocol to
fetch only those `Oid`s for those namespaces.
Add a new stage `SigrefsAnnouncement` that captures this behaviour by
only asking for the advertised `Oid`s and updating the corresponding
`rad/sigrefs` for each namespace. Note that this is a special case of
the `SpecialRefs` stage and so the protocol chooses between these two
by checking if there was announcement passed through. The following
stage, `DataRefs`, stays the same.
Note that this stage can only be activated via a `pull`, and `clone`
stays the same.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The RefsAnnouncement previous announced its complete set of
`rad/sigrefs`. This was useful in that verification can occur on the
receiveing side immediately by verifying the refs and proceeding with
the fetch -- note that it did nothing with the ref information it
received.
This can be minimised to only advertising the namespace, i.e. the
NID, and the new SHA of the `rad/sigrefs`. This is represented by a
new message type, `RefsAt`, which contains the NID and the Oid.
For checking if an announcement is stale or fresh, the receiving
side can check the existence of the SHA in its Git repository.
The tests are appropriately modified to use NIDs and Oids.
To prepare for a follow-up change, the announcement data is also
threaded through for the fetch protocol to focus fetching only the
advertised set of Oids.
N.B.: because the refs aren't communicated, the verification happens
when fetching since the data needs to be fetched before
verifying. This is a trade-off with minimising the message payload for
verifying earlier.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.
Change the error type from `git::ext::Error` to `git::raw::Error`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
cloudhead
Sebastian Martinez
Thomas Scholtes
Fintan Halpenny
Sebastian Thiel