It is possible that there can be errors while configuring a Radicle
`Repository` during `rad::init`, for example:
- configuring working copy remote fails
- signing references fails
- setting the identity head fails
- setting the project head fails
If any of these occur, a best effort is made to remove the
`Repository` and the working copy remote, so that the caller may fix
any issues and retry.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If the local peer does not have any `rad/sigrefs` then there is no
risk of any data corruption. In this case the repository can be
removed entirely.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `remove` method is no longer being used, with `clean` taking its
place.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a `clean` method to remove the remotes of a repository that are
not the local peer nor the delegates.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Distinguish between repos that are simply tracked, vs. repos that are
cloned (and thus forked). By default, we don't show repos that don't
have a local fork.
It can be useful to be able to check if a repository contains a given
commit. Add a `contains` method to the `ReadRepository` trait to
handle this.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It's possible that a temporary directory can be created on a different
mount, and when moved during a clone, creates an I/O error.
To prevent this, add a method to `Storage` which creates a temporary
directory and `Repository` with the path in the form
`<rid>.lock`.
This `.lock` path name can easily be skipped when listing
repositories. Further tooling can be added for detecting leftover
`.lock` directories if, for some reason, they were not cleaned up by
the `tempfile` crate.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of trying to handle cases where delegate branches diverge, we
simply throw an error in those cases, which prevents the canonical head
from updating.
This simplifies the code a lot, and forces the user to take action when
such an event occures.
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.
Change the error type from `git::ext::Error` to `git::raw::Error`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The fetch V2 improvements introduced a `UserInfo` type that is used to
set the Git config's name and email when cloning a repository. This
`UserInfo` can also be used when creating a new `Repository`.
The `Storage` type now takes a `UserInfo` so that it can be passed to
`Repository::create`, for creating a new repository.
For testing, this requires adding a new `fixtures::user` so that
testing `Storage` instances can be easily constructed.
It also means that the `radicle-cli` tests all changed with new SHAs
since the name and email are used as part of the hashing input.
N.b. the order of the timelines while testing the identity cob
changed. It's not clear why that is.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add the ability to remove a repository, given its RID.
This is useful in the case of attempting to clone a repository, and if
that process fails -- for example, if the repository is private --
then it's necessary to clean up the initially created repository.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Make the validation logic customisable by extracting it out into a
separate trait that can be defined by other crates.
The Repository validation logic stays the same.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of returning the unsigned references, the validation of a
repository's remotes returns the set of validation errors. The caller
is then expected to handle these validation errors appropriately, for
example, by logging them and re-raising an error.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a type to capture user information that can be used for Git
configuration and signatures.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
On push, we check whether the resulting state would cause a
divergence/fork in the canonical head, and if so, prevent the push from
happening.
This is to avoid situations where delegates have to then rollback their
heads.
Note that this doesn't prevent forks from happening altogether, as they
could happen asychronously, but it mitigates the problem.
We employ a better algorithm for determining which commit should be
canonical when there are multiple possible options.
The algorithm still doesn't work in certain cases when there are
multiple merge commits as candidates, with shared parents.
We leave some (ignored) failing tests to be resolved in the future.
Instead of returning histories from `radicle-cob`, we return an
evaluated object which we build during initial graph traversal.
We do this so that branches can be correctly pruned when operations are
invalid at the application level. This way, new operations are not
building on top of invalid ones contained in the history.
To achieve this, we introduce a new `Evaluate` trait that is implemented
by all COBs, and we make change graph evaluation fallible.
Doing this means that we get `apply` errors returned for free when an
invalid transaction is applied, and no longer need to check for action
validity in multiple places.
We also implement a new `Dag::prune` method to avoid having to copy
graph nodes during traversal.
Private repos are implemented by extending the identity document with a
`visibility` attribute, that can either be `"public"` (default) or
`"private"`.
In case of `private` visibility, only the delegates are allowed to view
the repo, as well as any DIDs added to the allow list.
To implement repo visibility, we simply block fetches from and
announcements to peers for whom the repo should remain invisible.
Private repos are also not announced in the `inventory` message, since
the full list of peers that *may* have the repo is retrievable from the
repo identity. This could cause errors if eg. a peer who is allowed to
view the repo doesn't actually have it. However this is an ok trade-off
for now to keep the complexity low. For repos to truly be private, it's
important that the RIDs don't leak either.
Finally, we modify `radicle-httpd` for now to only list public repos.
Eventually, we would want to change this depending on whether an allowed
peer is authenticated with the service or not.
---
It's also worth mentioning why this approach was taken, vs. end-to-end
encryption. The reasons are as follows:
1. Nodes that do not have access to a private repo will generally not want to
replicate encrypted data that they cannot examine or use.
2. The chosen solution is trivial, while encrypting git objects isn't.
3. Performance of the chosen solution is much better, there is no
overhead.
4. Privacy of the chosen solution is better: RIDs are never leaked, and
neither is the existence of a private repo, nor who has access to it.
There is one downside: Paying for storage of private repos is no better
in terms of privacy than what GitHub offers. Hosting providers will have
access to your private repos, if this solution is used.
This change allows for files (blobs) to be embedded into COB entries.
This allows for things like image attachments in issue comments for example.
For now, we only enable this in the `issue` COB.
The way it works is that relevant COB actions carry metadata about which
files are attached to them; and we store those files as blobs inside the
COB entry's *tree* object, under an `embeds/` folder.
The `Embed<T>` type is used for the above, and either carries actual
content, or carries a content-id. Retrieving the actual content is as
simple as asking the repository for a blob with that content-id.
As a possible future extension, MIME types could be stored alongside the
files in a "metadata" file. This could help clients display the content
appropriately.
This new trait is used when only signing refs is needed, and not general
write access to a repository.
We also implement `ReadRepository` and `SignRepository` for
`DraftStore`.
The additional flexibility in choosing a backend for the COB store will
allow things like "draft" COBs that are stored in a different location,
without changing much of the logic.
When applying a `Merge` action, we verify that the commit is in the
history of the default branch of that user.
Since this is not the case when issuing a `rad merge`, and since that
command is no longer needed with the push-based merge, we remove the
`rad merge` command completely.
This adds the ability to the git-remote-helper to open and update
patches via a `git push` to the magic ref `refs/patches` and
`refs/heads/patches/<patch-id>`.
The main change is that we had to move away from offering the `connect`
capability in the remote helper, to offering `push` and `fetch`
individually.
To get a feel for how this works, see the `rad-patch-via-push.md`
example.
This change updates to the latest radicle-git-ext, which required a
few changes within the radicle family of crates.
One set of changes is that the radicle-git-ext crate subsumes
git-commit, git-trailers, and git-ref-format -- so all those imports
go through radicle-git-ext.
The commit code requires that parent `Oid`s point to commits, which
made some tests fail. One of the reasons for these failures is that
the patch tests used fabricated `Oid`s and so the verification would
fail. This is fixed by allowing the test context code to create
`PatchRequest`s that create valid commits.
The other reason for failure was that the identity `Action` passed an
`Oid` that pointed to a blob. This is fixed by removing that code and
making a note of it in documentation.
The final major change was moving the Refspec type into git-ref-format
and removing the AsRefspecs trait. The trait was not required and
could its usage could be replaced by simpler code.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We were allowing remotes to be constructed via a function that didn't
check its inputs.
This is now fixed by moving the public key into the `SignedRefs` struct.
Fetch project information using the delegates fork of the project. This
avoids expecting a user's fork to exist locally.
Delegate project information is expected to remain in sync.
With this change, we not only verify the signed refs before
the repository "transfer", but also validate that all refs in the
repo are signed and there is no discrepancy with the signed refs
file.
When a merge is applied to a patch, we check whether the threshold
of delegates has been reached, at which point we set the patch status
to `merged`.
This requires threading the identity document oid through the "apply"
process.
Introduce a canonical reference for refs/rad/id to aid in discovering
information about a repository without knowing a peer -- e.g. if a
peer wants to clone a new repository it needs to find out which peers
are delegates for that project, but does not know what their NIDs are
up front.
The approach mirrors the canonical `head` for a repository and
introduces useful constructors for `Doc`/`DocAt`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Periodically, make sure we try to fetch inventory that is tracked
but missing from local storage.
We decide not to fetch immediately on `TrackRepo`, as this limits
the command's flexibility. For example, when `rad clone` is used,
we want to be able to fetch manually after tracking.
Before this change, the identity document would be loaded for the
COB signer. This meant that a fork would be needed for that signer.
After this change, it's no longer necessary, since the identity doc
head is computed from available remotes.
While making those changes, it was also apparent that some of the
identity-related functions had bad names and error types, this
was fixed as well.
The signatures for Radicle identities were using the documents
content.
It is standard to use the hash of the content instead, since signing
and verification is faster on a smaller set of data.
Use the resulting Git blob's object id, i.e. `git hash-object`, as the
signing and verifying payload.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
cloudhead
Fintan Halpenny
Alexis Sellier
Slack Coder