Shell out to `git`, avoiding using `git2`. This is to avoid an
issue where somewhere within the fetch there is an attempt to lookup a
`rad/sigrefs` object, which says that the object is missing. It is
suspected that this is due to the object being localised in the same
packfile as other objects we are fetching. Since the `rad/sigrefs`
object is never needed nor used in the working copy, this will always
result in the object missing. This seems to only be an issue with
`libgit2`/`git2` and not `git` itself.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add an enum to represent the special references, `rad/id` and
`rad/sigrefs`. These can be used for cases where it's more manageable
to work with the enum type over checking for reference string equality.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a type to capture user information that can be used for Git
configuration and signatures.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Similar to the `id` function, add a `sigrefs` function to construct
the reference name for a given remote's `rad/sigrefs`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
This storage backend for COBs stores changes in a `draft/cobs/*` namespace,
which allows for some of the features needed for code review. For
example, users can draft comments and later decide to publish
them.
Remove this command in favor of the `git push` flow.
We also remove the `rad/patches` remote configuration which wasn't
working properly with `git fetch --all`.
When applying a `Merge` action, we verify that the commit is in the
history of the default branch of that user.
Since this is not the case when issuing a `rad merge`, and since that
command is no longer needed with the push-based merge, we remove the
`rad merge` command completely.
The `refs/tmp` namespace is used by Git for the purpose of storing
temporary refs. We use it here instead of `refs/patch` to indicate
that the refs can be safely deleted.
This adds the ability to the git-remote-helper to open and update
patches via a `git push` to the magic ref `refs/patches` and
`refs/heads/patches/<patch-id>`.
The main change is that we had to move away from offering the `connect`
capability in the remote helper, to offering `push` and `fetch`
individually.
To get a feel for how this works, see the `rad-patch-via-push.md`
example.
This change updates to the latest radicle-git-ext, which required a
few changes within the radicle family of crates.
One set of changes is that the radicle-git-ext crate subsumes
git-commit, git-trailers, and git-ref-format -- so all those imports
go through radicle-git-ext.
The commit code requires that parent `Oid`s point to commits, which
made some tests fail. One of the reasons for these failures is that
the patch tests used fabricated `Oid`s and so the verification would
fail. This is fixed by allowing the test context code to create
`PatchRequest`s that create valid commits.
The other reason for failure was that the identity `Action` passed an
`Oid` that pointed to a blob. This is fixed by removing that code and
making a note of it in documentation.
The final major change was moving the Refspec type into git-ref-format
and removing the AsRefspecs trait. The trait was not required and
could its usage could be replaced by simpler code.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The previous fetching logic had some pitfalls regarding its logic:
* Fetched 'rad' references could end up not being verifiable and leaving
the storage in a corrupt state
* Forced pushes would be prohibited, since they would be rejected by
the client side due to the refspecs being used.
The new approach is to stage a fetch in 2 steps. The first step is to
fetch the relevant `rad/id` and `rad/sigrefs` for verification. For
all remotes that are verified, fetch all references listed in their
respective `rad/sigrefs`, using the `+` marker in the refspec,
allowing to update for any force pushes -- note that this becomes a
safe operation since the `sigrefs` are signed by the remote that
created them.
To prevent unverifiable 'rad' references from polluting storage, all
fetch negotiations are done in a staging repository set up in a
temporary directory. All successful fetches are then transferred,
using the 'file://' protocol, from the temporary directory into the
radicle storage.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Rust 1.67 was announced[0]. Update the necessary files for running
1.67 and fix the clippy suggestions for string interpolation.
[0]: https://blog.rust-lang.org/2023/01/26/Rust-1.67.0.html
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This will allow us to run commands other than `rad`
and to have tests that switch between different users.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Added the `parse_refname` function for parsing a reference into its
`TypeName` and `ObjectId`, if possible.
This replaces the `cob_suffix` function in `radicle`.
This change also fixes the `FromStr` instance of `ObjectId` to use
`from_str` instead of `TryFrom` for bytes.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Use the existing conventions for passing a signer to functions.
* Allow replacing git refs when updating cobs.
* Fix the parsing of a cob id.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
To wire up radicle-cob, it is necessary to implement the traits for
the following types:
* `Repository`: `change::Storage`, `object::Storage`, and `Store`
* `project::Identity`: `identity::Identity`
For `Repository`, the `change::Storage` implementation is trivial, as
it uses the underlying `git2::Repository` implementation. Then
`object::Storage` simply defines the required references names and
does lookups or updates.
The aforementioned reference naming scheme is:
refs/namespaces/<ns>/refs/cobs/<typename>/<object_id>
For `project::Identity`, the `identity::Identity` implementation is
straight-forward.
To finalise the work, a simplified API is exposed as `radicle::cob`
where the `create`, `update`, `get`, and `list` functions are
redefined in terms of `radicle` specific types.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We move the identity branch out of `heads/` and to the old location:
refs/rad/id
We move the signed refs branch to:
refs/rad/sigrefs
Keeping both "special" branches under the same hierarchy makes things a
bit easier to work with, and tells the user that they should be updated
with special tooling.
We also revert to `rad/` because we no longer have the issue of having
`rad/rad/id` when doing a checkout of the remote branch in a working
copy.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
The crypto types are useful to use in other components that are
separate from the radicle crate, e.g. the incoming radicle-cobs
crate.
Separate out the types into a radicle-crypto crate. This crate defines
the usual PublicKey, SecretKey, and Signature types. It also provides
the ssh functionality, under the `ssh` feature. It is also necessary
to provide the `From` impl for `Component` here, which again is
guarded by the `git-ref-format` feature.
The `Arbitrary` implementation are moved. The arbitrary decision of
redefining `ByteArray` in both radicle-crypto and radicle was made.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The changes in this commit are all intertwined and it was hard to
split them into smaller commits:
* Use custom transports everywhere
Instead of `file://` and instead of using a child process for fetching
and pushing; we go through our custom transports, which use the `rad://`
and `heartwood://` schemes.
* Introduce a 'mock' remote transport
To test the remote transport without needing to spawn TCP streams,
we add a mock transport that works between storages on the file
system.
* Get rid of node's `git-url`
Instead of git urls, nodes simply use their node-ids to replicate.
Since the `HEAD` reference is trusted (we set it locally), use that
when possible, instead of re-computing it.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
We set the `HEAD` and eg. `refs/heads/master` refs when updating
repositories, which allows users to easily clone with git.
For now, `HEAD` is chosen based on a unanimity quorum of delegates.
This can be made more sophisticated in the future.
We've also had to ignore some refs during verification, since these
have cropped in for eg. due to the staging copy clone and the
changes included here.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
The `git-ref-format`[0] and `radicle-git-ext`[1] crates are now
published to crates.io.
Switch the dependency to rely on them. This also required to update
the `git2` dependency to `0.15.0` as well as using the
`vendored-libgit2` feature -- avoiding the openssl dependency too.
[0]: https://crates.io/crates/git-ref-format
[1]: https://crates.io/crates/radicle-git-ext
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
The previous iteration of the storage layout for projects organised
peer's subtrees by the `refs/remotes` category. This layout leads to
problems when linking a working copy to the storage project. The fetch
refspec would look something like:
fetch = +refs/remotes/abc/heads/*:refs/remotes/rad/*
This works perfectly fine when running:
git fetch rad
but will fail when one tries to specify:
git fetch rad main
Instead, the command would need to be:
git fetch remotes/abc/heads/main
This patch proposes to move the use of `refs/namespaces` for each
peer. The layout looks similar in that each peer has a subtree, but
instead it is under `refs/namespaces/<peer>/refs/...`.
The previous refspec would look more familiar in its new form:
fetch = +refs/heads/*:refs/remotes/rad/*
With this in place running:
git --namespace=abc rad main
Will work as expected and place the reference under
`refs/remotes/rad/main`. Similary, `git --namespace=abc push` will
work as expected.
Another benefit of this approach is that the use of namespacing means
that a tag is pushed it will be placed under the respective namespace
rather than in the global `refs/tags` category.
The actual implementation needed to work around the fact that
`git2`/`libgit2` does not have any options for fetching or pushing
using namespaces. The working copy code uses processes to shell out to
`git` for namespacing needs.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
To be able to support git commits with multiple signature
headers, we have to implement our own parsing and formatting.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
cloudhead
Fintan Halpenny
Alexis Sellier
Alexis Sellier
Slack Coder