JSON Schema extraction via `schemars` is provided for configurations.
There are other interfaces that use JSON for (de-)serialization, such as
the communication with `radicle-node` via the control socket.
To ease implementation of tools that want to communicate via the control
socket, we add the respective `schemars` annotations.
The motivation of this change is to move away from tying signing to be
specifically for ed25519. The reason being that the protocol will want move
towards two different kinds of signing – the node signing artifacts, and an
author signing artifacts.
This change captures the former, node signing, by introducing a `Device` type
that is the `NodeId` – a PublicKey underneath the hood – and a signing
mechanism. This allows the replacement of the `Signer` trait being used – which
always assumed a `PublicKey`. Instead, the `Device` is constructed with
`NodeId`.
In `radicle-cob`, a signer is expected to implement
`signature::Signer<ExtendedSignature>`, and everywhere in `radicle`,
`radicle-node`, `radicle-cli`, and `radicle-remote-helper` is expected the
signer is expected to implement `signature::Signer<Signature>`.
A `Device` implements both of these but only requires
`signature::Signer<Signature>` to do so – since an `ExtendedSignature` is
essentially `(PublicKey, Signature)`, and the `NodeId` of the `Device` can be
used.
The aim of this change is to make the `Doc` type more safe to use by approaching
the design via [Parse don't validate][[0]] approach.
The problem with the previous approach was that all field were `pub` and thus a
`Doc<Verified>` could easily be mutated and serialized. Granted, the code that
used the serialization would tend to verify the `Doc` first, however, this
approach *ensures* that only a verified `Doc` can be serialized. It also meant
that trying to add new data that would follow the parse approach would require
more generic parameters on top of the existing `PhantomData` parameter, i.e. we
need to do something like: `Doc<RawField, V> -> Doc<ValidField, V>`.
The new approach splits the type into two separate types: `RawDoc` and `Doc`.
The former is allowed to be mutated at will, and uses types that are less
strict. The latter is the valid type that can only be constructed by validating
a `RawDoc` (or the `initial` constructor). The `Doc` type's fields can then only
be accessed by read-only methods.
Solves the problems above by only allowing mutations to `RawDoc`, as well as,
new fields being added to `RawDoc` which are then validated via
`RawDoc::verified`.
[0]: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
We ensure that a `rad/root` ref is included in the signed refs file
under `rad/sigrefs` for all remotes. This prevents a certain kind of
"grafting" attack where signed refs can be copied between repositories,
by having the peer sign over the identity root together with the data refs.
When verifying signed refs, we ensure that the ref is present and points
to an identity branch root that matches the repository identity containing
the signed refs.
Alternatives: lots of alternatives were considered, but this one doesn't
introduce any changes to the signing. The `rad/id/root` name was
considered but is invalid due to `rad/id`.
* Try to use constants instead of strings
* Move crypto seed override out of radicle-crypto
* Use GIT_COMMITTER_DATE for commit overrides
* Use RAD_LOCAL_TIME as general time override
* Allow variables to be used in release mode
Implement this function on `RemoteRepository`, and move `repositories`
to the trait level.
This function will be used to quickly get the `RefsAt` of all repository
remotes.
When deciding whether or not to fetch a repo based on a refs
announcement, we need to know whether our signed refs are stale or not,
compared to the ones in the announcement. This can be quite expensive as
it requires multiple reads from Git's ODB, which often results in
packfile loading.
To remedy this, we introduce a refs cache in our node database that is
consulted when an announcement is received.
We also remove the logic that generates `info` messages, since they are
rarely needed and introduced more complexity.
Additionally, we revert commit 3ad2b4431f,
which introduced logic to minimze wants and haves with additional logic,
since it potentially also increases the load on the ODB.
This change limits the amount of `wants` and `haves` data that is sent
to the serving side of a fetch.
When a `RefsAt` announcement is sent, the fetching peer can calculate
the difference between the `rad/sigrefs` they're aware of -- if it
exists -- and the newly advertised `rad/sigrefs`. This means they can
efficiently ask for the `wants` and `haves` of references that have
changed -- saving some data sent over to the serving side.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It can be useful to return the `RefsAt` that was announced for this
node, for example, to compare it to node events.
Change the `Handle` method to return `RefsAt` so that when it is
called it returns the local node's `RefsAt`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of creating the tree to check if `rad/sigrefs` has changed, it
is possible to check that the signature of the signed refs being
updated is the same as the old one. If so, then there are no
changes. Otherwise, there must be changes and the update can continue
to create the necessary Git objects.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The RefsAnnouncement previous announced its complete set of
`rad/sigrefs`. This was useful in that verification can occur on the
receiveing side immediately by verifying the refs and proceeding with
the fetch -- note that it did nothing with the ref information it
received.
This can be minimised to only advertising the namespace, i.e. the
NID, and the new SHA of the `rad/sigrefs`. This is represented by a
new message type, `RefsAt`, which contains the NID and the Oid.
For checking if an announcement is stale or fresh, the receiving
side can check the existence of the SHA in its Git repository.
The tests are appropriately modified to use NIDs and Oids.
To prepare for a follow-up change, the announcement data is also
threaded through for the fetch protocol to focus fetching only the
advertised set of Oids.
N.B.: because the refs aren't communicated, the verification happens
when fetching since the data needs to be fetched before
verifying. This is a trade-off with minimising the message payload for
verifying earlier.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.
Change the error type from `git::ext::Error` to `git::raw::Error`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.
The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.
[0]: https://github.com/Byron/gitoxide
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `identity_doc_at` and `identity_doc` methods on `ReadRepository`
returned the unverified identity document. Every usage of these
methods would always verify the document after.
Simplify the code by calling verified in the methods and directly return the
verified identity document.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduce a SignedRefsAt type that combines a SignedRefs along with the Oid
it was loaded at.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
When updating sigrefs with `storage::refs::save` we weren't checking
the `RAD_COMMIT_TIME` variable, so the sigrefs commit was using current
time, and we weren't able to reproduce a deterministic sigrefs commit.
Make `refs::save` reproducible in debug mode using `debug_assertions`
and checking the `RAD_COMMIT_TIME` environment variable.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
This change updates to the latest radicle-git-ext, which required a
few changes within the radicle family of crates.
One set of changes is that the radicle-git-ext crate subsumes
git-commit, git-trailers, and git-ref-format -- so all those imports
go through radicle-git-ext.
The commit code requires that parent `Oid`s point to commits, which
made some tests fail. One of the reasons for these failures is that
the patch tests used fabricated `Oid`s and so the verification would
fail. This is fixed by allowing the test context code to create
`PatchRequest`s that create valid commits.
The other reason for failure was that the identity `Action` passed an
`Oid` that pointed to a blob. This is fixed by removing that code and
making a note of it in documentation.
The final major change was moving the Refspec type into git-ref-format
and removing the AsRefspecs trait. The trait was not required and
could its usage could be replaced by simpler code.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We were allowing remotes to be constructed via a function that didn't
check its inputs.
This is now fixed by moving the public key into the `SignedRefs` struct.
Rust 1.67 was announced[0]. Update the necessary files for running
1.67 and fix the clippy suggestions for string interpolation.
[0]: https://blog.rust-lang.org/2023/01/26/Rust-1.67.0.html
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The latter is a fork of the former, which adds const generics
and a bunch of other quality of life improvements that aren't
in the original crate due to it being unmaintained.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
To support ssh-agent, add a signing method that can fail.
Also cleanup usage so that we always accept signer references.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
We move the identity branch out of `heads/` and to the old location:
refs/rad/id
We move the signed refs branch to:
refs/rad/sigrefs
Keeping both "special" branches under the same hierarchy makes things a
bit easier to work with, and tells the user that they should be updated
with special tooling.
We also revert to `rad/` because we no longer have the issue of having
`rad/rad/id` when doing a checkout of the remote branch in a working
copy.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
The crypto types are useful to use in other components that are
separate from the radicle crate, e.g. the incoming radicle-cobs
crate.
Separate out the types into a radicle-crypto crate. This crate defines
the usual PublicKey, SecretKey, and Signature types. It also provides
the ssh functionality, under the `ssh` feature. It is also necessary
to provide the `From` impl for `Component` here, which again is
guarded by the `git-ref-format` feature.
The `Arbitrary` implementation are moved. The arbitrary decision of
redefining `ByteArray` in both radicle-crypto and radicle was made.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `git-ref-format`[0] and `radicle-git-ext`[1] crates are now
published to crates.io.
Switch the dependency to rely on them. This also required to update
the `git2` dependency to `0.15.0` as well as using the
`vendored-libgit2` feature -- avoiding the openssl dependency too.
[0]: https://crates.io/crates/git-ref-format
[1]: https://crates.io/crates/radicle-git-ext
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
To build an address book, peers communicate via gossip, sending
`NodeAnnouncement` messages.
These need to be handled by storing the addresses in an address book
and the addresses need to be persisted to disk.
Node announcement messages should also be relayed to peers like other
gossip messages.
The previous iteration of the storage layout for projects organised
peer's subtrees by the `refs/remotes` category. This layout leads to
problems when linking a working copy to the storage project. The fetch
refspec would look something like:
fetch = +refs/remotes/abc/heads/*:refs/remotes/rad/*
This works perfectly fine when running:
git fetch rad
but will fail when one tries to specify:
git fetch rad main
Instead, the command would need to be:
git fetch remotes/abc/heads/main
This patch proposes to move the use of `refs/namespaces` for each
peer. The layout looks similar in that each peer has a subtree, but
instead it is under `refs/namespaces/<peer>/refs/...`.
The previous refspec would look more familiar in its new form:
fetch = +refs/heads/*:refs/remotes/rad/*
With this in place running:
git --namespace=abc rad main
Will work as expected and place the reference under
`refs/remotes/rad/main`. Similary, `git --namespace=abc push` will
work as expected.
Another benefit of this approach is that the use of namespacing means
that a tag is pushed it will be placed under the respective namespace
rather than in the global `refs/tags` category.
The actual implementation needed to work around the fact that
`git2`/`libgit2` does not have any options for fetching or pushing
using namespaces. The working copy code uses processes to shell out to
`git` for namespacing needs.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Lorenz Leutgeb
Fintan Halpenny
Lars Wirzenius
cloudhead
Sebastian Martinez
Alexis Sellier
Alexis Sellier
xphoniex