Similar to the `Fetcher` sans-IO approach, an `Announcer` sans-IO approach is
introduced. The previous `Handler::announce` was already quite close to using a
sans-IO approach, but using internally held state. This change makes the
business logic more reusable – and testable.
Similarly to the `Fetcher`, the `Announcer` is configured and then driven by
telling it when nodes have been synchronized with. It will continuously check if
it has reached its target, yielding to the caller to decide whether to keep
going. In contrast, it also provides a way to mark the process as timed out,
since announcing relies on waiting for events to come back in a timely manner.
The motivation of this change is to move away from tying signing to be
specifically for ed25519. The reason being that the protocol will want move
towards two different kinds of signing – the node signing artifacts, and an
author signing artifacts.
This change captures the former, node signing, by introducing a `Device` type
that is the `NodeId` – a PublicKey underneath the hood – and a signing
mechanism. This allows the replacement of the `Signer` trait being used – which
always assumed a `PublicKey`. Instead, the `Device` is constructed with
`NodeId`.
In `radicle-cob`, a signer is expected to implement
`signature::Signer<ExtendedSignature>`, and everywhere in `radicle`,
`radicle-node`, `radicle-cli`, and `radicle-remote-helper` is expected the
signer is expected to implement `signature::Signer<Signature>`.
A `Device` implements both of these but only requires
`signature::Signer<Signature>` to do so – since an `ExtendedSignature` is
essentially `(PublicKey, Signature)`, and the `NodeId` of the `Device` can be
used.
In issue #[2836] of `libgit2`, the speed of `libgit2`'s `file://` protocol for
git operations is found to be very slow. This is further corrobarated by `rad
init` taking 13 hours to initialise the hardenedbsd [ports] repository.
There are two areas where the `heartwood` project uses `libgit2` to push using
the file protocol. The first is when via the `trasnport::local` smart transport
registration and the second is the final push to storage in the `git-remote-rad`
binary.
When both these `push` operations are changed to use the `git` binary instead,
the [ports] repository can be initialised in less than 10 minutes (nearly 100x
speed up).
This change is clearly required if `heartwood` wishes to support larger
repositories.
[2836]: https://github.com/libgit2/libgit2/issues/2836
[ports]: https://git.hardenedbsd.org/hardenedbsd/ports
Adds the new `migrate` sub-command to migrate the COB database.
Checks for version when opening the COBs database and suggests the
command on out of date version.
Also adds the 2nd migration to the COBs cache, which updates the patch
JSON schema.
The `no quorum found` error can be opaque and lead to a lot of confusion. It is
better to give more information for this error so that it leads to easier
debugging, since it can be quite a common warning/error.
Instead of a single error, there are two cases provided: `NoCandidates` and
`Diverging`, to capture the two different variants of errors that can occur.
`Display` implementations are provided for both to provide more information.
The aim of this change is to make the `Doc` type more safe to use by approaching
the design via [Parse don't validate][[0]] approach.
The problem with the previous approach was that all field were `pub` and thus a
`Doc<Verified>` could easily be mutated and serialized. Granted, the code that
used the serialization would tend to verify the `Doc` first, however, this
approach *ensures* that only a verified `Doc` can be serialized. It also meant
that trying to add new data that would follow the parse approach would require
more generic parameters on top of the existing `PhantomData` parameter, i.e. we
need to do something like: `Doc<RawField, V> -> Doc<ValidField, V>`.
The new approach splits the type into two separate types: `RawDoc` and `Doc`.
The former is allowed to be mutated at will, and uses types that are less
strict. The latter is the valid type that can only be constructed by validating
a `RawDoc` (or the `initial` constructor). The `Doc` type's fields can then only
be accessed by read-only methods.
Solves the problems above by only allowing mutations to `RawDoc`, as well as,
new fields being added to `RawDoc` which are then validated via
`RawDoc::verified`.
[0]: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
When creating a new upstream for the patch workflow, the remote name is always
assumed to be `rad`.
It's possible for users to use another remote name but still use a `rad://` URL
for pushing. To allow for this, the function that creates the upstream entry now
takes a remote name. In the remote helper, `rad patch checkout`, and `rad patch
set` the remote can be optionally specified and fall back to the `rad` remote
name as a default.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When pushing to the default branch and updating the canonical head,
check to see if any merged patches are reverted due to this change.
If so, make sure the COB cache reflects this.
The rules for checking the quorum of the remote helper, when pushing to the
canonical branch, can prevent the operator from even attempting to try converge
to a state of agreeance.
Instead, the checking of the quorum rules are relaxed. It is still the case that
if the peer is pushing a diverging commit while in a working state they are
warned and the push does not go through. However, there are two cases where it
is fine to force push a change:
1. When pushing a new tip, as a delegate, there is still a canonical tip and the
new tip still converges with the history, i.e. it is equal to, behind, or
ahead of another delegate tip.
2. If there is no quorum in the current state, for example there were multiple
concurrent pushes, then the peer is free to continue to push to their default
branch in attempt to converge with the other peers.
The example tests ensure that these cases are covered, while also preserving the
old behaviour.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Remove the old `quorum` function in favour of using the newly introduced
`Canonical` code. Port over the tests to use the `Canonical` type. The change is
minimised by introducing a helper function called `quorum` in the test module
that acts like the old function.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
There are two areas where we can be more tolerant of delegate namespaces
being missing or not validating:
1. Calculating the canonical HEAD
2. Fetching from a remote
In 1. the protocol is tolerant in that if the local node does not have
the default branch for a delegate, it will still attempt to use any of
the delegates it does have to reach the threshold.
This is made safe by ensuring that if the threshold is being updated
then the node performing the update must have a threshold of delegates
locally in their storage. It also made safe by 2.
In 2. the protocol is tolerant by allowing delegates to be missing
from the serving side, as long as they can still meet a threshold of
delegates. This is further tolerant, when validating the received
data, a threshold of delegates are valid to consider the fetch
successful -- otherwise it will fail.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Implement caching for the `Issue` and `Patch` COB. This is achieved by
having the `Transaction::initial` and `Transaction::commit` methods
take a cache which can update the newly created/updated object. The
`Store::remove` method also takes the cache for removing the object
from the cache, as well as the repository.
This meant that the `*Mut` types for the respective COBs are required
to carry a cache alongside the repository store. Identities will not
be cached, and so this is always set to `NoCache` -- which performs no
caching, and always succeeds.
To perform cache reading for issues and patches, the `cache::Issues`
and `cache::Patches` traits are introduced. They capture the minimal
amount of methods that are needed for the `rad issue` and `rad patch`
CLI commands.
The implementor for each of these traits is there respective `Cache`
types, which combines their backing repository store and the SQLite
database. They both provide convenience methods:
- `create` (and `draft` for patches)
- `get_mut`
- `remove`
which can be used instead of their repository store counterparts.
All uses of the repository stores (where needed) are replaced with
`Cache` types.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Shows you updates on local repositories.
Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.
The `rad inbox` command can then list items in the database and delete
them.
To allow `rad sync` to break early from the call to `announce`,
control flow is introduced for the callback.
In `rad sync`, the number of synced nodes is checked against the
`replicas` -- if specified -- and sends a `Break`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The hint for updating a patch incorrectly printed:
git push rad -f HEAD:rad/patches/<patch id>
The remote, `rad`, is already specified so the the right hand side of
the refspec should only be `patches/<patch id>`.
Fix this by stripping the remote from the name of the branch in the
remote helper.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It's not currently possible to force-push rad/master if it's not a
rollback, even if you are the only delegate. Since there is no risk
in doing that, we allow it in this patch.
This makes the divergence check on push slightly more lenient, by
allowing rolling back to a previous commit in the canonical branch
history.
Without this, it can become impossible to roll-back commits in certain
situations.
When specifying a `patch.base` to point to for stacking patches, it's
useful to accept any kind of revspec, e.g. `HEAD^`.
Change the argument parsing to use the `Rev` type instead, and change
the `rad-patch-ahead-behind` to use `HEAD^` to test the example usage.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
The `graph_descendant_of` method is strict in that it will return
false if the two `Oid`s are equal -- they are strictly not
descendants.
If the `head` is the same as the `canonical_oid`, no error should be
returned and the `push` can happen without any issues.
Add a check for `head != canonical_oid` before the
`graph_descendant_check`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
As a delegate, you shouldn't be able to delete your default branch.
This would most likely happen by mistake, and it prevents the canonical
head from being calculated correctly.
On push, we check whether the resulting state would cause a
divergence/fork in the canonical head, and if so, prevent the push from
happening.
This is to avoid situations where delegates have to then rollback their
heads.
Note that this doesn't prevent forks from happening altogether, as they
could happen asychronously, but it mitigates the problem.
The `Patch` API and semantics are changed a little bit:
* It's now valid again to propose a revision without being the patch
author.
* To get revisions by the patch author, `Patch::updates` is used.
* `Patch::latest` now only looks at revisions by the patch author.
* Added `Patch::revisions_by`, `Patch::authors` and `Patch::latest_by`.
* The merge behavior on git-push is improved to favor revisions that
merge more recent commits. We also allow for merging the non-latest
revision.
Sets the current branch's upstream to point to the given patch.
This allows for pushes to update the patch.
When opening a patch, if the current branch already has an upstream,
it isn't updated automatically. This is why this command exists.
It happened that a small bug in the remote helper caused a patch not to
merge correctly on push. Hence, I implemented a `rad-merge` tool for
these cases where we need to fix broken state.
> I'm going to be making a set of breaking changes to COBs in order to
stabilize the data formats. This is hopefully a one-time change that
bundles various breaking changes.
All COBs have been reworked: issue, patch, id.
The changes included are:
* Revise the assign and tag actions to take a single list of
assignees/tags to set, instead of an "add" and a "remove" list. This
makes API usage simpler when editing issues, and simplifies the apply
function
* Rename "tags" to "labels", and the tag action to label. This is
because tag is confusing in the context of git, as it could mean a git
tag. Using label removes that confusion.
* Use DIDs instead of PublicKeys for assignees -- this is more
future-proof
* Modify the manifest file format in the COB tree. Mainly, remove the
`history_type` key which is redundant, and use camelCase for keys
* Flatten the `Thread` actions into the parent action type
* Ensure that operations on redacted objects do not fail, since
redactions could have happened concurrently
* Use a consistent naming scheme for actions, using `.` as separator
* Consolidate comment types and remove `CodeComment`, by adding an
optional `location` field to `Comment`
* Add many placeholder actions that are not yet implemented
To preserve backwards compatibility, a `legacy` module is created with
the old `apply` function. When loading the manifest, we check whether it
is a legacy COB or a "stable" COB, and in the legacy case, use the
legacy code to materialize the state and then convert the object into
the stable type. Eventually, we'll delete the legacy code.
Lorenz Leutgeb
Fintan Halpenny
Lorenz Leutgeb
cloudhead
Alexis Sellier