This change is to help separate the fact the namespacing by a `NodeId` is a
Radicle specific concern. The identifier can, in theory, be any kind of path
component, as long it is valid in Git.
The motivation for the change is to help separate the idea of a device's
`NodeId` vs an agent being an author – for future agent repository work.
The motivation of this change is to move away from tying signing to be
specifically for ed25519. The reason being that the protocol will want move
towards two different kinds of signing – the node signing artifacts, and an
author signing artifacts.
This change captures the former, node signing, by introducing a `Device` type
that is the `NodeId` – a PublicKey underneath the hood – and a signing
mechanism. This allows the replacement of the `Signer` trait being used – which
always assumed a `PublicKey`. Instead, the `Device` is constructed with
`NodeId`.
In `radicle-cob`, a signer is expected to implement
`signature::Signer<ExtendedSignature>`, and everywhere in `radicle`,
`radicle-node`, `radicle-cli`, and `radicle-remote-helper` is expected the
signer is expected to implement `signature::Signer<Signature>`.
A `Device` implements both of these but only requires
`signature::Signer<Signature>` to do so – since an `ExtendedSignature` is
essentially `(PublicKey, Signature)`, and the `NodeId` of the `Device` can be
used.
Update the rust toolchain version to 1.85.
Note that this is not the latest, however, it is the latest available in nixpgs,
so this version is good enough for now.
The `radicle-cob` crate provides a feature flag, `stable-commit-ids` to allow
for `Oid`s to remain stable for testing by always using the same author and
timestamp when creating commits.
This can be an issue when it interplays with COB related tests that are relying
on the ordering for the DAG walk. If the `Oid` values change and get re-ordered
then the test will fail. The DAG for COBs are now ordered via the `Oid` and
`Timestamp`.
In this change, a global timestamp is introduced that is initialised
per thread. The value is read, via `read_timestamp`, and can be
advanced using `with_advanced_timestamp` – while performing an action
that creates a new commit.
The aim of this change is to make the `Doc` type more safe to use by approaching
the design via [Parse don't validate][[0]] approach.
The problem with the previous approach was that all field were `pub` and thus a
`Doc<Verified>` could easily be mutated and serialized. Granted, the code that
used the serialization would tend to verify the `Doc` first, however, this
approach *ensures* that only a verified `Doc` can be serialized. It also meant
that trying to add new data that would follow the parse approach would require
more generic parameters on top of the existing `PhantomData` parameter, i.e. we
need to do something like: `Doc<RawField, V> -> Doc<ValidField, V>`.
The new approach splits the type into two separate types: `RawDoc` and `Doc`.
The former is allowed to be mutated at will, and uses types that are less
strict. The latter is the valid type that can only be constructed by validating
a `RawDoc` (or the `initial` constructor). The `Doc` type's fields can then only
be accessed by read-only methods.
Solves the problems above by only allowing mutations to `RawDoc`, as well as,
new fields being added to `RawDoc` which are then validated via
`RawDoc::verified`.
[0]: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
We ensure that a `rad/root` ref is included in the signed refs file
under `rad/sigrefs` for all remotes. This prevents a certain kind of
"grafting" attack where signed refs can be copied between repositories,
by having the peer sign over the identity root together with the data refs.
When verifying signed refs, we ensure that the ref is present and points
to an identity branch root that matches the repository identity containing
the signed refs.
Alternatives: lots of alternatives were considered, but this one doesn't
introduce any changes to the signing. The `rad/id/root` name was
considered but is invalid due to `rad/id`.
To facilitate edit actions, take URIs instead of the actual blobs. This
means API callers don't have to load all the blobs just for them to be
re-hashed when an edit action is submitted.
There are some peculiarities when dealing with the `Identity` COB since
the embed is the identity document itself. We handle that special case.
When evaluating a COB, expose to the `apply` function the operations
concurrent to the current one being applied. These are essentially
"related" nodes in the graph, that are neither ancestors or descendants.
This will let us for example have different failure modes depending on
whether there were conflicting operations, as well as expose conflicts
to the user.
Previously we panicked if a revision wasn't found after an update. But
it can happen that the update goes through yet a concurrent update
prevents the revision from being valid.
To partially deal with this, we don't try to lookup the revision after
an update, we simply return the ID. This means a subsequent lookup can
fail, and the error can be handled there.
The fetch V2 improvements introduced a `UserInfo` type that is used to
set the Git config's name and email when cloning a repository. This
`UserInfo` can also be used when creating a new `Repository`.
The `Storage` type now takes a `UserInfo` so that it can be passed to
`Repository::create`, for creating a new repository.
For testing, this requires adding a new `fixtures::user` so that
testing `Storage` instances can be easily constructed.
It also means that the `radicle-cli` tests all changed with new SHAs
since the name and email are used as part of the hashing input.
N.b. the order of the timelines while testing the identity cob
changed. It's not clear why that is.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `stable-commit-ids` feature flag was being included in non-dev,
non-test builds because of how cargo works (or doesn't work).
For now, we ignore the tests that this affects, and they can be run with
the `RAD_COMMIT_TIME=1514817556` environment.
`cargo clippy --all --all-targets` is warning that there are multiple
redundant clones.
Apply the suggestions by removing the calls to `clone`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
Instead of returning histories from `radicle-cob`, we return an
evaluated object which we build during initial graph traversal.
We do this so that branches can be correctly pruned when operations are
invalid at the application level. This way, new operations are not
building on top of invalid ones contained in the history.
To achieve this, we introduce a new `Evaluate` trait that is implemented
by all COBs, and we make change graph evaluation fallible.
Doing this means that we get `apply` errors returned for free when an
invalid transaction is applied, and no longer need to check for action
validity in multiple places.
We also implement a new `Dag::prune` method to avoid having to copy
graph nodes during traversal.
This change allows for files (blobs) to be embedded into COB entries.
This allows for things like image attachments in issue comments for example.
For now, we only enable this in the `issue` COB.
The way it works is that relevant COB actions carry metadata about which
files are attached to them; and we store those files as blobs inside the
COB entry's *tree* object, under an `embeds/` folder.
The `Embed<T>` type is used for the above, and either carries actual
content, or carries a content-id. Retrieving the actual content is as
simple as asking the repository for a blob with that content-id.
As a possible future extension, MIME types could be stored alongside the
files in a "metadata" file. This could help clients display the content
appropriately.
> I'm going to be making a set of breaking changes to COBs in order to
stabilize the data formats. This is hopefully a one-time change that
bundles various breaking changes.
All COBs have been reworked: issue, patch, id.
The changes included are:
* Revise the assign and tag actions to take a single list of
assignees/tags to set, instead of an "add" and a "remove" list. This
makes API usage simpler when editing issues, and simplifies the apply
function
* Rename "tags" to "labels", and the tag action to label. This is
because tag is confusing in the context of git, as it could mean a git
tag. Using label removes that confusion.
* Use DIDs instead of PublicKeys for assignees -- this is more
future-proof
* Modify the manifest file format in the COB tree. Mainly, remove the
`history_type` key which is redundant, and use camelCase for keys
* Flatten the `Thread` actions into the parent action type
* Ensure that operations on redacted objects do not fail, since
redactions could have happened concurrently
* Use a consistent naming scheme for actions, using `.` as separator
* Consolidate comment types and remove `CodeComment`, by adding an
optional `location` field to `Comment`
* Add many placeholder actions that are not yet implemented
To preserve backwards compatibility, a `legacy` module is created with
the old `apply` function. When loading the manifest, we check whether it
is a legacy COB or a "stable" COB, and in the legacy case, use the
legacy code to materialize the state and then convert the object into
the stable type. Eventually, we'll delete the legacy code.
It turns out that the CRDT formed by the union of Git DAGs
is enough to guarantee everything we need for COBs.
This changes the following things:
* COB operations no longer need to be commutative
* COB histories are traversed in the same deterministic order on all
replicas
* It's now possible to implement RSMs on top of COBs, eg. scripting
* Lamport clocks have been removed
* `radicle-crdt` is no longer a dependency of `radicle`
* COBs are no longer instances of `Semilattice`
* The `Ops` type was removed in favor of having `Op` contain multiple
actions
The additional flexibility in choosing a backend for the COB store will
allow things like "draft" COBs that are stored in a different location,
without changing much of the logic.
Though these are perhaps not the final error types, we name them the same
across COBs and have them work the same way.
All COB modules now have a `Error` enum.
In a future change, we may want the `FromHistory` trait to support
multiple error types, eg. one for `apply` and one for `validate`.
This adds a simple `validate` function to COBs that is run on load. It
allows COBs to check whether they are valid, once all operations are
loaded, and return an error if not.
This change updates to the latest radicle-git-ext, which required a
few changes within the radicle family of crates.
One set of changes is that the radicle-git-ext crate subsumes
git-commit, git-trailers, and git-ref-format -- so all those imports
go through radicle-git-ext.
The commit code requires that parent `Oid`s point to commits, which
made some tests fail. One of the reasons for these failures is that
the patch tests used fabricated `Oid`s and so the verification would
fail. This is fixed by allowing the test context code to create
`PatchRequest`s that create valid commits.
The other reason for failure was that the identity `Action` passed an
`Oid` that pointed to a blob. This is fixed by removing that code and
making a note of it in documentation.
The final major change was moving the Refspec type into git-ref-format
and removing the AsRefspecs trait. The trait was not required and
could its usage could be replaced by simpler code.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This allows change commits to have more than one parent that isn't
part of the DAG.
We can use this to create dependencies towards git objects such as
commits, in the case of patch actions.
When a merge is applied to a patch, we check whether the threshold
of delegates has been reached, at which point we set the patch status
to `merged`.
This requires threading the identity document oid through the "apply"
process.
Using LWWReg::from will always set the clock value to the default,
which can end up being used wrong.
Instead, introduce a constructor method `initial` to indicate that it
should be used on initial construction, while `new` should be used for
newer values of the register.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
By introducing a small limitation: only allowing entries in the change
graph to be addressable, instead of individual operations; we
drastically simplify the CRDT implementation.
There are four advantages:
1. Op ids are just regular SHA-1s
2. There's no need for relative IDs, ops never refer to other ops within the
same commit
3. There's no need for a nonce, since commits can't collide, and neither can op IDs
4. `OpId` can just be an alias of `EntryId`
The disadvantage of course, is that we have to be mindful of how we
create op transactions, since each transaction creates an addressable
unit. For example, we must not include multiple patch revisions in the
same transaction.
It's fairly easy for a user to (by mistake or intentionally) create two
operations with the same OpId. This patch makes it much less likely,
and ensures that if the OpId is equal, it's because the operations
are identical.
Instead of showing just public keys we need to show the fully qualified
spec for a NID which is prefixed by `did🔑` While we want to maintain
`Author` and `ActorId` for internal use.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
In order to make identity changes where the threshold is higher than
1, it is necessary to gather signatures of delegates to surpass that
threshold.
To achieve this the Proposal cob is introduced for proposing changes
and gathering the signatures over those changes.
Once a quorum is reached the Proposal can be published, committing the
new identity document to storage.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Lorenz Leutgeb
Fintan Halpenny
Lorenz Leutgeb
cloudhead
Alexis Sellier
Sebastian Martinez