Previously we only syntax highlighted files that were blobs inside the
git repo. This doesn't work when diffing files in the working copy.
We solve that by looking for files in the working copy if they aren't
found in the ODB.
* `args::rev` now parses things like `master`, `HEAD`, `3f25ea5`.
* `args::oid` is for parsing full or short oids, like `3f25ea5`.
Both return a `Rev`, since short oids can't be properly represented with
the `git2::Oid` type.
The `Element::render` function now takes a constraint object. This
allows the parent element to influence how the child is rendered. This
is useful eg. to have elements take up more space than they need to fill
some space.
We also make various changes needed for pretty diffs, such as merging
styles, a new `Filled` type, and some color theming.
Changes the behavior of `git fetch` by running a radicle network-level
fetch (ie. `rad sync --fetch`) before fetching refs from storage. This
avoids having to run `rad sync` before `git fetch`; and tries to make
sure that refs are always fresh.
As a delegate, you shouldn't be able to delete your default branch.
This would most likely happen by mistake, and it prevents the canonical
head from being calculated correctly.
On push, we check whether the resulting state would cause a
divergence/fork in the canonical head, and if so, prevent the push from
happening.
This is to avoid situations where delegates have to then rollback their
heads.
Note that this doesn't prevent forks from happening altogether, as they
could happen asychronously, but it mitigates the problem.
The `Patch` API and semantics are changed a little bit:
* It's now valid again to propose a revision without being the patch
author.
* To get revisions by the patch author, `Patch::updates` is used.
* `Patch::latest` now only looks at revisions by the patch author.
* Added `Patch::revisions_by`, `Patch::authors` and `Patch::latest_by`.
* The merge behavior on git-push is improved to favor revisions that
merge more recent commits. We also allow for merging the non-latest
revision.
Define types to support Diff of Diffs leaning on the Radicle Surf types.
Use Radicle Surf's Hunk type, but custom Diff, FileDiff, and
Modification types, and avoid Serde serialization as it does not yet
seem necessary.
Parse Git's unified diff format into Radicle Surf's types. Support to
and from the format helps testing and debugging.
Split the UnifiedDiff trait into Decode and Encode traits so we
can offer Decode in cases where no data could be expected..
For example Option<Hunk<_>> and Hunk<_> allowing partial and repeated
decoding on a byte stream.
Avoid decoding `Diff`, `DiffContent::Binary`, `FileDiff`, and `FileHeader`
types. `Diff` keeps fields and methods private which are necessary for
decoding.
Simplify the trait by defining it on the concrete type, which appears to
be the standard rust approach. Remove some oddities in the process (refs
of refs).
We remove the YAML front-matter, since that was annoying to use.
Issues are now edited like patches and commit messages.
Labeling and assigning are done via dedicated commands.
When `rad issue show` is run, the issue comments are shown by default.
This can be tweaked using the `--format` option. A value of "header"
only shows the header, while the default which is "full", shows the
header and comments.
Sets the current branch's upstream to point to the given patch.
This allows for pushes to update the patch.
When opening a patch, if the current branch already has an upstream,
it isn't updated automatically. This is why this command exists.
This allows a user to choose private or public visibility on `rad init`.
It also improves the output to tell the user what happened due to this
choice.
It didn't make sense to get eg. `Patch failed` when listing patches
failed. We change all fatal errors to simply "Error:", giving more
control over the individual commands.
After cloning, we show a cool little summary with some repo stats.
We had to make some adjustments to the way visual width is calculated
due to a bug surfaced around emoji width calculation. Unfortunately
there isn't a perfect solution that works across terminals yet, so the
"variation selector 16" code-point is filtered out for now.
When a fetch is user-requested, a timeout can be supplied that is passed
down to the worker.
When a fetch is service-requested, a default timeout is used.
Private repos are implemented by extending the identity document with a
`visibility` attribute, that can either be `"public"` (default) or
`"private"`.
In case of `private` visibility, only the delegates are allowed to view
the repo, as well as any DIDs added to the allow list.
To implement repo visibility, we simply block fetches from and
announcements to peers for whom the repo should remain invisible.
Private repos are also not announced in the `inventory` message, since
the full list of peers that *may* have the repo is retrievable from the
repo identity. This could cause errors if eg. a peer who is allowed to
view the repo doesn't actually have it. However this is an ok trade-off
for now to keep the complexity low. For repos to truly be private, it's
important that the RIDs don't leak either.
Finally, we modify `radicle-httpd` for now to only list public repos.
Eventually, we would want to change this depending on whether an allowed
peer is authenticated with the service or not.
---
It's also worth mentioning why this approach was taken, vs. end-to-end
encryption. The reasons are as follows:
1. Nodes that do not have access to a private repo will generally not want to
replicate encrypted data that they cannot examine or use.
2. The chosen solution is trivial, while encrypting git objects isn't.
3. Performance of the chosen solution is much better, there is no
overhead.
4. Privacy of the chosen solution is better: RIDs are never leaked, and
neither is the existence of a private repo, nor who has access to it.
There is one downside: Paying for storage of private repos is no better
in terms of privacy than what GitHub offers. Hosting providers will have
access to your private repos, if this solution is used.
Instead of having two types, we simply have one, called `Entry`, which
we rename from `Change`.
There was no real benefit to having two almost identical types, but lots
of added complexity. This patch simplifies the crate by removing one of
the types.
Add `deny.toml` with a few exceptions, and update certain dependencies
to get rid of duplicates.
Running `cargo deny check` will yield some warnings still, but no
errors.
This change allows for files (blobs) to be embedded into COB entries.
This allows for things like image attachments in issue comments for example.
For now, we only enable this in the `issue` COB.
The way it works is that relevant COB actions carry metadata about which
files are attached to them; and we store those files as blobs inside the
COB entry's *tree* object, under an `embeds/` folder.
The `Embed<T>` type is used for the above, and either carries actual
content, or carries a content-id. Retrieving the actual content is as
simple as asking the repository for a blob with that content-id.
As a possible future extension, MIME types could be stored alongside the
files in a "metadata" file. This could help clients display the content
appropriately.
This adds functionality so that on `rad sync` and `rad clone`, we
connect to known seeds if necessary, before attempting to fetch.
In passing, some related changes were made:
* The `rad sync` command's arguments were reworked:
`--replicas <count>` can be used to specify a replica count when
fetching
* The `rad patch` command no longer has a `--fetch` option
* It's now possible to tell the node not to automatically connect to
peers in the background
* `AddressBook` is being used underneath `Seeds` to allow for shuffled
iteration
Only return once the connection is established or failed.
It's still not perfect, as several failure scenarios are not handled,
but it's an improvement.
We move the session address to the top-level struct, since it's needed
in a bunch of places. This change is required for the rate-limiting
code that is coming next.
As a starting point, we populate the address book (if empty) with some
bootstrap nodes which are trusted.
To prevent these nodes being connected to during tests (including e2e
tests), we create a new constructor for `Config` that is used in tests.
Note that a `cfg(test)` check is not enough given that e2e tests don't
have that set.
> I'm going to be making a set of breaking changes to COBs in order to
stabilize the data formats. This is hopefully a one-time change that
bundles various breaking changes.
All COBs have been reworked: issue, patch, id.
The changes included are:
* Revise the assign and tag actions to take a single list of
assignees/tags to set, instead of an "add" and a "remove" list. This
makes API usage simpler when editing issues, and simplifies the apply
function
* Rename "tags" to "labels", and the tag action to label. This is
because tag is confusing in the context of git, as it could mean a git
tag. Using label removes that confusion.
* Use DIDs instead of PublicKeys for assignees -- this is more
future-proof
* Modify the manifest file format in the COB tree. Mainly, remove the
`history_type` key which is redundant, and use camelCase for keys
* Flatten the `Thread` actions into the parent action type
* Ensure that operations on redacted objects do not fail, since
redactions could have happened concurrently
* Use a consistent naming scheme for actions, using `.` as separator
* Consolidate comment types and remove `CodeComment`, by adding an
optional `location` field to `Comment`
* Add many placeholder actions that are not yet implemented
To preserve backwards compatibility, a `legacy` module is created with
the old `apply` function. When loading the manifest, we check whether it
is a legacy COB or a "stable" COB, and in the legacy case, use the
legacy code to materialize the state and then convert the object into
the stable type. Eventually, we'll delete the legacy code.
Some of the old patches had missing objects and that causes problems
during the ref advertisement, as the object is not found.
In this patch, we check that the object exists and also that the patch
is still open. Otherwise a remote tracking branch is created for every
single patch ref.
Move code for writing radicle_surf diff's in git's unified diff format
into its own module. Lean on the type system by defining a
`UnifiedDiff` trait to provide a method `encode` to encode to the
format and another default method to convert to a git 'unified' diff
string.
Additionally a `Header` struct is added to support processing Unified
Diff files. The Header's line numbers must be tracked and modified to
produce legal diff files.
This is in preparation for DDiff support. Upcoming improvements will
include parse unified diff's into these types to support test data.
If you accidently merge updated patch code before updating the patch in
storage, you end up in this weird state where you can't update the patch
any longer. This fixes it.
This commit addresses the inconsistent parsing of the command by fixing the
partial matching behavior for the `--seed` flag. Previously, specifying
the seed value resulted in an invalid option error, as shown in the example below:
```
➜ ~ rad sync --seed z6MksmpU5b1dS7oaqF2bHXhQi1DWy2hB7Mh9CuN7y1DN6QSz
✗ Error: rad sync: invalid option '--seed'
```
With this fix, there are no restrictions on when the seed can be specified.
In a future change, we plan to implement a custom error message when a
match case does not meet specific requirements, such as `rad --fetch --announce`,
which will return a more informative error message. However, this
enhancement is left for a derived solution.
```
➜ ~ rad sync --fetch --announce
✗ Error: rad sync: invalid option '--announce'
```
Fixes: d6cebf613f
Suggested-by: Slack Coder
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Due to differences between how BSD sed and GNU sed work, we disable for
now tests that use sed on macos
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
It turns out that the CRDT formed by the union of Git DAGs
is enough to guarantee everything we need for COBs.
This changes the following things:
* COB operations no longer need to be commutative
* COB histories are traversed in the same deterministic order on all
replicas
* It's now possible to implement RSMs on top of COBs, eg. scripting
* Lamport clocks have been removed
* `radicle-crdt` is no longer a dependency of `radicle`
* COBs are no longer instances of `Semilattice`
* The `Ops` type was removed in favor of having `Op` contain multiple
actions
Help users confirm all their radicle binary versions by providing the
'--version' flag.
Move the print_version function to the radicle crate for use by the node
and remote-rad binaries, and remove tests to confirm the output format
as it is deemed redundant.
Co-Developed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This storage backend for COBs stores changes in a `draft/cobs/*` namespace,
which allows for some of the features needed for code review. For
example, users can draft comments and later decide to publish
them.
The additional flexibility in choosing a backend for the COB store will
allow things like "draft" COBs that are stored in a different location,
without changing much of the logic.
The tracking database can be opened in read-only or read-write
mode. Knowing which is being used is significant in concurrent
scenarios, e.g. it's safer to have a read-only handle on multiple
threads.
Provide static type information by signifying what kind of database
handle is being worked with, read or read-write.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This is a fairly substantial change, which adds a new configuration file
to the user's `$RAD_HOME` that includes node configuration, including
the alias, and also makes node aliases required.
When running `rad auth`, the user is now prompted for an alias, which
defaults to `$USER`.
When running `rad self`, the alias is now shown.
If the user runs radicle without a config file, the defaults are loaded,
and `$USER` is used as the alias.
This allows the CLI to create a reaction to a specific comment without showing the
comment selector.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
This patch incorporates several important changes to code review:
* Code review verdict is no longer editable
* Made reviews redactable instead
* Renamed review top-level comment to "summary"
* Renamed inline comments to "comments"
* Made inline comments addressable by `EntryId`
* This is to allow for editing
* Removed inline comments from review creation
* Added a `CodeComment` action instead
* Removed `Semilattice` instance from `Review`
* Added a `EditReview` action instead to have more control
* Made it valid for revisions to be redacted concurrently with a review/merge
Most of these changes are needed to support proper inline code comments.
* Add `--home` flag
* Organize the information better
Now looks like this:
DID did🔑z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
└╴Node ID (NID) z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
SSH running (823)
├╴Key (hash) SHA256:UIedaL6Cxm6OUErh9GQUzzglSk7VpQlVTI1TAFB/HWA
└╴Key (full) ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHahWSBEpuT1ESZbynOmBNkLBSnR32Ar4woZqSV2YNH1
Home [..]
├╴Storage [..]/storage
├╴Keys [..]/keys
└╴Node [..]/node
Remove most of the user-friendly functionality and simplify this command
so that it is only used for scripting.
`git push rad` is now the proper way of updating patches.
We consolidate the setup done by `rad checkout`, `rad clone` and `rad
remote add`, so that they all check for node aliases, and are all
capable of setting up tracking branches etc.
There was an extra newline in the patch creation message due to not
accounting for commit messages with newlines at the end.
This is fixed and the tests are updated to be more realistic.
Remove this command in favor of the `git push` flow.
We also remove the `rad/patches` remote configuration which wasn't
working properly with `git fetch --all`.
Help the user when creating a patch by listing the messages of all
commits associated with the patch in almost the same format as git uses
for squashing.
This commit includes a fix for another occurrence of
the missing `EDITOR` variable. This time, it was
inside the `rad edit` command.
The fix is trivial, as it simply returns the error to the caller.
Co-Developed-by: Slack Coder <slackcoder@server.ky>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Verify special YAML strings '~' and 'null' are filtered out when
processing the input from the editor.
Serde replaces empty strings with their values causing an empty title to
become '~'. Best to avoid them entirely and be transparent about it.
The solution is not meant to be long term. This editor format for
getting issue data will be replaced in the near term.
When a user is creating an issue via the editor, it is very easy to
accidentally create an issue.
To fix this, make the default description also 'empty' and bail out if
either it or the title are empty.
Retain the default 'description' message by providing it as a comment.
Allows for setting a patch message during a push. Eg.:
$ git push -o patch.message="My Title" -o patch.message="My description" \
rad HEAD:refs/patches
Works the same way as the `-m` flag of `git commit`.
This allows third party integrations to read the obtained sessionId and
the signed payload without having to implement it on their own.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
This adds a simple `validate` function to COBs that is run on load. It
allows COBs to check whether they are valid, once all operations are
loaded, and return an error if not.
We change `Patch::latest` to always return something, because a patch is
malformed if it doesn't have at least one revision, and thus would not
load in the first place.
This also implies that there is a latest revision in the face of
concurrent updates.
Note that currently, traversal order for concurrent updates is still random.
This will be fixed in another patch.
When applying a `Merge` action, we verify that the commit is in the
history of the default branch of that user.
Since this is not the case when issuing a `rad merge`, and since that
command is no longer needed with the push-based merge, we remove the
`rad merge` command completely.
To allow third party tools read the routing table entries filtered by
rid or nid and if needed in json format.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
Move `merges` from `Revision` to `Patch`, and key them by actor.
This ensures that only one merge is possible per actor. We also make
sure that only one revision can be merged at a time, and signal a
conflict if more than one revision is considered merged.
Finally, we make sure that merges have the same target-branch commit
to be considered equal.
Allows third party implementations to listen to the event stream of a
node.
In case of a serialization error we exit the event subscription and end
the process.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
Allows patches to have their state updated to `Merged` with a `git push`
to the default branch.
Currently, due to the limitations of the COB store, this requires a
linear search through all patches. This will eventually become a problem
when projects have thousands of patches, so we should look into
long-term solutions.
'git pull' in workflow/6-pulling-contributor causes a message to be
emitted for some git versions. It asks to configure 'git pull's default
action.
Use '--ff' as a workaround as a git config file is unsupported, and no
environment variable exists for this configuration.
This adds the ability to the git-remote-helper to open and update
patches via a `git push` to the magic ref `refs/patches` and
`refs/heads/patches/<patch-id>`.
The main change is that we had to move away from offering the `connect`
capability in the remote helper, to offering `push` and `fetch`
individually.
To get a feel for how this works, see the `rad-patch-via-push.md`
example.
This change updates to the latest radicle-git-ext, which required a
few changes within the radicle family of crates.
One set of changes is that the radicle-git-ext crate subsumes
git-commit, git-trailers, and git-ref-format -- so all those imports
go through radicle-git-ext.
The commit code requires that parent `Oid`s point to commits, which
made some tests fail. One of the reasons for these failures is that
the patch tests used fabricated `Oid`s and so the verification would
fail. This is fixed by allowing the test context code to create
`PatchRequest`s that create valid commits.
The other reason for failure was that the identity `Action` passed an
`Oid` that pointed to a blob. This is fixed by removing that code and
making a note of it in documentation.
The final major change was moving the Refspec type into git-ref-format
and removing the AsRefspecs trait. The trait was not required and
could its usage could be replaced by simpler code.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If the 'rad' remote is still in the git config, then some of the CLI
tooling will think the project exists -- even after running `rad rm`.
For example, in the current state, if `rad rm` is followed by `rad
init` then the operation will fail.
During `rad rm`, attempt to delete the 'rad' remote.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, `rad rm` would require the node to be running to untrack
the repository.
This is no longer necessary since the tracking db is exposed via the
`radicle` crate. Instead, use the db directly to untrack the RID
during `rad rm`, as long as the node is not already running.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Now logic for parsing and supporting user facing Patch messages is
shared for creation and editing, refactor and move supporting code to
terminal::patch.
This fixes a bug where the title may be incorrectly parsed, resulting in
it containing multiple lines.
We fix the failing fetch tests by:
1. Not verifying our own refs, unless we're cloning, since we're
otherwise not fetching our own refs.
2. Always force-fetching `sigrefs` from remotes into the staging copy.
3. Making sure that sigref updates are fast-forward before transfering
the remote into the production copy.
Avoid having ssh-agent a hard dependency of `rad-auth`.
Only add the user's key in `auth::init()` to ssh-agent if it is running,
instead of exiting with error.
For `auth::authenticate()` which is used to authenticate the user with
Radicle, assume using `RAD_PASSPHRASE` is a fallback authentication
method. If ssh-agent is not-detected, verify the RAD_PASSPHRASE is
present and correct. Treat neither being present as an error.
Support this by clarifying on `ssh::agent::connect()`s error which are
due to it not being present.
This change ensures that we include as a parent to the initial commit
of a patch COB, the head of the branch we are proposing as a patch.
This was already happening for patch updates, but we forgot to also
include it for the initial patch revision.
This ensures that the code is always fetched alongside the patch.
Ensure that the deletion of cobs is respected by testing a scenario
where a peer creates and then deletes an issue, while another peer
fetches and also has the issue be present and then removed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Sebastian Martinez
cloudhead
Alexis Sellier
Slack Coder
Vincenzo Palazzo
xphoniex
Michail Zampetakis
Fintan Halpenny