With the increase in traffic on the network after launch, the old
defaults don't make sense anymore.
Longer term, we should look into throttling outgoing messages as well,
prioritizing original messages over relayed messages.
If multiple ids are specified for `rad inbox clear`, it would report
that only 1 item was cleared, when in fact multiple were.
This was found to be due to the SQL code returning a change count of
`1`. This is likely due to the way the statement is being reset each
time. The `sqlite` library does not seem to easily support the usage
of `IN` in a `WHERE` clause. So instead, the `count` is aggregated in
the loop and returned instead.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a `Patch` or `Issue` is entirely removed from the Git repository,
then there is no update to be made -- instead it should also be
removed from the cache.
This covers the case where the original author of the COB deletes the
COB.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Add `--force` flag to force a fetch even if the node is not a seed.
* Don't fail if we can't connect to a node during a fetch.
* More accurately exit when we hit our replica count.
When `--private` is passed to `clone`, we force a fetch from nodes that
aren't known seeds.
When it isn't specified, we skip nodes that aren't known seeds, even if
they are in the preferred seeds list. Previously, we would still fetch
from them, which would cause errors.
Add an example of changing a public repository to a private
repository, noting that any replicated data will be public, but any
new changes will not be replicated.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
There are two areas where we can be more tolerant of delegate namespaces
being missing or not validating:
1. Calculating the canonical HEAD
2. Fetching from a remote
In 1. the protocol is tolerant in that if the local node does not have
the default branch for a delegate, it will still attempt to use any of
the delegates it does have to reach the threshold.
This is made safe by ensuring that if the threshold is being updated
then the node performing the update must have a threshold of delegates
locally in their storage. It also made safe by 2.
In 2. the protocol is tolerant by allowing delegates to be missing
from the serving side, as long as they can still meet a threshold of
delegates. This is further tolerant, when validating the received
data, a threshold of delegates are valid to consider the fetch
successful -- otherwise it will fail.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When running `rad patch list`, show the set of delegate reviews for
each patch, with the following meaning for the symbols:
- ✔: delegate has accepted the patch
- ✗: delegate has rejected the patch
- -: delegate has not made a verdict on the patch
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If the user passes a <path> argument to init, it is incorrect and a
bit confusing to reference the "current directory" or "this directory"
e.g. "run rad . in current directory" when the repo was not created in the
current directory and was created with a path argument instead.
Use the path provided by the user in help text, otherwise reference
the current directory.
If `rad id` is called with no arguments, or just `--title` and
`--description`, then it will fail because there were no changes made
to the identity.
Check if the `proposal` document is the same the `current` document,
and if so supply a hint and a better error.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Help the user know which revision they currently have checked out when
running `rad patch checkout`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If an ancestry check of an annotated tag was triggered then the fetch
from that peer would fail. This is due to `graph_ahead_behind` only
working for commit-ish objects.
To allow for annotated tags, and other object that peel to commits,
the check now peels the two objects to commits before getting their
`Oid`s. The call to `graph_ahead_behind` can take these two `Oid`s
instead.
A test for annotated tags is included as part of the `examples/git`
suite in the CLI tests.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
No tests for `rad patch delete` existed.
Add a test where two peers interact on a patch and one decides to
delete the patch. It ensures that the patch is still visible to the
peer that did not delete the patch.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To make things a little more consistent, move the `rad self --address`
command to `rad node config --addresses`, since it's more about
externally-facing configuration of the node, rather than your local user
or profile.
Previously, authorization was only checked for ops after the root op.
We also avoid creating empty actions for `assign` and `label`, which
was needed for non-authorized users to be able to create issues/patches.
This makes the default policy `block` with scope `all`. The good thing
about this is that you don't have to manage individual nodes in the
beginning. You simply seed the repos you want, and everything is
followed within those repos. This matches user expectations, as `rad
seed` does what you expect it to do.
For open-policy seed nodes, it also means that simply changing `block`
to `allow` from the default config is all you need.
This leaves the non-default `followed` scope as a more "advanced"
setting, since it is much more cumbersome: it forces you to follow a
user before you can see their issues/patches, which is not something
most users expect coming from old-shcool forges.
Shows you updates on local repositories.
Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.
The `rad inbox` command can then list items in the database and delete
them.
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
The command now works with both a replica target *and* a seeds target.
This is especially useful to eg. ensure that a preferred seed has been
synced, while still trying to hit a higher replica count.
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.
This is used for pinning repositories in the httpd `/projects` handle.
To allow returning all repositories, an `all` query parameter is also
included.
This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
In the case where we don't find any seeds, display a better error
message.
We also don't bail out on error, since the announcement step might still
be worth trying.
Previously, COB change commits would not distinguish between their
parents. This means that they would try to load source code commits
(eg. for patch COBs) and then fail since those commits wouldn't have a
manifest or `Rad-Resource`.
We introduce a backwards compatible change here which is that parent
oids that are not COB commits are accompanied by a commit trailer called
`Rad-Related`. This is for all related content that should be pulled in
but is not part of the change graph proper.
When building the change graph, we simply ignore commits that are marked
as "related".
We were using the two pretty interchangeably. "Project" should only be
used to refer to the repository payload. "Repository" should be used
when referring to the resource that is fetched, synced, cloned, checked
out etc.
Add a test to show the behaviour of setting up 3 delegates, through two
seed nodes -- ensuring that commands are still operational after
accepting a delegate that the node is missing.
The two seeds are setup with two different scopes:
- `Scope::All`
- `Scope::Followed`
This is to show that the network has eventual consistency once there
is a way to fetch any other node's fork in the network.
The setup of the test is:
- Alice adds Bob as a delegate
- Bob adds Eve as a delegate
- Alice accepts Bob's changes
Alice is able to get Eve's fork through the `Scope::All` seed. The
`Scope::Followed` seed will then be able get Eve's fork once any other
node syncs with it, in this case Eve.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We cannot guarantee that the `rad sync status` will run before all the
nodes are synced.
Remove the call to `rad sync status` and explain that at least 1 of
the nodes will be synced.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To allow `rad sync` to break early from the call to `announce`,
control flow is introduced for the callback.
In `rad sync`, the number of synced nodes is checked against the
`replicas` -- if specified -- and sends a `Break`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, the head of the opened patch would not be displayed, and
so the user could not tell what it was if any updates were applied.
Show the head for the opened revision. The below shows the difference
between before and after, respectively:
```
├─────────────────────────────────────────────────────────────────────┤
│ ● opened by z6MknSL…StBU8Vi (you) now │
│ ↑ updated to 8d6692ce2b2219a54d09ccd5feefedb2bae33eba (02bef3f) now │
│ ↑ updated to 8438ab3940316abe65c93982a0a8f39688400359 (9304dbc) now │
╰─────────────────────────────────────────────────────────────────────╯
```
```
├─────────────────────────────────────────────────────────────────────┤
│ ● opened by z6MknSL…StBU8Vi (you) (8b0ea80) now │
│ ↑ updated to 8d6692ce2b2219a54d09ccd5feefedb2bae33eba (02bef3f) now │
│ ↑ updated to 8438ab3940316abe65c93982a0a8f39688400359 (9304dbc) now │
╰─────────────────────────────────────────────────────────────────────╯
```
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Allow `rad patch edit` to take a `--revision` option which will edit
the description of the specified revision, rather than the root
revision.
If the flag is not specified, then the original behaviour of editing
the root revision's title and description is used.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of having a timeline of a patch be global and sorted by
timestamp, organise it via revisions. Each revision has an entry in
the timeline, sorted by timestamp. Under each revision, a series of
updates are inlined, also sorted by their timestamps. Currently, these
updates can either be reviews or merges.
For example:
```
├────────────────────────────────────────────────────────────────────────────┤
│ ● opened by fintohaps (you) 1 week ago │
│ └── ✓ accepted by fintohaps (you) 1 week ago │
│ ↑ updated to 5f2a123e3f2aa60f2e03815df283563495b73085 (a1c3c0f) 1 week ago │
│ └── ✓ accepted by fintohaps (you) 1 week ago │
│ └── ✓ merged by cloudhead (z6Mkt67…v4N1tRk) now │
╰────────────────────────────────────────────────────────────────────────────╯
```
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
By default, this command now:
* Follows the peer if not already followed
* Syncs refs of the peer, if it wasn't followed before
* Fetches refs into working copy
This functionality can be disabled via `--no-sync` and `--no-fetch`.
Add functionality for lising the remotes that are in the storage
repository, but a working copy remote has not yet been created for
them.
This functionality is included in the `rad remote` command since it
can be directly used for creating Git remotes using the same command,
i.e. `rad remote add`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To keep behaviour consistently under subcommands, move the `rad node
seeding` behaviour under `rad seed`. When an `<rid>` isn't
provided then the seeded repositories are listed.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
To keep behaviour consistently under subcommands, move the `rad node
following` behaviour under `rad follow`. When an `<nid>` isn't
provided then the followed nodes are listed.
This includes filtering by alias, when `--alias` is used.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Shell out to `git`, avoiding using `git2`. This is to avoid an
issue where somewhere within the fetch there is an attempt to lookup a
`rad/sigrefs` object, which says that the object is missing. It is
suspected that this is due to the object being localised in the same
packfile as other objects we are fetching. Since the `rad/sigrefs`
object is never needed nor used in the working copy, this will always
result in the object missing. This seems to only be an issue with
`libgit2`/`git2` and not `git` itself.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The hint for updating a patch incorrectly printed:
git push rad -f HEAD:rad/patches/<patch id>
The remote, `rad`, is already specified so the the right hand side of
the refspec should only be `patches/<patch id>`.
Fix this by stripping the remote from the name of the branch in the
remote helper.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Allow the CLI user to update a patch's base commit through the `rad
patch update` command.
There was check to see if the head of the branch was the same as the
head of the revision, which would result in the update being skipped.
This was adapted to check that the head *and* the base were the same.
A test was added in the examples to showcase this use.
Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When we perform updates to cobs we tend to want to sync those changes
to the network after. This patch automatically does this for
operations that perform updates to issues and patches.
Note that `--announce` becomes the default for the `rad patch`
command.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Allow the user to specify a directory to clone into when
performing `rad clone`.
Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It's actually not such a great idea to sync on git-fetch and
git-ls-remote, as syncing is supposed to happen in the background
already, and if you want to force a sync, you can just call `rad sync`.
Previously, it would be annoying that when you called `rad sync` and
then `git fetch`, it would still attempt another sync, for example.
Moving forwards, it might make sense to call `git-fetch` on `rad sync`
(so the other way around), because `git-fetch` is free (it doesn't talk
to peers), and you almost always want your working copy up to date with
your storage.
In some cases, there's a chance for the patch to not have arrived
before trying to check it out in the `alice` repo.
Ensure that the patch is present by first running `rad sync -f`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Since forking creates your `rad/sigrefs`, which then needs to be backed
up forever, we avoid forking on `rad clone`. This should make little or
no difference for users, as the "fork" will be created on `git push`.
Split the commands and change the naming.
* Tracking a repo is now called "seeding"
* Tracking a peer is now called "following"
In this commit, we don't change most of the internal code, only the
user-facing part.
It is more natural for the review command to exist as a subcommand of
patch, since the reviews are for patches.
Remove the `rad review` command, moving its functionality to the `rad
patch` command. The rad-patch.md and rad-review-by-hunk.md tests are
modified to use the `rad patch review` command.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Rename the --remove options to --delete for assign and label, to be
more consistent with review, and let this be the standard.
Clean up any typos around labels/dids and issues/patches.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Make the CLI easier by unifying the label and unlabel, using `--add`
and `--remove` options instead.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It is more natural for the assign and unassign commands to exist as
subcommands of the object they are assigning to, i.e. issues and
patches.
Remove the rad assign and rad unassign commands, moving their
functionality to the rad issue and rad patch commands. The
rad-issue.md and rad-patch.md tests are modified to include a assign
test.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It is more natural for the label and unlabel commands to exist as
subcommands of the object they are labelling, i.e. issues and
patches.
Remove the `rad label` and `rad unlabel` commands, moving their
functionality to the `rad issue` and `rad patch` commands. The
`rad-issue.md` and `rad-patch.md` tests are modified to include a
label test.
The `rad patch show` command did not support showing labels as part of
its output. This was also included to confirm the test above was working.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
My use case for this is to have a script that lists my outstanding
patches so I can ping people for review or merge if there's no
movement for a long time.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
Allow the caller of `rad sync` to `--sort-by` the NID, alias, or
status of the seeds.
This prevents random ordering, due to timestamps, in CLI testing.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
It's necessary for a delegate's references to be present, e.g.
verification, calculating of canonical data, etc.
To ensure that a user does not get themselves into a state where they
have added a delegate without their references, prevent adding a
delegate to the identity document if their `rad/sigrefs` is not
present.
As well as this, it ensures that if there is a `rad/sigrefs` that it
has the default branch -- since it's possible for a peer to have created
`rad/sigrefs` through COBs without a default branch.
This change meant that the `rad-id` tests required the test scenarios
to be set up with all nodes having each other's references. A test
case to add a missing delegate was also added to `rad-id.md`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The call to `rad sync -a` can end up with different output if the `git
push` has already announced to the network -- ending up with the nodes
already be in sync.
Instead, for consistency, leave the syncing up to the `git push` and
remove the call to `rad sync`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If the local peer does not have any `rad/sigrefs` then there is no
risk of any data corruption. In this case the repository can be
removed entirely.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Removing a repository completely can result in data corruption for
users, since a user's `rad/sigrefs` are the ground truth of their
state, and any fork of them results in errors in the protocol.
Instead of `rad rm`, we replace it with `rad clean` that removes all
remotes other than the local peer's namespace and the delegates'. The
delegates are also necessary to keep, since the repository's identity
is dependent on them.
Note that the remotes that were cleaned will be populated again upon
fetching, as long as they are tracked.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Distinguish between repos that are simply tracked, vs. repos that are
cloned (and thus forked). By default, we don't show repos that don't
have a local fork.
cloudhead
Fintan Halpenny
Alexis Sellier
josibake
Fintan Halpenny
Lars Wirzenius