Commit Graph

1566 Commits

Author SHA1 Message Date
Fintan Halpenny 47a440a384
radicle: expose shared logger
To be able to use log setup across the different components of
heartwood, expose a `logger` module from `radicle`. This also includes
a `test` logger.

These modules are guarded behind a `logger` feature flag, so that
other crates are not bringing any uneeded dependencies.

This allows logging to be setup in `radicle-cli-test` so that when a
test fails, the log output is present for debugging purposes.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-20 15:11:10 +01:00
Fintan Halpenny f9d9bdf8cd
cli: fix update hint
The hint for updating a patch incorrectly printed:

    git push rad -f HEAD:rad/patches/<patch id>

The remote, `rad`, is already specified so the the right hand side of
the refspec should only be `patches/<patch id>`.

Fix this by stripping the remote from the name of the branch in the
remote helper.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-20 15:10:13 +01:00
cloudhead 9cf6fab7c8
node: Small fix to allow reconnections
We weren't allowing reconnections after a failed attempt previously.
2023-12-20 12:16:31 +01:00
Fintan Halpenny 8eac221f4c
node: Properly handle dequeued fetch results
When a peer is performing a background and foreground fetch from two
different peers, the background fetch result was being used as the
result for the foreground fetch. This results in inconsistencies,
since the foreground peer has new updates from themselves.

The reason for this issue was the fetches that were queued were not
correctly being handled, and the other peer's results were being
returned instead.

This fixes the queueing and dequeueing by ensuring that the queued
fetch occurs, passing the correct result channel. This fetch will then
be subsequently dequeued correctly and the foreground fetch obtains
the correct result.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:02:14 +01:00
Fintan Halpenny a6ba1b0103
node: inspect outdated sigrefs test
This test is flakey and requires further investigation. Some extra
assertions are added to ensure that the expected behaviour is
happening upon fetching. It is unknown if the fetch queue is behaving
properly, and it's suspected that some existing fetch results might be
used for queued fetches.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:02:14 +01:00
Fintan Halpenny 48aa725bea
cli: Add `--base` option to `rad patch update`
Allow the CLI user to update a patch's base commit through the `rad
patch update` command.

There was check to see if the head of the branch was the same as the
head of the revision, which would result in the update being skipped.
This was adapted to check that the head *and* the base were the same.

A test was added in the examples to showcase this use.

Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:00:27 +01:00
Fintan Halpenny f90421a6bb
cli: sync on patch and issue updates
When we perform updates to cobs we tend to want to sync those changes
to the network after. This patch automatically does this for
operations that perform updates to issues and patches.

Note that `--announce` becomes the default for the `rad patch`
command.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 14:19:55 +01:00
cloudhead 13db55014b
cli: Show URL on `rad init` 2023-12-19 14:15:30 +01:00
cloudhead 6eb2772235
node: Improve flexibility of test environment
Allows us to pass full configs (profile or node) when creating test
nodes or profiles.
2023-12-19 14:15:30 +01:00
cloudhead 67c950954d
cli: Improve `rad sync status` output
To make it less confusing, clearly mark the local node and don't show
sync status for it.
2023-12-19 14:14:11 +01:00
cloudhead 5e3990d411
cli: `rad ls` shouldn't show unseeded by default
Unseeded repos should not show up in `rad ls`, by default. Only with
`--all`. We also use a special visibility of "local" for those projects.
2023-12-19 14:14:11 +01:00
cloudhead 59f506dbb5
radicle: Fix bug in node.db query 2023-12-19 13:26:17 +01:00
Thomas Scholtes 48dedc6e7e
`rad-web` runs HTTP API and connects web app
Instead of `rad-web` just creating a session for an already running HTTP
API the command now starts the HTTP API, creates a session and opens the
authentication link.

Signed-off-by: Thomas Scholtes <geigerzaehler@axiom.fm>
2023-12-19 13:14:07 +01:00
cloudhead 6f7c2dca91
node: Introduce connection penalty system
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.

We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.

When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
2023-12-15 16:00:09 +01:00
Sebastian Martinez bdcaa60474
httpd: Update axum and hyper dependency 2023-12-15 12:08:43 +01:00
cloudhead d9a84de5ee
radicle: Simple database migration system
We create a system for migrating the our databases, as well as a first
migration, for adding a `penalty` field to the `nodes` table, which we
are going to need soon, as an example.
2023-12-15 12:04:54 +01:00
Fintan Halpenny f4dc224b3a
crypto: update ssh-key
There are two vulnerabilities that were due to `ssh-key`'s
dependencies[[0]][[1]].

Updating to ssh-key-0.6.3 removes the vulnerability[[0]], however,
there is no existing patch for [[1]] but it is being worked on.

[0]: https://rustsec.org/advisories/RUSTSEC-2022-0093
[1]: https://rustsec.org/advisories/RUSTSEC-2023-0071

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-14 16:13:29 +00:00
Fintan Halpenny 4afe235a3c
nix: simplify devShell and envrc
There is no need for the ';' in the `.envrc` file.

Do not run checks in devShell so that it's possible to enter the shell
while code is being edited and is not compiling. Also include
`cargo-nextest` so that one can test in the shell.

Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-14 10:52:04 +01:00
Fintan Halpenny 76c969c755
node: relax converge criteria
Some test flakes would occur due to the `converge` function looping
forever. It was noticed that in these cases the log output would look
like:

    test: Node z6MkqTKLVnFdVF24PQiDoQffbTPT7U5igghS3XWuFJLfah1P has
    {(Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
    PublicKey(z6MkemMxfWhRQFKy4UPX7FsN8GjLAZ9T65rbmDsUn7fchioP)),
    (Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
    PublicKey(z6MkpdPJp7naAurUjCpwef1BDD5jfPhAvwfinaGd7pVh227j)),
    (Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
    PublicKey(z6MkqTKLVnFdVF24PQiDoQffbTPT7U5igghS3XWuFJLfah1P))}

with further investigation, the variable `all_routes` was found to be
smaller than the calculated routes.

Instead of checking `==`, instead check that
`routes.is_superset(&all_routes)`, which means that `routes` must be
equal to `all_routes`, but allows it have extra entries.

Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-14 10:52:04 +01:00
Fintan Halpenny 070e912b3e
cli: allow directory in rad clone
Allow the user to specify a directory to clone into when
performing `rad clone`.

Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-14 10:52:04 +01:00
cloudhead c761d03a5a
cli: Review builder fixes
Diff hunks were not being applied properly due to the line offsets
changing when a hunk was skipped.
2023-12-14 10:50:38 +01:00
cloudhead 99aeae19ea
node: Fix node announcements not being received
Since node announcements are stored and retrieved from disk (due to the
PoW), the timestamp on it is never updated. This means it can be too old
to cache and not sent/received by nodes. Fix this by having a max age.
2023-12-14 10:41:59 +01:00
cloudhead 9ada30255b
cli: Don't sync on git-fetch
It's actually not such a great idea to sync on git-fetch and
git-ls-remote, as syncing is supposed to happen in the background
already, and if you want to force a sync, you can just call `rad sync`.

Previously, it would be annoying that when you called `rad sync` and
then `git fetch`, it would still attempt another sync, for example.

Moving forwards, it might make sense to call `git-fetch` on `rad sync`
(so the other way around), because `git-fetch` is free (it doesn't talk
to peers), and you almost always want your working copy up to date with
your storage.
2023-12-13 14:47:14 +01:00
Fintan Halpenny 63469fd3bc
node: init policies DB early
Instead of creating the DB policies handle when needed, construct the
handle when the `Worker` pool is being constructed.

This replaces the usage of the DB handle with the `Worker`'s field
instead.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-13 14:20:15 +01:00
Fintan Halpenny e894b6512d
node: prevent fetch response if no repo policy
If a node had removed a seeding policy and another node fetched from
it, then it would perform the fetch.

Instead, check the policy during the authorization step and prevent
the fetch is the resulting policy is `Block`.

Note that the policy will be `Block` iff the `Block` policy was
specifically set or the `Block` is the default policy for the node.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-13 14:19:46 +01:00
Sebastian Martinez 53c08f5d26
httpd: Cleaning up `api::json` and review comments 2023-12-13 12:37:10 +01:00
cloudhead cc380532c6
cli: Improve messaging when config doesn't load
Instead of telling users to setup their profile, point out the actual
problem with the config.
2023-12-13 12:30:11 +01:00
cloudhead 87debf380e
cli: Improve passphrase handling
Reduce redundancy in passphrase functions and give better error message
when not using a TTY.
2023-12-13 12:29:12 +01:00
cloudhead ebee41fbf1
cli: Only allow shell-friendly characters in names 2023-12-13 12:29:12 +01:00
cloudhead 376c3caca0
cli: Fix small clippy issue that snuck in 2023-12-13 12:27:49 +01:00
Fintan Halpenny 9f544ef8c1
nix: switch to use nix flakes
Nix Flakes[[0]] are a more modular way of expressing nix derivations.

The `flake.nix` file describes:
- How to build the project, or components of a projet, via `nix build`
- What "checks" can run for a project, which can be executed via
  `nix flake check`
- What binaries can be run, e.g. `nix run .#rad` to execute the CLI

This approach is preferable since a lot of the Nix ecosystem is using
Flakes -- despite it still being marked as experimental.

One interesting benefit, that needs confirming, is that a check can be
run from a remote Git source. So if someone creates a patch on
Radicle, someone else with the `nix` binary can execute:

         nix flake check git+https://app.radicle.xyz/nodes/seed.radicle.xyz/rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5?refs=patches/<patch id>

This means that anyone with `nix` can check patches without having to
fetch the patch itself.

Another obvious benefit will be that it allows the Radicle binaries to
enter the Nix ecosystem.

The big caveat to both of the above is that, currently, the tests are
failing while running `nix flake check`. They are currently disabled
with `doCheck = false`. They are potentially failing due to some
strange sandboxing issues which are being looked into.

[0]: https://nixos.wiki/wiki/Flakes

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-13 12:25:12 +01:00
Fintan Halpenny 44206cb6ca
cli: ensure patch is present in rad-push-and-pull-patches
In some cases, there's a chance for the patch to not have arrived
before trying to check it out in the `alice` repo.

Ensure that the patch is present by first running `rad sync -f`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-13 12:25:12 +01:00
cloudhead d773659d66
cli: Don't fork on clone
Since forking creates your `rad/sigrefs`, which then needs to be backed
up forever, we avoid forking on `rad clone`. This should make little or
no difference for users, as the "fork" will be created on `git push`.
2023-12-13 12:14:06 +01:00
cloudhead ecb59d687b
cob: Return `RevisionId` instead of `Revision`
Previously we panicked if a revision wasn't found after an update. But
it can happen that the update goes through yet a concurrent update
prevents the revision from being valid.

To partially deal with this, we don't try to lookup the revision after
an update, we simply return the ID. This means a subsequent lookup can
fail, and the error can be handled there.
2023-12-13 12:14:06 +01:00
cloudhead 57e3b2a154
cli: Add timeout option to `rad watch` 2023-12-13 12:14:06 +01:00
cloudhead 0e8a64d0f8
cli: Better handling of missing refs in `watch` 2023-12-13 12:14:06 +01:00
cloudhead 74a0d7a4d2
cli: Fix `rad seed --delete` command
There was a bug due to `--delete` not being treated properly as a
sub-command.
2023-12-12 11:02:07 +01:00
cloudhead bfdf82a69a
node: Small initialization changes
* Connect to nodes at the end, not the beginning
* Skip sync-status stuff if we're not seeding the repo
* Move debug message to be higher signal
2023-12-11 16:29:39 +01:00
cloudhead 472a1e6545
cli: Add `watch` command
Allows watching and waiting for refs to change. This is especially useful for
tests. For now, we don't show it in the output of `rad`.
2023-12-11 12:04:26 +01:00
cloudhead 5af8f70665
cli: Fix `rev` argument parsing
We were using the `oid` function incorrectly, which has been changed
to actually return an OID.
2023-12-11 12:04:26 +01:00
cloudhead 8196027f35
cli: Allow force-push with single delegates
It's not currently possible to force-push rad/master if it's not a
rollback, even if you are the only delegate. Since there is no risk
in doing that, we allow it in this patch.
2023-12-11 11:28:29 +01:00
cloudhead 9aee9b2436
cli: Corrections to patch/issue help text 2023-12-06 12:07:31 +01:00
cloudhead 6ca7da2768
radicle: Rename many more methods using new naming 2023-12-06 10:02:00 +01:00
cloudhead 1b026ae0ee
node: Rename node `tracking` module 2023-12-06 10:00:45 +01:00
cloudhead e65595c4c7
node: Rename commands with new "seed" nomenclature 2023-12-06 09:56:58 +01:00
cloudhead f453090e27
cli: Consolidate policy/scope variants
* Policy::Track -> Policy::Allow
* Scope::Trusted -> Scope::Followed
2023-12-06 09:56:58 +01:00
cloudhead 4605348d68
cli: From "tracking" to "seeding" and "following"
Split the commands and change the naming.

* Tracking a repo is now called "seeding"
* Tracking a peer is now called "following"

In this commit, we don't change most of the internal code, only the
user-facing part.
2023-12-06 09:56:57 +01:00
Fintan Halpenny ee9ee69130
cli: move rad review to rad patch
It is more natural for the review command to exist as a subcommand of
patch, since the reviews are for patches.

Remove the `rad review` command, moving its functionality to the `rad
patch` command. The rad-patch.md and rad-review-by-hunk.md tests are
modified to use the `rad patch review` command.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-06 09:51:55 +01:00
Fintan Halpenny 6a3a66ae9c
cli: simplify `Message` default
Use the derive and `#[default]` mechanism for deriving the default of
`Message`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-06 09:51:47 +01:00
Fintan Halpenny 3681c0e6df
cli: rename remove to delete
Rename the --remove options to --delete for assign and label, to be
more consistent with review, and let this be the standard.

Clean up any typos around labels/dids and issues/patches.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-06 09:48:02 +01:00