Similar to the `Fetcher` sans-IO approach, an `Announcer` sans-IO approach is
introduced. The previous `Handler::announce` was already quite close to using a
sans-IO approach, but using internally held state. This change makes the
business logic more reusable – and testable.
Similarly to the `Fetcher`, the `Announcer` is configured and then driven by
telling it when nodes have been synchronized with. It will continuously check if
it has reached its target, yielding to the caller to decide whether to keep
going. In contrast, it also provides a way to mark the process as timed out,
since announcing relies on waiting for events to come back in a timely manner.
This change improves the `rad clone` command in a few different ways.
The first improvement is to check if the repository already exists before
fetching. If it does exist, then there is no need for the fetch to be performed,
and can move straight to the checkout.
If the repository does not exist, then the fetch is performed, as per usual.
However, it uses a replication factor of 1, so that the fetch finishes as soon
as it has the repository in local storage.
The change also improves the output when failures occur while attempting to
perform the checkout phase. It will print the failure that occurred and provide
a hint message in each case to help debug the failure, or push the caller in the
direction of performing the correct action.
This required some changes in `test/commands.rs` setup. Some tests were using
the `rad-clone.md` to setup a clone, which would have different output if the
repository did not need to be fetched. In those cases, the `rad` method for
performing the clone is used instead.
Commit b608a788 (cli: `rad config schema` emits JSON Schema, 2025-04-28)
added a feature flag that enables the "schemars" optional dependency.
Crate radicle-node doesn't use this feature.
Crate radicle fails to compile when the feature is disabled.
This is reproducible with "flake check".
Fix it.
Note that a "cargo build" of the entire workspace doesn't fail, because the
feature is enabled due to "unification", since "radicle-cli" does use this
feature. From https://doc.rust-lang.org/cargo/reference/resolver.html#features:
> When building multiple packages in a workspace (such as with --workspace
> or multiple -p flags), the features [of the dependencies] of all of those
> packages are unified. If you have a circumstance where you want to avoid
> that unification for different workspace members, you will need to build
> them via separate cargo invocations.
Reword the help message of `--replicas` and `--max-replicas`, while also
improving the documentation on how the fetch and announce functionality works.
This patch changes separates the business logic of fetching from the process of
fetching itself. It does this through a sans-IO approach, where a `Fetcher`
provides the necessary state to help drive a fetch forward, without performing
any of the IO itself.
What the `Fetcher` cares about is:
- What nodes are going to be attempted to be fetched from
- In what order should they be attempted
- When should the fetching process be considered finished
The `Fetcher` is then used in `radicle-cli` to drive forward the `sync::fetch`
function, allowing it to only care about the IO.
Leverage `schemars` to generate a JSON Schema from our structs for
configurations and those occurring within them.
The output of `rad config schema` can be used by editors to provide a
smoother editing experience for the configuration file. Discovery
attributes (both keys and values) is greatly improved with the schema
helping in the background.
Refer to:
- https://json-schema.org
- https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings
- https://schemastore.org
- Print link direction.
- Shorten NIDs.
- Use single chars to indicate status, legend is provided via hint.
On nodes with only outgoing connections, link direction won't be very
interesting. But on seed nodes it sometimes *is* of interest in which
direction the connections are going.
Also, I hope that in the future we will enable more and more nodes to
connect directly to each other.
The table header "Dir." is an abbreviation for "Link Direction".
This change is to help separate the fact the namespacing by a `NodeId` is a
Radicle specific concern. The identifier can, in theory, be any kind of path
component, as long it is valid in Git.
The motivation for the change is to help separate the idea of a device's
`NodeId` vs an agent being an author – for future agent repository work.
The motivation of this change is to move away from tying signing to be
specifically for ed25519. The reason being that the protocol will want move
towards two different kinds of signing – the node signing artifacts, and an
author signing artifacts.
This change captures the former, node signing, by introducing a `Device` type
that is the `NodeId` – a PublicKey underneath the hood – and a signing
mechanism. This allows the replacement of the `Signer` trait being used – which
always assumed a `PublicKey`. Instead, the `Device` is constructed with
`NodeId`.
In `radicle-cob`, a signer is expected to implement
`signature::Signer<ExtendedSignature>`, and everywhere in `radicle`,
`radicle-node`, `radicle-cli`, and `radicle-remote-helper` is expected the
signer is expected to implement `signature::Signer<Signature>`.
A `Device` implements both of these but only requires
`signature::Signer<Signature>` to do so – since an `ExtendedSignature` is
essentially `(PublicKey, Signature)`, and the `NodeId` of the `Device` can be
used.
This declares the minimum supported Rust version, independently of the
`rust-toolchain.toml` file, which specifies the Rust version and
toolchain components the crate/workspace should be built with. The
MSRV affects other crates that depend on anything in this workspace,
but the toolchain file does not seem to affect them. This means the
MSRV is useful for those who build or develop dependents.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
Debian policy is to install to /usr/bin, not /bin, so do that. This
silences a lintian warning without using an override.
Also fix removal of .crates*.toml files that cargo install creates.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
In fresh environments, without a global git config, the `user.name` and
`user.email` will not be set for the fixture repository. This causes issues when
tests are run and attempt to call `git2::Signature::now` – it will fail to read
a user name and email for the commit.
To remedy this, the name and email are set for the repository when initialising
it.
These hook take relatively long on my machine, and my impatience has
lead to me doing `git commit --no-verify`, which defeats the purpose of
having these hooks.
Note that others (like `alejandra`, `cargo fmt` and `shellcheck`) are
reasonably fast and will keep doing good.
To keep expectations similar to the seed command after the changes in
the companion patch:
rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5/patch/3be0e68b8948e65431a288f225454bafd93de34a
This introduces rate limits for the `ChannelReader` to limit DDoS attacks and
attempts to upload repositories that are larger than a node is will to permit.
The limiter sets the total number of bytes it is will to accept in a single
exchange, defaulting to 500MB. This means that initial fetches will prevent
large repositories, but is plenty for new packfile data to be sent in subsequent
fetch exchanges.
The limit can be configured within the node's config file, under the limits.
If an upload-pack is of a considerable size, the writer thread will be busy
writing bytes to the receiving side. During this period, the receiving side will
not be sending any bytes.
To ensure that the reader does not exit while the writing thread is performing
writes, the reading side will check when the last time a write was performed. If
it has not reached the timeout, then it can continue attempting to read.
Otherwise, it will break out of the loop; killing the upload-pack process safely.
Similarly to pushing using the `file://` protocol and `libgit2`, fetching will
be slow – so instead use `git fetch` directly to perform the `checkout`.
This gives the possibility to see the inventory of another node than self.
This does replicatee the functionality of 'rad node routing --nid', but
feels like a more intuitive command to try.
In issue #[2836] of `libgit2`, the speed of `libgit2`'s `file://` protocol for
git operations is found to be very slow. This is further corrobarated by `rad
init` taking 13 hours to initialise the hardenedbsd [ports] repository.
There are two areas where the `heartwood` project uses `libgit2` to push using
the file protocol. The first is when via the `trasnport::local` smart transport
registration and the second is the final push to storage in the `git-remote-rad`
binary.
When both these `push` operations are changed to use the `git` binary instead,
the [ports] repository can be initialised in less than 10 minutes (nearly 100x
speed up).
This change is clearly required if `heartwood` wishes to support larger
repositories.
[2836]: https://github.com/libgit2/libgit2/issues/2836
[ports]: https://git.hardenedbsd.org/hardenedbsd/ports
Update the rust toolchain version to 1.85.
Note that this is not the latest, however, it is the latest available in nixpgs,
so this version is good enough for now.
This resolves issues:
- 111cf90 (`handler` is not async-signal-safe)
- c8e0300 (`libc::signal` should not be used)
- b91c6c3 (Mutex can cause signals to be missed)
See also: patch 8e05702.
The API of `radicle-signals` is the same as before, and so uses of it
don't need to be and aren't changed. The behavior is slightly different
in that:
- If the channel is full then signals will not be lost, which is an
improvement. This is achieved without blocking in the signal handler.
This is possible because of the counters approach along with the
internal receipts-processing thread of the `signals_receipts` crate.
- `install()` and `uninstall()` might block very briefly if necessary to
acquire the mutex, which is now internal to and managed by the
`signals_receipts` crate, only if there are concurrent calls to them
(which is unlikely), but such blocking is guaranteed to be bounded to
be very brief. This is done so they no longer can fail to do their
purpose, which is an improvement. They still return errors if the
handling is already installed or uninstalled, respectively, which
preserves the previous use cases.
- The new `finish()` function is introduced. This is provided in case
it's ever needed to completely clean-up the facility, by terminating
the internal receipts-processing thread, to be like it hadn't been
installed before.
- The user must ensure that the notifications channel is disconnected,
by dropping the receiver(s), when doing `uninstall()` or `finish()`.
Such dropping usually occurs naturally, and already occurs for all the
preexisting uses of `radicle_signals`. (The `signals_receipts` crate
is capable of a more robust approach, but this commit doesn't use
that, to avoid changing the preexisting uses of `radicle_signals`.)
- The `TryFrom` impl for `Signal` is of `SignalNumber` which is `c_int`,
instead of `i32`, because `c_int` (the type of signal numbers) might
not be `i32` on all platforms (POSIX only requires `c_int` to be at
least 32-bit).
The version of `radicle_signals` is incremented, to reflect those
changes and the substantially different internal implementation.
The new dependency on the `base64` crate is needed by the
`signals_receipts` crate for it to work on macOS, because its dependency
on the `sem_safe` crate uses `base64` as part of creating anonymous
semaphores on macOS (which lacks support for unnamed semaphores (in
violation of POSIX)).
Signed-off-by: Derick Eddington <kcired@pm.me>
Signed-off-by: Lorenz Leutgeb <lorenz@leutgeb.xyz>
The help shows that `--emoji` is optional, however it was not.
Instead of making `--emoji` mandatory, I added an emoji picker
for the emojis featured in the web ui.
`--emoji` is now (really) optional, but can be used to react
with other emojis not included in the selector or the web ui.
We don't have network access under Ambient, so we need to run "cargo
install" with the "--offline" option to prevent it from trying to
check current available versions of crates.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
In the 'clone' command, a 'signer' is instanciated just to get the
public key. In the 'job' command, a 'signer' is instanciated early,
even when the subcommand doesn't use it.
That's unnecessary, and infers the potential use of RAD_PASSPHRASE.
In those cases, avoid instanciating a 'signer'.
Fintan Halpenny
Johannes Altmanninger
Lorenz Leutgeb
n4ch7
Lorenz Leutgeb
Lars Wirzenius
Sekhat Temporus
Richard Levitte
Sebastian Thiel
Derick Eddington
tippfehlr