Previously `test_fetch_emits_canonical_ref_update` was passing by
catching a leftover `CanonicalRefUpdated` event from the initial
repository fetch. Furthermore, the new commit was written to the
un-namespaced root. It can be proven to be the case by commenting out
the `alice.commit_to(...)` line and watching the test still pass.
Fix the test by committing to the namespaced default branch which will
trigger the `CanonicalRefUpdated` that is expected by Bob. Drains events
so we don't catch the wrong event, and explicitly announces.
The `commit_to` method should namespace the reference, so that node is
only ever committing the namespace of that particular node.
This prevents the node from committing to another node's namespace or
the canonical namespace.
Previously a rule like `refs/heads/releases-*` would be rewritten
incorrectly to `refs/heads/releases-**/*` matching the 'trailing
asterisk' rule.
Simplify the rewrite rule so that a trailing `*` becomes a prefix
matching rule. Add tests.
Reorganises the rules test suite into a `test` module and a `helper`
sub-module.
The former holds the original tests, removing the `test_` prefix.
The latter holds helper functions for test setup.
- Drop `type='ipv6'` rows whose bracket-stripped inner part contains
no `:`, since every valid IPv6 textual form has at least one
- Cover the offender (`[]:8776`), bracketed garbage (`[abc]:8776`),
the unspecified (`::`), loopback, and full forms in tests
- Keep `dns` and `ipv4` rows untouched
Migration 8 retyped any `[..]:N` dns row to ipv6 by bracket pattern
alone, without checking the inner part is parseable
- Decode each row independently so one corrupt entry no longer aborts
the whole address-book iterator
- Log a warn with the raw `value` and continue past parse failures
- Add a regression test that mimics the post-migration-8 state where a
legacy `[]:8776` row lands as `type='ipv6'`
A pre-`df8e4e6c` parser admitted `[]:8776` as `HostName::Dns("[]")`.
Migration 8 retyped any `[..]:N` dns row to `ipv6` without validating
the inner part, so on read `Address::from_str` rejects the row with
"invalid IPv6 address syntax", which bubbled up and caused
`available_peers()` to silently return empty, breaking peer discovery
for the affected node.
Update the output of `rad_jj_colocated_patch` so that it passes with the current version of `jj` being used:
```
jj --version
jj 0.35.0
```
Mark `rad_jj_bare` as `ignore` since it currently cannot determine the
RID from the `rad` remote.
The name `print_inline` was used because, previously, the `print`
function name was taken.
This has now changed, allowing the rename of this function to happen.
The name `print` was used because, previously, the `println`
function name was taken.
This has now changed, allowing the rename of this function to happen.
Prevent `rad` from panicking when its stdout is a closed pipe, e.g.
when running `rad config | head`.
The broken pipes are handled at the output boundaries using three
layers of defence:
1. Inner: Route all stdout output through `term::print()` and
`term::print_inline()`, which use explicit `writeln!`/`write!` to
a locked stdout and silently ignore write errors. All `println!`
calls in `radicle-cli` and `radicle-term` are converted.
2. Middle: Catch `BrokenPipe` errors that propagate up via `anyhow`
from subcommands in `main::run()`, and exit cleanly with code 0.
3. Outer: Install a panic hook that intercepts `println!` panics
containing "Broken pipe" (from dependencies like clap) and exits
cleanly, chained in front of human_panic's hook.
To prevent regressions, `#![deny(clippy::print_stdout)]` is added to
`radicle-cli`, requiring all future stdout output to go through the
safe `term::` functions.
Note: `eprintln!` calls (stderr) are not handled, as broken stderr
pipes are extremely rare in practice.
Rust (since 1.62) ignores EPIPE by default (see [Rust #62569]),
causing writes to closed pipes to return `io::ErrorKind::BrokenPipe`
errors. The `println!` macro panics on these errors, producing a
confusing backtrace instead of the silent exit expected of Unix CLI
tools.
To avoid the use of `print!` and `println!`, the `radicle-term`
helper functions use `write!` and `writeln!`. This lays the groundwork
for the `rad` CLI to transition to using only `radicle-term`
functions.
There was a first attempt made, which is documented here. A
process-wide SIGPIPE reset (`SIG_DFL`) was ruled out because `rad`
manages child processes via libgit2 pipes and communicates with the
radicle node over Unix sockets. Resetting SIGPIPE globally caused
`rad` itself to be killed during internal pipe operations (e.g. during
`rad remote add --fetch`), producing flaky test failures.
[Rust #62569] https://github.com/rust-lang/rust/issues/62569
Add tests that verify the `rad` binary does not panic when its stdout
is a broken pipe (e.g., `rad config | head -1`).
Rust (since 1.62) ignores SIGPIPE by default (see [Rust #62569]),
causing `println!` to panic with "failed printing to stdout: Broken
pipe" instead of exiting silently. This is the standard Unix behaviour
expected of CLI tools.
These tests currently fail on `rad config` and `rad self`, and are set to `ignore`, to be enabled when fixed.
[Rust #62569]: https://github.com/rust-lang/rust/issues/62569
Many usages of `Oid::from_sha1` are actually in the context of
generating arbitrary `Oid`s. In these cases, `impl Arbitrary for Oid`
may be used directly.
The length of a SHA-1 object identifier is defined in various locations.
Instead of this redundancy, make the definition in `radicle-oid` public, and
use it in `radicle{-{core,protocol},}`.
A small refactor to move this constant closer to the `enum` that it is
about. An associated constant constant fits more nicely.
Reason for changing the name from `SHA1_*_LEN` to `LEN_SHA1`: As we
think about the object format in the future becoming variable, we
would have that the length of an OID is parameterized by the object
format. So, we imagine a function `fn len(format: ObjectFormat)`, to
be called as `Oid::len(ObjectFormat::Sha1)`. This is the new order
we use.
When the underlying hash function is not important, but only the fact
whether the OID is the zero sentinel, use `Oid::is_zero`. This is more
flexible, as it is forward compatible with the SHA-256 object format.
`fn sha1_zero` can be `const fn`, and since it does not take arguments,
it can just be a `const`. There is no good reason to not have a constant.
Justification for changing the order to first specify "zero" and then
"SHA1": In the future we will support multiple object formats. A
function that produces a zero OID would be parameterized by the object
format then. Imagine `fn zero(format: ObjectFormat)` which would be
called as `Oid::zero(ObjectForma::Sha1)`. In anticipation, we change the
ordering to match.
Because we changed our main domain from radicle.xyz to radicle.dev, we
should also change our systemd credential identifiers.
Handling is move into a new function, which also checks the old names
for backwards compatibility.
With `connect: std::collections::HashSet`, we suffer unpredictable reordering
of values in our test cases.
Use `connect: indexmap::IndexSet` instead, which preserves insertion
order for iteration.
Fixes an intermittent failure caused by a race condition with background
gossip messages.
Previously Eve's `RefsAnnounced` message could be processed by Bob
after the test updated the identity document to include Alice-Laptop
as a delegate. This caused Bob to unexpectedly fetch the issue in the
background, failing the `!has_issue` assertion.
We now explicitly wait for Bob to receive Eve's announcement before
updating the identity document.
Avoid relying on a thread sleep to wait for the nodes to connect.
Instead rely on the connection condition being established by using a polling loop.
The loop is controlled by a number of iterations, panicking if the
number of iterations is exceeded.
Addresses intermittent failures in the `test_connection_crossing` e2e test,
which were particularly prevalent on slower CI environments (such as
`rust:trixie`).
Previously, the test spawned two threads to make Alice and Bob dial each
other concurrently and strictly asserted that the "preferred" peer (the
one with the higher Node ID) would always win the `Outbound` link direction.
However, this assumption is flawed in real-world, OS-level network execution
due to two race conditions:
1. Thread Scheduling: One thread could execute and fully establish a
connection before the other thread even began processing its dial command.
2. Reactor Event Ordering: A node's reactor might wake up and process an
incoming TCP connection from its peer *before* it processes the `Connect`
command sent by the test. When it finally processes the `Connect` command,
it sees a session already exists and skips dialing entirely.
In both scenarios, a true "simultaneous crossing" never occurs. Instead, a
standard sequential connection happens, meaning the link direction is dictated
by whoever dialed first, not by the "preferred" peer logic. This caused the
strict `left: Outbound, right: Inbound` assertions to panic.
To fix this, two changes were made:
- Introduced a `std::sync::Barrier` to synchronize the two test threads.
This forces both threads to wait for each other before calling `.connect()`,
maximizing the probability of a true simultaneous dial.
- Relaxed the final assertions. Because OS-level TCP handshakes and reactor
polling can never guarantee perfect simultaneity, we no longer assert
*which* peer gets the `Outbound` link. Instead, we assert the core invariant:
that exactly one connection is established between the nodes, and that their
link directions are opposite (`s1.link != s2.link`).
Two tests assumed that `refs/remotes/rad/HEAD` would automatically be
crated. It turns out, however, that this only the case if executed
with Git 2.48.0 or newer. Git versions older than 2.48.0 do not touch
'refs/remotes/<remote>/HEAD'.
Git 2.48.0 and newer will update `refs/remotes/<remote>/HEAD` by
default. This can be disabled by setting
`remote.<remote>.followRemoteHEAD` to "never". Thus, with these
versions, setting that configuration to "never" emulates the behaviour
of older versions.
Therefore, to ensure that test results are consistent across Git
versions before and after 2.48.0, the fixture of test repositories
now include setting 'remote.rad.followRemoteHEAD = never'.
The affected tests are adjusted accordingly.
This change can possibly be reverted once Radicle requires usage of Git
2.48.0 or newer.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
Multiple commands take an RID as a command line argument. However, the
handling, which usually involves falling back to detection of the
repository associated with the current directory, is duplicated many
times over.
Introduce `fn rid_or_cwd` which can be used by many commands.
Many commands take RIDs as arguments. Most of them call the
corresponding struct member `repo`, some call it `rid`.
For consistency, name these members `repo` for all commands.
This change is intended to be invisible to the user. That is why,
in some cases, the "long" version of the argument must be renamed to
match `--rid`.
The main change that needs adjustment on our side is that
`impl EcPk for ec25519::PublicKey` changed from `Compressed = [u8; 32]`
to `Compressed = amplify::Bytes32`.
Also remove `impl Deref for PublicKey` to avoid dependents of
`radicle-crypto` depend on a particular implementation, and make the
anonymous member private.
See
<f42396139a>.
`radicle-cli-test` initiates `cargo` builds under the assumption
that the target directory is always within the source directory.
However, the user may have a different idea, specified with the
environment variable `CARGO_TARGET_DIR` (supported by `cargo`, see
https://doc.rust-lang.org/cargo/reference/environment-variables.html).
Add support for `CARGO_TARGET_DIR`, with a fallback to 'target'
at the top of the source tree if `CARGO_TARGET_DIR` is undefined.
The value of `RAD_SOCKET` influences the outcome of `cargo test`.
Users that set this environment variable to, e.g. point `rad` at
a socket at a custom location, were surprised that execution of
tests would interact with their node.
Split `Home::socket` into two functions. `Home::socket_default` is
"pure" and only dependent on the path represented by `Home`. On the
other hand, `Home::socket_from_env` interprets the environment.
Most users will want to use `Home::socket_from_env`.
There is only one type that can really construct canonical references,
which is the the identity document. Remove the corresponding method
from the trait and rename it to `GetRawCanonicalRefs` accordingly.
Separate module for the concern of protecting `refs/rad`.
Note that this also fixes a bug, as previously all refs
*starting with* `refs/rad`, such as `refs/radieschen` were protected.
The method `set_head` does two things:
1. Compute the canonical head and set the default branch to target.
2. Set the symbolic reference `HEAD` to target the default branch.
Split these two concerns into:
1. `set_default_branch_to_canonical_head`
2. `set_head_to_default_branch`
The main motivation behind this change is to get just a little
telemetry information from nodes on the network, namely the version
of Radicle they are running.
This is achieved by rewriting `impl Default for UserAgent` which now
uses the version information provided by the build script also used
in other crates.
Also, a new configuration option `node.userAgent` is added, which
allows users to override the user agent if they so please, or set
the value `null`, which will in turn send the user agent
`/radicle/`, which is not really helpful, and the default prior to
this commit.
Creations of `UserAgent` in the whole workspace is cleaned up. In
order to do that `UserAgent::test` is introduced.
This makes the `radicle-ssh` crate obsolete.
Signed-off-by: Wiktor Kwapisiewicz <wiktor@metacode.biz>
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
The `fxhash` crate has become unmaintained and was a transitive
dependency of `inquire`.
The `inquire` crate moved to use `stdlib` instead of `fxhash`[^0].
Update `inquire` to 0.9.4 to get the latest version of the crate.
`cargo deny check` output:
```
error[unmaintained]: fxhash - no longer maintained
┌─ /home/fintohaps/Developer/heartwood/Cargo.lock:111:1
│
111 │ fxhash 0.2.1 registry+https://github.com/rust-lang/crates.io-index
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
│
├ ID: RUSTSEC-2025-0057
├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0057
├ The fxhash crate is no longer maintained.
The repository is stale and owner is no longer active on GitHub.
Please take a look at [rustc-hash](https://github.com/rust-lang/rustc-hash) instead.
```
├ Announcement: https://github.com/cbreeden/fxhash/issues/20
├ Solution: No safe upgrade is available!
├ fxhash v0.2.1
└── inquire v0.7.5
└── radicle-term v0.17.0
└── radicle-cli v0.19.0
└── radicle-remote-helper v0.15.0
```
[^0]: 3d5b65422a/CHANGELOG.md?plain=1#L72
The `paste` crate is no longer maintained, and the `pastey` fork is a
drop-in replacement.
`cargo deny check` output:
```
error[unmaintained]: paste - no longer maintained
┌─ /home/fintohaps/Developer/heartwood/Cargo.lock:239:1
│
239 │ paste 1.0.15 registry+https://github.com/rust-lang/crates.io-index
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
│
├ ID: RUSTSEC-2024-0436
├ Advisory: https://rustsec.org/advisories/RUSTSEC-2024-0436
├ The creator of the crate `paste` has stated in the [`README.md`](https://github.com/dtolnay/paste/blob/master/README.md)
that this project is not longer maintained as well as archived the repository
## Possible Alternative(s)
- [`pastey`]: a fork of paste and is aimed to be a drop-in replacement with additional features for paste crate
- [`with_builtin_macros`]: crate providing a [superset of `paste`'s functionality including general `macro_rules!` eager expansions](https://docs.rs/with_builtin_macros/0.1.0/with_builtin_macros/macro.with_eager_expansions.html) and `concat!`/`concat_idents!` macros
[`pastey`]: https://crates.io/crates/pastey
[`with_builtin_macros`]: https://crates.io/crates/with_builtin_macros
├ Announcement: https://github.com/dtolnay/paste
├ Solution: No safe upgrade is available!
├ paste v1.0.15
└── (dev) radicle-protocol v0.6.0
└── radicle-node v0.18.0
└── (dev) radicle-cli v0.19.0
└── radicle-remote-helper v0.15.0
```
Each revision now shows a `{base}..{head}` range instead of just the
head. This is meant to clarify when a patch is rebased, and to be
helpful when one might want to observe changes using 'git range-diff'.
Fixes two earlier mistakes:
1. In `fb18083`, `fn null_to_default` was added, which is only
conditionally used if the feature "tor" is enabled, so also
only conditionally compile the function.
2. In `1e13268`, the `impl Arbitrary` for AddressType` was not
properly adjusted for the case where the newly introduced
feature "tor" is disabled.
Instead of passing the signer as an argument to many methods on `Store`,
scope the `Store` itself to a signer.
This further allows to differentiate two different access modes on the
store in `radicle::cob::store::access`: `WriteAs` (which requires signer)
and `ReadOnly` (which does not require a signer).
The caches for issues and patches in `radicle::cob::{issue,patch}::Cache`
are concretised by removing the first type parameter, since it was
specific to issues and patches anyway. This was done in this commit as
it touches very similar usage sites.
Make `Device` less prominent, and instead lean more heavily towards
traits from the `signature` crate, such as `Keypair` and `Verifier`
in addition to `Signer`. Trait bounds regarding `Signer` could be
simplified, but this is left for the future.
In `radicle-cli`, the function `term::cob::patches_mut`, which generates
errors with a hint is used instead of the lower-level `Profile::patches_mut`.
Commands `rad issue cache` and `rad patch cache` now construct a
writeable cache on top of a read-only store.
Many knock-on changes are handled as well, to arrive at a clean state.
The intent to drop outgoing connections is modeled as `Option::None`
which is brittle and easy to miss.
Extend `enum AddressConfig` with a variant that is more explicit.
A timeout of PT3S was empirically found to be too short to allow for the
verification of Signed References, given that this verification is now
potentially much more costly compared to earlier versions.
PT30S should allow fetching reasonably sized histories, even if they
are on feature level "root".
Note that this is not a timeout for the overall fetch process, but
only for inactivity on the channel during an ongoing fetch.
When upgrading Signed References via `fn upgrad_sigrefs`, the value of
`RepositoryInfo::synced_at` was not updated. Thus, the caller could not
continue with migrated repositories as usual.
Now that enough information about Signed References is propagated, such
as the head of Signed References after upgrade, is available, it is
possible to construct `SyncedAt`.
For stable commits, use the same timestamp as the CLI tests do.
This preserves that commits that are created in the Signed References
module appears as "now" in the output.
Instead of repairing downgrades, automatic migration could fail.
This occurs when attempting to load signed references, and a downgrade
is detected while coalescing `RepositoryInfo`.
Similarly to how fetch recovers, automatic migration must also recover
by detecting for the downgrade error, and allowing the migration to
happen in this case.
Teach the `rad sync` and `rad clone` commands to accept the
`--signed-refs-feature-level` option. This option configures that
fetch to use the minimum feature level provided when fetching from
other nodes. This overrides the value of
`node.fetch.signedReferences.featureLevel.minimum`, and should only be
used in scenarios where it is necessary to downgrade the security of a
fetch for backwards compatibility.
To achieve this, the `Handle::fetch` method introduces an optional
parameter for threading through the `FeatureLevel`.
The configuration of the fetch process was only achieved at the setup
of the of the node.
This limits the ability to pass configuration options to the fetch
worker while the node service is running.
This change introduces `radicle_protocol::fetcher::state::FetchConfig`
to allow configuration options to be kept track of. This includes the
previous `timeout` option, but now also includes the
`radicle_fetch::Config`.
The service now produces the `FetchConfig` and passes it through to
the fetch worker.
After feature levels are introduced to Signed References, and downgrade
attack protections are implemented, new errors are encountered when
reading Signed References that are in a downgraded state.
Such downgrade errors are recoverable by advancing history in a way that
restores the feature level that the history requires.
However, the implementation in `radicle-fetch` must be adjusted to not
abort fetching on downgraded histories, but rather inspect the error in
this case and decide whether continuing the fetching process would
actually help recover from the downgraded state.
The two cases are treated slightly differently for non-delegates and
delegates. Since delegates are crucial for evaluating repository
identity, also the fact that they cannot recover from a downgraded state
is logged at the "info" level.
Add a configuration knob `node.fetch.signedReferences.featureLevel.minimum` to
allow node operators to enforce more strict verification when fetching
Signed References.
In order to allow automatic migration to the feature level 'parent',
introduce a way to force writing signed references, even if the
user-controlled refs are unchanged.
To detect the special case of a Signed References history with a single
and root commit (which would otherwise be detected as feature level
'parent', even though the `refs/rad/sigrefs-parent` ref cannot be
written), keep information about the parent commit in `SignedRefs`.
The output of `rad inspect --sigrefs` is changed to match the more
strict interpretation of the feature level for histories with a single
and root commit.
In hopes of nudging users to upgrade, print a warning if older feature
levels are encountered. However, only do this for delegates, as these
are more critical for the repository, to not log too many warnings.
In order for users to be able to inspect the feature level of Signed
References in a repository, have `rad inspect --sigrefs` attempt to
load Signed References, and, if successful, print the feature level.
To prevent from inability to write sigrefs in case of a verification
error, swallow verification errors about the head commit, log, and
carry on writing.
In order to respond to the caller with the correct feature level in
case sigrefs are to be saved but unchanged, the write side needs to
care about feature levels. This is done by sharing more code with the
read side.
To be able to reason about backwards compatibility, downgrade attacks,
and future configurable enforcement of minimum required (security)
features, introduce the concept of a "feature level" in Signed
References.
In this implementation feature levels are assumed to be monotonic, i.e.,
that higher feature levels behave like lower ones, and only add
semantics on top.
Implementing this as an `enum` that is `Ord` flows nicely.
Treatment of the write-side is left for the next commit. Currently,
the write side "promises" `FeatureLevel::Parent`, but is not able
to "keep the promise", since it does not actually inspect the
feature level of the head in case it is unchanged.
CHANGELOG
The scenario A₁ ← B₁ ← A₂ ← B₂ would previously have resulted in A₁
being loaded. This is undesriable, since the latest non-replayed commit
is B₁.
The information regarding the order of duplicates, which is preserved by
`seen`, can be used to recover the first commit.
Due to a bug introduced in `df8e4e6c88a8bfb6c1ec6b07dcda64093b477cbe`, IPv6
addresses in the configuration file might have ended up in the database
tagged with `type = "dns"`, which is incorrect. The bug was fixed in
`a2e72b48e79d090d33f6c13c485239947de0522e`.
However, Radicle 1.7.0 was released in between those two versions, so a
this migration to repair the entries is added.
Tests are added in `test::migration_8` module. This requires some test
setup, mainly a `Database::memory_up_to_migration` constructor that is only
available to test code.
More sigrefs on the network than previously thought actually are missing
the reference `refs/rad/root`. Revert the change to make this an error,
which was part of commit `d3bc868e84c334f113806df1737f52cc57c5453d`.
IPv6 addresses are already persisted in `config.json` files.
The parsing of these old addresses, e.g. `FE80::0202:B3FF:FE1E:8329:5976` should remain valid.
Emit a warning when they are found.
The output of the address will still enclose the host in `[]`.
Previously, trying to load `SignedRefs` for any given remote would
result in a fetch failure.
Teach the fetch to be more resilient by pruning remotes that result in
an error when loading `SignedRefs`, and add the error to the
validation failures.
Split up signed references into its read and write components.
On the write side:
- Preserve the old behavior of writing references to the blob `/refs`
and sign over the blob.
- Ensure `refs/rad/root` is contained in the `/refs` blob.
- Ensure `refs/rad/sigrefs` is *not* contained in the `/refs` blob.
- Introduce a new (internal) reference `refs/rad/sigrefs-parent`
so that no two `/refs` blob are equal, even if they contain
the same set of (non-internal) refs.
On the read side:
- Preserve the verification of the signature in `/signature` and
the reference `refs/rad/root` (if present).
- Fail verification of `refs/rad/root` is not present.
- Protect against replay attacks by walking the history of the
head of `refs/rad/sigrefs`, skipping interpretation of `/refs`
blobs in case they are identical to a previous `/refs` blob.
This is achieved by searching for repeated contents of the
`/signature` blob.
The reference `refs/rad/sigrefs-parent` is never read from or written to
the Git repository in storage.
The pre-existing implementation of signed references did not include
a nonce, thus duplicate but legitimate sets of references could not
be distinguished from maliciously replayed sets of references.
The new implementation uses `radicle-git-metadata` which is moved from
`dev-dependencies` to `dependencies`.
In issue
29c6c6fc8171287faa0079798ba2d6e3e7fd86f3
was noted that it would be nice to use value parsers for the timeouts in
the CLI.
This patch implements this.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Co-Authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Adrian Duke
Fintan Halpenny
Daniel Norman
Josh Soref
Lorenz Leutgeb
:raw`
Lorenz Leutgeb
Richard Levitte
Wiktor Kwapisiewicz
Defelo
Matthias Beyer