Recent versions of Windows support Unix Domain Sockets natively, see
<https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/>.
By using that feature instead of Windows named pipes, the difference
for handling communication via the control socket comparing Unix-like
systems and Windows becomes smaller:
1. No special paths like `\.\pipe\…` have to be handled.
2. Not two I/O mechanisms are abstracted (named pipe and UDS) but
just one.
3. `winpipe` relies on background threads while `uds_windows` does
not.
Once the feature `windows_unix_domain_sockets` (which is tracked at
https://github.com/rust-lang/rust/issues/150487) stabilizes, it will
likely be possible to just drop the dependency `uds_windows` and use
the implementation in `std::os::windows::net` directly.
Command line parsing differs on Windows vs. POSIX compliant operating
systems (just consider the differences in handling `\` for paths vs. as
a marker for escape sequences). Thus, on Windows use `winsplit` to
split command arguments instead of `shlex` for Unix-like OSes.
`winsplit` is a small crate with no other dependencies.
This patch introduces the `FetcherState` which encapsulates the logic
and state for keeping track of fetches.
It uses a sans-IO approach, where the state of fetches transition
based on the current state and provided input.
Callers can then decide to perform I/O based on the returned event.
Introduce the `radicle-core` crate for housing data types that are at
the core of the Radicle protocol. The initial data type being added is
`RepoId`. To make the crate as lightweight as possible many of
dependencies are optional and gated by feature flags.
The crate is marked as `no_std` even though it unconditionally depends
on `radicle-crypto`, which is *not* `no_std`. This is to invite
refactoring of `radicle-crypto` to `no_std` at a later time.
The `localtime` crate was defined by cloudhead, and is a minimal
repository with a single `lib.rs`.
Instead of using it as an external dependency, copy the code directly
into a new workspace crate `radicle-localtime`.
The default `serde` implementation goes through the `LocalTime`'s
seconds values rather than milliseconds, since this is the more common
format. This allows the removal of the `serde_ext` functions.
The one place milliseconds was used was for the
`radicle::cob::common::Timestamp` type, where the `Serialize` and
`Deserialize` implementations are manually written.
It also adds a `schemars` feature to remove `schemars_ext` functions
in `radicle` as well.
Update the above crates to their latest version, which now use MIT OR Apache-2.0 licensing.
This allows us to remove the exception entries in `deny.toml`.
Introduce the use of `clap_completion` for generating static completion.
The shells that are supported are the ones that are included in `clap_complete`:
- `bash`,
- `elvish`,
- `zsh`,
- `fish`, and
- `powershell`.
Co-authored-by: Michael Uti <utimichael9@gmail.com>
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
Update the `gix` family of crates to avoid the vulnerability reported in
[CVE-2025-31130].
Since `gix-hash` is used in two places, its version definition was moved to the
top-level `Cargo.toml`. `cargo` warned that `default-features` not being defined
in the top-level could result in a future error, so that was carried along with
it. This did not affect the build of `radicle-fetch`.
[CVE-2025-31130]: https://www.cve.org/CVERecord?id=CVE-2025-31130
Make `git2` an *optional* dependency of `radicle-cob`, and refactor
`radicle` to depend on crates that in turn do not depend on `git2`
*non-optionally*
The main offending dependency of `radicle-cob` is `radicle-git-ext`
from the `radicle-git` workspace in repository
(rad:z6cFWeWpnZNHh9rUW8phgA3b5yGt) which *non-optionally* depends on
`git2`.
So, to achieve removal of this dependency:
1. The crate is refactored to depend on the new crates
`radicle-git-ref-format` `radicle-git-metadata`, and
`radicle-oid` introduced in the previous commits, instead of
`radicle-git-ext`.
2. Some code from the `radicle-git-ext` crate in the `radicle-git`
workspace in repository (rad:z6cFWeWpnZNHh9rUW8phgA3b5yGt) is
copied. See `crates/radicle-cob/src/backend/git/commit.rs`.
This cascades to `radicle` and its dependents.
Firstly, the there is an
`impl Deref<Target=git2::Oid> for radicle_git_ext::Oid`. This made
it very convenient to just deref to obtain the wrapped `git2::Oid`,
so there are many expressions of the shape `*oid` in `radicle` and
its dependents. However `radicle-oid` does not provide
`impl Deref<Target=git2::Oid> for radicle_oid::Oid`, as notably,
`Target` is an associated type, and not a type parameter, so an
implementation of `Deref` would tie the new `Oid` too tightly to a
particular implementation.
Instead, work with `impl From<radicle_oid::Oid> for git2::Oid`
(which can be enabled using the feature flag `radicle-oid/git2`).
This explains the changes from `*oid` to `oid.into()` at every
transition to "`git2` land".
Secondly, `radicle` and its dependents are refactored to also depend
on `radicle-git-ref-format` and (much less prominently)
`radicle-git-metadata` instead of `radicle-git-ext`
This is to avoid pulling in `git2` via these dependencies.
Thirdly, as the re-exports in `crates/radicle/src/git.rs` change,
they are at the same time also cleaned up. Notably, the types from
`radicle-git-ref-format` are re-exported under `fmt` only, not "twice".
While overall this obviously is very much a breaking change, these
changes should mostly amount to changing from `Deref` to `Into`, i.e.,
`*oid` → `oid.into()` and rewriting imports for `radicle::fmt`.
This is indicated by the mostly mechanical nature of the changes to
`crates/radicle-{cli,node,remote-helper}`.
This is just enough to get heartwood going.
Some code from the `radicle-git-ext` crate in the `radicle-git`
workspace in repository (rad:z6cFWeWpnZNHh9rUW8phgA3b5yGt) is
copied.
For further details on the crate see its documentation in
`crates/radicle-oid/src/lib.rs`.
Changes to `crates/radicle{,-crypto}/Cargo.toml` are to preserve the
default features of their dependencies `serde` and `schemars`,
which are disabled to support `radicle-oid` being `no_std`.
The dependencies `netservices`, `io-reactor` and `popol` served us well,
however they do not support Windows and are not actively maintained.
This change removes the aforementioned dependencies (and `libc` along
with them). It reuses the integration with `cyphernet` from
`netservices` for Noise and SOCKS.
The new module `reactor` is a rewrite of `io-reactor` on top of Mio.
Note that no tests were changed.
The `git-ref-format-core` crate is enough to implement everything
required by `radicle-crypto`, and has a much smaller dependency
footprint than `radicle-git-ext`. Notably, it does not depend on
`git2` and also does not depend on procedural macros.
Register a panic handler that logs a backtrace via the `backtrace`
crate, instead of setting `RUST_BACKTRACE`.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xy>
Optionally depend on crate `structured-logger`. Allow enabling it via
`--log-logger structured`. For future compatibility, provide
`--log-format` which currently only supports JSON.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
As a library crate, it is bad to return such generic errors.
To get there, errors from `inquire` are downcasted.
This allowed to clean up the public interface of `radicle-term`, so that
it does also not leak the `inquire::InquireError` in its public API.
`anyhow` is only used to capture a handful of errors. It also turns out that
usage of `lexopt` was wrong, since it features `parse_with`, which neatly
integrates with `FromStr`. The types, luckily, implement this trait already –
making for cleaner parsing.
For now, the execution errors are all "transparent", which shouldn't be
much of a regression, if at all.
In many tests we assert that encoding and decoding gives the same
object. Let's call that a "roundrip", and also provide a proc macro to
generate such tests.
As `std::os::unix` is obviously not available on Windows, resort to the
`winpipe` crate, which implements a very similar API for named pipes.
Apart from simple changes to imports, we need to jump through a few
hoops because the `std::io::{Read, Write}` impls are on `&mut` for
`winpipe`, thus yielding different mutability requirements on Unix vs.
Windows.
We resort to cloning, but as this fallible, swallow the added complexity
of a platform-dependent implementation to not risk panics on Unix.
Microsoft Windows has Universal Naming Convention (UNC) paths, see
<https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#unc-paths>
The Rust standard library normalizes paths to this form when calling
`std::fs::canonicalize`, see
<https://doc.rust-lang.org/std/fs/fn.canonicalize.html>
However, some programs, do not parse these paths correctly, including
our ally `git`:
$ git clone \\?\C:\Users\lorenz\tmp
Cloning into 'tmp'...
ssh: Could not resolve hostname \\\\?\\c: No such host is known.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The `dunce` crate solves the problem of having to deal with
normalization and avoiding weird UNC paths for us. Note that on
non-Windows platforms, it *behaves exactly like*
`std::fs::canonicalize`!
The upgrade of the `radicle-crypto` crate was missed when upgrading
semver versions.
Unfortunately, since it is at the base of the dependency tree, all
crates that depend on it require an additional upgrade.
This is morally a revert of 70fb0d3fef
while additionally getting to compile
`crates/radicle-term/src/editor.rs` on Windows.
To get the changes that were made on top of the revert, check
git diff 70fb0d3fef~1 <this commit> -- crates/radicle-term/src/editor.rs
There were reports of the inquire editor being unrealiable.
Introducing the new enum `TerminalFile` to model which standard stream
stream the terminal uses allows us to remove the dependency on `libc`
which does not build on Windows.
Mostly changes of `use`s and moving components to `radicle-protocol`,
with the goal of just getting `radicle-node` to work on top of the new
`radicle-protocol` crate.
Introduce the `rules` module beneath `canonical`. The implementation follows
the Canonical References RIP[^0].
A `Rule` defines the DIDs and threshold required to get the canonical tip
for a reference (or set of references). Since a `Rule` has some specific
validation rules, there can be two versions of a `Rule`. The `RawRule` is one
that can be easily constructed, serialized, and deserialized. It must then be
validated into a `ValidRule`, which validates any constraints, and can be used
in the computing of a canonical tip.
For convenience, a set of `ValidRule`s is wrapped into the `Rules` type, which
also has a further constraint that each rule refers to a specific `refspec`.
[0]: 1d1ce874f7
Prepare the crates for releasing to crates.io:
- radicle-remote-helper: 0.10 -> 0.11
- radicle-term: 0.12 -> 0.13
- radicle-cli: 0.13 -> 0.14
- radicle-node: 0.11 -> 0.12
- radicle-fetch: 0.11 -> 0.12
- radicle: 0.15 -> 0.16
It was also necessary to specify the `version` in the workspace Cargo.toml file,
since this is required for publishing to crates.io.
While running `cargo-semver-checks`, there was an issue that was surfaced
regarding `serde(into)`, `serde(try_from)` and `schemars(transparent)` being
used, and they are incompatible.
This appeared to be a semver issue, on the side of `schemars`, since the issue
would only appear when adding `radicle` as a fresh dependency. The intuition for
this is that once `schemars` was updated locally, the same error appeared when
attempting to build the `radicle` crate.
To avoid this, `schemars` provides a `!` operator for the macro[^0]:
> You can also “unset” serde attributes by including them with a ! prefix in a
> schemars attribute, which will make schemars ignore the corresponding serde
> attribute item
[^0]: https://docs.rs/schemars_derive/latest/schemars_derive/derive.JsonSchema.html#attributes
The `Review::summary` was limited in behaviour, since it was only an
`Option<String>`.
The field is now changed to `NonEmpty<Edit>` so that it can enable an edit
history, as well as being able to supply embeds as part of the text.
`Reactions` are also introduced to `Review` and, necessarily, introducing an
`Action::ReviewReact`.
Note that some machinery was added under `actions` and `encoding` to enable
backwards-compatibility.
For `actions`, a new `ReviewEdit` is introduced to keep
track of the different versions of editing actions – supporting the old actions
while also introducing the new variant.
For `encoding`, the machinery exploits the fact that an `Option<String>` can be
converted into a `NonEmpty<Edit>` given that an `ActorId` and `Timestamp` are
available, and that `None` can be converted into an empty string. The struct
`patch::Review` now deserializes via this `encoding::review::Review` type and,
importantly, preserves backwards-compatibility.
The `radicle-tools` binaries were specialised binaries for early development of
the `heartwood` set of crates. Choose to remove the crate so that it no longer
needs to build as part of our workspace.
The `radicle-crdt` crate is no longer used by any of the other crates. Choose to
remove the crate so that it no longer needs to build and test as part of our
workspace.
Relax the set of radicle-git depedencies to allow for patches. In
particular, the dependencies introduced some small improvements with
`0.*.1` updates.
All dependencies that are dependencies of multiple crates within the
workspace were pulled up to the root `Cargo.toml`. Dependencies that are
dependencies of just one crate within the workspace remain in the
`*/Cargo.toml` of that particular crate.
Generally, this helps us keep an overview of dependencies:
- If multiple crates depend on a crate, we will see the reference to
the workspce.
- Otherwise, we will see a line that specifies the version.
It is up to the maintainers to recognize when a dependency becomes a
dependency of multiple crates in the workspace, at which point it should
be moved to the workspace.
The tool `cargo-autoinherit` can be used to automatically pull *ALL*
dependencies to the workspace. I used this, and in a separate, manual
step, filtered out those dependencies that are dependencies of a single
crate only, moving those back manually.
At the same time, I also unified depending on sibling crates in the
workspace. This is now also done via workspace dependencies.
See:
- https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#inheriting-a-dependency-from-a-workspace
- https://github.com/mainmatter/cargo-autoinherit
Extracting JSON Schemas from the Radicle crate can be done via
`radicle-cli` for `radicle::profile::Config` by executing:
rad config schema
However, for other JSON Schema metadata, this is not possible.
To allow other tools to extract JSON schemas as part of their build
process, introduce a new crate that only depends on `radicle`,
`schemars`, and `serde`, with a tiny binary that will reproduce various
schemas.
This prepares the crate versions for a new release. The version bumps were
determined by the `cargo-semver-checks` tool.
However, there were some issues with the `radicle-fetch` crate – where when
attempting to check it, it was unable to compile due to the previous version
expecting the kind of `Doc` with a generic parameter. I believe this is due to
the version for `radicle` being specified as simply `0`. This prompted changing
the version specifications to use `major.minor` for all the internal crates.
This, for example, would mean that `radicle-fetch` that relies on `0.15` can use
`>=0.15.0,<0.16.0` – allowing any patch versions but must require a minor
version bump.
Leverage `schemars` to generate a JSON Schema from our structs for
configurations and those occurring within them.
The output of `rad config schema` can be used by editors to provide a
smoother editing experience for the configuration file. Discovery
attributes (both keys and values) is greatly improved with the schema
helping in the background.
Refer to:
- https://json-schema.org
- https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings
- https://schemastore.org
The motivation of this change is to move away from tying signing to be
specifically for ed25519. The reason being that the protocol will want move
towards two different kinds of signing – the node signing artifacts, and an
author signing artifacts.
This change captures the former, node signing, by introducing a `Device` type
that is the `NodeId` – a PublicKey underneath the hood – and a signing
mechanism. This allows the replacement of the `Signer` trait being used – which
always assumed a `PublicKey`. Instead, the `Device` is constructed with
`NodeId`.
In `radicle-cob`, a signer is expected to implement
`signature::Signer<ExtendedSignature>`, and everywhere in `radicle`,
`radicle-node`, `radicle-cli`, and `radicle-remote-helper` is expected the
signer is expected to implement `signature::Signer<Signature>`.
A `Device` implements both of these but only requires
`signature::Signer<Signature>` to do so – since an `ExtendedSignature` is
essentially `(PublicKey, Signature)`, and the `NodeId` of the `Device` can be
used.
This introduces rate limits for the `ChannelReader` to limit DDoS attacks and
attempts to upload repositories that are larger than a node is will to permit.
The limiter sets the total number of bytes it is will to accept in a single
exchange, defaulting to 500MB. This means that initial fetches will prevent
large repositories, but is plenty for new packfile data to be sent in subsequent
fetch exchanges.
The limit can be configured within the node's config file, under the limits.
This resolves issues:
- 111cf90 (`handler` is not async-signal-safe)
- c8e0300 (`libc::signal` should not be used)
- b91c6c3 (Mutex can cause signals to be missed)
See also: patch 8e05702.
The API of `radicle-signals` is the same as before, and so uses of it
don't need to be and aren't changed. The behavior is slightly different
in that:
- If the channel is full then signals will not be lost, which is an
improvement. This is achieved without blocking in the signal handler.
This is possible because of the counters approach along with the
internal receipts-processing thread of the `signals_receipts` crate.
- `install()` and `uninstall()` might block very briefly if necessary to
acquire the mutex, which is now internal to and managed by the
`signals_receipts` crate, only if there are concurrent calls to them
(which is unlikely), but such blocking is guaranteed to be bounded to
be very brief. This is done so they no longer can fail to do their
purpose, which is an improvement. They still return errors if the
handling is already installed or uninstalled, respectively, which
preserves the previous use cases.
- The new `finish()` function is introduced. This is provided in case
it's ever needed to completely clean-up the facility, by terminating
the internal receipts-processing thread, to be like it hadn't been
installed before.
- The user must ensure that the notifications channel is disconnected,
by dropping the receiver(s), when doing `uninstall()` or `finish()`.
Such dropping usually occurs naturally, and already occurs for all the
preexisting uses of `radicle_signals`. (The `signals_receipts` crate
is capable of a more robust approach, but this commit doesn't use
that, to avoid changing the preexisting uses of `radicle_signals`.)
- The `TryFrom` impl for `Signal` is of `SignalNumber` which is `c_int`,
instead of `i32`, because `c_int` (the type of signal numbers) might
not be `i32` on all platforms (POSIX only requires `c_int` to be at
least 32-bit).
The version of `radicle_signals` is incremented, to reflect those
changes and the substantially different internal implementation.
The new dependency on the `base64` crate is needed by the
`signals_receipts` crate for it to work on macOS, because its dependency
on the `sem_safe` crate uses `base64` as part of creating anonymous
semaphores on macOS (which lacks support for unnamed semaphores (in
violation of POSIX)).
Signed-off-by: Derick Eddington <kcired@pm.me>
Signed-off-by: Lorenz Leutgeb <lorenz@leutgeb.xyz>
Version 0.5.13 was yanked, resulting in the following warning when
building with --locked:
warning: package `crossbeam-channel v0.5.13` in Cargo.lock is yanked
in registry `crates-io`, consider running without --locked
The nix flake's checks were failing for various reasons. The following fixes are
applied to get it up to date again.
- Update to release-24.11, since `crane` requires it – also to keep up-to-date
- Add `.json` to the source filters to allow for testing patch migration
- Ensure that the `rad` and `git-remote-rad` binaries are built for the nextest
check. N.b. the binaries need to be in the target dir of `radicle-cli` since
that is where the nextest check runs from.
- Add `cargo-audit` and `cargo-deny`, to the devShell, to allow for running those
tools locally
- Bump `gix-path` to 0.10.11 to avoid a vulnerability[^0]
[^0]: https://rustsec.org/advisories/RUSTSEC-2024-0371
Extend the ranges of emojis that are supported, listed
[below](#new-emojis-supported).
To ensure that single `char` emojis are supported, the `emojis` crate
is added as a dev dependency, using it to check that a `Reaction` can
be constructed for the two groups `SmileysAndEmotion` and
`PeopleAndBody`.
### New Emojis Supported
'🩵' 0x1FA75
'🩶' 0x1FA76
'🩷' 0x1FA77
'🫀' 0x1FAC0
'🫁' 0x1FAC1
'🫂' 0x1FAC2
'🫃' 0x1FAC3
'🫄' 0x1FAC4
'🫅' 0x1FAC5
'' 0x1FAC6
'🫠' 0x1FAE0
'🫡' 0x1FAE1
'🫢' 0x1FAE2
'🫣' 0x1FAE3
'🫤' 0x1FAE4
'🫥' 0x1FAE5
'🫦' 0x1FAE6
'🫨' 0x1FAE8
'' 0x1FAE9
'🫰' 0x1FAF0
'🫱' 0x1FAF1
'🫲' 0x1FAF2
'🫳' 0x1FAF3
'🫴' 0x1FAF4
'🫵' 0x1FAF5
'🫶' 0x1FAF6
'🫷' 0x1FAF7
'🫸' 0x1FAF8
Ensures dead connections are not kept around longer than necessary.
Note that the ping/pong mechanism is more expensive and checks that the
session is active and healthy, while TCP_KEEPALIVE is lower level,
cheaper, and checks that the connection is still alive.
Upgrade the family of `gix` crates and resolve any breaking API changes.
Fixes RUSTEC-2024-0335[[0]].
[0]: https://rustsec.org/advisories/RUSTSEC-2024-0335.html
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Emit the `UploadPack` events from the `upload_pack` method.
A wrapper type, `Reporter`, is introduced for implementing a `Write` instance
that will parse the given bytes as `gix_protocol::RemoteProgress`, if possible,
and emit it via the `Emitter`, and then write the bytes to the underlying
writer. This allows any subscribers to see the progress of an upload-pack, i.e.
`Compressing objects`, `Counting object`, `Enumerating objects`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This is a pager (like `less`) that takes an `Element`, and dynamically
re-renders the element when the terminal size changes. This ensures that
the content is always properly wrapped and rendered.
Many (most?) system clocks are not monotonic, and even though we're
getting a `LocalTime`, which *is* monotonic when using `now()`, it is
constructed from a `SystemTime` which isn't.
Lorenz Leutgeb
Fintan Halpenny
Sebastian Martinez
Erik Kundt
Matthias Beyer
Tobias Hunger
n4ch7
Sebastian Thiel
Derick Eddington
Ivan Stanković
Lorenz Leutgeb
cloudhead
Alexis Sellier
Dr Maxim Orlovsky
