On Windows, all attempts to clone repositories failed with
Fetch failed for rad:… from z6Mk…: Access is denied. (os error 5)
The reason is that, other than Unix-like systems, it forbids that
directories that are in use are moved.
To improve the situation, take back control over what is moved and
removed exactly by implementing `cleanup` instead of relying
on `impl Drop for tempfile::TempDir`.
A new type `TempRepository` is introduced to capture a repository that can be
used temporarily, and either cleaned up on failure, or moved on success.
This `TempRepository` can be constructed using `Storage::temporary_repository` –
renamed from `Storage::lock_repository` since it is only creating a temporary
resource and not locking it.
In the future (once Rust 1.89 is a little less cutting edge and more
widely available), we may opt for actual locking via
`std::fs::File::lock`.
Allow the fetch interface to accept anything that implements
`AsRef<Repository>`. This allows flexibility in the types that the `Handle` can
accept.
This change is motivated by wanting to introduce a type that is a temporary
repository that wraps a `Repository`.
With this change, the location of the secret SSH key can be configured
through `${RAD_HOME}/config.json` so that the node key does not have to
be placed under `${RAD_HOME}/keys` anymore.
Further, there is now an option to override `config.json` directly when
executing `radicle-node` via the command line argument
`--secret`.
The primary motivation is more flexible deployments, for example
leveraging external secret management solutions, like
<https://systemd.io/CREDENTIALS/>.
The secret key is fingerprinted by taking the fingerprint of the public
key corresponding to the secret key. This allows to detect when the
secret key changes.
When using Jujutsu and a non-colocated Git repository, the detection
using `git2` directly fails (just as `git` in a shell would fail)
because there is no `.git` directory found by traversing up the
filesystem hierarchy.
Add an invocation of `jj git root` as a fallback.
When run with a secret and public key that do not cryptographically
match, `fn radicle_cli::terminal::io::signer` would prompt the user
for a password to unlock the secret key, and then carry on with a
mismatching key pair.
Fix this by verifying that the keys match in `MemorySigner::load`
and only swallow errors initializing the signer in cases where
prompting for a password makes sense. It does not make sense in the
case of mismatched keys.
Previously it was possible to create an issue with an empty description
(i.e. an empty "root" comment), and also to change it later to a
non-empty string, however it was not possible to clear it again.
`radicle::run` interprets the `Output` returned by the `git`
process. However, depending on the application we have different
requirements for interpreting the output, e.g., how to handle errors.
Make `radicle::run` not interpret `Output`, but move this
logic to the respective binary crates (`radicle-remote-helper` and
`radicle-cli`).
The `verbosity` option is ignored, which makes it more difficult to
obtain debugging information if the "internal" push, i.e., the
`git push` invocation by `git-remote-rad` fails.
Provide two types for verbosity, and wire them up.
Ideally, we would like to use a name that is compliant with RFC 2606,
such as "radicle.example.com", but this would change *lots* of hashes,
which is not worth the hassle.
Instead, make changing this in the future as easy as changing one
constant.
This makes the default Radicle Explorer URL configurable at compile-time
via the environment variable `RADICLE_EXPLORER`.
In order to stay backwards compatible, we still default to
"app.radicle.xyz"
As a library crate, it is bad to return such generic errors.
To get there, errors from `inquire` are downcasted.
This allowed to clean up the public interface of `radicle-term`, so that
it does also not leak the `inquire::InquireError` in its public API.
Whenever the node fetches new updates, it checks if canonical references can be
updated. The node has learned how to return these results and emit them as node
events. This is a breaking change since it adds a new variant the `Event` type,
which is not forwards-compatible.
This change was inspired by the a story as old as time:
𝕃𝕖𝕥 𝕦𝕤 𝕞𝕚𝕩 𝕠𝕦𝕣 𝕓𝕦𝕤𝕚𝕟𝕖𝕤𝕤 𝕝𝕠𝕘𝕚𝕔 𝕨𝕚𝕥𝕙 𝕠𝕦𝕣 𝕀𝕆!
It was motivated by the fact that the canonical quorum logic was spread across
two modules and also two different repositories (in the API sense). The
`radicle-remote-helper` contained special logic for computing the quorum, and
also relied on the logic with `radicle` itself. It would also mix using the
Radicle storage repository and the working copy repository – resulting in issues
where objects could exist in one and not the other.
The change begins by separating away the IO away from any of the business logic
in the `git::canonical` module. To follow along, there are two important submodules:
1. `voting` captures the different type of voting processes for commits and tags
a. Tags are simple, where one `Oid` means one vote
b. Commits are slightly more complicated. They begin with one `Oid` means one
vote, but then it is expected that merge bases are calculated for pairs of
commits. These merge bases are used to increase a vote for an `Oid` if it
is the merge base of another commit.
2. `quorum` builds on top of `voting` and uses the voting processes as well as
the `threshold` to find the quorum for the tag or commit reference.
a. For tags, the first past the threshold wins, but if multiple pass then it
is an error.
b. For commits, the merge base process should be used to increase the votes,
until the caller is ready to find the quorum. At this point, the commits
that pass the `threshold` are then compared to find the commit that is the
child-most commit of all other candidates. If they diverge, then an error
is returned.
There is also a `convergence` module that captures the logic that is required
for the `radicle-remote-helper`. It essentially checks if a candidate object
matches the expected objects, tags or commits, and performs the necessary
convergence logic – checking that the candidate commit is converging with
at least of the other `Did`s.
These two quorum processes, and the convergence process, essentially act as
state machines and can be driven by the use of a Git repository for finding the
merge bases. This is where the `effects` module comes in. The `effects` capture
the necessary traits that are required to drive the state machines of the quorum
processes. It is expected that a Git repository implements these, and in fact, a
`git2::Repository` implementation is provided. The traits are useful, since it
means that a `git2::Repository` can be swapped out and the logic would stay the
same.
This all culminates into the new and improved `Canonical` and
`CanonicalWithConvergence`. Both of which have methods `find_quorum` for
performing the quorum process using a *single* provided repository.
This resulted in the semantic change of requiring that the
`radicle-remote-helper` pushes the candidate commit, irregardless of whether it
will be a fast-forward. For now, this is something that will be accepted while
the UX can be improved in the future by providing detailed warnings of the
divergence and ways to fix it. The benefit is that the tooling will never stop
someone from diverging if that is in fact what they want to do.
I was greeted by `rad patch redact` with
called `Result::unwrap()` on an `Err` value:
Error { code: None, message: Some("failed to convert") }
stack backtrace:
[…]
3: core::result::Result<T,E>::unwrap
at …/rust-1.88.0/lib/rustlib/src/rust/library/core/src/result.rs:1137:23
4: sqlite::cursor::Row::read
at …/index.crates.io-1949cf8c6b5b557f/sqlite-0.32.0/src/cursor.rs:136:9
5: radicle::cob::patch::cache::query::find_by_revision
at ./crates/radicle/src/cob/patch/cache.rs:624:65
6: <… as radicle::cob::patch::cache::Patches>::find_by_revision
at ./crates/radicle/src/cob/patch/cache.rs:553:9
7: radicle_cli::commands::rad_patch::redact::run
at ./crates/radicle-cli/src/commands/patch/redact.rs:23:9
8: radicle_cli::commands::rad_patch::run
at ./crates/radicle-cli/src/commands/patch.rs:1026:13
[…]
It turns out that `sqlite::cursor::Row::read` is the panicky version of
`sqlite::cursor::Row::try_read` (which returns a `Result`).
While it is somewhat rare that SQLite reads fail, it is not unheard of.
In `radicle-cli` it might not be critical, but also `radicle-protocol`
and `radicle-fetch` are affected, and they could potentially panic a
`radicle-node` process.
Use `try_read` instead, and propagate down the error handling.
The `winpipe` crate which is our current best bet for support on Windows
requires that named pipes are at the magical location `\\.pipe\`.
Provide a default on Windows that matches this, and use our freshly
assigned IANA service name for it.
This is a breaking change, as we remove a `pub const`, it should be
reasonably easy to fix by dependents, and probably it was a mistake to
make the default `pub const` as dependents should call `Home::socket`
anyway.
Microsoft Windows has Universal Naming Convention (UNC) paths, see
<https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#unc-paths>
The Rust standard library normalizes paths to this form when calling
`std::fs::canonicalize`, see
<https://doc.rust-lang.org/std/fs/fn.canonicalize.html>
However, some programs, do not parse these paths correctly, including
our ally `git`:
$ git clone \\?\C:\Users\lorenz\tmp
Cloning into 'tmp'...
ssh: Could not resolve hostname \\\\?\\c: No such host is known.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The `dunce` crate solves the problem of having to deal with
normalization and avoiding weird UNC paths for us. Note that on
non-Windows platforms, it *behaves exactly like*
`std::fs::canonicalize`!
The stream tests did not take into account that the tips of previous COB entries
need to be inserted so that a Git history can be formed.
This was surfaced by the added regression test.
If the tests are run without a global Git config, then the tests will error with
no name or email set for the commits.
Instead, use the test fixture setup to ensure there is a name and email for
running the tests.
This patch adds the `cob::common::Title` struct, this allows instead of
using `std::str::String` or generics and traits for it, to define more
precisely what a title should be.
It trims the provided string and makes sure it contains no carriage
return or new line characters.
Where a `std::str::String` makes more sense so it's easier to mutate it,
we keep the code as is.
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Add type aliases to specialise the `Stream` type for each of `Patch`, `Issue`,
and `Identity`.
An `init` constructor is also added for each to make it easier to construct each
stream.
Introduce a new `stream` module to `radicle::cob`. The purpose of this module is
to provide an API for iterating over a COB's operations, given a range of
commits.
The `CobStream` trait provides the generic API for implementing and using, while
`Stream` provides a concrete implementation using the `git2` crate.
The approach is to use a `git2::Revwalk` for walking over each commit in the
range, then loading the `Op` for that commit – skipping any commits that are
unrelated to that COB type, in the case of associated COBs.
The `Stream` is tested using some properties of the API.
The next steps are to introduce specialised types for `Patch` and `Issue` to
ensure that it works for those.
Adds an `Op::load` helper method for loading an `Entry`, given its `Oid` and the
`store` it is stored in.
This can be useful for downstream consumers to inspect operations on COBs given
a single `Oid`, without having to load the entire object and/or graph.
It can also be useful when looking at implementing COB stream primitives.
Add a helper method `Op::manifest_of`, to allow the caller to inspect the
`Manifest` of the entry. This can be used to avoid attempting to call `Op:load`
with a manifest that would not match the expected `Action`s of the `Op`.
This patch adds an implementation of the Iterator::size_hint() trait
function so that the compiler can optimize allocations on the collecting
side of the iterator better.
From the trait function docs:
> size_hint() is primarily intended to be used for optimizations such
> as reserving space for the elements of the iterator, but must not be
> trusted to e.g., omit bounds checks in unsafe code. An incorrect
> implementation of size_hint() should not lead to memory safety
> violations.
Source: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.size_hint
And:
> Vec may use any or none of the following strategies, depending on the
> supplied iterator:
>
> * preallocate based on Iterator::size_hint() and panic if the number
> of items is outside the provided lower/upper bounds
Source: https://doc.rust-lang.org/stable/std/iter/trait.FromIterator.html
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
When initializing the `radicle-node` process, see
`radicle-node/src/runtime.rs`, an attempt is made to read from a node
announcement file.
This file is never written, therefore all of these reads fail.
Even though this caching mechanism might make sense or might have made
sense at some point, it is dead code now, and therefore removed.
Also, the fact that the file name was defined in the `radicle` crate is
ugly.
Ensures that preferred seeds that are not synced are made part of the `to_sync`
set, and the Announcer is constructed without an error.
As well as this, ensure that the `synced` and `to_sync` sets do not overlap by
constructing the `to_sync` and removing any `synced` elements from it.
Making a note of that `Announcer::can_continue` might never be able to hit the
`Break` case. It could be defensive programming, so I'm going to leave it in for
the time being.
Adds a test case to show that the Announcer is allowed to mark an "unknown" node
as synced.
There may be a case where gossip reached a node that the current node was not
aware of. This should be allowed and should count towards the final replication
factor.
The `Announcer` logic was subtly different to the fetching logic, where the
preferred seeds need to be announced to *and* the replication factor had to be
reached.
This patch relaxes that constraint, and allows it to hit either targets before
exiting.
This also surfaced an issue with the with the `Progress` calculation, where
calculating the `unsynced` used the `synced` value to subtract. However, this
was not required because `to_sync` is mutated, so the value is simply
`to_sync.len()`. A test case was added to ensure this is correct by checking
that invariant that the total between synced and unsynced nodes always matches
the total of synced and unsynced progress.
The upgrade of the `radicle-crypto` crate was missed when upgrading
semver versions.
Unfortunately, since it is at the base of the dependency tree, all
crates that depend on it require an additional upgrade.
The logic for computing canonical references conflated the semantics of
annotated and lighweight tags, yielding confusing/wrong results. The
main culprit was the call to `peel_to_commit` in
`impl ReadRepository for Repository`, peeling tag objects away.
To resolve this, we separate out quroum computation per object type: one
implementation for commits, and one for tags.
Also move move canonical error types and methods into their own
module to have a cleaner file structure for the main logic.
Capture the `BTreeMap<Oid, u8>` type into a `Votes` struct, with its own API.
We were getting the objects from the repository twice – once in `Canonical::new`
and once again in `ensure_commit_or_tag` – so the latter was removed.
Use an `enum CanonicalObject` to specify which object types the canonical
process supports.
During `converges`, separate commits and tags, and ensure that we are only
looking at objects of one type. Note, that Fintan think this is actually the
*wrong* place to do it because we already filter the objects – so we should
keep track of that information higher up the callstack.
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
After upgrading from Rust 1.85 to 1.88 in 586eefc, clippy started
warning about large error types. Box action to make
`radicle::cob::patch::Error` lighter.
To avoid that users on Windows have to explcitly set `HOME`, also accept
`USERPROFILE` which is considered equivalent.
To avoid repeating to join `.radicle`, split the logic up into two
phases. Also take care to provide a good error message, as this error is
fatal and user-facing.
When using `rad init` the default branch for the project can be predetermined by
inspecting the working copy repository.
To improve the experience, it will first check `init.defaultBranch` in the Git
configuration file.
If it does not find this, it falls back to choosing the default head of the
repository at the time. This may be HEAD, and in this case it will error and
provide helpful information.
The repository may also not have any default head at the time, and this also
results in an error and hint.
This set of changes refactors the canonical handling by isolating the business
logic into another type, `Canonical`.
The functionality remains the same, but certain error scenarios are better
handled. Primarily, the commits of the inner `Canonical` type are checked to see
if they exist in the working copy, and if not, provide a more useful message
about which commit was missing or invalid for which DID.
Pay more attention to when a Node ID is displayed in compact form. As
this was sometimes used to save space, e.g. in `rad node status` in
combination with long node addresses (like .onion names), also implement
a compact format for node addresses and use that instead in some cases.
Progress notifications in spinners keep the short form.
Ensure that the default branch is only ever synthesized from the `Doc`'s
`threshold` and `delegates` value. If the rule is ever written into a `RawDoc`,
then the update of the that `RawDoc` must go through the `update::veryify`
function, which checks that the rule is not present.
The business logic of modifying the identity document should not live in the CLI
code for updating said document.
Instead, we extract out that logic into separate update functions and these are
using from the CLI instead.
Introduce the `rules` module beneath `canonical`. The implementation follows
the Canonical References RIP[^0].
A `Rule` defines the DIDs and threshold required to get the canonical tip
for a reference (or set of references). Since a `Rule` has some specific
validation rules, there can be two versions of a `Rule`. The `RawRule` is one
that can be easily constructed, serialized, and deserialized. It must then be
validated into a `ValidRule`, which validates any constraints, and can be used
in the computing of a canonical tip.
For convenience, a set of `ValidRule`s is wrapped into the `Rules` type, which
also has a further constraint that each rule refers to a specific `refspec`.
[0]: 1d1ce874f7
Downstream clients require access to the data for a `ReviewEdit` – for example,
the desktop client builds a representation of `Action`s for TypeScript
generation.
To allow these kinds of use cases, methods are added to `ReviewEdit` to allow
access to its underlying data.
Prepare the crates for releasing to crates.io:
- radicle-remote-helper: 0.10 -> 0.11
- radicle-term: 0.12 -> 0.13
- radicle-cli: 0.13 -> 0.14
- radicle-node: 0.11 -> 0.12
- radicle-fetch: 0.11 -> 0.12
- radicle: 0.15 -> 0.16
It was also necessary to specify the `version` in the workspace Cargo.toml file,
since this is required for publishing to crates.io.
While running `cargo-semver-checks`, there was an issue that was surfaced
regarding `serde(into)`, `serde(try_from)` and `schemars(transparent)` being
used, and they are incompatible.
This appeared to be a semver issue, on the side of `schemars`, since the issue
would only appear when adding `radicle` as a fresh dependency. The intuition for
this is that once `schemars` was updated locally, the same error appeared when
attempting to build the `radicle` crate.
To avoid this, `schemars` provides a `!` operator for the macro[^0]:
> You can also “unset” serde attributes by including them with a ! prefix in a
> schemars attribute, which will make schemars ignore the corresponding serde
> attribute item
[^0]: https://docs.rs/schemars_derive/latest/schemars_derive/derive.JsonSchema.html#attributes
The `Review::summary` was limited in behaviour, since it was only an
`Option<String>`.
The field is now changed to `NonEmpty<Edit>` so that it can enable an edit
history, as well as being able to supply embeds as part of the text.
`Reactions` are also introduced to `Review` and, necessarily, introducing an
`Action::ReviewReact`.
Note that some machinery was added under `actions` and `encoding` to enable
backwards-compatibility.
For `actions`, a new `ReviewEdit` is introduced to keep
track of the different versions of editing actions – supporting the old actions
while also introducing the new variant.
For `encoding`, the machinery exploits the fact that an `Option<String>` can be
converted into a `NonEmpty<Edit>` given that an `ActorId` and `Timestamp` are
available, and that `None` can be converted into an empty string. The struct
`patch::Review` now deserializes via this `encoding::review::Review` type and,
importantly, preserves backwards-compatibility.
Changes for the `Canonical` type:
- Reorder arguments so that `delegates` and `threshold` are grouped
after `refname`
- Add `Canonical::is_empty` to detect when `Canonical` was constructed
with no tips
- Consume `Canonical` when calling `Canonical::quorum` to indicate
that it should be the final step
The Canonical type is generally constructed with the threshold, needed
for the quorum computation, in scope. To make it easier to use and
encapsulate its functionality, it now holds the threshold as an
additional field.
Lorenz Leutgeb
Fintan Halpenny
Defelo
Matthias Beyer
Sebastian Martinez
Sekhat Temporus
Tobias Hunger
Erik Kundt
Jakob Kirsch
