Our implementation for the control socket is based on AF_UNIX, and as
the note (which is removed) suggests, we should actually check that the
received socket really is of that domain.
This check was not implemented because it is not exposed via `std`, and
a bit cumbersome to do via `libc`.
I did not realize that `socket2` which neatly sits between `std` and
`libc` in terms of its abstractions and is cross-platform allows us to
do this, and we even already depend on it!
So, add the suggested check.
While at it, refactor the function to return early in cases. Now the
progression from a file descriptor to a socket to a listener is nicely
captured in the types and not obstructed too much by indentation.
Also log at the error level.
I was greeted by `rad patch redact` with
called `Result::unwrap()` on an `Err` value:
Error { code: None, message: Some("failed to convert") }
stack backtrace:
[…]
3: core::result::Result<T,E>::unwrap
at …/rust-1.88.0/lib/rustlib/src/rust/library/core/src/result.rs:1137:23
4: sqlite::cursor::Row::read
at …/index.crates.io-1949cf8c6b5b557f/sqlite-0.32.0/src/cursor.rs:136:9
5: radicle::cob::patch::cache::query::find_by_revision
at ./crates/radicle/src/cob/patch/cache.rs:624:65
6: <… as radicle::cob::patch::cache::Patches>::find_by_revision
at ./crates/radicle/src/cob/patch/cache.rs:553:9
7: radicle_cli::commands::rad_patch::redact::run
at ./crates/radicle-cli/src/commands/patch/redact.rs:23:9
8: radicle_cli::commands::rad_patch::run
at ./crates/radicle-cli/src/commands/patch.rs:1026:13
[…]
It turns out that `sqlite::cursor::Row::read` is the panicky version of
`sqlite::cursor::Row::try_read` (which returns a `Result`).
While it is somewhat rare that SQLite reads fail, it is not unheard of.
In `radicle-cli` it might not be critical, but also `radicle-protocol`
and `radicle-fetch` are affected, and they could potentially panic a
`radicle-node` process.
Use `try_read` instead, and propagate down the error handling.
Remove `fn deserialize` and instead provide a trait `fn decode_exact`
for testing.
Refactor errors, so that we can now cleanly distinguish between
`Error::UnexpectedEnd` which means that decoding might work given
more data, and `Error::Invalid`, which means that no amount of
additional data will decode.
While at it, cleanup the variants of `Error::Invalid`, and provide `ControlType`.
What made the issue fixed in the parent commit harder to spot
for me was the multitude of concepts and terms involved in
encoding: What's *serializing* vs. *encoding*? Why is there
this odd `Frame::to_bytes`?
So I decided to clean up a bit. I removed `fn serialize` so that the
logic stays close to the data, and instead of it provide
`fn encode_vec` in `trait Encode` so that we don't add up with
one-offs like `Frame::to_bytes` again.
Also we make use of `encode_vec` in the tests.
In 3c5668edd2 I mistakenly moved the check
the length of an encoded `wire::Message` from its `impl wire::Encode` to
the `fn serialize`, so that it would not apply only to messages (as
originally intended), but to *any* argument of `serialize`.
This is not a problem for gossip messages, but catastrophic for Git
streams, which tend to send a lot of data.
The fix is simple: Move the check back into `impl Encode for Message`.
As `std::os::unix` is obviously not available on Windows, resort to the
`winpipe` crate, which implements a very similar API for named pipes.
Apart from simple changes to imports, we need to jump through a few
hoops because the `std::io::{Read, Write}` impls are on `&mut` for
`winpipe`, thus yielding different mutability requirements on Unix vs.
Windows.
We resort to cloning, but as this fallible, swallow the added complexity
of a platform-dependent implementation to not risk panics on Unix.
The `winpipe` crate which is our current best bet for support on Windows
requires that named pipes are at the magical location `\\.pipe\`.
Provide a default on Windows that matches this, and use our freshly
assigned IANA service name for it.
This is a breaking change, as we remove a `pub const`, it should be
reasonably easy to fix by dependents, and probably it was a mistake to
make the default `pub const` as dependents should call `Home::socket`
anyway.
The signals crate does not build on non-Unix-like platforms, such as
Windows. It uses constants for signals that are not exported from
`libc` on Windows, and transitive dependencies such as `sem_safe` fail
to compile.
Resolve this, by guarding most of the crate, certainly all real
features, by `cfg(unix)`.
To require less uses of the `cfg` macro, move all affected parts into a
new module, and re-export it.
The crate now builds on Windows (although it does not do anything
interesting on that platform) and its Unix interface remains unchanged.
Changes are integrated into `radicle-node`, but only affect Windows.
Microsoft Windows has Universal Naming Convention (UNC) paths, see
<https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#unc-paths>
The Rust standard library normalizes paths to this form when calling
`std::fs::canonicalize`, see
<https://doc.rust-lang.org/std/fs/fn.canonicalize.html>
However, some programs, do not parse these paths correctly, including
our ally `git`:
$ git clone \\?\C:\Users\lorenz\tmp
Cloning into 'tmp'...
ssh: Could not resolve hostname \\\\?\\c: No such host is known.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The `dunce` crate solves the problem of having to deal with
normalization and avoiding weird UNC paths for us. Note that on
non-Windows platforms, it *behaves exactly like*
`std::fs::canonicalize`!
The `fn io_other` disguises all sorts of errors as I/O errors, making
errors in the transport component of `radicle-fetch` harder to
understand.
Instead, expose a new error type that allows to discriminate.
The stream tests did not take into account that the tips of previous COB entries
need to be inserted so that a Git history can be formed.
This was surfaced by the added regression test.
If the tests are run without a global Git config, then the tests will error with
no name or email set for the commits.
Instead, use the test fixture setup to ensure there is a name and email for
running the tests.
This patch adds the `cob::common::Title` struct, this allows instead of
using `std::str::String` or generics and traits for it, to define more
precisely what a title should be.
It trims the provided string and makes sure it contains no carriage
return or new line characters.
Where a `std::str::String` makes more sense so it's easier to mutate it,
we keep the code as is.
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Add type aliases to specialise the `Stream` type for each of `Patch`, `Issue`,
and `Identity`.
An `init` constructor is also added for each to make it easier to construct each
stream.
Introduce a new `stream` module to `radicle::cob`. The purpose of this module is
to provide an API for iterating over a COB's operations, given a range of
commits.
The `CobStream` trait provides the generic API for implementing and using, while
`Stream` provides a concrete implementation using the `git2` crate.
The approach is to use a `git2::Revwalk` for walking over each commit in the
range, then loading the `Op` for that commit – skipping any commits that are
unrelated to that COB type, in the case of associated COBs.
The `Stream` is tested using some properties of the API.
The next steps are to introduce specialised types for `Patch` and `Issue` to
ensure that it works for those.
Adds an `Op::load` helper method for loading an `Entry`, given its `Oid` and the
`store` it is stored in.
This can be useful for downstream consumers to inspect operations on COBs given
a single `Oid`, without having to load the entire object and/or graph.
It can also be useful when looking at implementing COB stream primitives.
Add a helper method `Op::manifest_of`, to allow the caller to inspect the
`Manifest` of the entry. This can be used to avoid attempting to call `Op:load`
with a manifest that would not match the expected `Action`s of the `Op`.
This patch clearifies (in the help message of the command) how the
"clear" subcommand behaves when passing ids.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
This patch optimizes how the issues are collected.
Before, the issues where pushed to a new instance of `Vec`, which was
not preallocated to the size of the iterator.
This could be slow when a large number of issues is returned by the
iterator.
With the `Iterator::collect` function, the `FromIterator` implementation
of `Vec` is used, which makes use of `Iterator::size_hint` for
optimizing how much space is preallocated when collecting into the
`Vec`.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
This patch adds an implementation of the Iterator::size_hint() trait
function so that the compiler can optimize allocations on the collecting
side of the iterator better.
From the trait function docs:
> size_hint() is primarily intended to be used for optimizations such
> as reserving space for the elements of the iterator, but must not be
> trusted to e.g., omit bounds checks in unsafe code. An incorrect
> implementation of size_hint() should not lead to memory safety
> violations.
Source: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.size_hint
And:
> Vec may use any or none of the following strategies, depending on the
> supplied iterator:
>
> * preallocate based on Iterator::size_hint() and panic if the number
> of items is outside the provided lower/upper bounds
Source: https://doc.rust-lang.org/stable/std/iter/trait.FromIterator.html
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Symbols signaling status of a process and similar are duplicated in
three different crates. Also two different symbols to signal success are
used:
1. ✓ ( Check Mark, U+2713)
2. ✔ (Heavy Check Mark, U+2714)
Work against that by referring to `radicle-term` from the other crates.
When initializing the `radicle-node` process, see
`radicle-node/src/runtime.rs`, an attempt is made to read from a node
announcement file.
This file is never written, therefore all of these reads fail.
Even though this caching mechanism might make sense or might have made
sense at some point, it is dead code now, and therefore removed.
Also, the fact that the file name was defined in the `radicle` crate is
ugly.
Ensures that preferred seeds that are not synced are made part of the `to_sync`
set, and the Announcer is constructed without an error.
As well as this, ensure that the `synced` and `to_sync` sets do not overlap by
constructing the `to_sync` and removing any `synced` elements from it.
Making a note of that `Announcer::can_continue` might never be able to hit the
`Break` case. It could be defensive programming, so I'm going to leave it in for
the time being.
Adds a test case to show that the Announcer is allowed to mark an "unknown" node
as synced.
There may be a case where gossip reached a node that the current node was not
aware of. This should be allowed and should count towards the final replication
factor.
The `Announcer` logic was subtly different to the fetching logic, where the
preferred seeds need to be announced to *and* the replication factor had to be
reached.
This patch relaxes that constraint, and allows it to hit either targets before
exiting.
This also surfaced an issue with the with the `Progress` calculation, where
calculating the `unsynced` used the `synced` value to subtract. However, this
was not required because `to_sync` is mutated, so the value is simply
`to_sync.len()`. A test case was added to ensure this is correct by checking
that invariant that the total between synced and unsynced nodes always matches
the total of synced and unsynced progress.
The upgrade of the `radicle-crypto` crate was missed when upgrading
semver versions.
Unfortunately, since it is at the base of the dependency tree, all
crates that depend on it require an additional upgrade.
If one attempted to compile the test target for `radicle-node` by itself, it
would result in a compiler error – due to missing trait implementations that are
provided by `radicle-protocol/test`.
It would compile fine when `--all` is used due to cargo unifying feature sets,
so it went unnoticed until now.
This fixes the issue by adding `radicle-protocol` to the `dev-dependencies` and
enabling the `test` feature flag.
This is morally a revert of 70fb0d3fef
while additionally getting to compile
`crates/radicle-term/src/editor.rs` on Windows.
To get the changes that were made on top of the revert, check
git diff 70fb0d3fef~1 <this commit> -- crates/radicle-term/src/editor.rs
There were reports of the inquire editor being unrealiable.
Only the latest log was kept as `node.log.old`. Now, log files are
numbered (`node.log.1`, `node.log.2` and so on). Also `node.log` is now
a hard link to the current log file. `rad node stop` will delete
`node.log` (note that `node.log.x` stays intact) and `rad node start`
will delete `node.log` before it creates `node.log.y` in case the node
crashed.
Also, when running in foreground mode, now a log file is created. It
just contains the hint that the node was started in foreground mode,
just to avoid confusion.
The implementation is split into a sans I/O part as `struct LogRotator`,
while the I/O counterpart is captured by `struct LogRotatorFileSystem`.
The logic for computing canonical references conflated the semantics of
annotated and lighweight tags, yielding confusing/wrong results. The
main culprit was the call to `peel_to_commit` in
`impl ReadRepository for Repository`, peeling tag objects away.
To resolve this, we separate out quroum computation per object type: one
implementation for commits, and one for tags.
Also move move canonical error types and methods into their own
module to have a cleaner file structure for the main logic.
Capture the `BTreeMap<Oid, u8>` type into a `Votes` struct, with its own API.
We were getting the objects from the repository twice – once in `Canonical::new`
and once again in `ensure_commit_or_tag` – so the latter was removed.
Use an `enum CanonicalObject` to specify which object types the canonical
process supports.
During `converges`, separate commits and tags, and ensure that we are only
looking at objects of one type. Note, that Fintan think this is actually the
*wrong* place to do it because we already filter the objects – so we should
keep track of that information higher up the callstack.
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
When we fail to get the info for the binary names of `radicle-node` and
`git-remote-rad`, we still mention the binary name which is usually a part of
their output.
After upgrading from Rust 1.85 to 1.88 in 586eefc, clippy started
warning about large error types. Box action to make
`radicle::cob::patch::Error` lighter.
Provide a default message that informs the user of the `(e)`, `(enter)`, and
`(esc)` functionality.
Unfortunately, it is not possible to provide any message that is better
customised to each case, due to `inquire` requiring a `'a` lifetime on all its
inputs – this rules out providing any kind of `String` input.
The first paramter of `Editor::new` can already be used to provide more
contextual information. The second parameter is left configurable, but for now
`Editor::HELP` is used everywhere.
To avoid that users on Windows have to explcitly set `HOME`, also accept
`USERPROFILE` which is considered equivalent.
To avoid repeating to join `.radicle`, split the logic up into two
phases. Also take care to provide a good error message, as this error is
fatal and user-facing.
The trait `radicle_ssh::agent::client::ClientStream` trait was
simplified, because `winpipe::WinStream` is so similar to
`std::os::unix::net::UnixStream` that we can just use the same
implementation for both platforms.
However, obtaining the process identifier of the SSH agent process is
not common on Windows, so we instead track the path that we used to
connect to the agent. This is also helpful on Unix. In some cases, the
path will even contain the process identifier. The path is printed by
`rad self`.
Errors are cleaned up, and error messages are improved.
Introducing the new enum `TerminalFile` to model which standard stream
stream the terminal uses allows us to remove the dependency on `libc`
which does not build on Windows.
`radicle-term` includes quite a lot of rather "low-level" logic to spawn
the preferred editor of the user. In particular, this logic is
platform-dependent and only works on Unix-like platforms.
Also, `radicle-term` already depends on `inquire` which features an
editor prompt. It is a cross-platform solution to spawn the editor.
This commit changes the implementation of `Editor` to be a wrapper of
`inquire::Editor`, keeping the interface mostly intact.
Downsides:
- We cannot edit a file "in place" this way, instead we have to read
it, as `inquire` only supports editing temporary files. This can also
be viewed as a benefit, as we lessen the risk of corruption of these
files.
- `inquire::Editor` contains borrows, so it is quite cumbersome to
extend it with a custom mechanism for looking up the preferred
editor. The lookup logic was kept, but needs to be invoked by the
caller. In the future we might consider just using the logic provided
by `inquire` or requesting a change to `inquire`.
- The prompt is a bit strange. It does not show the current state of
the output, see <https://github.com/mikaelmello/inquire/issues/280>
and <https://github.com/mikaelmello/inquire/pull/271>.
The `radicle-signals` crate only supports Unix, so using it prevents
`radicle-term` to build on Windows.
Of course, this means that non-Unix platforms will not support any
signal handling at all, but at least we can build.
Directory builder extensions are platform specific and only available on
Unix. Move their use into a conditionally compiled block, so that we can
build on Windows.
When using `rad init` the default branch for the project can be predetermined by
inspecting the working copy repository.
To improve the experience, it will first check `init.defaultBranch` in the Git
configuration file.
If it does not find this, it falls back to choosing the default head of the
repository at the time. This may be HEAD, and in this case it will error and
provide helpful information.
The repository may also not have any default head at the time, and this also
results in an error and hint.
This set of changes refactors the canonical handling by isolating the business
logic into another type, `Canonical`.
The functionality remains the same, but certain error scenarios are better
handled. Primarily, the commits of the inner `Canonical` type are checked to see
if they exist in the working copy, and if not, provide a more useful message
about which commit was missing or invalid for which DID.
Reduce the nesting of if statements by using a `PushAction` enum that is
constructed by checking the `dst` reference and parsing the correct action, one
of:
- `OpenPatch`
- `UpdatePatch`
- `PushRef`
This makes the intent of the code a little bit more clear and reduces the tabbing
of big code blocks. It also captures the error specifically for handling the
parsing of this `dst` reference.
This allows any revision that can resolve to an object. The `src` side of the
refspec is resolved to its corresponding `Oid` which is further used for remote
helper pushing and patch creation/update.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
Bare git repositories are meant to fetch into/push from and
fetching works, but pushing from a bare repo into radicle-storage
was prohibited.
This is a problem for me as I want to use `jj`, which uses
a bare git repository in the back (when not co-located into
a "normal" git repository).
`radicle-crypto` is the only crate that actually depends on `ssh-key`. I
overlooked that previously and added it as a workspace dependency. Now I
am limiting its scope again.
Pay more attention to when a Node ID is displayed in compact form. As
this was sometimes used to save space, e.g. in `rad node status` in
combination with long node addresses (like .onion names), also implement
a compact format for node addresses and use that instead in some cases.
Progress notifications in spinners keep the short form.
The `[T]` and `[I]` are interpretted as links when running `cargo doc`. This
table is surrounded in a code fence to prevent this, but also provide a better
view of the table when rendered to HTML.
Mostly changes of `use`s and moving components to `radicle-protocol`,
with the goal of just getting `radicle-node` to work on top of the new
`radicle-protocol` crate.
Add a test to exercise adding canonical reference rules and pushing new
canonical tags.
A change in `rad id` was required to allow adding a new payload to the document,
i.e. one that does not already exist.
Adding rules is also documented in the man page for `rad id`.
Ensure that the default branch is only ever synthesized from the `Doc`'s
`threshold` and `delegates` value. If the rule is ever written into a `RawDoc`,
then the update of the that `RawDoc` must go through the `update::veryify`
function, which checks that the rule is not present.
The business logic of modifying the identity document should not live in the CLI
code for updating said document.
Instead, we extract out that logic into separate update functions and these are
using from the CLI instead.
Introduce the `rules` module beneath `canonical`. The implementation follows
the Canonical References RIP[^0].
A `Rule` defines the DIDs and threshold required to get the canonical tip
for a reference (or set of references). Since a `Rule` has some specific
validation rules, there can be two versions of a `Rule`. The `RawRule` is one
that can be easily constructed, serialized, and deserialized. It must then be
validated into a `ValidRule`, which validates any constraints, and can be used
in the computing of a canonical tip.
For convenience, a set of `ValidRule`s is wrapped into the `Rules` type, which
also has a further constraint that each rule refers to a specific `refspec`.
[0]: 1d1ce874f7
Refactor the `connect` method to return a `ConnectError` so that it can be
reported via logging at call sites, and returned as an error from the node
handle.
Downstream clients require access to the data for a `ReviewEdit` – for example,
the desktop client builds a representation of `Action`s for TypeScript
generation.
To allow these kinds of use cases, methods are added to `ReviewEdit` to allow
access to its underlying data.
Prepare the crates for releasing to crates.io:
- radicle-remote-helper: 0.10 -> 0.11
- radicle-term: 0.12 -> 0.13
- radicle-cli: 0.13 -> 0.14
- radicle-node: 0.11 -> 0.12
- radicle-fetch: 0.11 -> 0.12
- radicle: 0.15 -> 0.16
It was also necessary to specify the `version` in the workspace Cargo.toml file,
since this is required for publishing to crates.io.
While running `cargo-semver-checks`, there was an issue that was surfaced
regarding `serde(into)`, `serde(try_from)` and `schemars(transparent)` being
used, and they are incompatible.
This appeared to be a semver issue, on the side of `schemars`, since the issue
would only appear when adding `radicle` as a fresh dependency. The intuition for
this is that once `schemars` was updated locally, the same error appeared when
attempting to build the `radicle` crate.
To avoid this, `schemars` provides a `!` operator for the macro[^0]:
> You can also “unset” serde attributes by including them with a ! prefix in a
> schemars attribute, which will make schemars ignore the corresponding serde
> attribute item
[^0]: https://docs.rs/schemars_derive/latest/schemars_derive/derive.JsonSchema.html#attributes
The `Review::summary` was limited in behaviour, since it was only an
`Option<String>`.
The field is now changed to `NonEmpty<Edit>` so that it can enable an edit
history, as well as being able to supply embeds as part of the text.
`Reactions` are also introduced to `Review` and, necessarily, introducing an
`Action::ReviewReact`.
Note that some machinery was added under `actions` and `encoding` to enable
backwards-compatibility.
For `actions`, a new `ReviewEdit` is introduced to keep
track of the different versions of editing actions – supporting the old actions
while also introducing the new variant.
For `encoding`, the machinery exploits the fact that an `Option<String>` can be
converted into a `NonEmpty<Edit>` given that an `ActorId` and `Timestamp` are
available, and that `None` can be converted into an empty string. The struct
`patch::Review` now deserializes via this `encoding::review::Review` type and,
importantly, preserves backwards-compatibility.
This change better separates the different errors that can occur during an
upload. It does this by:
- Separating the authorization error into its own enum and having a variant
simply for that
- The `PacketLine` error was never constructed, so the error returned from the
`upload_pack::pktline::git_request` is now wrapped in this variant.
- The error from `upload_pack::upload_pack` is now wrapped in an `UploadPack`
variant.
- The `Io` variant is removed.
- Other redundant variants are also removed.
Allow the `rad node connect` command to use only a `NodeId` when connecting.
The `NodeId` is used to connect to known addresses, and will stop at the first
address that is connected to.
I learned that there is an implementation of a pager in
`crates/radicle-term/src/pager.rs`. It does not make sense to me to
maintain a pager in `heartwood`, this is too far from the core of what
we are trying to achieve and adds unnecessary maintenance burden. We can
allow ourselves to defer to the paging done by `git` or just call the
user's `$PAGER`. Even though falling back to unpaged output might not be
the nicest thing, I still prefer that over maintaining the pager.
It also was frustrating to me that the pager does not support things
like search (like `less` does).
This also significantly decreases our dependence on `termion`.
The pager was originally implemented by cloudhead in
40f4383bbc
and integrated into `rad diff` in
6dd52c94fb
This commit restores the behaviour introduced in
84a95848b0
(before there was a custom pager implementation).
The `radicle-tools` binaries were specialised binaries for early development of
the `heartwood` set of crates. Choose to remove the crate so that it no longer
needs to build as part of our workspace.
The `radicle-crdt` crate is no longer used by any of the other crates. Choose to
remove the crate so that it no longer needs to build and test as part of our
workspace.
Writing a comment, only the most recent comment replied to is shown.
Editing a comment, current contents are not prepared. Manual copy required.
To improve:
- Chase all replies to the root and quote them. Gracefully handle if
this proces fails, i.e. a comment in the chain cannot be resolved.
- If editing, copy the current contents.
A `HashMap` does not guarantee a consistent iteration order. It is
desirable that all environment variables are printed in a consistent
order.
Use a `BTreeMap` which guarantees a consistent iteration order.
Changes for the `Canonical` type:
- Reorder arguments so that `delegates` and `threshold` are grouped
after `refname`
- Add `Canonical::is_empty` to detect when `Canonical` was constructed
with no tips
- Consume `Canonical` when calling `Canonical::quorum` to indicate
that it should be the final step
The Canonical type is generally constructed with the threshold, needed
for the quorum computation, in scope. To make it easier to use and
encapsulate its functionality, it now holds the threshold as an
additional field.
Lorenz Leutgeb
Fintan Halpenny
Sebastian Martinez
Matthias Beyer
Erik Kundt
Sekhat Temporus
Tobias Hunger
Lorenz Leutgeb
Johannes Kühlewindt
Jakob Kirsch