Make `git2` an *optional* dependency of `radicle-cob`, and refactor
`radicle` to depend on crates that in turn do not depend on `git2`
*non-optionally*
The main offending dependency of `radicle-cob` is `radicle-git-ext`
from the `radicle-git` workspace in repository
(rad:z6cFWeWpnZNHh9rUW8phgA3b5yGt) which *non-optionally* depends on
`git2`.
So, to achieve removal of this dependency:
1. The crate is refactored to depend on the new crates
`radicle-git-ref-format` `radicle-git-metadata`, and
`radicle-oid` introduced in the previous commits, instead of
`radicle-git-ext`.
2. Some code from the `radicle-git-ext` crate in the `radicle-git`
workspace in repository (rad:z6cFWeWpnZNHh9rUW8phgA3b5yGt) is
copied. See `crates/radicle-cob/src/backend/git/commit.rs`.
This cascades to `radicle` and its dependents.
Firstly, the there is an
`impl Deref<Target=git2::Oid> for radicle_git_ext::Oid`. This made
it very convenient to just deref to obtain the wrapped `git2::Oid`,
so there are many expressions of the shape `*oid` in `radicle` and
its dependents. However `radicle-oid` does not provide
`impl Deref<Target=git2::Oid> for radicle_oid::Oid`, as notably,
`Target` is an associated type, and not a type parameter, so an
implementation of `Deref` would tie the new `Oid` too tightly to a
particular implementation.
Instead, work with `impl From<radicle_oid::Oid> for git2::Oid`
(which can be enabled using the feature flag `radicle-oid/git2`).
This explains the changes from `*oid` to `oid.into()` at every
transition to "`git2` land".
Secondly, `radicle` and its dependents are refactored to also depend
on `radicle-git-ref-format` and (much less prominently)
`radicle-git-metadata` instead of `radicle-git-ext`
This is to avoid pulling in `git2` via these dependencies.
Thirdly, as the re-exports in `crates/radicle/src/git.rs` change,
they are at the same time also cleaned up. Notably, the types from
`radicle-git-ref-format` are re-exported under `fmt` only, not "twice".
While overall this obviously is very much a breaking change, these
changes should mostly amount to changing from `Deref` to `Into`, i.e.,
`*oid` → `oid.into()` and rewriting imports for `radicle::fmt`.
This is indicated by the mostly mechanical nature of the changes to
`crates/radicle-{cli,node,remote-helper}`.
For further details on the crate see its documentation in
`crates/radicle-oid/src/lib.rs`.
Changes to `crates/radicle{,-crypto}/Cargo.toml` are to preserve the
default features of their dependencies `serde` and `schemars`,
which are disabled to support `radicle-oid` being `no_std`.
The `git-ref-format-core` crate is enough to implement everything
required by `radicle-crypto`, and has a much smaller dependency
footprint than `radicle-git-ext`. Notably, it does not depend on
`git2` and also does not depend on procedural macros.
As a library crate, it is bad to return such generic errors.
To get there, errors from `inquire` are downcasted.
This allowed to clean up the public interface of `radicle-term`, so that
it does also not leak the `inquire::InquireError` in its public API.
Microsoft Windows has Universal Naming Convention (UNC) paths, see
<https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#unc-paths>
The Rust standard library normalizes paths to this form when calling
`std::fs::canonicalize`, see
<https://doc.rust-lang.org/std/fs/fn.canonicalize.html>
However, some programs, do not parse these paths correctly, including
our ally `git`:
$ git clone \\?\C:\Users\lorenz\tmp
Cloning into 'tmp'...
ssh: Could not resolve hostname \\\\?\\c: No such host is known.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The `dunce` crate solves the problem of having to deal with
normalization and avoiding weird UNC paths for us. Note that on
non-Windows platforms, it *behaves exactly like*
`std::fs::canonicalize`!
The upgrade of the `radicle-crypto` crate was missed when upgrading
semver versions.
Unfortunately, since it is at the base of the dependency tree, all
crates that depend on it require an additional upgrade.
`radicle-crypto` is the only crate that actually depends on `ssh-key`. I
overlooked that previously and added it as a workspace dependency. Now I
am limiting its scope again.
Prepare the crates for releasing to crates.io:
- radicle-remote-helper: 0.10 -> 0.11
- radicle-term: 0.12 -> 0.13
- radicle-cli: 0.13 -> 0.14
- radicle-node: 0.11 -> 0.12
- radicle-fetch: 0.11 -> 0.12
- radicle: 0.15 -> 0.16
It was also necessary to specify the `version` in the workspace Cargo.toml file,
since this is required for publishing to crates.io.
While running `cargo-semver-checks`, there was an issue that was surfaced
regarding `serde(into)`, `serde(try_from)` and `schemars(transparent)` being
used, and they are incompatible.
This appeared to be a semver issue, on the side of `schemars`, since the issue
would only appear when adding `radicle` as a fresh dependency. The intuition for
this is that once `schemars` was updated locally, the same error appeared when
attempting to build the `radicle` crate.
To avoid this, `schemars` provides a `!` operator for the macro[^0]:
> You can also “unset” serde attributes by including them with a ! prefix in a
> schemars attribute, which will make schemars ignore the corresponding serde
> attribute item
[^0]: https://docs.rs/schemars_derive/latest/schemars_derive/derive.JsonSchema.html#attributes
Relax the set of radicle-git depedencies to allow for patches. In
particular, the dependencies introduced some small improvements with
`0.*.1` updates.
All dependencies that are dependencies of multiple crates within the
workspace were pulled up to the root `Cargo.toml`. Dependencies that are
dependencies of just one crate within the workspace remain in the
`*/Cargo.toml` of that particular crate.
Generally, this helps us keep an overview of dependencies:
- If multiple crates depend on a crate, we will see the reference to
the workspce.
- Otherwise, we will see a line that specifies the version.
It is up to the maintainers to recognize when a dependency becomes a
dependency of multiple crates in the workspace, at which point it should
be moved to the workspace.
The tool `cargo-autoinherit` can be used to automatically pull *ALL*
dependencies to the workspace. I used this, and in a separate, manual
step, filtered out those dependencies that are dependencies of a single
crate only, moving those back manually.
At the same time, I also unified depending on sibling crates in the
workspace. This is now also done via workspace dependencies.
See:
- https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#inheriting-a-dependency-from-a-workspace
- https://github.com/mainmatter/cargo-autoinherit
Extracting JSON Schemas from the Radicle crate can be done via
`radicle-cli` for `radicle::profile::Config` by executing:
rad config schema
However, for other JSON Schema metadata, this is not possible.
To allow other tools to extract JSON schemas as part of their build
process, introduce a new crate that only depends on `radicle`,
`schemars`, and `serde`, with a tiny binary that will reproduce various
schemas.
The `schemars` crate is a dependency of multiple workspace crates in the
same version. Its version number is repeated multiple times in the
respective `*/Cargo.toml` files. This requires more maintenance effort
and risks versions drifting.
As long as all crates depend on the same version, it makes more sense to
have `schemars` as a workspace dependency.
See: <https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table>
This declares the minimum supported Rust version, independently of the
`rust-toolchain.toml` file, which specifies the Rust version and
toolchain components the crate/workspace should be built with. The
MSRV affects other crates that depend on anything in this workspace,
but the toolchain file does not seem to affect them. This means the
MSRV is useful for those who build or develop dependents.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
Nix Flakes[[0]] are a more modular way of expressing nix derivations.
The `flake.nix` file describes:
- How to build the project, or components of a projet, via `nix build`
- What "checks" can run for a project, which can be executed via
`nix flake check`
- What binaries can be run, e.g. `nix run .#rad` to execute the CLI
This approach is preferable since a lot of the Nix ecosystem is using
Flakes -- despite it still being marked as experimental.
One interesting benefit, that needs confirming, is that a check can be
run from a remote Git source. So if someone creates a patch on
Radicle, someone else with the `nix` binary can execute:
nix flake check git+https://app.radicle.xyz/nodes/seed.radicle.xyz/rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5?refs=patches/<patch id>
This means that anyone with `nix` can check patches without having to
fetch the patch itself.
Another obvious benefit will be that it allows the Radicle binaries to
enter the Nix ecosystem.
The big caveat to both of the above is that, currently, the tests are
failing while running `nix flake check`. They are currently disabled
with `doCheck = false`. They are potentially failing due to some
strange sandboxing issues which are being looked into.
[0]: https://nixos.wiki/wiki/Flakes
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.
The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.
[0]: https://github.com/Byron/gitoxide
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We're using petgraph to encode an operation based CRDT (each graph node
is an operation) as a DAG, and one of the important things to ensure
is that the final state is not influenced by the order of concurrent
operations. In this case a concurrent operation would mean two potential
graph traversal orders, as there's no direct edge between the two
concurrent ops. This is essential a "partial order" of operations.
To be able to test this, we'd want some kind of control over how
neighbors are iterated over I guess, when there's more than one possible
sort order. Say we have a graph like this:
┌────────b◄─────┐
▼ │
a d
▲ │
└────────c◄─────┘
There are two possible topological traverse orders: [a, b, c, d] and [a,
c, b, d]. Having a way to go through these different orders would
be super handy.
One option would be to allow random order traversal. This would allow us
to test all orders by running the test enough times to likely test all
permutations.
Since petgraph doesn't support this, we implement our own simple DAG in
`radicle-dag`, which implements random-order topological orders.
For now, we don't make explicit use of these improvements, we simply
replace the underlying graph with our own.
This is a large change that introduces transport encryption via
a Noise_XK handshake. The inner `Service` now works directly
with `NodeId` and doesn't need to interact with socket addresses.
The transport logic takes place in `wire::transport`, and the
established sessions are passed down to `Service`.
Note that the i/o implementation in the `poll-reactor` crate still
needs testing, and there is still work to be done to integrate the
Git transport as part of this architecture.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
This is a minimal port of the collaborative objects code from
radicle-link[0].
A `CollaborativeObject` is made up of a `History` of `Change`s that
are expected to be stored in a `git` backend.
The library exposes a CRU series of functions `create`, `get`, `list`,
and `update`.
This implementation differs, to previous one, in that it does not
assume any kind of identity system -- its only assumption is that
identities have an identifier and can be addressed in the `git`
database. This means that it does not require an `author` to be passed
in and it only uses the `resource` (previously `authorizing_identity`)
to store its content-address in the `Change`.
It also foregoes any caching to simplify the port and not make any
early optimisation assumptions.
[0]: https://github.com/radicle-dev/radicle-link/tree/master/cob
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
The crypto types are useful to use in other components that are
separate from the radicle crate, e.g. the incoming radicle-cobs
crate.
Separate out the types into a radicle-crypto crate. This crate defines
the usual PublicKey, SecretKey, and Signature types. It also provides
the ssh functionality, under the `ssh` feature. It is also necessary
to provide the `From` impl for `Component` here, which again is
guarded by the `git-ref-format` feature.
The `Arbitrary` implementation are moved. The arbitrary decision of
redefining `ByteArray` in both radicle-crypto and radicle was made.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new `radicle-httpd` crate which will serve as the basis
for heartwood's HTTP backend.
The first part of this is a git server which allows projects to be
fetched from storage using plain `git`.
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
The `git-ref-format`[0] and `radicle-git-ext`[1] crates are now
published to crates.io.
Switch the dependency to rely on them. This also required to update
the `git2` dependency to `0.15.0` as well as using the
`vendored-libgit2` feature -- avoiding the openssl dependency too.
[0]: https://crates.io/crates/git-ref-format
[1]: https://crates.io/crates/radicle-git-ext
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Lorenz Leutgeb
Fintan Halpenny
Matthias Beyer
Lars Wirzenius
Lorenz Leutgeb
cloudhead
Erik Kundt
Alexis Sellier
xla
xphoniex
Han Xu
Alexis Sellier
Dr. Maxim Orlovsky
Adam Szkoda
Dr Maxim Orlovsky
Dr Maxim Orlovsky
