Previously the pre-commit hook shelled out to `rad lfs store` once per
staged LFS file, each a separate process. For private repos, each of
those needs a keystore passphrase for an ECDH key-agreement operation,
so a multi-file commit meant a separate interactive passphrase prompt
per file -- observed live against the blog repo (9+ prompts in a row).
Add `rad lfs precommit`, which reads every staged file's oid/size/path
from stdin and processes them all in one process, loading the signer
at most once and reusing it across the batch. `store::run` now
delegates to a shared `store_object` helper that both it and
`precommit::run` call.
Found live while migrating the blog's food-gallery images into LFS,
diagnosed in NOTES-lfs-store-note-write-bug.md, confirmed and fixed here.
Root cause: store.rs/rekey.rs wrote local notes to the bare
refs/notes/rad-lfs, but the earlier fetch-bug fix (1c65ee62) has the
fetch refspec populate refs/notes/rad-lfs/<peer> siblings locally --
including your own peer's copy, fetched back after your own push. A bare
refs/notes/rad-lfs ref can't coexist with refs/notes/rad-lfs/<peer> in
the same git ref namespace (a leaf ref vs. a directory prefix at the same
path) -- so the very first commit+push+fetch cycle on a repo left it
permanently unable to `rad lfs store` again, failing with "failed to
write LFS note" and no further detail (git itself hits the same conflict
from the fetch direction and just refuses that one ref cleanly; it's
specifically libgit2's ref-write path used here that fails hard instead).
Fix: stop writing to the bare ref locally at all. New
lfs_crypto::LOCAL_NOTES_REF (refs/notes/rad-lfs/local) is what
store.rs/rekey.rs write to now -- living under the same
refs/notes/rad-lfs/ prefix as every fetched peer ref makes it just
another sibling leaf ref, structurally incapable of conflicting with
them ("local" can never collide with an actual peer ID). NOTES_REF
(bare) stays exactly as it was for the remote-side/push-destination
role, unchanged. rad lfs init's push refspec becomes asymmetric
(+refs/notes/rad-lfs/local:refs/notes/rad-lfs) instead of symmetric;
fetch refspec is untouched.
Added a migration step (migrate_local_notes_ref, run by rad lfs init)
for already-affected repos: reads any existing bare-ref notes out,
deletes the bare ref, then re-writes them under the new local ref --
also removes the old symmetric push refspec config entry the same way
the earlier fetch-refspec migration did, so re-running rad lfs init
fully repairs an already-broken repo with no manual git surgery needed.
(First version of the migration itself had the identical bug -- tried to
write the new ref before deleting the old one, hit the same conflict
self-inflicted -- fixed by reading all notes into memory before deleting
the bare ref.)
Verified locally: reproduced the exact failure in the precise order that
triggers it, confirmed the fix resolves that exact same failing call in
the same repo, verified the migration path preserves note content
exactly, and confirmed a full commit->push->fetch->decrypt round trip
plus `rad lfs rekey` all still work correctly afterward.
Private-repo LFS operations need a signer capable of ECDH, which
ssh-agent (the normal day-to-day flow after `rad auth`) can't provide --
that's a hard protocol limitation (standard ssh-agent only implements
signing, there's no key-agreement request type), not a gap in our code,
so there's no way to make it work "through the agent" the way signing
does.
Previously the only alternative to an unencrypted keystore was setting
RAD_PASSPHRASE, which breaks the normal ssh-agent-based workflow and
means putting a password in an environment variable -- not ergonomic,
and not how any other `rad` command behaves.
`rad` already has an established fallback for exactly this situation:
`crate::terminal::io::signer` tries ssh-agent first, and if that's
unavailable/unregistered and we're connected to a TTY, prompts
interactively for the passphrase via `inquire` (reusing the existing
`PassphraseValidator`). `lfs_crypto::load_signer` was missing this last
step entirely -- it only tried "unencrypted keystore" then
"RAD_PASSPHRASE" then gave up. Added the same interactive-prompt
fallback, reusing the exact same validator/prompt helpers already used
elsewhere, before falling back to a hard failure.
This covers the common case (an interactive `git commit` triggering the
pre-commit hook, which inherits the terminal's TTY) with no environment
variable needed at all. Only genuinely non-interactive contexts (CI, a
script with no TTY) still need RAD_PASSPHRASE -- and get a fast, clear
failure rather than hanging, since `inquire` returns `Ok(None)` on
`NotTTY` rather than blocking (verified: piping /dev/null as stdin still
fails immediately, not a timeout).
Verified against a real encrypted keystore through an actual pty (via
`script`, since a plain non-interactive shell doesn't have a TTY to
prompt on): the masked "Passphrase:" prompt appears, accepts the typed
passphrase, and produces a correctly encrypted note -- with
RAD_PASSPHRASE unset the entire time.
Found in a real deployment (mbator.pl blog pipeline): rad lfs init
configures a client-side fetch refspec for refs/notes/rad-lfs, but
git-remote-rad's for_fetch() never advertised any such ref -- notes only
exist per-peer, under refs/namespaces/<peer>/refs/notes/rad-lfs, with no
canonicalization step the way refs/heads/refs/tags get one. Requesting an
unadvertised refspec fails the *entire* git fetch, not just that one ref,
so any repo with `rad lfs init` run could no longer git fetch/pull at
all, LFS or not.
Fix: expose each peer's refs/notes/rad-lfs individually, as
refs/notes/rad-lfs/<peer-id> (list.rs's for_fetch), matching how patch
refs already work rather than trying to canonicalize -- notes are
per-peer contributions (any peer can commit an LFS-tracked file, not
just delegates), so unlike heads there's no single "correct" value to
resolve to; canonicalizing would silently drop non-delegate contributors'
objects. rad lfs init's fetch refspec becomes a wildcard
(+refs/notes/rad-lfs/*:refs/notes/rad-lfs/*); push is unchanged (already
worked -- for_push() turns out to be purely informational for git's
list-for-push handshake, not an enforcement gate, so the actual send-pack
call was never restricted to heads/tags despite what for_push() lists).
rad lfs store/fetch/rekey and seed/unseed's pinning (ipfs.rs::lfs_cids)
now read across every notes ref instead of assuming one canonical ref,
merging recipient lists for notes that share a CID (e.g. after a rekey
performed by a different peer than the original committer) while keeping
genuinely different ciphertexts (from two peers independently encrypting
byte-identical content) as separate candidates to try in turn.
Also added a migration step so re-running `rad lfs init` on an
already-configured repo actually fixes it, by removing the old
non-wildcard fetch refspec rather than just adding the wildcard
alongside a still-broken entry.
Verified through the real git-remote-rad transport (not the storage-copy
shortcut used for earlier testing, since this bug lives specifically in
that transport): fresh identities, a real rad:// remote matching exactly
what `rad init` configures, a genuine `git fetch rad` that used to fail
with "fatal: couldn't find remote ref refs/notes/rad-lfs" now succeeds
and pulls the notes ref, and `git lfs pull` afterward correctly decrypts
private content end-to-end (SHA-256 verified). Re-verified the rekey
flow through the same real path too.
Radicle's Visibility::Private is enforced entirely by access control at
the replication layer -- git objects for a private repo are plain,
unencrypted objects, protected only by "unauthorized peers are never
given them" (identity/doc.rs). That protection has no jurisdiction over
IPFS's own block-exchange layer: once a legitimate collaborator's IPFS
daemon pins content and joins the network, anyone who obtains the CID by
any means can fetch the plaintext directly.
Adds envelope encryption for private repos: a random per-object content
key encrypts the file (XChaCha20-Poly1305), wrapped once per authorized
recipient (delegates + private allow-list) via ECDH between Radicle
identity keys (already Ed25519, no separate keypair needed) plus HKDF.
The wrapped keys travel in the same refs/notes/rad-lfs note as the CID.
New:
- crates/radicle-cli/src/lfs_crypto.rs -- encryption core (envelope
schema, encrypt/decrypt, key wrapping, recipient resolution, signer
loading).
- `rad lfs store`/`rad lfs fetch` -- plumbing commands that now own the
entire IPFS add/cat + git-notes read/write step (encrypting/decrypting
as needed), replacing what the pre-commit hook and rad-lfs-transfer
used to do directly. This collapses what would've become a third copy
of the note-format logic down to one canonical implementation.
- `rad lfs rekey` -- since Radicle's access control is retroactive (a
newly-authorized DID can replicate full history) but encrypted objects
aren't automatically re-wrapped for new recipients, this command lets
an already-authorized collaborator grant a newly-authorized one access
to previously-committed objects, without re-encrypting or re-uploading
content. Revocation remains unsolved, matching Radicle's own model
(already-replicated/decrypted content can't be un-known).
Operational requirement: private-repo LFS operations need a signer
capable of ECDH, which only an unencrypted keystore or RAD_PASSPHRASE
provides (an ssh-agent-only signer can sign but not do key agreement).
Fails fast with an actionable error rather than hanging on a passphrase
prompt inside a non-interactive git hook.
Breaking change: the refs/notes/rad-lfs note format moves from bare
`cid=<cid>` text to JSON (`{"v":1,"cid":"...","enc":...}`). No migration
path -- prototype, no external users yet.
Verified end-to-end with real Radicle profiles/identities (not just the
transfer-agent protocol): a private repo's IPFS-stored content is
confirmed ciphertext (byte-different from plaintext, +16 bytes AEAD tag);
an authorized peer decrypts correctly; an unauthorized peer is denied
with a clear error and no leaked content; RAD_PASSPHRASE is required and
enforced with a fast, clear failure; and rekey correctly grants a
newly-authorized peer access to a pre-existing object while leaving other
recipients' entries untouched.
Adds Git LFS support for repositories, with large file content stored on
each contributor's own local IPFS node instead of a seed-hosted server:
- `rad lfs init`: one-time setup (transfer-agent config, pre-commit hook,
notes-ref push/fetch refspecs, IPFS daemon reachability check).
- `refs/notes/rad-lfs` carries the oid -> CID mapping, replicating with
the repository via Radicle's existing ref sync (verified against
`references_of` in crates/radicle/src/storage/git.rs: every ref is
replicated except refs/tmp/heads/*, so no protocol changes were needed).
- `rad seed`/`rad unseed` now pin/unpin a repo's known LFS objects in IPFS,
mirroring Radicle's existing "seeding = keep a full copy" model, and
warning (not failing) if no local IPFS daemon is reachable.
The actual byte transfer is handled by a separate binary, rad-lfs-transfer
(added here as a submodule), which implements the Git LFS custom-transfer-
agent protocol against a local IPFS daemon. See LFS-IPFS.md for the full
design and setup instructions.
Building and using `rad` for anything other than `rad lfs` requires no
IPFS dependency at all; `rad lfs init` checks for a local daemon upfront
and fails with an actionable message rather than proceeding silently.
On `Action::RevisionAccept` use `has_active_sibling_accept` to check for a sibling accept.
If one is found, the `RevisionAccept` is logged and skipped.
This preserves the invariant of a single accept per delegate on active, sibling
revisions without throwing an error on existing histories.
The evaluation of the reposioty identity is rewritten to better handle cases
where there are child and sibling revisions that are active at the same time.
In particular, `fn Identity::action` is now free of any references to
`self.current`.
This is achieved through two main improvements. The first is that the `State`
of revisions changes to improve clarity:
1. `Active` and `Accepted` remain, and their meanings also remain the same.
2. `Stale` is removed entirely.
3. `Rejected` is improved to also contain `RejectedBy`, to keep track of the
reason for rejection.
4. `Redacted` is promoted to a state. Similarly, the reason for
redaction is tracked by `RedactedBy`.
The transition of a revision from `Active` to one of the other states now
influences siblings and children.
If a revision transitions to `Accepted`, then the sibling revisions can no
longer transition to `Accepted`. They are considered `Rejected` where the reason
is `Sibling`. This cascades: Children of siblings are `Rejected`
recursively, tracking `RejectedBy::Parent`.
Any children of the `Accepted` revision are also evaluated to see if they can be
similarly transitioned to the `Accepted` state; since they may have already been
voted on.
If a revision was rejected by a majority, then the it transitions to `Rejected`
with `RejectedBy::Vote`. Dually to acceptance, the children of this revision
are also rejected with the reason of `Ancestor`.
Finally, when the author of a revision redacts a revision, it transitions
to `Redacted`, tracking `RedactedBy::Author`, and its children are redacted
tracking `RedactedBy::Parent`.
The test is adjusted to `remove_delegate_concurrent` reflect that concurrently
proposed revisions are retained in the timeline and explicitly marked as rejected,
rather than being dropped entirely (as before).
Introduces support for Gerrit-style magic push references via
`refs/for/<branch>`. Pushing to this ref automatically extracts the
target branch and opens a patch against it, bypassing the need for the
push option `patch.target`. Example:
```
$ git push rad HEAD:refs/for/accepted
```
Will open a patch with its `patch.target` set to `refs/heads/accepted`.
Extend `enum MergeTarget` to include a new variant, `Branch`.
The intended use of this new variant is to allow a `Patch` to have a
target branch other than the default branch.
The `Branch` variant holds a `TargetBranch` which, in turn, is ensured
to be a `Qualified` reference that begins with `refs/heads`, i.e. a
Git branch.
The `rad_cob_update` test accidentally use the `rad-cob-log` file
instead of `rad-cob-log`.
This change fixes that, and updates the `rad-cob-update` example file
to the latest patch show format.
The normalization of paths was converting valid `\n` characters to `/n`.
This had not been noticed until codespell suggest the change of
`/ndefined` to be `/undefined`.
Use `Assert::normalize_paths(false)` to disable this behaviour.
The documentation for `RateLimit::fill_rate` and `RateLimit::capacity`
were missing.
Add the documentation to help users decided on values based on how the
rate limiting works.
Add custom check for ellipses "...", asking for replacement of "…".
Git ranges and the CLI wildcard matches are ignored.
This change includes fixes to all sites that did not pass the check.
When invoked without any of `--patch`, `--delete`, `--accept`,
or `--reject`, the command panicked.
It will now correctly return an error to use `--accept`, `--reject`,
or to supply a message.
Note that `--patch` and `--delete` are removed as these commands are
now obsolete.
The new doc test also documents two previously-untested behaviors
of `rad patch review`: neutral reviews (verdict=None with summary)
and the COB layer's silent no-op when a duplicate review is submitted
(see `test_patch_review_duplicate` in radicle-cli/tests/commands/patch.rs).
Fixes 8ceff5d4b8fa94bcbe243f9f8ef52138f65e495d
Signed-off-by: Arthur Mariano <arthvm@proton.me>
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Exercise the edge case where Bob, as a delegate, has never pushed to the default branch.
He then merges Alice's patch, and it is correctly marked as merged.
When delegates collaborate on the repository identity, their voting
narrows the tree of active proposals to the chain of accepted
proposals. Thus, they crucially must understand which proposals are
siblings, i.e., which proposals share the same parent, because no
two sibling proposals should ever be accepted at the same time, as
this would mean a fork.
To ease collaboration of delegates and improve their overview, also
print the parent revision, if present. Note that the initial revision
is the only revision that has no parent.
Parsing of IPv6 addresses not enclosed in square brackets causes a
warning to be logged.
This is too strict, since the user can not influence IPv6 addresses
received over the network, but only those in their configuration.
Remove the warning from parsing, and instead carry a boolean to warn the
user later.
Move logging-related module `radicle::logging` into its own crate.
While at it, remove the "logger" feature flag from `radicle`.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
`cargo check` fails because `human-panic` and `sysinfo` require at least
1.88.0.
+++ command cargo check --release --locked --all-targets
error: rustc 1.85.0 is not supported by the following packages:
human-panic@2.0.6 requires rustc 1.88
sysinfo@0.37.2 requires rustc 1.88
Bump MSRV to fix this.
1.88.0 introduced [let chains], which in turn has `clippy` warn about
nested if statements. All of these sites are fixed in this change.
1.87.0 introduced [`is_multiple_of`], which is a more readable version
of `x % y == 0`.
[let chains]: https://blog.rust-lang.org/2025/06/26/Rust-1.88.0/#let-chains
[`is_multiple_of`]: https://doc.rust-lang.org/std/primitive.usize.html#method.is_multiple_of
- Fixes a bug where the log level set in the config file was
ignored: `Logger` and `StderrLogger` captured the level in a
`self.level` field at construction time and checked it in
`Log::enabled`. After config was loaded, the global
`log::set_max_level` was updated but `self.level` was not,
so verbose messages were dropped by the per-instance filter
even when the global filter allowed them.
- Make `log::set_max_level` the single source of truth: remove
the `level` field and have `Log::enabled` defer to
`log::max_level()` so the two filters can no longer drift.
- Update call sites to construct loggers without a level.
- Disable the structured logger's internal filter (set to "trace") so
that it also falls back to `log::set_max_level`.
Canonical references can not be used to model symbolic references.
Relax this restriction by adding another member to the payload, named
"symbolic". Its key-value/name-target pairs then translate directly to
canonical symbolic references.
Care is taken to not allow circular references, and that there always
is a rule that could generate the target of a symbolic reference
(or a chain of symbolic references). Still, a symbolic reference may
dangle (for example when its target reference cannot be computed
because of divergence), but at least it can be prevented that a
symref *always* dangles, because there is no rule that would produce
its target.
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Update the output of `rad_jj_colocated_patch` so that it passes with the current version of `jj` being used:
```
jj --version
jj 0.35.0
```
Mark `rad_jj_bare` as `ignore` since it currently cannot determine the
RID from the `rad` remote.
The name `print_inline` was used because, previously, the `print`
function name was taken.
This has now changed, allowing the rename of this function to happen.
The name `print` was used because, previously, the `println`
function name was taken.
This has now changed, allowing the rename of this function to happen.
Prevent `rad` from panicking when its stdout is a closed pipe, e.g.
when running `rad config | head`.
The broken pipes are handled at the output boundaries using three
layers of defence:
1. Inner: Route all stdout output through `term::print()` and
`term::print_inline()`, which use explicit `writeln!`/`write!` to
a locked stdout and silently ignore write errors. All `println!`
calls in `radicle-cli` and `radicle-term` are converted.
2. Middle: Catch `BrokenPipe` errors that propagate up via `anyhow`
from subcommands in `main::run()`, and exit cleanly with code 0.
3. Outer: Install a panic hook that intercepts `println!` panics
containing "Broken pipe" (from dependencies like clap) and exits
cleanly, chained in front of human_panic's hook.
To prevent regressions, `#![deny(clippy::print_stdout)]` is added to
`radicle-cli`, requiring all future stdout output to go through the
safe `term::` functions.
Note: `eprintln!` calls (stderr) are not handled, as broken stderr
pipes are extremely rare in practice.
Rust (since 1.62) ignores EPIPE by default (see [Rust #62569]),
causing writes to closed pipes to return `io::ErrorKind::BrokenPipe`
errors. The `println!` macro panics on these errors, producing a
confusing backtrace instead of the silent exit expected of Unix CLI
tools.
To avoid the use of `print!` and `println!`, the `radicle-term`
helper functions use `write!` and `writeln!`. This lays the groundwork
for the `rad` CLI to transition to using only `radicle-term`
functions.
There was a first attempt made, which is documented here. A
process-wide SIGPIPE reset (`SIG_DFL`) was ruled out because `rad`
manages child processes via libgit2 pipes and communicates with the
radicle node over Unix sockets. Resetting SIGPIPE globally caused
`rad` itself to be killed during internal pipe operations (e.g. during
`rad remote add --fetch`), producing flaky test failures.
[Rust #62569] https://github.com/rust-lang/rust/issues/62569
Add tests that verify the `rad` binary does not panic when its stdout
is a broken pipe (e.g., `rad config | head -1`).
Rust (since 1.62) ignores SIGPIPE by default (see [Rust #62569]),
causing `println!` to panic with "failed printing to stdout: Broken
pipe" instead of exiting silently. This is the standard Unix behaviour
expected of CLI tools.
These tests currently fail on `rad config` and `rad self`, and are set to `ignore`, to be enabled when fixed.
[Rust #62569]: https://github.com/rust-lang/rust/issues/62569
`fn sha1_zero` can be `const fn`, and since it does not take arguments,
it can just be a `const`. There is no good reason to not have a constant.
Justification for changing the order to first specify "zero" and then
"SHA1": In the future we will support multiple object formats. A
function that produces a zero OID would be parameterized by the object
format then. Imagine `fn zero(format: ObjectFormat)` which would be
called as `Oid::zero(ObjectForma::Sha1)`. In anticipation, we change the
ordering to match.
Two tests assumed that `refs/remotes/rad/HEAD` would automatically be
crated. It turns out, however, that this only the case if executed
with Git 2.48.0 or newer. Git versions older than 2.48.0 do not touch
'refs/remotes/<remote>/HEAD'.
Git 2.48.0 and newer will update `refs/remotes/<remote>/HEAD` by
default. This can be disabled by setting
`remote.<remote>.followRemoteHEAD` to "never". Thus, with these
versions, setting that configuration to "never" emulates the behaviour
of older versions.
Therefore, to ensure that test results are consistent across Git
versions before and after 2.48.0, the fixture of test repositories
now include setting 'remote.rad.followRemoteHEAD = never'.
The affected tests are adjusted accordingly.
This change can possibly be reverted once Radicle requires usage of Git
2.48.0 or newer.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.xyz>
Maciek "mab122" Bator
Fintan Halpenny
Lorenz Leutgeb
Fred Gobry
Adrian Duke
stefan
Arthur Mariano
Fabrice Bellamy
Lorenz Leutgeb
Daniel Norman
Josh Soref
Richard Levitte