`rad lfs init` previously only configured a push refspec for the
notes ref (`+refs/notes/rad-lfs/local:refs/notes/rad-lfs`) -- once
any explicit push refspec exists on a remote, git stops falling back
to push.default-driven behavior for a bare `git push rad`, so it only
ever pushed the notes mapping, never the branch. That's the exact
footgun that just caused a real "blog did not update" report: `git
push rad` (bare) after a commit only sent the notes ref, silently
leaving the commit itself unpushed.
Add a second push refspec, `+refs/heads/*:refs/heads/*`, mirroring
the existing `+refs/heads/*:refs/remotes/rad/*` fetch refspec already
configured. A bare `git push rad` now pushes every local branch and
the notes mapping together in one command.
Only takes effect once `rad lfs init` has been (re-)run with this
build -- it's an additive, idempotent config change via the existing
`ensure_refspec` helper, no migration/cleanup needed since there's no
stale conflicting entry to remove this time. Updated `rad lfs init`'s
own success/info messages plus README.md/LFS-IPFS.md/
radicle-lfs-transfer's README to match -- the "push gotcha" workaround
they documented is now the *old*-repository fallback path, not the
primary instructions.
There is no `rad push`/`rad pull` subcommand -- `rad`'s actual
subcommand set is auth/checkout/clone/.../sync/watch (see `rad
--help`); pushing/pulling git refs (branch, notes) is done via plain
`git push rad`/`git pull rad`, same as any git remote. LFS-IPFS.md's
"Use" section and `rad lfs init`'s own printed success message both
said `rad push`/`rad pull`, which doesn't exist and doesn't even
gesture at the real push gotcha (branch and notes ref need two
separate pushes).
Also updates LFS-IPFS.md's "How it works" section, which still
described the pre-batching design (a pre-commit hook calling `rad lfs
store` once per file, and the bare `refs/notes/rad-lfs` as if it were
the local write target) -- both stale since the LOCAL_NOTES_REF fix
and the `rad lfs precommit` batching added this session. `rad lfs
init`'s own success message now proactively states the push gotcha,
rather than only documenting it after the fact in Troubleshooting.
Adds a copy-pasteable day-to-day cheatsheet (init, track/commit/push,
clone-elsewhere, private-repo rekey) right after the existing "Zero to
usable" build section. Also corrects/expands the existing
git-push-rad troubleshooting row in LFS-IPFS.md based on today's live
round-trip test: `rad lfs init` only configures a push refspec for the
notes ref, not the branch, so pushing a commit needs both
`git push rad <branch>` and a separate bare `git push rad` -- missing
the second step is what produces "no CID recorded for oid" on a
collaborator's clone.
Private-repo LFS operations need a signer capable of ECDH, which
ssh-agent (the normal day-to-day flow after `rad auth`) can't provide --
that's a hard protocol limitation (standard ssh-agent only implements
signing, there's no key-agreement request type), not a gap in our code,
so there's no way to make it work "through the agent" the way signing
does.
Previously the only alternative to an unencrypted keystore was setting
RAD_PASSPHRASE, which breaks the normal ssh-agent-based workflow and
means putting a password in an environment variable -- not ergonomic,
and not how any other `rad` command behaves.
`rad` already has an established fallback for exactly this situation:
`crate::terminal::io::signer` tries ssh-agent first, and if that's
unavailable/unregistered and we're connected to a TTY, prompts
interactively for the passphrase via `inquire` (reusing the existing
`PassphraseValidator`). `lfs_crypto::load_signer` was missing this last
step entirely -- it only tried "unencrypted keystore" then
"RAD_PASSPHRASE" then gave up. Added the same interactive-prompt
fallback, reusing the exact same validator/prompt helpers already used
elsewhere, before falling back to a hard failure.
This covers the common case (an interactive `git commit` triggering the
pre-commit hook, which inherits the terminal's TTY) with no environment
variable needed at all. Only genuinely non-interactive contexts (CI, a
script with no TTY) still need RAD_PASSPHRASE -- and get a fast, clear
failure rather than hanging, since `inquire` returns `Ok(None)` on
`NotTTY` rather than blocking (verified: piping /dev/null as stdin still
fails immediately, not a timeout).
Verified against a real encrypted keystore through an actual pty (via
`script`, since a plain non-interactive shell doesn't have a TTY to
prompt on): the masked "Passphrase:" prompt appears, accepts the typed
passphrase, and produces a correctly encrypted note -- with
RAD_PASSPHRASE unset the entire time.
Found in a real deployment (mbator.pl blog pipeline): rad lfs init
configures a client-side fetch refspec for refs/notes/rad-lfs, but
git-remote-rad's for_fetch() never advertised any such ref -- notes only
exist per-peer, under refs/namespaces/<peer>/refs/notes/rad-lfs, with no
canonicalization step the way refs/heads/refs/tags get one. Requesting an
unadvertised refspec fails the *entire* git fetch, not just that one ref,
so any repo with `rad lfs init` run could no longer git fetch/pull at
all, LFS or not.
Fix: expose each peer's refs/notes/rad-lfs individually, as
refs/notes/rad-lfs/<peer-id> (list.rs's for_fetch), matching how patch
refs already work rather than trying to canonicalize -- notes are
per-peer contributions (any peer can commit an LFS-tracked file, not
just delegates), so unlike heads there's no single "correct" value to
resolve to; canonicalizing would silently drop non-delegate contributors'
objects. rad lfs init's fetch refspec becomes a wildcard
(+refs/notes/rad-lfs/*:refs/notes/rad-lfs/*); push is unchanged (already
worked -- for_push() turns out to be purely informational for git's
list-for-push handshake, not an enforcement gate, so the actual send-pack
call was never restricted to heads/tags despite what for_push() lists).
rad lfs store/fetch/rekey and seed/unseed's pinning (ipfs.rs::lfs_cids)
now read across every notes ref instead of assuming one canonical ref,
merging recipient lists for notes that share a CID (e.g. after a rekey
performed by a different peer than the original committer) while keeping
genuinely different ciphertexts (from two peers independently encrypting
byte-identical content) as separate candidates to try in turn.
Also added a migration step so re-running `rad lfs init` on an
already-configured repo actually fixes it, by removing the old
non-wildcard fetch refspec rather than just adding the wildcard
alongside a still-broken entry.
Verified through the real git-remote-rad transport (not the storage-copy
shortcut used for earlier testing, since this bug lives specifically in
that transport): fresh identities, a real rad:// remote matching exactly
what `rad init` configures, a genuine `git fetch rad` that used to fail
with "fatal: couldn't find remote ref refs/notes/rad-lfs" now succeeds
and pulls the notes ref, and `git lfs pull` afterward correctly decrypts
private content end-to-end (SHA-256 verified). Re-verified the rekey
flow through the same real path too.
Radicle's Visibility::Private is enforced entirely by access control at
the replication layer -- git objects for a private repo are plain,
unencrypted objects, protected only by "unauthorized peers are never
given them" (identity/doc.rs). That protection has no jurisdiction over
IPFS's own block-exchange layer: once a legitimate collaborator's IPFS
daemon pins content and joins the network, anyone who obtains the CID by
any means can fetch the plaintext directly.
Adds envelope encryption for private repos: a random per-object content
key encrypts the file (XChaCha20-Poly1305), wrapped once per authorized
recipient (delegates + private allow-list) via ECDH between Radicle
identity keys (already Ed25519, no separate keypair needed) plus HKDF.
The wrapped keys travel in the same refs/notes/rad-lfs note as the CID.
New:
- crates/radicle-cli/src/lfs_crypto.rs -- encryption core (envelope
schema, encrypt/decrypt, key wrapping, recipient resolution, signer
loading).
- `rad lfs store`/`rad lfs fetch` -- plumbing commands that now own the
entire IPFS add/cat + git-notes read/write step (encrypting/decrypting
as needed), replacing what the pre-commit hook and rad-lfs-transfer
used to do directly. This collapses what would've become a third copy
of the note-format logic down to one canonical implementation.
- `rad lfs rekey` -- since Radicle's access control is retroactive (a
newly-authorized DID can replicate full history) but encrypted objects
aren't automatically re-wrapped for new recipients, this command lets
an already-authorized collaborator grant a newly-authorized one access
to previously-committed objects, without re-encrypting or re-uploading
content. Revocation remains unsolved, matching Radicle's own model
(already-replicated/decrypted content can't be un-known).
Operational requirement: private-repo LFS operations need a signer
capable of ECDH, which only an unencrypted keystore or RAD_PASSPHRASE
provides (an ssh-agent-only signer can sign but not do key agreement).
Fails fast with an actionable error rather than hanging on a passphrase
prompt inside a non-interactive git hook.
Breaking change: the refs/notes/rad-lfs note format moves from bare
`cid=<cid>` text to JSON (`{"v":1,"cid":"...","enc":...}`). No migration
path -- prototype, no external users yet.
Verified end-to-end with real Radicle profiles/identities (not just the
transfer-agent protocol): a private repo's IPFS-stored content is
confirmed ciphertext (byte-different from plaintext, +16 bytes AEAD tag);
an authorized peer decrypts correctly; an unauthorized peer is denied
with a clear error and no leaked content; RAD_PASSPHRASE is required and
enforced with a fast, clear failure; and rekey correctly grants a
newly-authorized peer access to a pre-existing object while leaving other
recipients' entries untouched.
Cross-repo markdown links (radicle-lfs-transfer) now use full https URLs
instead of relative paths, since Forgejo can't resolve ../other-repo across
repositories the way a plain filesystem or the git submodule mechanism can.
Adds Git LFS support for repositories, with large file content stored on
each contributor's own local IPFS node instead of a seed-hosted server:
- `rad lfs init`: one-time setup (transfer-agent config, pre-commit hook,
notes-ref push/fetch refspecs, IPFS daemon reachability check).
- `refs/notes/rad-lfs` carries the oid -> CID mapping, replicating with
the repository via Radicle's existing ref sync (verified against
`references_of` in crates/radicle/src/storage/git.rs: every ref is
replicated except refs/tmp/heads/*, so no protocol changes were needed).
- `rad seed`/`rad unseed` now pin/unpin a repo's known LFS objects in IPFS,
mirroring Radicle's existing "seeding = keep a full copy" model, and
warning (not failing) if no local IPFS daemon is reachable.
The actual byte transfer is handled by a separate binary, rad-lfs-transfer
(added here as a submodule), which implements the Git LFS custom-transfer-
agent protocol against a local IPFS daemon. See LFS-IPFS.md for the full
design and setup instructions.
Building and using `rad` for anything other than `rad lfs` requires no
IPFS dependency at all; `rad lfs init` checks for a local daemon upfront
and fails with an actionable message rather than proceeding silently.
Maciek "mab122" Bator