`rad lfs fetch` still exists for single-object/manual use, but
`radicle-lfs-transfer` used to spawn a fresh one per object during
`git lfs pull`/checkout -- for a private repo, each one independently
loaded the signer, so N files meant N passphrase prompts. Same bug
class `rad lfs precommit` already fixed for commits, unfixed here
until now.
Unlike precommit (which knows every staged file upfront and can read
stdin until EOF), fetch can't collect a batch before starting:
git-lfs's custom-transfer protocol requests objects one at a time.
`rad lfs fetch-batch` instead stays alive for as long as its caller
keeps its stdin open, answering one "<oid> <size> <out-path>" request
at a time with a JSON response line, reusing the same unwrapped
signer across all of them -- paired with radicle-lfs-transfer's new
FetchWorker, which spawns this once per transfer session instead of
once per object.
Also factors `fetch::run`'s core logic out into `fetch_object`,
taking a `&mut Option<MemorySigner>` the same way `store::store_object`
already does, so both the single-shot `rad lfs fetch` and the new
batch worker share one implementation.
Two known gaps, both flagged as follow-up work in LFS-IPFS.md:
`rad unseed` unconditionally unpinned every CID a repository's LFS
notes referenced. Two repositories that happen to track
byte-identical file content get the same content-addressed CID, so
unseeding one could silently evict content another still-seeded
repository needed -- that repo's next `git lfs fetch` would then fail
with no obvious cause. Fixed by checking every other currently-seeded
repository's own LFS notes before unpinning, and skipping any CID
still referenced elsewhere. No persistent refcount is kept anywhere;
this recomputes the still-needed set from git-notes each time,
consistent with the rest of this design treating notes as the single
source of truth rather than maintaining separate bookkeeping that
could drift out of sync.
`rad lfs backfill` retroactively pins any LFS-tracked file at HEAD
that doesn't yet have a note -- the situation a `--no-verify` commit
or a commit made while `ipfs`/`rad` weren't on `PATH` leaves behind
(the pre-commit hook soft-skips pinning in the latter case rather
than blocking the commit entirely). Content is read from Git LFS's
own local object cache (populated by the clean filter on `git add`
regardless of whether the hook ran), falling back to the working-tree
file. Reports what it backfilled, what already had a note, and what
it couldn't find content for locally (meaning some other peer that
does have it needs to run this instead).
Previously the pre-commit hook shelled out to `rad lfs store` once per
staged LFS file, each a separate process. For private repos, each of
those needs a keystore passphrase for an ECDH key-agreement operation,
so a multi-file commit meant a separate interactive passphrase prompt
per file -- observed live against the blog repo (9+ prompts in a row).
Add `rad lfs precommit`, which reads every staged file's oid/size/path
from stdin and processes them all in one process, loading the signer
at most once and reusing it across the batch. `store::run` now
delegates to a shared `store_object` helper that both it and
`precommit::run` call.
Radicle's Visibility::Private is enforced entirely by access control at
the replication layer -- git objects for a private repo are plain,
unencrypted objects, protected only by "unauthorized peers are never
given them" (identity/doc.rs). That protection has no jurisdiction over
IPFS's own block-exchange layer: once a legitimate collaborator's IPFS
daemon pins content and joins the network, anyone who obtains the CID by
any means can fetch the plaintext directly.
Adds envelope encryption for private repos: a random per-object content
key encrypts the file (XChaCha20-Poly1305), wrapped once per authorized
recipient (delegates + private allow-list) via ECDH between Radicle
identity keys (already Ed25519, no separate keypair needed) plus HKDF.
The wrapped keys travel in the same refs/notes/rad-lfs note as the CID.
New:
- crates/radicle-cli/src/lfs_crypto.rs -- encryption core (envelope
schema, encrypt/decrypt, key wrapping, recipient resolution, signer
loading).
- `rad lfs store`/`rad lfs fetch` -- plumbing commands that now own the
entire IPFS add/cat + git-notes read/write step (encrypting/decrypting
as needed), replacing what the pre-commit hook and rad-lfs-transfer
used to do directly. This collapses what would've become a third copy
of the note-format logic down to one canonical implementation.
- `rad lfs rekey` -- since Radicle's access control is retroactive (a
newly-authorized DID can replicate full history) but encrypted objects
aren't automatically re-wrapped for new recipients, this command lets
an already-authorized collaborator grant a newly-authorized one access
to previously-committed objects, without re-encrypting or re-uploading
content. Revocation remains unsolved, matching Radicle's own model
(already-replicated/decrypted content can't be un-known).
Operational requirement: private-repo LFS operations need a signer
capable of ECDH, which only an unencrypted keystore or RAD_PASSPHRASE
provides (an ssh-agent-only signer can sign but not do key agreement).
Fails fast with an actionable error rather than hanging on a passphrase
prompt inside a non-interactive git hook.
Breaking change: the refs/notes/rad-lfs note format moves from bare
`cid=<cid>` text to JSON (`{"v":1,"cid":"...","enc":...}`). No migration
path -- prototype, no external users yet.
Verified end-to-end with real Radicle profiles/identities (not just the
transfer-agent protocol): a private repo's IPFS-stored content is
confirmed ciphertext (byte-different from plaintext, +16 bytes AEAD tag);
an authorized peer decrypts correctly; an unauthorized peer is denied
with a clear error and no leaked content; RAD_PASSPHRASE is required and
enforced with a fast, clear failure; and rekey correctly grants a
newly-authorized peer access to a pre-existing object while leaving other
recipients' entries untouched.
Adds Git LFS support for repositories, with large file content stored on
each contributor's own local IPFS node instead of a seed-hosted server:
- `rad lfs init`: one-time setup (transfer-agent config, pre-commit hook,
notes-ref push/fetch refspecs, IPFS daemon reachability check).
- `refs/notes/rad-lfs` carries the oid -> CID mapping, replicating with
the repository via Radicle's existing ref sync (verified against
`references_of` in crates/radicle/src/storage/git.rs: every ref is
replicated except refs/tmp/heads/*, so no protocol changes were needed).
- `rad seed`/`rad unseed` now pin/unpin a repo's known LFS objects in IPFS,
mirroring Radicle's existing "seeding = keep a full copy" model, and
warning (not failing) if no local IPFS daemon is reachable.
The actual byte transfer is handled by a separate binary, rad-lfs-transfer
(added here as a submodule), which implements the Git LFS custom-transfer-
agent protocol against a local IPFS daemon. See LFS-IPFS.md for the full
design and setup instructions.
Building and using `rad` for anything other than `rad lfs` requires no
IPFS dependency at all; `rad lfs init` checks for a local daemon upfront
and fails with an actionable message rather than proceeding silently.
Maciek "mab122" Bator