radicle-heartwood-lfs/LFS-IPFS.md

Git LFS support, backed by IPFS

This is a fork of Radicle's heartwood adding Git LFS (large file) support, with large file content stored on each contributor's own local IPFS node rather than on a central server.

Why not a server?

Radicle's own FAQ notes that its first-generation protocol was built on IPFS and moved away from it — IPFS is a content-addressed store, and repositories are mutable, so it wasn't a fit for Radicle's core storage. That objection doesn't apply here: Git LFS's own pattern already separates a small, mutable pointer file (which Radicle replicates exactly as it replicates any other git object) from immutable large-file content addressed by hash — which is precisely what IPFS is good at. This fork only uses IPFS for that second part.

There's no server anywhere in this design: every peer uses their own local IPFS node, and the file-CID mapping travels with the repository itself, avoiding a single point of failure.

How it works

• The mapping: a cid=<cid> git note on refs/notes/rad-lfs, attached to each LFS pointer file's own git blob hash (not the LFS oid — a different value). Verified against this repo's actual replication code (references_of in crates/radicle/src/storage/git.rs) that Radicle replicates every ref under a peer's namespace except refs/tmp/heads/* — there's no fixed allowlist, so this notes ref replicates the same way refs/heads/refs/tags/refs/cobs do, once the push/fetch refspecs below are configured.
• On commit: a pre-commit hook (installed by rad lfs init) adds each staged LFS-tracked file to your local IPFS daemon, pins it, and writes the note above.
• On push/pull: the actual bytes move through rad-lfs-transfer, a Git LFS custom transfer agent — see that repo for details. It's a separate binary/repository since it has nothing Radicle-specific in it; it only needs a local IPFS daemon and a git repository with the notes above.
• Seeding: rad seed now also pins a repository's known LFS objects in your local IPFS node (mirroring "seeding = keep a full copy"); rad unseed unpins them. Known prototype limitation: no cross-repository pin refcounting, so unseeding one repository can unpin content still needed by another seeded repository that happens to share the same CID.

Prerequisites

• A Rust toolchain (stable; see rust-toolchain.toml for the exact version this workspace pins), Git, and OpenSSH.
• Git LFS (git-lfs on your PATH) — only needed to use rad lfs, not to build or install anything.
• A local IPFS (Kubo) daemon — also only needed to use rad lfs, not to build/install. Install Kubo and make sure ipfs is on your PATH.

On Arch Linux:

sudo pacman -S --needed rust git openssh base-devel   # build
sudo pacman -S --needed git-lfs kubo                  # use `rad lfs`

Build & install

Follows upstream heartwood's own install convention (cargo install --root ~/.radicle, one shared install root for the whole stack), with radicle-lfs-transfer added the same way:

git clone --branch rad-lfs-ipfs --recurse-submodules \
  ssh://git@git.hswro.org:9022/mab122/radicle-heartwood-lfs.git
cd radicle-heartwood-lfs

cargo install --path crates/radicle-cli --force --locked --root ~/.radicle
cargo install --path crates/radicle-node --force --locked --root ~/.radicle
cargo install --path crates/radicle-remote-helper --force --locked --root ~/.radicle
cargo install --path radicle-lfs-transfer --force --locked --root ~/.radicle

Add ~/.radicle/bin to your PATH (e.g. in ~/.bashrc/~/.zshrc):

export PATH="$HOME/.radicle/bin:$PATH"

Confirm everything landed correctly:

rad --version
command -v rad-lfs-transfer   # just needs to resolve; it only speaks git-lfs's stdin/stdout protocol, no --help output

If you already cloned without --recurse-submodules, run git submodule update --init before building — otherwise radicle-lfs-transfer/ will be an empty directory and that last cargo install step will fail.

Run

Nothing extra to run for rad/radicle-node themselves — use them exactly as upstream (rad auth, radicle-node, etc.; see the main README and docs/). The only new moving part is the IPFS daemon, needed only when you use rad lfs:

ipfs init      # first time only, creates ~/.ipfs
ipfs daemon    # leave running in the background while you use `rad lfs`

Use

Inside a Radicle repository you're a contributor on (i.e. it already has a rad remote from rad init/rad clone):

# One-time per repository (with the IPFS daemon above already running)
rad lfs init

# Then it's just Git LFS, as normal
git lfs track "*.psd"          # or whatever large-file patterns you need
git add .gitattributes
git add my-large-file.psd
git commit -m "Add asset"      # pre-commit hook pins it to IPFS and records the CID here
rad push                       # pushes the commit and the refs/notes/rad-lfs mapping together

Someone else cloning the same repository:

rad clone rad:<repo-id>
cd <repo>
rad lfs init                   # one-time, same as above (needs their own IPFS daemon running)
git lfs pull                   # fetches large files via IPFS instead of git

rad seed/rad unseed additionally pin/unpin a repository's known LFS objects in your local IPFS node, so seeding a repository keeps a full copy of its large files too, not just its git history.

What happens without an IPFS daemon running

• rad lfs init checks upfront and refuses with a clear message (no IPFS (Kubo) daemon reachable at http://127.0.0.1:5001 — start one with 'ipfs daemon') rather than silently continuing.
• rad seed/rad unseed print a warning and skip pinning/unpinning — seeding/unseeding itself still succeeds.
• Nothing else in this fork needs IPFS at all. Building, installing, and using rad for anything other than rad lfs/pinning works exactly like upstream heartwood, with zero IPFS dependency.

Troubleshooting

Symptom	Cause / fix
`rad lfs init` must be run inside a Radicle repository working copy	Run it inside a directory that already has a rad remote (from rad init or rad clone), not a plain git repo.
Git LFS is not installed (the 'git-lfs' binary was not found on your PATH)	Install Git LFS.
no IPFS (Kubo) daemon reachable at ...	Start one: ipfs daemon (see Run above).
rad-lfs-transfer: command not found during git lfs push/pull	cargo install --path radicle-lfs-transfer didn't complete, or ~/.radicle/bin isn't on PATH.
Large file didn't get pinned (commit went through, but no rad-lfs note)	The pre-commit hook silently skips pinning if the ipfs CLI isn't on PATH — check command -v ipfs. Re-add the file and commit again once it is.

Status

This is a prototype, not upstream Radicle functionality. See the commits on the rad-lfs-ipfs branch for the full change set relative to upstream heartwood.