d67ac79f40
Private Radicle repos aren't encrypted at the object level -- access control happens at Radicle's replication layer, which has no jurisdiction over IPFS's own block-exchange layer. Once content is pinned and joins the public IPFS network, anyone who obtains the CID can fetch it. A companion change in radicle-heartwood-lfs adds client-side envelope encryption for private repos, via two new plumbing commands: `rad lfs store` and `rad lfs fetch`. This crate now shells out to those instead of talking to Kubo/git-notes directly: upload spawns `rad lfs store`, download spawns `rad lfs fetch`. Removed src/notes.rs entirely (no longer needed -- the Radicle-side commands own all of that, including note reads/writes) and trimmed kubo.rs down to just the init-event healthcheck. This crate still has no dependency on Radicle's identity/crypto stack, and no longer needs `git` on PATH at all -- only `rad` and a local IPFS daemon. |
||
|---|---|---|
| src | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| README.md | ||
README.md
radicle-lfs-transfer
A Git LFS custom transfer agent that backs large file storage with IPFS, for use with Radicle repositories.
This is the companion binary for the rad lfs support added in
radicle-heartwood-lfs (a fork of
Radicle's core node/CLI, heartwood). Most people want to start there, not here — that
repo has the full zero-to-usable setup guide. This repo is what you're reading if you want to
build/understand the transfer agent on its own, or you're following a link back from there.
rad-lfs-transfer is not meant to be invoked directly — git lfs spawns it automatically,
once a repository has been set up with rad lfs init (from the fork above).
How it works
Every peer uses their own local IPFS node — there's no shared server. All of the actual
IPFS add/pin work and git-notes bookkeeping (refs/notes/rad-lfs) lives in the rad lfs store/rad lfs fetch subcommands, implemented in the companion
radicle-heartwood-lfs fork — that's
also where content gets encrypted for private repositories before it ever reaches IPFS. This
crate deliberately has no dependency on Radicle's identity/crypto stack; it just shells out:
- On
git add/git commit, a pre-commit hook (installed byrad lfs init) runsrad lfs storeon each staged LFS-tracked file, which adds it to your local IPFS daemon, pins it, encrypts it first if the repo is private, and records the resulting CID as a git note. That notes ref travels with the repository via Radicle's normal replication, alongside everything else. rad-lfs-transferimplements the other half: whengit lfs push/pullruns, it's invoked per-object over stdin/stdout JSON (the standard Git LFS custom-transfer-agent protocol) and:- upload: shells out to
rad lfs store --oid <oid> --size <size> <path>, which is authoritative and idempotent — it always ends up added, pinned, and noted correctly, whether or not the pre-commit hook already did it (e.g. if it was bypassed with--no-verify). - download: shells out to
rad lfs fetch --oid <oid> --size <size> --out <path>, which looks up the note, fetches from IPFS (pulling from the wider network if not local yet), decrypts if needed, and writes the plaintext to<path>.
- upload: shells out to
There is no central server anywhere in this design — see
radicle-heartwood-lfs's LFS-IPFS.md
for the full rationale and how rad seed/rad unseed tie IPFS pinning to Radicle's existing
seeding lifecycle.
Dependencies
On Arch Linux:
# To build and install
sudo pacman -S --needed rust
# To actually use it (not needed to build/install — see "Requirements" below)
sudo pacman -S --needed kubo
On other distributions, install a Rust toolchain (e.g. via rustup) — and, only when you want to use LFS, not to build this — Kubo.
Requirements
None of these are needed to build or install this binary — only at runtime, when git lfs push/pull actually invokes it:
- The
radbinary (built from radicle-heartwood-lfs) on yourPATH, with the repository already set up viarad lfs init.rad-lfs-transfershells out torad lfs store/rad lfs fetchfor every object — it does not talk to git notes or IPFS directly itself anymore (that's all inside those subcommands now), and it has no dependency ongiteither. - A local IPFS (Kubo) daemon reachable at
http://127.0.0.1:5001(or whereverKUBO_API_URLpoints) — used both byrad lfs store/fetchthemselves and by this binary's own startup healthcheck on theinitevent, so a daemon-not-running failure is reported clearly up front (ipfs daemon) rather than confusingly mid-transfer. - Git LFS installed (
git-lfson yourPATH) —git-lfspackage on Arch (sudo pacman -S --needed git-lfs).
Build & install
git clone ssh://git@git.hswro.org:9022/mab122/radicle-lfs-transfer.git
cd radicle-lfs-transfer
cargo install --path .
This installs the rad-lfs-transfer binary to ~/.cargo/bin by default — make sure that's on
your PATH. If you're installing it alongside radicle-heartwood-lfs (the usual case), install
it to that project's shared root instead so everything ends up in one place:
cargo install --path . --force --locked --root ~/.radicle
(You normally don't need to clone this repo separately at all for that — radicle-heartwood-lfs
includes it as a git submodule and its own setup instructions cover installing both together.)
Configuration
| Environment variable | Default | Purpose |
|---|---|---|
KUBO_API_URL |
http://127.0.0.1:5001 |
Base URL of your local Kubo HTTP RPC API |
You normally don't need to set this yourself — rad lfs init wires up everything else
(the transfer-agent config, the pre-commit hook, and the notes-ref refspecs) automatically.
License
Licensed under either of Apache License, Version 2.0 or MIT license at your option.