Delegate IPFS/notes work to rad lfs store/fetch, add encryption support

Private Radicle repos aren't encrypted at the object level -- access
control happens at Radicle's replication layer, which has no jurisdiction
over IPFS's own block-exchange layer. Once content is pinned and joins
the public IPFS network, anyone who obtains the CID can fetch it. A
companion change in radicle-heartwood-lfs adds client-side envelope
encryption for private repos, via two new plumbing commands: `rad lfs
store` and `rad lfs fetch`.

This crate now shells out to those instead of talking to Kubo/git-notes
directly: upload spawns `rad lfs store`, download spawns `rad lfs fetch`.
Removed src/notes.rs entirely (no longer needed -- the Radicle-side
commands own all of that, including note reads/writes) and trimmed
kubo.rs down to just the init-event healthcheck. This crate still has no
dependency on Radicle's identity/crypto stack, and no longer needs `git`
on PATH at all -- only `rad` and a local IPFS daemon.

README.md

@@ -16,19 +16,27 @@ once a repository has been set up with `rad lfs init` (from the fork above).
 
 ## How it works
 
-Every peer uses their own local IPFS node — there's no shared server:
+Every peer uses their own local IPFS node — there's no shared server. All of the actual
+IPFS add/pin work and git-notes bookkeeping (`refs/notes/rad-lfs`) lives in the `rad lfs
+store`/`rad lfs fetch` subcommands, implemented in the companion
+[**radicle-heartwood-lfs**](https://git.hswro.org/mab122/radicle-heartwood-lfs) fork — that's
+also where content gets encrypted for private repositories before it ever reaches IPFS. This
+crate deliberately has no dependency on Radicle's identity/crypto stack; it just shells out:
 
-- On `git add`/`git commit`, a pre-commit hook (installed by `rad lfs init`) adds each staged
+- On `git add`/`git commit`, a pre-commit hook (installed by `rad lfs init`) runs `rad lfs
-  LFS-tracked file to your local IPFS daemon, pins it, and records the resulting CID as a git
+  store` on each staged LFS-tracked file, which adds it to your local IPFS daemon, pins it,
-  note (`refs/notes/rad-lfs`) attached to the LFS pointer file's own git blob hash. That notes
+  encrypts it first if the repo is private, and records the resulting CID as a git note. That
-  ref travels with the repository via Radicle's normal replication, alongside everything else.
+  notes ref travels with the repository via Radicle's normal replication, alongside everything
+  else.
 - `rad-lfs-transfer` implements the other half: when `git lfs push`/`pull` runs, it's invoked
   per-object over stdin/stdout JSON (the standard Git LFS custom-transfer-agent protocol) and:
-  - **upload**: confirms the object is already pinned in IPFS (it should be, from the
+  - **upload**: shells out to `rad lfs store --oid <oid> --size <size> <path>`, which is
-    pre-commit hook) or adds it if the hook was bypassed.
+    authoritative and idempotent — it always ends up added, pinned, and noted correctly,
-  - **download**: reconstructs the expected pointer blob hash from the requested oid/size,
+    whether or not the pre-commit hook already did it (e.g. if it was bypassed with
-    looks up its `cid=` note, and fetches the content from your local IPFS daemon (which will
+    `--no-verify`).
-    pull it from the wider IPFS network if it isn't local yet).
+  - **download**: shells out to `rad lfs fetch --oid <oid> --size <size> --out <path>`, which
+    looks up the note, fetches from IPFS (pulling from the wider network if not local yet),
+    decrypts if needed, and writes the plaintext to `<path>`.
 
 There is no central server anywhere in this design — see
 [radicle-heartwood-lfs's `LFS-IPFS.md`](https://git.hswro.org/mab122/radicle-heartwood-lfs/src/branch/rad-lfs-ipfs/LFS-IPFS.md)
@@ -41,22 +49,30 @@ On Arch Linux:
 
 ```sh
 # To build and install
-sudo pacman -S --needed rust git
+sudo pacman -S --needed rust
 
 # To actually use it (not needed to build/install — see "Requirements" below)
 sudo pacman -S --needed kubo
 ```
 
-On other distributions, install a Rust toolchain (e.g. via [rustup](https://rustup.rs)), Git,
+On other distributions, install a Rust toolchain (e.g. via [rustup](https://rustup.rs)) —
-and — only when you want to use LFS, not to build this — [Kubo](https://docs.ipfs.tech/install/).
+and, only when you want to use LFS, not to build this — [Kubo](https://docs.ipfs.tech/install/).
 
 ## Requirements
 
-- Only needed **at runtime**, not to build or install this binary: a local IPFS (Kubo) daemon
+None of these are needed to build or install this binary — only at runtime, when `git lfs
-  reachable at `http://127.0.0.1:5001` (or wherever `KUBO_API_URL` points). If it's not
+push`/`pull` actually invokes it:
-  running when `git lfs push`/`pull` tries to use it, you'll get a clear error telling you to
+
-  start one (`ipfs daemon`) — nothing fails silently, and nothing about building or installing
+- The `rad` binary (built from
-  this binary requires IPFS to be present at all.
+  [**radicle-heartwood-lfs**](https://git.hswro.org/mab122/radicle-heartwood-lfs)) on your
+  `PATH`, with the repository already set up via `rad lfs init`. `rad-lfs-transfer` shells out
+  to `rad lfs store`/`rad lfs fetch` for every object — it does not talk to git notes or IPFS
+  directly itself anymore (that's all inside those subcommands now), and it has no dependency
+  on `git` either.
+- A local IPFS (Kubo) daemon reachable at `http://127.0.0.1:5001` (or wherever `KUBO_API_URL`
+  points) — used both by `rad lfs store`/`fetch` themselves and by this binary's own startup
+  healthcheck on the `init` event, so a daemon-not-running failure is reported clearly up
+  front (`ipfs daemon`) rather than confusingly mid-transfer.
 - [Git LFS](https://git-lfs.com) installed (`git-lfs` on your `PATH`) — `git-lfs` package on
   Arch (`sudo pacman -S --needed git-lfs`).
 

src/kubo.rs

@@ -1,18 +1,19 @@
 use anyhow::{Context, bail};
-use serde::Deserialize;
 
+/// Thin client for the local Kubo (IPFS) daemon's HTTP RPC API.
+///
+/// All actual add/pin/cat work for LFS objects has moved to `rad lfs
+/// store`/`rad lfs fetch` (see `main.rs`), which run with access to the
+/// user's Radicle identity and can decide whether to encrypt content for
+/// private repos. The only thing this crate still talks to Kubo about
+/// directly is a startup healthcheck, so a daemon-not-running failure is
+/// reported clearly up front rather than confusingly mid-transfer.
 #[derive(Clone)]
 pub struct Kubo {
     client: reqwest::Client,
     api_url: String,
 }
 
-#[derive(Deserialize)]
-struct AddResponse {
-    #[serde(rename = "Hash")]
-    hash: String,
-}
-
 impl Kubo {
     pub fn new(api_url: String) -> Self {
         Self {
@@ -21,66 +22,6 @@ impl Kubo {
         }
     }
 
-    /// Adds bytes to IPFS using Kubo's normal chunked UnixFS add (default
-    /// params — no raw-leaves / single-block tricks) and returns the CID.
-    pub async fn add(&self, bytes: Vec<u8>) -> anyhow::Result<String> {
-        let part = reqwest::multipart::Part::bytes(bytes).file_name("blob");
-        let form = reqwest::multipart::Form::new().part("file", part);
-
-        let resp = self
-            .client
-            .post(format!("{}/api/v0/add", self.api_url))
-            .multipart(form)
-            .send()
-            .await
-            .context("failed to reach Kubo daemon for /api/v0/add")?;
-
-        if !resp.status().is_success() {
-            bail!("kubo add failed: HTTP {}", resp.status());
-        }
-
-        let body = resp.text().await?;
-        let line = body
-            .lines()
-            .find(|l| !l.trim().is_empty())
-            .context("empty response from kubo add")?;
-        let parsed: AddResponse = serde_json::from_str(line)
-            .with_context(|| format!("failed to parse kubo add response: {line}"))?;
-        Ok(parsed.hash)
-    }
-
-    /// Explicitly (recursively) pins a CID, decoupled from add-time pinning.
-    pub async fn pin_add(&self, cid: &str) -> anyhow::Result<()> {
-        let resp = self
-            .client
-            .post(format!("{}/api/v0/pin/add", self.api_url))
-            .query(&[("arg", cid), ("recursive", "true")])
-            .send()
-            .await
-            .context("failed to reach Kubo daemon for /api/v0/pin/add")?;
-
-        if !resp.status().is_success() {
-            bail!("kubo pin/add failed for {cid}: HTTP {}", resp.status());
-        }
-        Ok(())
-    }
-
-    /// Streams the full reassembled file content for a CID via /api/v0/cat.
-    pub async fn cat(&self, cid: &str) -> anyhow::Result<reqwest::Response> {
-        let resp = self
-            .client
-            .post(format!("{}/api/v0/cat", self.api_url))
-            .query(&[("arg", cid)])
-            .send()
-            .await
-            .context("failed to reach Kubo daemon for /api/v0/cat")?;
-
-        if !resp.status().is_success() {
-            bail!("kubo cat failed for {cid}: HTTP {}", resp.status());
-        }
-        Ok(resp)
-    }
-
     /// Verifies the local Kubo daemon is actually reachable, for a clear
     /// upfront error instead of a confusing failure mid-transfer.
     pub async fn healthcheck(&self) -> anyhow::Result<()> {

src/main.rs

@@ -1,8 +1,8 @@
 mod kubo;
-mod notes;
 
 use serde_json::{Value, json};
 use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
+use tokio::process::Command;
 
 use kubo::Kubo;
 
@@ -32,11 +32,11 @@ async fn main() -> anyhow::Result<()> {
                 write_line(&mut stdout, &resp).await?;
             }
             "upload" => {
-                let resp = handle_upload(&kubo, &msg).await;
+                let resp = handle_upload(&msg).await;
                 write_line(&mut stdout, &resp).await?;
             }
             "download" => {
-                let resp = handle_download(&kubo, &msg).await;
+                let resp = handle_download(&msg).await;
                 write_line(&mut stdout, &resp).await?;
             }
             "terminate" => break,
@@ -53,8 +53,8 @@ async fn main() -> anyhow::Result<()> {
     Ok(())
 }
 
-async fn handle_upload(kubo: &Kubo, msg: &Value) -> Value {
+async fn handle_upload(msg: &Value) -> Value {
-    match upload_inner(kubo, msg).await {
+    match upload_inner(msg).await {
         Ok(oid) => json!({"event": "complete", "oid": oid}),
         Err((oid, e)) => json!({
             "event": "complete",
@@ -64,42 +64,47 @@ async fn handle_upload(kubo: &Kubo, msg: &Value) -> Value {
     }
 }
 
-async fn upload_inner(kubo: &Kubo, msg: &Value) -> Result<String, (String, anyhow::Error)> {
+/// Delegates entirely to `rad lfs store`, which owns add/pin/note-write
+/// (and encrypts first, for private repos) on the Radicle side. It's
+/// authoritative and idempotent, so there's no separate "already pinned by
+/// the pre-commit hook" fast path to maintain here anymore.
+async fn upload_inner(msg: &Value) -> Result<String, (String, anyhow::Error)> {
     let oid = field_str(msg, "oid").map_err(|e| (String::new(), e))?;
     let size = field_i64(msg, "size").map_err(|e| (oid.clone(), e))?;
     let path = field_str(msg, "path").map_err(|e| (oid.clone(), e))?;
 
-    let blob_sha = notes::pointer_blob_hash(&oid, size)
+    let output = Command::new("rad")
+        .args(["lfs", "store", "--oid", &oid, "--size", &size.to_string(), "--", &path])
+        .output()
         .await
-        .map_err(|e| (oid.clone(), e))?;
+        .map_err(|e| {
+            (
+                oid.clone(),
+                anyhow::Error::from(e).context("failed to spawn `rad lfs store`"),
+            )
+        })?;
 
-    // The pre-commit hook should already have added+pinned this and written
+    if !output.status.success() {
-    // the note. If it did, just confirm the pin (idempotent). If not (e.g.
+        return Err((
-    // the hook was bypassed with `--no-verify`), self-heal by adding it now.
+            oid,
-    match notes::read_cid(&blob_sha).await {
+            anyhow::anyhow!("rad lfs store failed: {}", String::from_utf8_lossy(&output.stderr)),
-        Ok(Some(cid)) => {
+        ));
-            kubo.pin_add(&cid).await.map_err(|e| (oid.clone(), e))?;
+    }
-        }
+
-        Ok(None) | Err(_) => {
+    // `rad lfs store` prints only the resulting CID on success. The custom-transfer
-            let bytes = tokio::fs::read(&path).await.map_err(|e| {
+    // protocol's "complete" response must echo back the requested oid (not the CID) so
-                (
+    // git-lfs can match it to the in-flight transfer, but we still sanity-check that a
-                    oid.clone(),
+    // CID actually came back rather than trusting the exit code alone.
-                    anyhow::Error::from(e).context(format!("failed to read {path}")),
+    let cid = String::from_utf8_lossy(&output.stdout).trim().to_string();
-                )
+    if cid.is_empty() {
-            })?;
+        return Err((oid, anyhow::anyhow!("rad lfs store exited successfully but printed no CID")));
-            let cid = kubo.add(bytes).await.map_err(|e| (oid.clone(), e))?;
-            kubo.pin_add(&cid).await.map_err(|e| (oid.clone(), e))?;
-            notes::write_cid(&blob_sha, &cid)
-                .await
-                .map_err(|e| (oid.clone(), e))?;
-        }
     }
 
     Ok(oid)
 }
 
-async fn handle_download(kubo: &Kubo, msg: &Value) -> Value {
+async fn handle_download(msg: &Value) -> Value {
-    match download_inner(kubo, msg).await {
+    match download_inner(msg).await {
         Ok((oid, path)) => json!({"event": "complete", "oid": oid, "path": path}),
         Err((oid, e)) => json!({
             "event": "complete",
@@ -109,44 +114,40 @@ async fn handle_download(kubo: &Kubo, msg: &Value) -> Value {
     }
 }
 
-async fn download_inner(kubo: &Kubo, msg: &Value) -> Result<(String, String), (String, anyhow::Error)> {
+/// Delegates entirely to `rad lfs fetch`, which looks up the note, fetches
+/// from IPFS, and decrypts (if needed) on the Radicle side.
+async fn download_inner(msg: &Value) -> Result<(String, String), (String, anyhow::Error)> {
     let oid = field_str(msg, "oid").map_err(|e| (String::new(), e))?;
     let size = field_i64(msg, "size").map_err(|e| (oid.clone(), e))?;
 
-    let blob_sha = notes::pointer_blob_hash(&oid, size)
+    let tmp_path = std::env::temp_dir().join(format!("rad-lfs-{oid}"));
-        .await
-        .map_err(|e| (oid.clone(), e))?;
 
-    let cid = notes::read_cid(&blob_sha)
+    let output = Command::new("rad")
+        .args([
+            "lfs",
+            "fetch",
+            "--oid",
+            &oid,
+            "--size",
+            &size.to_string(),
+            "--out",
+        ])
+        .arg(&tmp_path)
+        .output()
         .await
-        .map_err(|e| (oid.clone(), e))?
+        .map_err(|e| {
-        .ok_or_else(|| {
             (
                 oid.clone(),
-                anyhow::anyhow!(
+                anyhow::Error::from(e).context("failed to spawn `rad lfs fetch`"),
-                    "no CID recorded for oid {oid} (no rad-lfs note on pointer blob {blob_sha}); \
-                     the peer that has this object needs to push its refs/notes/rad-lfs ref"
-                ),
             )
         })?;
 
-    let mut resp = kubo.cat(&cid).await.map_err(|e| (oid.clone(), e))?;
+    if !output.status.success() {
-
+        return Err((
-    let tmp_path = std::env::temp_dir().join(format!("rad-lfs-{oid}"));
+            oid,
-    let mut file = tokio::fs::File::create(&tmp_path)
+            anyhow::anyhow!("rad lfs fetch failed: {}", String::from_utf8_lossy(&output.stderr)),
-        .await
+        ));
-        .map_err(|e| (oid.clone(), anyhow::Error::from(e)))?;
-
-    while let Some(chunk) = resp
-        .chunk()
-        .await
-        .map_err(|e| (oid.clone(), anyhow::Error::from(e)))?
-    {
-        file.write_all(&chunk)
-            .await
-            .map_err(|e| (oid.clone(), anyhow::Error::from(e)))?;
     }
-    file.flush().await.map_err(|e| (oid.clone(), anyhow::Error::from(e)))?;
 
     Ok((oid, tmp_path.to_string_lossy().to_string()))
 }

src/notes.rs

@@ -1,89 +0,0 @@
-use anyhow::{Context, bail};
-use tokio::process::Command;
-
-pub const NOTES_REF: &str = "refs/notes/rad-lfs";
-
-/// Computes the git blob hash for the standard 3-line LFS pointer text for a
-/// given oid/size, without needing the file itself or a repo search — the
-/// pointer format is fully deterministic. Shells out to `git hash-object`
-/// rather than hand-rolling the hash so this automatically respects the
-/// repo's actual object format (SHA-1 vs SHA-256).
-pub async fn pointer_blob_hash(oid: &str, size: i64) -> anyhow::Result<String> {
-    let pointer = pointer_text(oid, size);
-
-    let mut child = Command::new("git")
-        .args(["hash-object", "--stdin", "-t", "blob"])
-        .stdin(std::process::Stdio::piped())
-        .stdout(std::process::Stdio::piped())
-        .stderr(std::process::Stdio::piped())
-        .spawn()
-        .context("failed to spawn `git hash-object`")?;
-
-    {
-        use tokio::io::AsyncWriteExt;
-        let stdin = child.stdin.as_mut().context("no stdin for git hash-object")?;
-        stdin.write_all(pointer.as_bytes()).await?;
-    }
-
-    let output = child.wait_with_output().await?;
-    if !output.status.success() {
-        bail!(
-            "git hash-object failed: {}",
-            String::from_utf8_lossy(&output.stderr)
-        );
-    }
-    Ok(String::from_utf8(output.stdout)?.trim().to_string())
-}
-
-pub fn pointer_text(oid: &str, size: i64) -> String {
-    format!("version https://git-lfs.github.com/spec/v1\noid sha256:{oid}\nsize {size}\n")
-}
-
-/// Reads the `cid=<cid>` note attached to a blob, if any.
-pub async fn read_cid(blob_sha: &str) -> anyhow::Result<Option<String>> {
-    let output = Command::new("git")
-        .args(["notes", "--ref", NOTES_REF, "show", blob_sha])
-        .output()
-        .await
-        .context("failed to spawn `git notes show`")?;
-
-    if !output.status.success() {
-        // Non-zero exit typically means no note exists for this object.
-        return Ok(None);
-    }
-
-    let text = String::from_utf8(output.stdout)?;
-    Ok(parse_cid(&text))
-}
-
-/// Writes/overwrites the `cid=<cid>` note on a blob.
-pub async fn write_cid(blob_sha: &str, cid: &str) -> anyhow::Result<()> {
-    let output = Command::new("git")
-        .args([
-            "notes",
-            "--ref",
-            NOTES_REF,
-            "add",
-            "-f",
-            "-m",
-            &format!("cid={cid}"),
-            blob_sha,
-        ])
-        .output()
-        .await
-        .context("failed to spawn `git notes add`")?;
-
-    if !output.status.success() {
-        bail!(
-            "git notes add failed: {}",
-            String::from_utf8_lossy(&output.stderr)
-        );
-    }
-    Ok(())
-}
-
-fn parse_cid(note: &str) -> Option<String> {
-    note.lines()
-        .find_map(|line| line.strip_prefix("cid="))
-        .map(|s| s.trim().to_string())
-}