Skip to content

Abstract IPFS pinning behind a PinningBackend trait (unify Kubo + Pinata sinks) #364

Description

@andreolf

Context

The node currently has two independent IPFS pinning sinks, wired in side-by-side:

  • crates/gitlawb-node/src/ipfs_pin.rs — pins each new git object to a local Kubo node via /api/v0/add (no-op when ipfs_api is empty).
  • crates/gitlawb-node/src/pinata.rs — uploads the same objects to Pinata (Filecoin-backed warm tier) via the v3 API (no-op when GITLAWB_PINATA_JWT is empty).

Both are good, but they're hardcoded as separate call paths in the push flow. Adding a third sink (another pinning service, direct Filecoin deals, an S3-compatible gateway, incentivized peer pinning) means touching push logic again, and there's no shared notion of pin status/redundancy across sinks.

Proposal

Unify the existing sinks behind a single trait so pinning is provider-agnostic and composable:

#[async_trait]
pub trait PinningBackend: Send + Sync {
    async fn pin(&self, cid: &Cid, data: &[u8]) -> Result<PinReceipt, PinError>;
    async fn unpin(&self, cid: &Cid) -> Result<(), PinError>;
    async fn status(&self, cid: &Cid) -> Result<PinStatus, PinError>;
}
  • Wrap the current Kubo path as KuboBackend and the Pinata path as PinataBackend implementing this trait — pure refactor, no behavior change.
  • Drive the push flow off a Vec<Box<dyn PinningBackend>> selected from config, so operators choose which sinks run and future providers slot in without editing push logic.
  • Optional follow-up: expose aggregate pin status (how many sinks hold a given CID) for redundancy visibility.

Why this matters

It turns "we happen to call Kubo and Pinata" into a documented extension point, and directly answers the common "isn't durability just Pinata?" question by making the redundancy model explicit and swappable. Groundwork for Filecoin deals / incentivized pinning later without another push-path refactor.

Scope for a first PR

Just the trait + wrapping the two existing sinks behind it (no new providers), so it's reviewable and non-breaking. Follow-ups add providers and aggregate status.

Questions for maintainers

  1. Is there an existing storage/sink abstraction you'd prefer I extend rather than introducing a new trait?
  2. Any objection to async-trait, or do you prefer a different async pattern here?
  3. Should pin fan-out be best-effort-any (succeed if ≥1 sink pins) or all-must-succeed? Happy to make it configurable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    crate:nodegitlawb-node — the serving node and REST APIkind:refactorRestructure, behavior preservedsev:lowCosmetic, cleanup, or nice-to-havesubsystem:storageBlob/object store, Arweave, IPFS, archives

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions