Internals

These pages document the design decisions inside bcmr that aren't visible from the CLI surface — the measurements behind the shipped ones, the work designed but not shipped, and the explicit non-goals we've decided against. The structure mirrors the layers of the tool:

• Streaming Checkpoint Copy is the algorithm at the heart of every single-file copy: 4 MiB blocks, inline BLAKE3, 64 MB checkpoint with crash-safe write ordering, $\mathcal{O}(1)$ resume verification, and the kernel fast paths (copy_file_range, clonefile/reflink) we fall through to when available.

• Local Multi-File Performance covers the parts above that algorithm: per-file fsync gating, file-level --jobs concurrency, and skipping the whole-source hash when nothing reads it. These are the changes that took bcmr from "13× slower than cp on a 2100-file repo" to "1.5× faster than cp, 2× faster than rsync" without touching the per-file path.

• Wire Protocol & Remote Transfers is the binary frame protocol that drives bcmr serve, the per-worker SSH connection design, the negotiated wire compression (LZ4 + Zstd-3 with auto-skip), and the content-addressed dedup for repeat PUTs.

• Path B: Direct-TCP + AEAD Data Plane is the design of the optional direct-TCP data plane that bypasses SSH's single-stream crypto ceiling using AES-256-GCM over a rendezvous-established TCP connection.

• Non-Goal: Rolling-Checksum Delta-Sync explains why bcmr will not implement rsync's signature feature. Three pieces: the 1996 trade-off has flipped, the workloads that need byte-precise delta already moved to CDC-based tools, and the engineering cost doesn't convert us into an rsync replacement afterward.

• Open Questions lists work that's designed but not shipped: splice(2) zero-copy, io_uring reads, CAS LRU eviction, and the failed pipelined-hashing experiment.

See Also (design discussion in context)

Not every design rationale lives under Internals — some is in documents that serve a different primary audience:

• README — What bcmr does that cp and scp don't is the user-facing capability narrative (integrity defaults, resume semantics, unified local/SSH CLI, JSON for agents) that this Internals section measures and justifies.
• Remote Copy guide — How It Works is the two-paragraph summary of the serve vs legacy split, aimed at a user deciding which mode their transfer will pick.
• SECURITY.md lists which internal components are in-scope for vulnerability reports (direct-TCP data plane, rendezvous auth, session file format, CAS). The threat-model discussion for AEAD framing and session keys lives on the Path B page above.

Cross-Cutting Summary

Most rows below have their own experiment with measured numbers; click through to the relevant page for the table. Every number here is from a specific workload — the benefit column notes the conditions so the row can't be lifted context-free. Rows marked non-goal are design decisions without a measurement (the reasoning lives on the linked page).