Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Chapter 1 · Foundations

Foundations: CAS + CDC + COW

A filesystem state should be complete logically but incremental physically. That is a requirement—not a later optimization—when many agents explore from the same environment.

Why storage efficiency is a design constraint

LayerFS is designed for one environment to support many isolated agent workspaces. Each workspace must behave like a complete filesystem: tools can modify it, checkpoint it, fork it, resume it, compare it, or roll it back.

The straightforward implementation is to copy the current filesystem whenever an agent forks or records a checkpoint. That provides isolation, but it gives these operations the wrong cost model: a small edit to one workspace can require storing another copy of the entire workspace.

Multi-agent development makes that mismatch fundamental rather than incidental:

  • Shared origin. Agents usually begin from the same repository, dependencies, and generated environment.
  • Sparse change. A tool call normally changes a small set of files relative to that shared state.
  • High fan-out and history. Parallel attempts, environment experiments, and MCTS-style rollouts create many related states, while checkpoints retain earlier ones.

If every logical state owns its bytes, storage grows with the total size of every workspace. LayerFS instead chooses a different invariant: the physical cost of a new state should follow what changed, not the size of the filesystem that state exposes.

This choice shapes the filesystem model. A workspace starts from an immutable root. Reads reuse objects reachable from that root; writes create new objects without modifying the shared base; a checkpoint records a new root that still references everything unchanged. Isolation comes from giving each agent its own evolving root—not from duplicating all of its bytes.

How CAS + CDC + COW implement the choice

Structural sharing can be lost at the object, file, or tree level. LayerFS therefore applies reuse at all three levels, and Chapter 1 builds them in that order:

1.1 · Available now
Content-Addressed Storage Object reuse · CAS stores equal bytes once

Content-addressed storage derives identity from bytes. Equal immutable objects converge on one identity and one stored copy.

What remains: changing one byte gives a whole-file object a new identity.

1.2 · Available now
Content-Defined Chunking File-region reuse · CDC keeps a local edit local

Content-defined chunking gives unchanged regions stable boundaries, so a small insertion or deletion replaces nearby chunks rather than the entire file.

What remains: a complete filesystem state still needs a new root without copying every file and directory.

1.3 · Planned
Copy-on-Write Filesystem Trees Tree reuse · COW rebuilds only the changed path

Copy-on-write creates a new file manifest and new directory records along the edited path. Every unrelated file and subtree remains shared.

Result: each checkpoint is a complete immutable root whose physical cost follows the change, not the workspace size.