CapDAG documentation

Relay topology

Relay slaves and relay masters, and how relay chains bridge cartridge hosts to the relay switch.

In this section

RelaySlave, RelayMaster, and how relay chains connect components.

Relay pair

A RelaySlave/RelayMaster pair forms a transparent frame bridge over a Unix socket. The pair connects a CartridgeHostRuntime (managing cartridge processes) to the RelaySwitch (routing cap invocations).

graph LR
    SW["RelaySwitch"] <-->|Unix socket| RM["RelayMaster"]
    RM <-->|Unix socket| RS["RelaySlave"]
    RS <--> PH["CartridgeHostRuntime"]

    class RM,RS accent

All regular frames pass through unchanged. Only two frame types are intercepted, RelayNotify and RelayState, which carry control information between the switch and the host.

Source: capdag-rs/src/bifaci/relay.rs.

RelaySlave

RelaySlave sits at the CartridgeHostRuntime end of the Unix socket. It runs two concurrent tasks:

Task 1 (socket → local): Reads frames from the socket (sent by the RelaySwitch via the master). Intercepts RelayState frames and stores their payload. Forwards all other frames to the local CartridgeHostRuntime. Applies a ReorderBuffer to ensure frames arrive in sequence order.

Task 2 (local → socket): Reads frames from the CartridgeHostRuntime. Drops any RelayState frames (cartridges must not originate these). Forwards everything else to the socket.

The slave can also inject RelayNotify frames into the socket stream on demand, triggered when the CartridgeHostRuntime’s capability set changes.

pub struct RelaySlave<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> {
    local_reader: FrameReader<R>,
    local_writer: FrameWriter<W>,
    resource_state: Arc<Mutex<Vec<u8>>>,
}

The resource_state is shared with the CartridgeHostRuntime via resource_state_handle(), giving the host access to the latest RelayState payload (model paths, GPU info, etc.).

Source: relay.rs (RelaySlave).

RelayMaster

RelayMaster sits at the RelaySwitch end of the Unix socket. During initialization, the RelaySwitch reads the first frame from the socket, a RelayNotify from the slave advertising its capabilities and limits.

After initialization, the RelaySwitch’s reader task reads frames directly from the master’s socket. The master stores the latest manifest and limits, which the switch queries for routing decisions.

The master also applies a ReorderBuffer and exposes methods for sending frames (send_state for RelayState) and reading frames (read_frame, which intercepts control frames and delivers only data frames).

Source: relay.rs.

Frame interception

sequenceDiagram
    participant PH as CartridgeHostRuntime
    participant RS as RelaySlave
    participant RM as RelayMaster
    participant SW as RelaySwitch

    Note over PH,SW: RelayNotify (slave → master)
    PH->>RS: Caps changed
    RS->>RM: RelayNotify (manifest, limits)
    Note over RM: Stores manifest, never forwarded
    RM--xSW: (intercepted)

    Note over PH,SW: RelayState (master → slave)
    SW->>RM: RelayState (resource info)
    RM->>RS: RelayState (payload)
    Note over RS: Stores in resource_state, never forwarded
    RS--xPH: (intercepted)

    Note over PH,SW: Regular frames
    PH->>RS: REQ / CHUNK / END / etc.
    RS->>RM: (pass-through)
    RM->>SW: (pass-through)

RelayNotify (slave → master)

RelayNotify frames carry capability advertisements from the slave’s host to the switch. They contain:

  • manifest (bytes): JSON-encoded aggregate manifest of all cartridges on the host. The payload has the structure: 
    {
  "caps": ["cap:effect=none", "cap:in=...;out=...;generate", ...],
  "installed_cartridges": []
}

    caps is a flat list of all capability URN strings from all healthy cartridges on the host. Every host must include CAP_IDENTITY (cap:effect=none); the RelaySwitch rejects connections that do not advertise it.

  • max_frame, max_chunk, max_reorder_buffer: Protocol limits.

When a CartridgeHostRuntime rebuilds its capability set (because a cartridge was added, died, or respawned), it sends a RelayNotify through the RelaySlave. The slave forwards it to the socket. The switch reads it and updates the master’s stored manifest and the aggregate cap table.

RelayNotify frames never leak through the relay to the opposite side. The slave’s Task 1 ignores incoming RelayNotify (from the socket), and the switch never sees them in its regular frame routing; they are consumed during initialization and by the reader task.

RelayState (master → slave)

RelayState frames carry host resource information from the engine to the host process. The payload is opaque (CBOR or JSON) and can contain:

  • macOS security bookmarks for file access.
  • GPU/accelerator availability.
  • Configuration settings.
  • Capability demands (what the engine needs from this host).

The engine sends RelayState through the switch to a specific master’s socket. The slave intercepts it in Task 1 (socket → local) and stores the payload in resource_state. The CartridgeHostRuntime reads this shared state and passes relevant information to cartridges.

RelayState frames never reach the CartridgeHostRuntime as frames; they are intercepted and stored.

Policy (switch → host, passed through)

A Policy is not intercepted. The switch sends it to a master as the first frame after that master’s RelayNotify, and again on every change. The master and the slave forward it unchanged to the CartridgeHostRuntime, which keeps it and gives it to its cartridges. A Policy coming from the host side is a protocol violation, and the relay ends on it: the slave’s Task 2 refuses one sent toward the switch, and a master refuses one read from its slave.

Connection lifecycle

sequenceDiagram
    participant E as Engine
    participant RM as RelayMaster
    participant RS as RelaySlave
    participant PH as CartridgeHostRuntime

    Note over E,PH: 1. Socket creation
    E->>RM: Create Unix socket pair
    E->>RS: Other end of socket

    Note over E,PH: 2. Initial advertisement
    RS->>RM: RelayNotify (caps, limits)

    Note over E,PH: 3. Identity verification
    RM->>RS: REQ(CAP_IDENTITY)
    RS->>PH: (forward)
    PH-->>RS: nonce echo
    RS-->>RM: (forward)

    Note over E,PH: 4. Spawn reader tasks
    Note over E,PH: 5. Bidirectional flow — relay is live
  1. Socket creation: The engine creates a Unix socket pair (or equivalent IPC channel).
  2. Endpoint assignment: One end goes to the RelaySlave (in the CartridgeHostRuntime’s process), the other to the RelaySwitch (in the engine’s process).
  3. Initial advertisement: The RelaySlave sends a RelayNotify on startup, advertising its host’s capabilities and limits. This is triggered by run() with an initial_notify parameter.
  4. Identity verification: The RelaySwitch reads the RelayNotify, then runs identity verification through the relay chain (REQ for CAP_IDENTITY flows through the slave, to the CartridgeHostRuntime, to a cartridge, and the echo response flows back).
  5. Reader tasks: After verification, both sides spawn reader tasks for ongoing frame forwarding.
  6. Bidirectional flow: The relay is now live. Regular frames flow in both directions. RelayNotify and RelayState frames are exchanged as needed.
  7. Shutdown: When either side closes the socket, the other side’s reader task detects EOF and shuts down.

Source: relay.rs, relay_switch.rs.

Multiple masters

A RelaySwitch manages an array of masters, each connected to a different host:

graph TD
    SW["RelaySwitch"]
    SW --> RM0["RM_0"]
    SW --> RM1["RM_1"]
    SW --> RM2["RM_2"]

    RM0 --> RS0["RS_0"] --> PH0["PH_0"]
    RM1 --> RS1["RS_1"] --> PH1["PH_1"]
    RM2 --> RS2["RS_2"] --> PH2["PH_2"]

    PH0 --> P0["P0"]
    PH0 --> P1["P1"]
    PH0 --> P2["P2"]
    PH1 --> P3["P3"]
    PH2 --> P4["P4"]
    PH2 --> P5["P5"]

Each master provides a different set of capabilities (though overlap is possible). The switch aggregates all capabilities into a single cap table and routes each REQ to the master whose cartridges can best handle the requested cap, using is_dispatchable matching and specificity ranking across all masters.

Masters are independent. A failure in one does not affect the others. The switch marks the failed master as unhealthy and continues routing to the remaining healthy masters.

Source: relay_switch.rs.

Swift equivalent

Relay.swift and RelaySwitch.swift in capdag-objc/Sources/Bifaci/ provide the same relay topology for Swift-based hosts. The frame interception rules (RelayNotify and RelayState) and the transparent forwarding behavior are identical.