CapDAG documentation

Cartridge runtime

The cartridge-side runtime: manifest construction, handler registration, mode detection, and the main frame-processing loop.

In this section

The cartridge-side entry point: handler registration, mode detection, and the main frame processing loop.

CartridgeRuntime

CartridgeRuntime is the only supported way for a cartridge to communicate with the host. A cartridge creates a runtime with its manifest, registers handlers for each capability it provides, and calls run(). The runtime handles everything else: handshake, frame parsing, request routing, response framing, and heartbeat replies.

let manifest = build_manifest();
let mut runtime = CartridgeRuntime::with_manifest(manifest);

runtime.register_op("cap:in=...;out=...;generate", || {
    Box::new(MyGenerateOp::default())
});

runtime.run().unwrap();

The struct itself is small:

pub struct CartridgeRuntime {
    handlers: HashMap<String, OpFactory>,   // cap URN → handler factory
    manifest_data: Vec<u8>,                 // JSON-encoded manifest (sent in HELLO)
    manifest: Option<CapManifest>,          // parsed manifest (for CLI mode)
    limits: Limits,                         // negotiated protocol limits
}

Source: capdag-rs/src/bifaci/cartridge_runtime.rs.

Manifest construction

The manifest tells the host what capabilities this cartridge provides. It is built with CapManifest:

let manifest = CapManifest::new(
    "mycartridge".to_string(),
    "1.0.0".to_string(),
    "Description of what this cartridge does".to_string(),
    vec![CapGroup::default_group(vec![identity_cap(), my_cap_1, my_cap_2])],
);

Each Cap in the list has:

  • A cap URN built via CapUrn::from_string() or a builder, which defines what inputs the cap accepts, what outputs it produces, and what operation tags it carries.
  • A title and slug for display and CLI subcommand naming.
  • Arguments (CapArg): each with a media URN, required flag, and one or more sources (ArgSource::Stdin, ArgSource::FilePath, ArgSource::Slot).

The manifest must always include identity_cap(), the identity capability required for handshake verification (see Handshake). CartridgeRuntime::new() panics if CAP_IDENTITY (cap:effect=none) is missing.

Source: capdag-rs/src/bifaci/manifest.rs (CapManifest); capdag-rs/src/cap/definition.rs (Cap, CapArg, ArgSource).

Handler registration

Two methods register handlers:

register_op(cap_urn, factory)

Registers a factory closure that creates a fresh Op<()> instance per request:

runtime.register_op("cap:in=...;out=...;describe", || {
    Box::new(DescribeOp::new(some_config))
});

The factory is called once per incoming REQ. This lets handlers hold per-request state.

register_op_type::<T>(cap_urn)

Registers a type that implements Op<()> + Default. Instances are created via T::default():

runtime.register_op_type::<ExtractMetadataOp>("cap:in=...;out=...;extract;target=metadata");

Handler lookup

When a REQ arrives, the runtime finds the handler by matching the request’s cap URN against registered cap URNs using is_dispatchable(candidate, request) (see Dispatch). When multiple handlers match, the runtime ranks by specificity, preferring exact matches, then refinements (more specific than requested), then fallbacks (more generic).

The cap URN string used in register_op must be parseable by CapUrn::from_string(). It does not need to be character-for-character identical to the manifest entry, but it must dispatch equivalently.

Source: cartridge_runtime.rs (register_op, register_op_type, find_handler).

Mode detection

flowchart TD
    A["runtime.run()"] --> B{CLI arguments?}
    B -->|No args| C["Cartridge CBOR Mode<br/>Read/write binary frames<br/>on stdin/stdout"]
    B -->|Has args| D{First arg?}
    D -->|"manifest"| E["Print JSON manifest"]
    D -->|"--help"| F["Print subcommands"]
    D -->|slug match| G["CLI Mode<br/>Parse args from cap defs<br/>Run handler, output RAW to stdout"]
    D -->|no match| H["UnknownSubcommand error"]

run() checks the process’s command-line arguments:

  • No arguments: Cartridge CBOR mode. The runtime reads binary frames from stdin and writes to stdout. This is how the host invokes cartridges.
  • Any arguments: CLI mode. The runtime parses arguments from cap definitions and runs the matching handler directly. Output goes to stdout.

This dual-mode design means a single binary serves as both a CapDAG cartridge (when launched by the host with no args) and a standalone CLI tool (when run directly by a user).

CLI mode supports:

  • manifest subcommand: prints the JSON manifest.
  • <slug> subcommand: finds the cap whose slug matches, parses arguments from the cap’s CapArg definitions, and invokes the handler.
  • --help: lists available subcommands.

Source: cartridge_runtime.rs (run).

Cartridge mode main loop

When running in cartridge CBOR mode, run() executes these steps:

  1. Handshake: Call handshake_accept() to exchange HELLO frames and negotiate limits. The host initiates; the cartridge responds with its manifest.

  2. Start the writer thread: A plain OS thread receives frames from a std::sync::mpsc channel and writes them to stdout. It is the only thing that touches stdout; handlers send through a tokio channel that a forwarding task drains into it.

2a. Start the reader thread (spawn_frame_reader): A second plain OS thread, with a small runtime of its own, reads frames from stdin. It ANSWERS A HEARTBEAT ITSELF, straight into the writer thread’s queue, and passes every frame — heartbeats included, in arrival order — to the main loop. Nothing between a probe arriving and its reply leaving runs on the handlers’ runtime.

  1. Read loop: Read frames from stdin in a loop.
    • REQ: Look up a handler by cap_urn, resolving the registered pattern that will serve it. Admit through the cap’s whole pool chain (see Concurrency Pools below); if any chain pool is full, enqueue on the cap’s singleton queue and send a level="queued" LOG naming the queue position and blocking pool. Admitted requests spawn a new handler task (tokio::spawn) with its own InputPackage and OutputStream. Record the request ID so continuation frames can be routed to this task.
    • Heartbeat: Already answered by the reader thread, which applied any desired_capacities the probe carried (an unknown pool refuses the whole batch with an UNKNOWN_POOL ERR in place of the reply) and replied with the same ID and the mandatory refreshed pools state map. The frame reaches this loop only so that it turns: a raised capacity admits what was waiting.
    • STREAM_START / CHUNK / STREAM_END / END: Route to the handler task that owns the request ID. These frames feed the handler’s InputPackage.
    • LOG / ERR (from peer responses): Route to the handler task’s peer response channel.
    • CloseStream: The tap-off (12.2 §CloseStream, 15.2 §Runs stop). Close the request’s live feed tap(s) (the one bound to the frame’s stream_id, or all of them when absent) and continue: the feeds end, the handler drains its input, and the request ends naturally with END and complete outputs. Nothing is cancelled. A CloseStream for a request holding no matching open feed is logged and ignored, never turned into an abort.
    • Cancel: Always cancels. The frame’s attribution (meta.code / meta.attribution_class / meta.message, 12.2 §Cancel) is read with Frame::cancel_reason(); an unattributed Cancel yields CancelReason::unattributed, never an error. Close any feeds the request holds, then cancel cooperatively: a queued request is removed and answered with the reason’s terminal ERR immediately; an in-flight handler has its input closed, its credit waiters released, its peer calls cancelled under the same reason, and its terminal ERR (the reason’s code, class and message: CANCELLED/user, ABORTED_COLLATERAL with the originating failure’s class, ABORTED with the host’s, or CANCELLED/internal for an unattributed cancel) is deferred until the handler finishes so no stream is left orphaned. The runtime keeps the reason per cancelled request; it never derives the ERR’s words from circumstance.
    • Policy: Replace the runtime’s live policy (12.2 §Policy). Every handler’s network() reads the new value at its next check, including handlers already running. A REQ before the first Policy is a protocol violation: the runtime answers it with a PROTOCOL_VIOLATION ERR and never dispatches it. The one exception is CAP_IDENTITY, the host’s identity check, which belongs to the handshake (12.3).
    • stdin EOF: Break the loop and shut down.
  2. Shutdown: Wait for all handler tasks to complete, then drop the writer task.

Source: cartridge_runtime.rs (run method).

Writer and reader threads

The writer is an OS thread that loops on a std::sync::mpsc::Receiver<Frame>:

graph LR
    HA["Handler A"] --> CH["mpsc channel"]
    HB["Handler B"] --> CH
    HC["Handler C"] --> CH
    CH --> WT["Writer thread"] --> OUT["stdout"]

All handlers and the runtime itself send frames through this channel. The writer serializes each frame to CBOR, prepends the 4-byte length, and writes to stdout.

Why neither is a task on the cartridge’s runtime. A heartbeat asks whether the cartridge is there, and the host allows ten seconds for the answer (14.1). While the frames were read by a runtime task, answered by the main loop on the same runtime and handed to the writer by a third task on it, the answer waited on whatever the handlers were doing: a handler that kept the runtime’s threads busy — a download’s worth of hashing, a long document rendered page after page — got its cartridge declared CARTRIDGE_UNHEALTHY in the middle of correct work, and the work was lost. The reply path now touches no handler-runtime thread at all.

What a heartbeat therefore says: the process is alive and reading its input. It does not say the handlers are progressing; that is the request’s own activity and stall tracking (13.4).

spawn_blocking still matters for a handler’s OWN frames: its progress and keepalive frames are sent from tasks on the handlers’ runtime, and a handler that blocks every worker thread (e.g. a synchronous FFI model load) cannot send them, so the engine’s activity timeout fires.

run_with_keepalive() solves this by moving blocking work to tokio::task::spawn_blocking (a separate thread pool), freeing the tokio workers so the writer task can flush keepalive frames. See Progress and logging.

Concurrency pools

The runtime’s one capacity concept (bifaci::pools). A cap IS a pool of one: the runtime materializes one singleton pool per registered handler pattern (keyed by the canonical cap URN), every shared pool the manifest declares (PoolDeclarations; memberships must resolve to registered patterns, and a mismatch is a hard startup error), and the reserved all pool containing every cap.

Each pool carries three numbers: declared (the manifest’s shipped default), configured (the operator’s number, delivered via heartbeat probes and starting equal to declared), and optional available (the cartridge’s self-report; absent means static, the normal case). The effective admission bound is min(configured, available) with 0 meaning unlimited everywhere.

A dispatch admits through its cap’s whole CHAIN (singleton, declared pools containing it, all) atomically: every pool must have room or the request waits in line (shared pools have no line of their own). Whenever anything changes (a handler finishes, the operator changes a limit, or the cartridge reports what it can serve), the runtime admits the request that has waited longest among those whose whole chain has room, then the next, until nobody in line can go: in order of arrival across all caps, never cap-biased. pool_handle(name) gives a handler a handle that writes available only (e.g. lowering gpu to 0-concurrency during a model load); it never touches declared or configured, and raising it wakes the run loop so a request in line goes at once rather than at the host’s next frame.

Who is admitted, and when, is the proved model’s decision (formal/CapDAG/Bifaci/Pools.lean, laws in Laws/Pools.lean): the runtime keeps the three numbers each pool carries on the wire and the queued requests themselves; the model keeps each pool’s limit, how many requests hold a slot in it, and the line. It is proved that no pool is ever over its limit, that a release undoes an admission, that the request admitted next is the first in line whose chain has room (nextUp_is_first), that no slot stays free while a request that could use it waits (admits_whenever_possible), and that an arriving request never overtakes: it is admitted at once exactly when its chain has room and nobody already in line could go (arrive_admits_iff).

The full pool-state map rides the HELLO meta and every heartbeat reply; the host folds it into roster stats, so the engine’s admission and both clients see the same numbers the runtime enforces.

Op trait

Handlers implement the Op<()> trait from the ops crate:

#[async_trait]
pub trait Op<T>: Send + Sync {
    async fn perform(&self, dry: &mut DryContext, wet: &mut WetContext) -> OpResult<T>;
    fn metadata(&self) -> OpMetadata;
}
  • perform() runs the handler logic. It receives a DryContext (control flags, dry-run state) and a WetContext (a typed key-value store). The WetContext holds an Arc<Request> under the key "request".
  • metadata() returns descriptive information about the operation.
flowchart LR
    REQ["REQ frame"] --> FH["find_handler<br/>(is_dispatchable)"]
    FH --> F["OpFactory"]
    F --> OP["Op instance"]
    OP --> P["perform()"]
    P -->|Ok| CL["close() → STREAM_END + END"]
    P -->|Err| ER["ERR frame"]

The runtime calls dispatch_op(), which:

  1. Creates a Request from the input, output, and peer invoker.
  2. Inserts the Request into a fresh WetContext.
  3. Calls op.perform().
  4. On success, closes the output stream (sends STREAM_END if the stream was started).
  5. On error, the runtime sends an ERR frame for the request.

Source: cartridge_runtime.rs (dispatch_op); ops crate.

Request object

The Request struct bundles the three things a handler needs:

pub struct Request {
    input: Mutex<Option<InputPackage>>,
    output: Arc<OutputStream>,
    peer: Arc<dyn PeerInvoker>,
}
  • take_input() → InputPackage: The bundle of all input argument streams. Can only be called once; a second call returns an error. This prevents double-consumption bugs.
  • output() → &OutputStream: For emitting response data, progress, and log messages.
  • peer() → &dyn PeerInvoker: For calling other cartridges’ capabilities. See Peer invocation.
  • network() → NetworkAccess: The host’s live network policy (12.2 §Policy). A cheap handle that can be cloned and passed to the code that makes connections, which asks it before each call: allow(Reach::ThirdParty) for an outside service, Reach::OurServices for MachineFabric’s own, Reach::Telemetry for analytics. A reach the policy does not allow returns a NetworkRefused that converts into the NETWORK_POLICY / user failure naming the policy, and the call is not made. The handle reads the runtime’s current value, never a copy taken at dispatch, so a long download or a stream that asks before each call stops at its next one once the host goes offline.

Handlers retrieve the Request from WetContext:

let req = wet.get_arc::<Request>(WET_KEY_REQUEST).unwrap();
let input = req.take_input()?;
let streams = input.collect_streams().await?;
let data = require_stream(&streams, "media:document;ext=pdf")?;
let out = req.output();
out.start(false, None)?;
out.write(result_bytes).await?;

Source: cartridge_runtime.rs (Request).

Swift equivalent

The Swift CartridgeRuntime in capdag-objc/Sources/Bifaci/CartridgeRuntime.swift follows the same design:

  • Construction: CartridgeRuntime(manifest: manifestData).
  • Registration: closures or Op types registered by cap URN string.
  • run() blocks via DispatchSemaphore while an internal async event loop handles frames.
  • Error types mirror the Rust variants (CartridgeRuntimeError with cases for handshake, noHandler, ioError, etc.).

Key differences from Rust:

  • Concurrency model: Swift uses structured concurrency (async/await with Task) instead of tokio. Handler dispatch uses Task + DispatchSemaphore.wait() rather than tokio::spawn.
  • I/O: Synchronous FileHandle reads on stdin instead of async I/O. The semaphore-based blocking model means the event loop runs on a single thread.
  • Deadlock risk: Because dispatchOp() wraps Op.perform() in Task + DispatchSemaphore.wait(), nesting another Task + semaphore inside perform() deadlocks the async executor. This constraint does not exist in Rust’s tokio model.

Source: capdag-objc/Sources/Bifaci/CartridgeRuntime.swift.