SDK affordances

SDK affordances

general patterns surfaced by studying sync 1.1 integration in zlay (an atproto relay). the question: what could zat (the atproto building-blocks library underneath it) provide that's broadly useful without being experimental or over-specific?

pattern 1: firehose frame parsing

the problem: zlay manually extracts fields from decoded CBOR Value maps — getString("repo"), get("prevData"), getString("since"), etc. this is repeated across frame_worker.zig and subscriber.zig with subtly different field names for #commit vs #sync vs #account events.

what exists: zat's FirehoseClient already decodes frames into typed FirehoseEvent structs. but zlay doesn't use it — zlay has its own subscriber that connects to individual PDSes, not upstream relays. it receives the same wire format but doesn't get the typed decode.

the affordance: separate the frame parsing from the frame receiving. a function like:

parseFirehoseFrame(header_bytes, payload_bytes) → FirehoseEvent | error

this is already mostly what the firehose client does internally. exposing it independently means any consumer with raw websocket frames can get typed events without adopting the full client lifecycle.

generality: high. any custom subscriber (relay, indexer, archiver) that handles its own websocket connection would benefit. the wire format is stable — it's the spec.

status: not yet justified — zlay is the only consumer (4 days old). wait for a second consumer to hit this need; declining now is reversible, a public API is forever.

pattern 2: operation type bridging

the problem: the firehose wire format and the MST internal format use different representations for the same concept:

field firehose (repoOp) MST (Operation) path path (single string) path new value cid (nullable CID) value (nullable raw bytes) old value prev (nullable CID) prev (nullable raw bytes) action action string inferred from value/prev nullability

zlay's extractOps manually converts between these. it got the field names wrong for months.

the affordance: a conversion function:

repoOpToMstOperation(repo_op) → MstOperation

or better: have verifyCommitDiff accept RepoOp[] directly, doing the conversion internally.

generality: high. this is a spec-level mapping, not an implementation detail.

pattern 3: chain state tracking

the problem: every consumer that verifies commits needs to:

store (rev, data_cid) per DID compare incoming (since, prevData) against stored state update state after successful verification handle chain breaks (log, mark desynchronized, queue resync)

this is ~50 lines of boilerplate that's easy to get wrong (race conditions on concurrent updates, multibase encoding of CIDs for comparison, rev ordering).

what the SDK could offer: not a full state machine (that's too opinionated), but:

a ChainState struct: { rev: []const u8, data_cid: Cid } a checkContinuity(stored: ChainState, incoming_since: ?[]const u8, incoming_prev_data: ?Cid) → .ok | .break_since | .break_prev_data | .first_commit let the caller decide what to do on break

generality: medium-high. the continuity check is pure logic — no I/O, no storage assumptions. but it's also simple enough that it's borderline "just write it." the value is in being correct — the field comparison has encoding subtleties.

hesitation: this might be too thin to justify SDK surface area. maybe it belongs in documentation/examples rather than code.

pattern 4: verify result → storage-ready output

the problem: both verifyCommitCar and verifyCommitDiff return result structs with raw CID bytes. consumers invariably need to:

multibase-encode the data CID for storage extract the rev string store both as the new chain state

zlay does this manually after every verify call.

the affordance: a method on the verify result:

result.chainState(allocator) → ChainState { .rev = "3jui...", .data_cid_encoded = "bafyrei..." }

or just make the verify results include the encoded form.

generality: medium. convenient, but it's two lines of code. might not clear the bar.

pattern 5: commit parsing without verification

the problem: sometimes you want to extract commit metadata (DID, rev, data CID) from a CAR without running the full verification pipeline. zlay does this in the chain continuity check path — it needs the fields for comparison before deciding whether to verify.

what exists: loadCommitFromCAR does exactly this, but it's currently internal to the verify module.

the affordance: make loadCommitFromCAR public, or provide a lightweight parseCommit(car_bytes) → CommitMetadata that stops after extracting fields.

generality: high. indexers, archivers, and debugging tools all want commit metadata without paying for verification.

what NOT to add

things that seem useful but are too specific or experimental:

stateful chain verifier with storage backend — too opinionated about persistence. every consumer has different storage (postgres, rocksdb, sqlite, in-memory). automatic resync orchestration — involves HTTP fetching, backoff, thundering herd mitigation. this is operational, not SDK-level. DID resolution caching — zlay's LRU cache is highly tuned to its concurrency model. a generic cache would either be too simple or too complex. rate limiting / admission control — purely operational. summary: what seems worth pursuing

ranked by (generality × value / complexity):

expose frame parsing independently — high value, low complexity, clearly general operation type bridging — prevents a real class of bugs, spec-level mapping make loadCommitFromCAR public — zero-cost, already implemented, broadly useful chain continuity check helper — useful but borderline; maybe docs/examples instead

the theme: make the SDK's internals composable. zlay doesn't use the firehose client but needs the frame parser. it doesn't need the full verify pipeline but needs commit metadata. the value is in unbundling, not in adding new features.