{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/misc/protobuffed-contracts/",
  "description": "Define service contracts with Protocol Buffers for non-gRPC systems. Generate serializers, maintain self-documented APIs, and ensure cross-language compatibility.",
  "path": "/misc/protobuffed-contracts/",
  "publishedAt": "2024-05-10T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "API",
    "Networking",
    "Distributed Systems"
  ],
  "textContent": "People typically associate Google's [Protocol Buffer] with [gRPC] services, and rightfully\nso. But things often get confusing when discussing protobufs because the term can mean\ndifferent things:\n\n- A binary protocol for efficiently serializing structured data.\n- A language used to specify how this data should be structured.\n\nIn gRPC services, you usually use both: the protobuf language in proto files defines the\nservice interface, and then the clients use the same proto files to communicate with the\nservices.\n\nHowever, protobuf can be used in non-gRPC contexts for anything that requires a strict\ninterface. You can optionally choose to use the more compact serialization format that gRPC\ntools offer, or just keep using JSON if you prefer. I've seen this use case in several\norganizations over the past few years, though I haven't given it much thought. It definitely\nhas its benefits!\n\nDefining your service contracts with protobuf:\n\n- Allows you to generate message serializers and deserializers in almost any language of\n  your choice.\n- You can choose from a set of serialization formats.\n- The service contracts are self-documented, and you can simply hand over the proto files to\n  your service users.\n- Different parts of a service or a fleet of services can be written in different languages,\n  as long as their communication conforms to the defined contracts.\n\nFor example, consider an event-driven application that sends messages to a message broker\nwhen an event occurs. A consumer then processes these messages asynchronously. Both the\nproducer and consumer need to agree on a message format, which is defined by a contract. The\nworkflow usually goes as follows:\n\n- Define the message contract using the protobuf DSL.\n- Generate the code for serializing/deserializing the messages in the language of your\n  choice.\n- On the publisher side, serialize the message using the generated code.\n- On the consumer side, generate code from the same contract and deserialize the message\n  with that.\n\nDefine the contract\n\nYou can define your service interface in a .proto file. Let's say we want to emit some\nevent in a search service when a user queries something. The query message structure can be\ndefined as follows:\n\nI'm using proto3 syntax, and you can find more about that in the [official proto3 guide].\nNext, you can install the gRPC tools for your preferred programming language to generate the\ninterfacing code that'll be used to serialize and deserialize the messages.\n\nHere's how it looks in Python:\n\n- Install grpcio-tools.\n- Generate the interface. From the directory where your proto files live, run:\n\n    \n\n- This will generate the following files in the root directory:\n\n    \n\nSerialize and publish\n\nOnce you have the contracts in place and have generated the interfacing code, here's how you\ncan serialize a message payload before publishing it to an event stream:\n\nThe code is structured in the following manner now:\n\nDeserialize and consume\n\nOn the consumer side, if you have access to the proto files, you can generate the interface\ncode again via the same commands as before and use it to deserialize the message as follows:\n\nYou can even save the proto files in a common repo, generate the interfacing code\nautomatically for multiple languages, and package them up via CI whenever some changes are\nmerged into the main branch. Then the services can just update those protocol packages and\nuse the serializers and deserializers as needed.\n\n\n\n\n[protocol buffer]:\n    https://protobuf.dev/\n\n[gRPC]:\n    https://grpc.io/\n\n[official proto3 guide]:\n    https://protobuf.dev/programming-guides/proto3/",
  "title": "Protobuffed contracts"
}