Transport protocols, gRPC and service meshes

Microservices and how they talk

The primer describes microservices as "a suite of independently deployable, small, modular services", each running its own process and communicating "through a well-defined, lightweight mechanism". Pinterest, for example, could split into user profile, follower, feed, search and photo upload services.

Two things follow. Services must find each other: service discovery systems such as Consul, etcd and ZooKeeper keep track of registered names, addresses and ports, usually with HTTP health checks. And every call that used to be a function call is now a network call. In the primer's words, remote calls "are usually slower and less reliable than local calls". The primer also warns that microservices "can add complexity in terms of deployments and operations".

RPC and REST

The primer names Protobuf, Thrift and Avro as RPC frameworks. gRPC is one widely used example. By default it uses Protocol Buffers as its interface definition language. It supports four kinds of method: unary, server streaming, client streaming and bidirectional streaming.

Why Protocol Buffers are smaller than JSON

Protobuf writes each field as a numeric tag, (field_number << 3) | wire_type, followed by the value. Integers are varints: 7 bits per byte, with the top bit marking whether another byte follows. So field 1 = 150 encodes as three bytes, 08 96 01. Strings are written as a length followed by their UTF-8 bytes.

JSON instead repeats every field name as text in every message and writes numbers as decimal digits. How much smaller protobuf is depends on the message: names that are long compared with their values favor protobuf most. The visual computes both sizes for its sample messages, so you can see the difference rather than take a quoted percentage.

Note that format and transport are separate choices. JSON can travel over HTTP/2, and protobuf over HTTP/1.1.

HTTP/1.1 head-of-line blocking and HTTP/2 streams

HTTP/1.1 allows pipelining: sending several requests without waiting for each response. But RFC 9112 requires the server to "send the corresponding responses in the same order that the requests were received". So one slow response holds up every response behind it on that connection. RFC 9113 calls this "application-layer head-of-line blocking" and notes that HTTP/1.x clients therefore "use multiple connections to a server to make concurrent requests".

HTTP/2 (RFC 9113) fixes the application layer:

• It allows "interleaving of messages on the same connection". Each request–response pair is its own stream, so a slow one does not delay the others.
• The number of concurrent streams is a setting. RFC 9113 recommends it be "no smaller than 100".
• Header fields are compressed with HPACK, which helps because HTTP fields "are often repetitive and verbose".
• It does not solve everything. RFC 9113: "TCP head-of-line blocking is not addressed by this protocol". A lost packet still delays every stream on that connection.

Service mesh: data plane and control plane

Retries, timeouts, encryption, discovery and telemetry are needed by every service. A service mesh moves them out of application code into proxies. Istio's architecture documentation describes two parts:

1. Data plane: "a set of intelligent proxies (Envoy) deployed as sidecars". Each service instance's inbound and outbound traffic passes through its sidecar.
2. Control plane: it "manages and configures the proxies to route traffic". In Istio this is istiod, which "provides service discovery, configuration and certificate management".
3. Mutual TLS: istiod signs certificates for each workload, and they are rotated periodically. Istio "automatically upgrades all traffic between two" sidecars to mutual TLS. By default the mesh is permissive, accepting plain text as well; strict mode accepts only mutual TLS.

The cost is an extra proxy hop on each side of every call, plus CPU and memory for every sidecar. Istio publishes measured figures for each release; for Istio 1.24 it reports about 0.20 vCPU and 60 MB per sidecar at 1,000 requests per second.