Learn gRPC and Protobuf in a Single Post: Complete Tutorial From Protobuf Schemas to Streaming RPC and Production

gRPC is a high-performance RPC framework: you define your service and messages in a Protobuf schema, a code generator produces typed stubs in a dozen languages, and calls happen over HTTP/2 with binary-framed, multiplexed transport. Where REST is the default for public HTTP APIs, gRPC is the default for internal service-to-service communication — where you want strict contracts, streaming, and speed. This single post teaches the whole stack in five stages, with hand-drawn diagrams and runnable proto + code.

Learning Roadmap

The roadmap moves from the schema (Stage 1), through codegen (Stage 2), the server (Stage 3), the client (Stage 4), and production concerns (Stage 5). You’ll want the REST API tutorial for the API-design context gRPC is an alternative to.

---

Stage 1 — Protobuf

The schema is the contract

A .proto file defines your messages (data shapes) and services (RPC methods). It’s the single source of truth — every language’s stubs are generated from it.

syntax = "proto3";
package shop.v1;

message OrderRequest { int64 id = 1; }
message Order {
  int64  id    = 1;
  string item  = 2;
  float  total = 3;
  enum Status { PENDING = 0; PAID = 1; SHIPPED = 2; }
  Status status = 4;
}

service Shop {
  rpc GetOrder(OrderRequest) returns (Order);
  rpc Subscribe(stream OrderRequest) returns (stream Order);
}

Messages and fields

• Scalar types: int32/int64, uint32, float/double, bool, string, bytes.
• Field numbers (= 1) are permanent — never change a number once in use (it breaks wire compatibility). Reuse of a number reads old data wrong.
• repeated for lists (repeated string tags = 5;), map<K,V> for maps, nested messages for structure.
• enum with an explicit zero (the default value).
• optional (proto3) for explicit presence tracking.

Pitfall: Field numbers are forever. Adding a field with a new number is safe; reusing or removing a number breaks old clients reading new data. To “remove” a field, keep the number and mark it reserved so nobody reuses it.

---

Stage 2 — Codegen

protoc generates typed stubs

# install protoc + language plugins
protoc --go_out=. --go-grpc_out=. --proto_path=. shop.proto
protoc --python_out=. --grpc_python_out=. shop.proto

From one .proto, protoc generates a matching stub set per language: message types, (de)serialization, and a client + server interface for the service. The contract is the file; the types are generated — no hand-writing JSON mappers per language.

// Go: the generated client
client := shopv1.NewShopClient(conn)
order, err := client.GetOrder(ctx, &shopv1.OrderRequest{Id: 42})

# Python: the generated client
client = shop_v1.ShopStub(channel)
order = client.GetOrder(shop_v1.OrderRequest(id=42))

Both are typed, both serialize to the same wire bytes. A Go client can talk to a Python server — the schema, not the language, is the contract.

---

Stage 3 — Service: RPC Types

Four RPC kinds

RPC	Pattern	Example
Unary	one request, one response	Get(id) -> Order
Server streaming	one request, stream of responses	Subscribe(filter) -> stream Event
Client streaming	stream of requests, one response	Upload(stream Chunk) -> Summary
Bidi streaming	stream of requests, stream of responses	Chat(stream Msg) -> stream Msg

service Shop {
  rpc GetOrder(OrderRequest) returns (Order);                       // unary
  rpc Watch(OrderRequest) returns (stream Order);                    // server stream
  rpc BulkCreate(stream Order) returns (Summary);                   // client stream
  rpc Chat(stream Message) returns (stream Message);                // bidi
}

A server (Go)

type shopServer struct { shopv1.UnimplementedShopServer; db *DB }

func (s *shopServer) GetOrder(ctx context.Context, req *shopv1.OrderRequest) (*shopv1.Order, error) {
    o, err := s.db.Find(req.Id)
    if err != nil { return nil, status.Error(codes.NotFound, "not found") }
    return o, nil
}

// server streaming
func (s *shopServer) Watch(req *shopv1.OrderRequest, stream shopv1.Shop_WatchServer) error {
    for o := range s.db.Watch(req.Id) {
        if err := stream.Send(o); err != nil { return err }
    }
    return nil
}

func main() {
    lis, _ := net.Listen("tcp", ":50051")
    s := grpc.NewServer()
    shopv1.RegisterShopServer(s, &shopServer{db: openDB()})
    s.Serve(lis)
}

Why HTTP/2

gRPC runs over HTTP/2: multiplexed (many concurrent calls on one TCP connection), binary-framed (not text), with header compression (HPACK). One connection serves thousands of in-flight RPCs — no “one TCP per request” overhead like HTTP/1.1 keep-alive-less connections.

---

Stage 4 — Client

Channel, stub, deadline, metadata

conn, _ := grpc.Dial("shop:50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
defer conn.Close()
client := shopv1.NewShopClient(conn)

// deadline: cancel if it takes too long
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()

// metadata (per-RPC headers)
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "bearer "+token)

order, err := client.GetOrder(ctx, &shopv1.OrderRequest{Id: 42})
if err != nil {
    st, _ := status.FromError(err)
    log.Printf("code=%v msg=%v", st.Code(), st.Message())   // codes.NotFound, etc.
}

• Channel — a connection to the server (pooled, reused).
• Stub — the typed client (NewShopClient); one method per RPC.
• Deadline — always set one; gRPC cancels the RPC (and propagates the cancel downstream) if it expires. No deadline = a call that can hang forever.
• Metadata — key/value pairs (like HTTP headers): auth, trace ID, request ID.
• Status codes — codes.OK/NotFound/InvalidArgument/Unavailable/... (like HTTP status, but gRPC’s own set).

Pitfall: A gRPC call with no deadline can hang forever if the server is stuck. Always pass a context.WithTimeout (or a deadline from the incoming request, propagated). gRPC propagates the deadline across service hops, so a slow leaf cancels the whole chain.

---

Stage 5 — Production

Interceptors — middleware for gRPC

func logging(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) {
    start := time.Now()
    resp, err := handler(ctx, req)
    log.Printf("%s %v err=%v", info.FullMethod, time.Since(start), err)
    return resp, err
}
s := grpc.NewServer(grpc.UnaryInterceptor(logging))

Interceptors are the gRPC analog of Express middleware: auth, logging, metrics, tracing, rate limiting — one function wraps every RPC. Use them for all cross-cutting concerns instead of repeating in each handler.

Health checking, reflection

• grpc.health.v1 — a standard health-check service; load balancers (Envoy, K8s) call it to decide whether to route traffic to a backend.
• Reflection — lets tools list your services/methods at runtime (used by grpcurl, the gRPC analog of curl).

Edges: gateway, web, Envoy

• grpc-gateway — a proxy that exposes your gRPC services as REST (/v1/orders/42 → the GetOrder RPC), generating OpenAPI. Use it when you need a REST edge for browsers/external clients.
• grpc-web — lets browsers call gRPC directly (with a small JS library + a proxy).
• Envoy — the proxy/load-balancer that speaks gRPC natively; it’s the default for gRPC load balancing (gRPC breaks normal L4 LBs because of HTTP/2 multiplexing — one connection carries many calls, so connection-based balancing concentrates load).
• Connect (connect-go) — a newer protocol that’s gRPC-compatible over HTTP/1.1 and HTTP/2, simpler in some ways.

Observability

OpenTelemetry trace context propagates through gRPC metadata automatically — a trace started at the edge carries its trace ID into every downstream RPC, so you see the full call graph in Jaeger/Tempo. An interceptor emits Prometheus metrics per RPC (latency histogram by method + status). gRPC + OTel + Envoy is the standard observable microservice stack.

---

Quick-Start Checklist

1. Install protoc + a language plugin (e.g. protoc-gen-go, grpcio-tools).
2. Write a .proto with one message + one unary RPC.
3. Generate stubs in two languages; confirm the types match.
4. Write a server (one RPC), run it on :50051.
5. Write a client with a 2s deadline; call it.
6. Add a streaming RPC (server-stream Watch) to feel the difference from REST.
7. Add an interceptor for logging or auth.
8. Enable health checking so a load balancer can route to you.
9. Put Envoy (or grpc-gateway) at the edge for load balancing / REST translation.
10. Add OpenTelemetry — watch the trace span across the client + server.

Common Pitfalls

• Reusing field numbers — breaks wire compatibility; reserved retired numbers.
• No deadline — a hung server hangs the client forever; always set + propagate a deadline.
• Removing a field — old clients reading new data get wrong (zero) values; keep the number, mark reserved.
• Normal L4 load balancing — breaks because HTTP/2 multiplexes; use Envoy/round-robin on RPCs not connections, or use client-side load balancing.
• Ignoring status codes — gRPC errors come back as status.Error(codes.X, ...); handle them, don’t just check err != nil.
• Forgetting UnimplementedXServer — embedding the generated “unimplemented” base lets you add RPCs later without recompiling every server.
• Blocking in a streaming handler — a Send that blocks (slow client) holds the server thread; use flow control / backpressure.
• No health check — without grpc.health.v1, load balancers can’t tell a ready backend from a dead one.

Further Reading

• gRPC Docs — official, per-language
• Protocol Buffers Docs — the schema language
• Buf — modern protoc replacement + schema registry + linting
• Connect (connect-go) — gRPC-compatible, simpler protocol
• grpc-gateway — REST edge for gRPC

Related guides

gRPC is the internal-service layer — these PyShine tutorials connect to it:

• Learn REST API in One Post — the alternative API style for public/edge; the comparison above.
• Learn Go in One Post — Go is the dominant gRPC language; the server/client snippets above are Go.
• Learn Python in One Post — grpcio / grpcio-tools for Python gRPC.
• Learn Kubernetes in One Post — gRPC services deploy to K8s; health checks wire to liveness probes.
• Learn Observability in One Post — OTel trace context propagates through gRPC metadata.

---

gRPC’s pitch is a strict, typed, polyglot contract with streaming and speed — one .proto file is the source of truth across a dozen languages, calls multiplex over HTTP/2, and deadlines + status codes give you the control REST lacks. The five stages here — proto, codegen, service, client, production — cover everything from a one-message unary call to a streaming, intercepted, health-checked, Envoy-fronted, trace-propagated microservice. The two habits that pay off: treat field numbers as immutable (wire compatibility depends on it), and always set a deadline (gRPC propagates it, so a slow leaf cancels the whole chain). Write a .proto, generate stubs in two languages, and watch a Go client talk to a Python server through one schema — that’s the moment polyglot services click.

Enjoyed this post? Never miss out on future posts by following us