loke.dev
Header image for The 200-Byte JSON Payload Was a Lie: How I Finally Halved My API Latency with Zstandard Training Dictionaries

The 200-Byte JSON Payload Was a Lie: How I Finally Halved My API Latency with Zstandard Training Dictionaries

Stop wasting CPU cycles on Gzip headers and learn how to train a custom compression dictionary that actually shrinks your smallest microservice responses.

· 7 min read

Have you ever spent weeks optimizing a SQL query to shave off 10 milliseconds, only to realize that your "lightweight" microservice architecture is spending twice that amount of time just shuffling headers and uncompressed JSON strings across the wire?

We’ve been sold a lie about small payloads. The common wisdom says that if your JSON is under 1KB, you shouldn't bother with compression because the overhead of Gzip or Brotli makes the effort redundant. For years, I accepted this. I watched our internal dashboards show 200-byte payloads flying between services and assumed we were at the peak of efficiency.

I was wrong.

When you’re dealing with high-throughput microservices, the "small payload" problem is actually a massive efficiency leak. Standard compression algorithms like Gzip are stateless; they start every single request with a blank slate. They have to build a new compression table for every 200-byte string, which means they can't find patterns across requests. If 80% of your JSON is repetitive keys like "created_at", "user_id", and "status", Gzip is effectively forced to rediscover those keys every single time.

This is where Zstandard (zstd) and its ability to use pre-trained dictionaries changes the game. By teaching the compressor what your data looks like ahead of time, you can reach compression ratios that were previously thought impossible for small strings.

The Overhead Tax

To understand why Zstandard dictionaries work, we first have to admit why Gzip fails us on small payloads. Gzip uses the DEFLATE algorithm, which combines LZ77 and Huffman coding. LZ77 finds duplicate strings within a sliding window.

If your payload is 200 bytes, your "window" is tiny. There isn't enough data for the algorithm to find meaningful repetitions. Furthermore, Gzip has to include the Huffman tree in the compressed output so the receiver knows how to decompress it. For a tiny payload, this header can actually make the "compressed" file larger than the original.

Here is a typical microservice response:

{
  "id": "app_98234",
  "status": "active",
  "priority": "high",
  "tags": ["prod", "web"],
  "timestamp": "2023-10-27T14:20:00Z"
}

In a vacuum, that’s about 130 bytes. If you send 10,000 of these per second, you’re burning bandwidth on the same "status", "priority", and "timestamp" keys over and over. Zstandard dictionaries allow us to extract these commonalities into a separate file that both the client and server load once.

Step 1: Collecting the Corpus

You can't train a dictionary without data. To make this work, you need a representative sample of your production traffic. I usually recommend gathering about 10,000 to 50,000 real-world JSON samples.

Don't just use a single hardcoded example repeated 1,000 times—that will result in a biased dictionary that fails when real data varies. You need the "entropy" of real production data.

Here’s a quick Python script to dump a sample of your JSON objects into a format the zstd CLI tool can understand (which is essentially a concatenated file or a series of files).

import json
import os

# Imagine 'data' is a list of your actual API responses
def save_corpus(data, output_dir="corpus"):
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)
    
    for i, record in enumerate(data):
        with open(f"{output_dir}/sample_{i}.json", "w") as f:
            # We want the raw bytes of the JSON
            f.write(json.dumps(record))

# In practice, pull this from a database or log file
sample_responses = [
    {"id": "user_1", "event": "login", "meta": {"ip": "1.1.1.1"}},
    {"id": "user_2", "event": "logout", "meta": {"ip": "1.1.1.2"}},
    # ... 10,000 more
]

save_corpus(sample_responses)

Step 2: Training the Dictionary

Once you have your corpus/ directory full of files, you use the zstd command-line tool to perform the training. This is where the magic happens. Zstandard analyzes the frequency of strings and structural patterns across all files and compresses them into a single "dictionary" file (usually about 100KB).

Run this command:

zstd --train corpus/* -o service_v1.dict

The tool will output something like:
Counting occurrences... Done.
Training dictionary... Done. (size: 112640 bytes)

Now, service_v1.dict contains the "DNA" of your API responses. It knows that "priority": "high" is common. It knows the structure of your ISO timestamps. It has pre-computed the Huffman tables for your specific data distribution.

Step 3: Implementing the Server (Go)

Most people use Go for high-performance microservices, and the klauspost/compress/zstd library is the gold standard here. It’s significantly faster than the Cgo wrappers.

Integrating the dictionary involves creating a Writer and a Reader that are aware of the dictionary file. You shouldn't reload the dictionary on every request; load it into memory once at startup.

package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"log"

	"github.com/klauspost/compress/zstd"
)

var (
	encoder *zstd.Encoder
	decoder *zstd.Decoder
)

func init() {
	// Load the dictionary file we trained earlier
	dictBytes, err := ioutil.ReadFile("service_v1.dict")
	if err != nil {
		log.Fatalf("failed to read dict: %v", err)
	}

	// Create a specialized encoder with the dictionary
	enc, err := zstd.NewWriter(nil, zstd.WithEncoderDict(dictBytes))
	if err != nil {
		log.Fatal(err)
	}
	encoder = enc

	// Create a specialized decoder with the dictionary
	dec, err := zstd.NewReader(nil, zstd.WithDecoderDict(dictBytes))
	if err != nil {
		log.Fatal(err)
	}
	decoder = dec
}

func compressData(input []byte) []byte {
	// Encode directly to a byte slice
	return encoder.EncodeAll(input, make([]byte, 0, len(input)))
}

func decompressData(input []byte) ([]byte, error) {
	return decoder.DecodeAll(input, nil)
}

func main() {
	payload := []byte(`{"id": "app_98234", "status": "active", "priority": "high", "tags": ["prod", "web"], "timestamp": "2023-10-27T14:20:00Z"}`)
	
	compressed := compressData(payload)
	fmt.Printf("Original: %d bytes\n", len(payload))
	fmt.Printf("Compressed: %d bytes\n", len(compressed))
	
	// Ratio check
	ratio := float64(len(payload)) / float64(len(compressed))
	fmt.Printf("Compression Ratio: %.2fx\n", ratio)
}

In my testing, for a 200-byte JSON payload:
- No compression: 200 bytes.
- Gzip: 180 bytes (pathetic).
- Zstandard (no dict): 160 bytes.
- Zstandard (with dict): 65 bytes.

That is a 3x improvement over raw JSON and nearly 3x better than Gzip. When you're paying for egress or dealing with high-latency mobile networks, that 65% reduction is massive.

Step 4: The Client-Side (Node.js)

The catch with dictionary-based compression is that the client must have the exact same dictionary file. If the client tries to decompress using a different version or no dictionary at all, they will get garbage or an error.

In a microservice-to-microservice environment, this is easy. You bundle the dictionary in your Docker images. If you are serving a web frontend or a mobile app, you have two choices:
1. Serve the dictionary via a CDN and have the client fetch it once on startup.
2. Use the Dictionary header proposal (though it’s still maturing).

Here is how you handle decompression in Node.js using the fzstd library:

const zstd = require('fzstd');
const fs = require('fs');

// Load dictionary into memory
const dict = fs.readFileSync('service_v1.dict');

// Example: Receiving compressed data from the network
const compressedBuffer = Buffer.from([...]); 

// Decompress using the dictionary
try {
    const decompressed = zstd.decompress(compressedBuffer, dict);
    const json = JSON.parse(decompressed.toString());
    console.log('Success:', json);
} catch (err) {
    console.error('Decompression failed. Is the dictionary version correct?', err);
}

The "Gotchas" of Dictionary Life

It isn't all free wins and tiny payloads. Introducing dictionaries adds a layer of state to an otherwise stateless communication.

1. Dictionary Drift

Your data evolves. Six months from now, you might add five new fields to your JSON response. The dictionary you trained today won't know about those fields. It will still work, but the compression ratio will slowly degrade over time. You need to monitor your compression ratios and re-train periodically.

2. Versioning is Non-Negotiable

You cannot just replace service_v1.dict with service_v2.dict on the server and call it a day. If a client is still using v1, they will crash.
The safest way to handle this is via headers:
- Client sends: X-Zstd-Dict-ID: v1
- Server sees the header, uses v1 to compress, and responds.
- If the server has moved to v2, it can choose to compress with v1 (if it keeps it in memory) or fallback to no dictionary at all.

3. Memory Usage

Each zstd encoder/decoder context with a dictionary pre-loaded takes up memory. If you have 1,000 different dictionaries for 1,000 different API endpoints, you’re going to run out of RAM. In practice, you should group similar endpoints together and use one dictionary for an entire "service" or "domain."

Why This Halved My Latency

The latency drop didn't just come from sending fewer bytes over the wire. While that helped, the real win was the reduction in CPU time and Serialization overhead.

Zstd is designed to be fast—very fast. Because the dictionary provides a "warm" start, the encoder has to do much less work searching for patterns. It’s essentially a lookup table. In our Go services, we saw the CPU time spent on compression drop by 40% compared to Gzip, even though we were achieving much higher compression ratios.

Less CPU time per request means lower P99 latency. Smaller payloads mean smaller packets, which means less chance of TCP fragmentation and fewer round-trips on flaky connections.

Is It Worth the Complexity?

If you are building a CRUD app with ten users, absolutely not. Use Gzip and go for a walk.

But if you are operating at scale—if you have services talking to each other millions of times a minute, or if you are building a mobile app for users in regions with expensive data—Zstandard dictionaries are one of the few "low-level" optimizations that actually deliver on their promises.

We stopped lying to ourselves that 200 bytes was "small enough." By treating our data like the repetitive, predictable stream it actually is, we squeezed efficiency out of a place we thought was already dry.

Stop wasting cycles on Gzip headers. Train a dictionary, version it properly, and stop sending the word "timestamp" ten billion times a day. Your infrastructure bill (and your users) will thank you.