Workload characterization and traffic modeling

Orientation

Definition

Workload characterization is measuring how a system is actually used: request rates over time, the mix of operations, how traffic spreads across keys and tenants, how large the objects are, and how users' sessions behave. Traffic modeling turns those measurements into a forecast that can be used to size and test a design.

Why it matters

Averages conceal almost everything that breaks production. A service averaging 1,000 requests per second may peak at 8,000 for 90 seconds, send 40% of its traffic to 0.1% of its keys, and serve a few objects 1,000 times larger than the median. Each of those features selects a different design: bursts call for queues or headroom, key skew calls for caching and hot-key handling, and size variance calls for separate paths for large objects. Without characterization, a load test can also pass while proving nothing, because it tests a workload that does not exist.

Prerequisites

Functional and non-functional requirements

Study time

About 1.5 hours, including the terminal lab and challenges. This is an estimate.

Related topics

Back-of-the-envelope capacity estimation, Hot keys, hot partitions, and rebalancing (not yet written), Little’s Law and practical queueing theory (not yet written), Constraints, assumptions, and architecture drivers

Learning objectives

• Describe a workload in terms of rate, mix, distribution, size, and time correlation.
• Compute peak-to-mean ratio, read ratio, key skew, and size percentiles from a trace.
• Explain why averages mislead for heavy-tailed distributions.
• Recognize open and closed workloads and how each one behaves under overload.
• Turn a measured profile into a growth forecast and a realistic test.

Production scenario

A photo-sharing service plans to shard its metadata store. The design uses a mean of 12,000 reads and 800 writes per second, hash-partitioned by photo ID across 32 shards, so each shard gets 400 requests per second. After launch, three shards regularly hit 90% CPU while the others stay at 15%.

A trace from the load balancer explains why. The peak-to-mean ratio is 3.4, because of a daily evening peak, and it reaches 11 during live events. The top 0.1% of photos receive 38% of reads, and a celebrity post can reach 40,000 reads per second by itself. Hash partitioning spreads keys evenly, but not load. Object sizes are bimodal: most metadata rows are near 400 bytes, but rows with long comment threads go past 1 MB, and those block the event loop for long stretches. None of this appeared in the mean, and two days of log analysis would have revealed all of it before the migration.

Mental model

Analogy

Consider planning a road network. Knowing that a city averages 50,000 trips per day tells you little. You need to know the rush-hour peak, which junctions get most of the traffic, the fraction of heavy trucks, and how a stadium event changes things. Traffic engineers count cars by junction, hour, and vehicle type for this reason.

Where the analogy breaks

Drivers react to congestion by choosing another route or staying home, and many software clients do the reverse: they retry, adding load exactly when the system is slow. Software traffic can also jump 100-fold in minutes, and a single bot may send more than a city's worth of users. Roads don't have traffic spikes caused by one celebrity's post.

Entities, state, flow, and invariants

• Entities: clients, sessions, requests, keys, tenants, objects, time windows.
• State: the current arrival rate, popularity ranking, and data size, all of which drift.
• Flow: raw logs, then a sample or a stream of counters, then a profile, then a model, then a forecast and test.
• Invariant 1: a profile states the window and population it came from.
• Invariant 2: a test workload reproduces the distribution, not only the mean.
• Invariant 3: forecasts carry a range and date, then get compared to what actually happened.

Foundations

Concepts and terminology

First-principles explanation

A workload is a joint distribution over who asks, for what, when, and how much. A single number compresses this and throws away the variation that governs system behavior. Queues form because of variation, not only because of the mean rate: even a system that has spare capacity on average will have queueing when requests clump, as covered in Little’s Law and practical queueing theory (not yet written).

Skew has opposite effects on different components. It helps caches, because a few keys account for most reads, and it hurts partitioning, because those keys land on a single shard. That's why you should measure instead of assuming "more skew is bad" or "more skew is good". The same applies to time: a 1-minute average can hide a 1-second burst that exhausts a connection pool.

The open or closed distinction matters enormously for testing. Most public web traffic is open-like: new users arrive regardless of how slow you are. A closed load generator with 100 threads backs off when the server slows, so it cannot produce the overload that real traffic does. See Benchmark design and coordinated omission (not yet written).

Mathematics

Peak-to-mean: PMR = peak rate ÷ mean rate. Always state the window. PMR at 1 s can far exceed PMR at 1 min.

Zipf share: with s = 1 and N keys, the share of traffic to the top k keys ≈ ln(k) + 0.577, divided by ln(N) + 0.577. For N = 10,000,000 and k = 10,000: (9.21 + 0.58) ÷ (16.12 + 0.58) = 9.79 ÷ 16.70 ≈ 58.6%. So 0.1% of keys get about 59% of requests under this model.

Coefficient of variation: CV = standard deviation ÷ mean. CV near 1 is roughly Poisson-like in interarrival times, and larger values indicate burstiness.

Growth: with g% monthly growth, load at month m = L0 × (1 + g)^m. At 8% monthly, doubling takes ln 2 ÷ ln 1.08 ≈ 9 months.

Complexity

An exact top-K over n requests and d distinct keys takes O(n) time and O(d) memory, which is infeasible for billions of keys. Space-Saving uses O(k) memory for k counters, with error bounded by n ÷ k. Exact percentiles require O(n log n) for sorting, and a histogram takes O(1) per sample at the cost of bucket-width error. A Count-Min sketch is another option, at O(w × d) memory. See Probabilistic data structures (not yet written).

Mechanism

Normal path

1. Choose a source: load balancer logs are best for external traffic since they see failures too. Add database slow logs and per-service traces for internal views.
2. Choose a window that includes a full cycle: at least a week, plus known events.
3. Extract timestamp, operation, key or route, tenant, size, status, and latency.
4. Compute rate at several resolutions (1 s, 1 min, 1 h), operation mix, skew, sizes, and session behavior.
5. Segment by route, tenant, region, and client type. Aggregates may hide a dominant client.
6. Build the model: distributions and parameters plus growth assumptions with a range.
7. Validate by replaying or generating traffic and comparing the resulting profile and resource use with production.

State transitions

stateDiagram-v2
    [*] --> Assumed: new feature with no traffic
    [*] --> Measured: trace of existing traffic
    Assumed --> Measured: first production data
    Measured --> Validated: replay matches production
    Validated --> InUse: drives sizing and tests
    InUse --> Stale: measured drift exceeds tolerance
    Stale --> Measured: new trace taken

A model starts either from data or from a stated assumption, and new features can only begin as assumptions. Validation is where most teams stop short: until a generated workload reproduces production resource use, you only have a hypothesis.

Concurrency behavior

Request concurrency is itself part of the workload: a fan-out page load sends 30 near-simultaneous requests, and they arrive as a clump. Correlated clients do this at larger scale: mobile apps polling on the hour, cron jobs at midnight, and retries following a blip. Look for spikes aligned to clock boundaries in the per-second data. Adding jitter at clients is frequently the cheapest capacity increase available.

Ordering and consistency

Logs from several machines arrive out of order and with clock differences, and analyses at 1-second resolution are sensitive to this. Sort by timestamp within a tolerance, or use one measurement point. Sampling also changes results: sample by key or session to keep correlations intact, because random request sampling loses session behavior and understates rare keys.

Ownership of data and control

The team that owns the service owns its profile. Infrastructure owns aggregate capacity forecasts, which depend on those profiles. Product and marketing own information about future events, and need a route into forecasting. Logs containing user identifiers are subject to privacy controls, so whoever analyzes them must follow data-handling rules.

Architecture

flowchart LR
    C["Clients"] -->|"HTTPS"| LB["Load balancer: access logs"]
    LB --> S["Services"]
    S --> DB["Datastores: slow logs"]
    LB -->|"log export"| L["Log store: restricted access"]
    DB -->|"log export"| L
    L --> A["Analysis job: anonymizes and aggregates"]
    A --> P["Workload profile: versioned"]
    MK["Product calendar: launches and events"] --> F["Forecast"]
    P --> F
    F --> CP["Capacity plan and design review"]
    P --> G["Load generator"]
    G -->|"replay or model"| T["Test environment"]

Taking data at the load balancer captures traffic the application never logged, including requests that timed out. The profile sits between raw logs and its consumers, so people working on capacity do not require direct access to sensitive logs. The product calendar arrow is the one that is usually missing, and that absence is why launches surprise capacity planners.

flowchart LR
    G["Python generator: sine, burst, Zipf, lognormal"] -->|"NDJSON file"| F["trace.ndjson"]
    G --> B["Python characterize: exact, in memory"]
    F -->|"read line by line"| I["TypeScript Profiler.ingest"]
    I --> C1["Per-second counts"]
    I --> C2["Space-Saving top-K: 1,000 counters"]
    I --> C3["Size histogram: power-of-two buckets"]
    C1 --> O["Profile JSON"]
    C2 --> O
    C3 --> O

The two analyzers show the common trade-off: exact answers using memory proportional to input, or bounded memory with bounded error. Comparing both on the same file is a good way to see how close an approximation is. The lab has you do so.

sequenceDiagram
    participant J as Scheduled job
    participant L as Log store
    participant P as Profile store
    participant O as Service owner
    participant T as Load test
    J->>L: Query 7 days of access logs
    L-->>J: Aggregated rate, mix, and key counts
    J->>P: Load last profile
    P-->>J: Prior version
    J->>J: Compute drift per metric
    J->>O: Ticket: peak-to-mean rose from 3.1 to 4.6
    O->>T: Run updated model at forecast peak
    T-->>O: Saturation at 78 percent of forecast
    O->>P: Publish new profile and capacity action

Refresh is automated, and the judgment is not. The job finds drift and the owner decides what it means. The load test uses the new shape, so it reveals a saturation point that the old profile would have missed.

flowchart TD
    A["Sized on mean: 400 rps per shard"] --> B["Celebrity post: 40,000 rps on one key"]
    B --> C["One shard saturates"]
    C --> D["Latency rises, clients retry"]
    D --> E["Shared connection pools fill up"]
    E --> F["Unrelated requests fail"]
    C -->|"detected: per-shard CPU imbalance"| G["Add hot-key cache and per-key limits"]
    G --> H["Load is flattened and shard recovers"]

The important step is the jump from one shard to unrelated requests, through shared pools. Skew begins as a local problem and spreads through shared resources. The detection path relies on per-shard metrics: a fleet average would hide this entirely. See Hot keys, hot partitions, and rebalancing (not yet written).

Trade-offs

Advantages

• Targets design effort at what actually stresses the system.
• Makes load tests and capacity plans believable.
• Often reveals cheap fixes such as adding client jitter, or removing one abusive client.

Costs

• Log retention and analysis cost money, and there are privacy obligations.
• Detailed profiles go stale, so they require ongoing refresh.
• Too much modeling produces false precision, especially with new products.

Alternatives

• Replay real traffic: the highest fidelity, but it has privacy and state issues. See Shadow traffic and traffic replay (not yet written).
• Synthetic parametric model: easy to scale and share, but misses correlations.
• Benchmark standards like YCSB: comparable across systems, but not your workload.
• Reasoned estimates: the only choice for new products. Replace them quickly.

When to use it

Before sharding, choosing storage, setting autoscaling policies, running a load test, or negotiating committed capacity. Also after any significant product change.

When not to use it

For a product with too little traffic to matter, a brief check of the logs is enough. For a brand-new product there is nothing to measure, so use estimates and plan a review after launch.

Reconsider when

• Peak-to-mean or skew has drifted by more than an agreed tolerance, such as 25%.
• A new client type, region, or high-volume tenant appears.
• Test results and production resource use disagree.

Capacity planning

Inputs and assumptions

Measured: mean of 12,000 reads and 800 writes per second, a 1-minute PMR of 3.4, top 0.1% of keys getting 38% of reads, a cache hit rate of 85% on reads. Assumed: 8% monthly growth for 12 months and 900 reads per second per shard at safe utilization.

Formula and units

• Future mean rate = current × (1 + monthly growth)^months, in requests per second.
• Peak rate = future mean × PMR, in requests per second.
• Database reads = peak reads × (1 − hit rate).
• Shards = database reads ÷ reads per shard.

Intermediate calculations

• Growth factor: 1.08^12 = 2.52.
• Future mean reads: 12,000 × 2.52 = 30,240 per second.
• Peak reads: 30,240 × 3.4 = 102,816 per second.
• Database reads: 102,816 × 0.15 = 15,422 per second.
• Shards: 15,422 ÷ 900 = 17.1, rounded up to 18.
• Using the mean alone gives 30,240 × 0.15 ÷ 900 = 5.04, or 6 shards. That is three times too few.

Headroom

Add 30% for forecast error: 18 ÷ 0.7 = 25.7, so 26 shards. Since shard counts are expensive to change, many teams round to a convenient power of two, here 32. Note also that a drop in hit rate from 85% to 75% raises database reads by 67%, from 15% to 25% of peak. So the cache's hit rate is itself a capacity assumption.

Failure capacity

If a cache node fails and hit rate drops to 60%, then database reads are 102,816 × 0.4 = 41,126 per second, needing 46 shards at safe load. Provisioning for this is costly, so the realistic alternatives are protecting the database through load shedding, with the failure budgeted, or keeping cache replicas warm. See Load shedding and admission control (not yet written) and Cache stampedes and request coalescing (not yet written).

Failure modes

Security and privacy

Threats

Access logs commonly contain IP addresses, user IDs, query strings, and sometimes tokens or email addresses in URLs. Bulk analysis copies that sensitive data into a place with broader access. Traffic data also reveals business information like customer sizes and launch timing.

Trust boundaries

Treat raw logs as sensitive and aggregated profiles as less so. Separate them: restricted store for raw logs, with analysis jobs writing aggregates only. Never send raw production logs into a test environment without review.

Authentication and authorization

Restrict raw-log queries to named roles, and log who queried what. Use service accounts for scheduled analysis with read access only to the needed tables and columns.

Tenant isolation

Per-tenant profiles reveal one customer's activity, so do not share them with other customers. Keep per-tenant rates and quotas in the profile, as this data is how noisy neighbors are found. See Tenant isolation and noisy-neighbor controls (not yet written).

Sensitive data

Scrub query strings and tokens at collection time. Hash keys with a secret salt when analysis requires only cardinality and frequency. Retain raw data for a stated period and expire it. Note that hashed identifiers can remain personal data under regulations such as GDPR. See Data classification, retention, and deletion (not yet written).

Abuse resistance

A workload profile is also an abuse detector. Scrapers, credential stuffing, and enumeration show unusual key distributions, session lengths, and failure rates. Also take care that attackers cannot poison forecasts: exclude identified bot traffic from growth estimates, while keeping it for capacity estimates, as it still consumes resources.

Observability

Metrics

Use a rate counter by route, tenant, and status; a size histogram by route; per-partition load; cache hit rate; and top-K keys from a sketch. Avoid using raw keys as metric labels, because the cardinality would overwhelm the metrics system. See Metric cardinality and histogram design (not yet written).

Logs

Standard fields: timestamp, route template rather than raw URL, status, duration, sizes, client class, hashed tenant ID. Keep templates low-cardinality. Log sampling must be recorded, so rates can be correctly estimated afterward.

Traces

Traces reveal the fan-out ratio: how many internal calls each external request makes. This multiplier is a key parameter of a workload model and it often drifts as features grow.

Dashboards

Show one workload page per service: rate at two resolutions, a week-over-week overlay, operation mix, top clients, partition imbalance (maximum ÷ mean), and profile drift compared with the plan.

Alerts

Page on symptoms like saturation, not on workload shifts themselves. Send tickets on drift: "maximum partition load has exceeded twice the mean for an hour", or "traffic exceeds the planned peak by 20%".

SLO examples

• Profiles for tier-1 services are refreshed within 30 days.
• Maximum-to-mean partition load stays under 2.0 for 99% of hours.
• Forecast error at 3 months stays within 25% for tier-1 services.

Runbook actions

On an unexplained traffic rise: segment by route, client, and region, check for a single source, check retry rate, and compare with the launch calendar. If it is a single client, apply a quota. If it is organic, scale and update the forecast.

Implementation

Architecture of the examples

Python generates a seeded synthetic trace and computes an exact in-memory profile. TypeScript reads the same trace as a stream using bounded state. Using a real readline stream shows how you would handle logs too big to load. No HTTP layer is included because these are offline tools.

Data structures

SpaceSaving keeps k counters. When full, it evicts the minimum and inherits its count, guaranteeing that any key with frequency above n ÷ k stays present. The size histogram uses power-of-two buckets, so the result is an upper bound. The Python generator precomputes a cumulative table and uses binary search for Zipf sampling.

Interfaces

python -m app.main [--seed N] [--zipf S] [--export PATH] prints a profile and optionally writes NDJSON. pnpm start <trace.ndjson> prints a streaming profile. The Profiler.ingest method takes one raw line, so it could consume from any source.

Error handling

Malformed lines are counted in skipped rather than crashing the run, since real logs always contain a few of them, and the count is reported so that high skip rates are visible. An empty trace raises an error because reporting zeros would look like a valid measurement.

Concurrency considerations

The stream processes one line at a time, and for await applies backpressure so a large file does not fill memory. To parallelize this, you would partition the file, build a profile for each part, and merge. Counts and histograms merge easily, while Space-Saving merges only approximately.

Educational limitations

The synthetic trace has independent keys and sizes, unlike real traffic. The per-second map grows with duration; production code would use a fixed window. The Poisson sampling is a normal approximation. Python keeps every request in memory, which is fine at this scale only.

Production hardening

Run analysis in a data warehouse or stream processor, add client and tenant dimensions, scrub sensitive fields, version profiles, compare with the previous version automatically, and validate them against production resource use.

Example source

// Streaming trace profile. Production logs may not fit in memory, so this reads NDJSON line by line
// with bounded state: per-second counts, a keyed count capped by a Space-Saving summary, and a
// fixed-bucket size histogram.
import { createReadStream } from 'node:fs';
import { createInterface } from 'node:readline';
import { fileURLToPath } from 'node:url';

export interface TraceLine { t: number; op: string; key: number; size: number }

export class SpaceSaving {
  private readonly counts = new Map<number, number>();
  private readonly capacity: number;
  constructor(capacity: number) { this.capacity = capacity; }

  add(key: number): void {
    const seen = this.counts.get(key);
    if (seen !== undefined) { this.counts.set(key, seen + 1); return; }
    if (this.counts.size < this.capacity) { this.counts.set(key, 1); return; }
    // Replace the current minimum and inherit its count. This overestimates but never misses
    // an item whose true frequency exceeds total / capacity.
    let minKey = -1;
    let min = Number.POSITIVE_INFINITY;
    for (const [k, c] of this.counts) if (c < min) { min = c; minKey = k; }
    this.counts.delete(minKey);
    this.counts.set(key, min + 1);
  }

  top(n: number): Array<[number, number]> {
    return [...this.counts.entries()].sort((a, b) => b[1] - a[1]).slice(0, n);
  }
}

export interface StreamProfile {
  lines: number;
  skipped: number;
  meanRps: number;
  peakRps: number;
  peakToMean: number;
  readRatio: number;
  topKeys: Array<[number, number]>;
  sizeP99UpperBound: number;
}

export class Profiler {
  private lines = 0;
  private skipped = 0;
  private reads = 0;
  private minT = Infinity;
  private maxT = -Infinity;
  private readonly perSecond = new Map<number, number>();
  private readonly keys = new SpaceSaving(1_000);
  private readonly sizeBuckets = new Array<number>(40).fill(0); // power-of-two buckets

  ingest(raw: string): void {
    let row: Partial<TraceLine>;
    try { row = JSON.parse(raw) as Partial<TraceLine>; } catch { this.skipped++; return; }
    const { t, op, key, size } = row;
    if (typeof t !== 'number' || typeof op !== 'string' || typeof key !== 'number' || typeof size !== 'number' || size < 1) { this.skipped++; return; }
    this.lines++;
    if (op === 'read') this.reads++;
    this.minT = Math.min(this.minT, t);
    this.maxT = Math.max(this.maxT, t);
    const sec = Math.floor(t);
    this.perSecond.set(sec, (this.perSecond.get(sec) ?? 0) + 1);
    this.keys.add(key);
    const bucket = Math.min(this.sizeBuckets.length - 1, Math.ceil(Math.log2(size)));
    this.sizeBuckets[bucket] = (this.sizeBuckets[bucket] ?? 0) + 1;
  }

  profile(): StreamProfile {
    if (this.lines === 0) throw new Error('No valid trace lines.');
    const mean = this.lines / Math.max(this.maxT - this.minT, 1);
    const peak = Math.max(...this.perSecond.values());
    const threshold = Math.ceil(this.lines * 0.99);
    let running = 0;
    let p99 = 0;
    for (let i = 0; i < this.sizeBuckets.length; i++) {
      running += this.sizeBuckets[i] ?? 0;
      if (running >= threshold) { p99 = 2 ** i; break; }
    }
    return {
      lines: this.lines, skipped: this.skipped, meanRps: round(mean), peakRps: peak,
      peakToMean: round(peak / mean), readRatio: round(this.reads / this.lines),
      topKeys: this.keys.top(5), sizeP99UpperBound: p99,
    };
  }
}

const round = (x: number): number => Math.round(x * 100) / 100;

export async function profileFile(path: string): Promise<StreamProfile> {
  const profiler = new Profiler();
  const lines = createInterface({ input: createReadStream(path), crlfDelay: Infinity });
  for await (const line of lines) if (line.trim()) profiler.ingest(line);
  return profiler.profile();
}

if (process.argv[1] === fileURLToPath(import.meta.url)) {
  const path = process.argv[2];
  if (!path) { console.error(JSON.stringify({ level: 'error', msg: 'usage: pnpm start <trace.ndjson>' })); process.exit(2); }
  profileFile(path).then(
    (p) => { console.log(JSON.stringify(p, null, 2)); },
    (err: unknown) => { console.error(JSON.stringify({ level: 'error', msg: 'profile.failed', error: String(err) })); process.exit(1); },
  );
}

"""Generate a synthetic request trace, then characterize it the way you would a production log.

The generator is seeded so results repeat. Traffic follows a daily sine curve with a burst, keys
follow a Zipf-like popularity distribution, and sizes are lognormal. The analyzer reports the
numbers a design needs: rates, peak-to-mean ratio, read ratio, key skew, and size percentiles.
"""

from __future__ import annotations

import argparse
import json
import math
import random
from bisect import bisect_left
from collections import Counter
from itertools import accumulate

from pydantic import BaseModel, Field


class Request(BaseModel):
    t: float  # seconds since trace start
    op: str
    key: int
    size: int


class Profile(BaseModel):
    requests: int
    duration_s: float
    mean_rps: float
    peak_rps: int
    peak_to_mean: float
    read_ratio: float
    distinct_keys: int
    top_1pct_key_share: float
    size_p50: int
    size_p99: int


class GeneratorConfig(BaseModel):
    seed: int = 7
    duration_s: int = Field(default=3_600, gt=0)
    mean_rps: float = Field(default=50, gt=0)
    keys: int = Field(default=10_000, gt=0)
    zipf_s: float = Field(default=1.0, ge=0)
    read_ratio: float = Field(default=0.9, ge=0, le=1)
    burst_at_s: int = 1_800
    burst_len_s: int = 60
    burst_multiplier: float = 5


def generate(cfg: GeneratorConfig) -> list[Request]:
    rng = random.Random(cfg.seed)
    cdf = list(accumulate(1 / (rank**cfg.zipf_s) for rank in range(1, cfg.keys + 1)))
    total = cdf[-1]
    out: list[Request] = []
    for second in range(cfg.duration_s):
        # One full sine cycle over the trace, plus a short burst.
        rate = cfg.mean_rps * (1 + 0.5 * math.sin(2 * math.pi * second / cfg.duration_s))
        if cfg.burst_at_s <= second < cfg.burst_at_s + cfg.burst_len_s:
            rate *= cfg.burst_multiplier
        for _ in range(_poisson(rng, rate)):
            out.append(Request(
                t=second + rng.random(),
                op="read" if rng.random() < cfg.read_ratio else "write",
                key=bisect_left(cdf, rng.random() * total),
                size=max(1, int(rng.lognormvariate(8, 1))),
            ))
    return out


def _poisson(rng: random.Random, lam: float) -> int:
    # A normal approximation is accurate for the rates used here and avoids an O(lam) loop.
    return max(0, round(rng.gauss(lam, math.sqrt(lam))))


def percentile(sorted_values: list[int], q: float) -> int:
    return sorted_values[min(len(sorted_values) - 1, math.ceil(q * len(sorted_values)) - 1)]


def characterize(trace: list[Request]) -> Profile:
    if not trace:
        raise ValueError("cannot characterize an empty trace")
    per_second = Counter(int(r.t) for r in trace)
    duration = max(r.t for r in trace) - min(r.t for r in trace)
    mean = len(trace) / max(duration, 1)
    peak = max(per_second.values())
    keys = Counter(r.key for r in trace)
    top_n = max(1, len(keys) // 100)
    top_share = sum(c for _, c in keys.most_common(top_n)) / len(trace)
    sizes = sorted(r.size for r in trace)
    return Profile(
        requests=len(trace),
        duration_s=round(duration, 1),
        mean_rps=round(mean, 1),
        peak_rps=peak,
        peak_to_mean=round(peak / mean, 2),
        read_ratio=round(sum(r.op == "read" for r in trace) / len(trace), 3),
        distinct_keys=len(keys),
        top_1pct_key_share=round(top_share, 3),
        size_p50=percentile(sizes, 0.50),
        size_p99=percentile(sizes, 0.99),
    )


def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--seed", type=int, default=7)
    parser.add_argument("--zipf", type=float, default=1.0, help="key skew; 0 is uniform")
    parser.add_argument("--export", help="write the trace as NDJSON for the TypeScript analyzer")
    args = parser.parse_args(argv)
    trace = generate(GeneratorConfig(seed=args.seed, zipf_s=args.zipf))
    if args.export:
        with open(args.export, "w", encoding="utf-8") as fh:
            for r in sorted(trace, key=lambda x: x.t):
                fh.write(r.model_dump_json() + "\n")
    print(json.dumps(characterize(trace).model_dump(), indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Terminal lab

Setup

cd topics/workload-characterization-and-traffic-modeling/examples/python
uv sync --quiet

Run

Characterize the default skewed trace, and write it out:

uv run python -m app.main --export /tmp/trace.ndjson | jq '{mean_rps, peak_rps, peak_to_mean, read_ratio, top_1pct_key_share, size_p50, size_p99}'

Compare with uniform keys:

uv run python -m app.main --zipf 0 | jq '{top_1pct_key_share}'

Profile the same file using the bounded-memory analyzer:

cd ../typescript && pnpm install --ignore-workspace --silent && pnpm --silent start /tmp/trace.ndjson | jq '{meanRps, peakRps, peakToMean, readRatio, topKeys: .topKeys[:2], sizeP99UpperBound}'

Try it on your own logs. This extracts per-second peak from a common access-log format:

awk '{print $4}' access.log | sort | uniq -c | sort -rn | head -5

Test

cd ../python && uv run pytest -q

cd ../typescript && pnpm test

Expected output

Measured with the default seed on the review date: about 192,000 requests, a mean of 53.3 rps, a peak of 273 rps, a peak-to-mean ratio of 5.12, and a read ratio of 0.90. The top 1% of keys get 52.6% of the traffic, while --zipf 0 drops that to 1.7%. The median size is about 3,000 bytes and p99 about 30,000 bytes, so the tail is ten times the median. The TypeScript profile agrees on rates and read ratio, and its sizeP99UpperBound of 32,768 is the next power of two above the exact p99. Your exact values may differ slightly with another Python version.

Cleanup

rm -f /tmp/trace.ndjson && rm -rf node_modules ../python/.venv

Troubleshooting

• The access.log command fails if you have no such file, and it assumes the timestamp is in field 4.
• No valid trace lines: run the export step first and check the file path.
• The generator takes a few seconds, since it creates about 192,000 records as validated models.

Practical challenges

Industry standards and trends

Standards and specifications

Few formal standards exist for workload modeling. For measurement mechanics, see W3C Trace Context, November 2021, for correlating requests, and OpenTelemetry semantic conventions for HTTP, a living specification for field names. RFC 9110, June 2022, defines the HTTP semantics such logs record.

Established practice

Research that shaped common practice includes YCSB, Cooper et al., 2010, a benchmark framework with skewed key distributions; Workload Analysis of a Large-Scale Key-Value Store, Atikoglu et al., 2012, based on real Memcached traces; and Wide Area Traffic: The Failure of Poisson Modeling, Paxson and Floyd, 1995. Findings in those papers are specific to the systems they studied, so measure your own.

Managed-service options

Cloud load balancers export access logs to object storage, which can be queried using warehouse engines. Observability vendors provide aggregations, usually sampled. Check how sampling affects tails and rare keys before you rely on them.

Emerging approaches

eBPF-based collection captures traffic data without application changes. AI workloads are changing assumptions, since requests may run for minutes and cost depends on token counts rather than on request counts, so rate alone describes them poorly. This is an observation from public systems, without a settled modeling standard.

Recent changes

OpenTelemetry's HTTP semantic conventions were declared stable in 2023, making field names more consistent across tools. Verify the version in use by your instrumentation.

Adoption constraints

Log retention costs and privacy rules limit how much raw data you can keep. Sampling reduces cost and damages tail analysis. Start with load-balancer logs, since they need no code changes.

Links in this section were checked on 2026-11-03 (access date). Items labelled as emerging or inferred are not settled facts and should be re-checked before you rely on them.

Interview preparation

Review

Key takeaways

• Describe workloads by rate, mix, distribution, size, and time.
• State a window with every rate. PMR at 1 second differs greatly from PMR at 1 minute.
• Skew helps caches and hurts partitions.
• Heavy tails make averages misleading.
• Closed-loop tests hide overload that open traffic causes.
• A model is unproven until generated traffic reproduces production resource use.

Decision checklist

• Does the data cover a full weekly cycle and known events?
• Have I segmented by route, tenant, region, and client?
• Do I know PMR at two or more resolutions?
• Do I know top-key share and per-partition imbalance?
• Do forecasts include a range and a calendar of events?

Production-readiness checklist

• Rate, size, and partition metrics exist per route.
• Traffic can be attributed to a client or tenant.
• Clients use jitter and scheduled jobs are spread out.
• Load tests are validated against production resource use.
• Sensitive fields are scrubbed before analysis.

Glossary

• PMR: peak-to-mean ratio, for a stated window.
• Zipf: a distribution in which frequency falls as a power of rank.
• Heavy tail: rare large values account for a significant share of the total.
• Working set: the data accessed in a window.
• Open workload: arrivals independent of completions.
• Fan-out: internal calls per external request.

Quiz

Further study

Continue with Back-of-the-envelope capacity estimation to convert a profile into capacity, Little’s Law and practical queueing theory (not yet written) for how variation creates queues, and Hot keys, hot partitions, and rebalancing (not yet written) for handling the hot keys this lesson reveals. Benchmark design and coordinated omission (not yet written) covers making tests reproduce these shapes faithfully.

Progress

Progress is stored only in this browser. Export it from the curriculum page.